Development of the Chunks and Tasks model and runtime library implementations with applications in electronic structure calculations
The goal of this project is to extend our work on the Chunks and Tasks parallel programming model and apply it to massively parallel calculations. See our article "Chunks and Tasks: a programming model for parallelization of dynamic algorithms", http://dx.doi.org/10.1016/j.parco.2013.09.006 , and our latest article "Locality-aware parallel block-sparse matrix-matrix multiplication using the Chunks and Tasks programming model", http://dx.doi.org/10.1016/j.parco.2016.06.005 which were recently published in Parallel Computing. The project regards the development of the Chunks and Tasks programming model for parallel implementation of methods that require dynamic distribution of both work and data. Such methods are difficult to implement using standard languages or libraries such as MPI that leave it to the user to provide the distribution of both work and data. In an application program that uses the Chunks and Tasks programming model, the user defines the algorithm in terms of chunks and tasks without specifying where the work should be performed or how the data should be distributed. Our pilot C++ Chunks and Tasks runtime library implementation uses MPI and pthreads to distribute work and data of Chunks and Tasks application programs on clusters of multicore machines. This project will be used for development and evaluation of the Chunks and Tasks model and runtime library implementations as well as for our ongoing work on distributed-memory parallelization of the Ergo quantum chemistry code (http://ergoscf.org) using Chunks and Tasks. Since the Chunks and Tasks model is intended to work well for massively parallel calculations and our main target application (linear scaling electronic structure calculations with the Ergo program) motivates very large calculations, it is important for us to carry out real calculations on as many nodes as possible. Our recent SNAC Large proposal "SNIC 2016/34-39" was denied, with the following comment: "Although your project is of good scientific quality, in the fierce competition of the limited amount of HPC resources we have viewed it as your demands are better suited for a medium allocation. We welcome you for medium sized applications, [...]" So, it was recommended that we should submit medium applications instead. We hope that the fact that our project according to that evaluation has good scientific quality can strengthen our case to get this medium application approved. Regarding previous resource usage, although we did not use all hours during previous months, our project is at a stage where we need large resources; we have already made significant efforts to make our code run efficiently on Beskow and results so far are very promising. Please consider also that a development project such as ours may have less homogeneous usage of resources compared to other projects, as discussed in [SNIC support #125492], but still need large resources.