Welcome to the Mochi project

The modern DOE scientific computing portfolio consists of a rich ecosystem of simulation, data analytics, and learning applications, with many distinct data management and analysis needs. The objective of the Mochi project is to design methodologies and tools that allow for the rapid development of distributed data services in support of DOE science. An important aspect of Mochi is composition: common capabilities such as communication, data storage, concurrency management, and group membership are provided under Mochi along with building blocks such as BLOB and key-value stores. These building blocks are mixed together to provide specialized service implementations catering to specific platforms and science needs. Current Mochi research directions include unifying management of disparate data classes from scientific campaigns and applying learning and artificial intelligence to improve the adaptability of data services on heterogeneous DOE platforms.
The Mochi project is a collaboration between Argonne National Laboratory, Los Alamos National Laboratory, Carnegie Mellon University, and the HDF Group. However, Mochi is also bigger than just these partners: Mochi is an open ecosystem enabling the development of a variety of services both within the DOE and internationally.

Quarterly newsletter, January 2021

Platform notes:

Logistics:

  • All of the ANL-hosted git repositories (https://xgitlab.cels.anl.gov/sds/) will be moving within the next few months. We will communicate when that happens.
    • There are no changes to policy or access (in fact, access will likely change for the better); it’s just that the xgitlab.cels instance that we are using is being decommissioned
  • We are working on landing a Spack PR (https://github.com/spack/spack/pull/20273) that will introduce a “mochi-margo” package, maintained by us, to replace the out-of-date “margo” package
    • Once this is done, we will likely start upstreaming more packages that depend on margo-mochi

Mochi service development news:

  • Work continues on a new component called “Bedrock” that can be used to more easily bootstrap microservice compositions (https://xgitlab.cels.anl.gov/sds/bedrock).
    • Bedrock is already available, and we are in the process of updating existing services to use it.
    • You can think of bedrock as a general-purpose Mochi daemon that takes a JSON configuration file describing how to spin up embedded microservices
  • We are actively working on performance tuning of “Benvolio”, which you can think of as a runtime I/O delegation service (i.e. that provides a more generic version of MPI-IO aggregation capabilities). https://xgitlab.cels.anl.gov/sds/benvolio

Upcoming training events:

  • We plan to host a BoF session at this year’s (virtual) ECP annual meeting
    in mid-April, with a mechanism for people to sign up for one on one
    sessions for more detailed interaction. (https://ecpannualmeeting.com/)
  • Please let us know what other kinds of outreach/training you are interested in this year.

Mochi at SC 2019

Highlighted events for the Mochi project at this year’s Supercomputing included:

  • Srinivasan Ramesh, Philip Carns, Robert Ross, Shane Snyder, and Allen Malony. “Profiling Composable HPC Data Services”, Work-In-Progress report at the 4th International Parallel Data Systems Workshop (PDSW 2019). LINK (paper) LINK (slides)
  • Jerome Soumagne et al. “Data Services for High Performance Computing”, Birds of a Feather session. LINK