Please join us for the next Mochi quarterly meeting on Thursday, July 27, 2023, at 10am CT. Mochi quarterly meetings are a great opportunity to learn about community activities, share best practices, get help with problems, and find out what’s new in Mochi.
Please suggest agenda items on the Mochi slack space or the [email protected] mailing list.
Microsoft Teams meeting
Join on your computer or mobile app
Click here to join the meeting
Or call in (audio only)
+1 630-556-7958,,254649841#
Mochi updates and agenda items
- Recent presentations
- The Mochi team presented two seminars in June 2023 as part of the Mathematics and Computer Science Division’s CS seminar series. These seminars provide an overview of the state of Mochi and how it can be used in 2023. The first covers high-level motivation, concepts, and key technologies, while the second describes the Mochi methodology for composable data services and highlights success stories in domain-specific data services (HEPnOS) and elastic in situ visualization (Colza).
- Philip Carns. “Mochi Project Overview: the Democratization of Data Services in HPC”, CS seminar series, Argonne National Laboratory Mathematics and Computer Science division, June 13, 2023. ABSTRACT PDF VIDEO
- Matthieu Dorier. “Mochi in Practice: Data Services for High-Energy Physics and Elastic In Situ Visualization Workflows”, CS seminar series, Argonne National Laboratory Mathematics and Computer Science division, June 20, 2023. ABSTRACT PDF VIDEO
- The Mochi team presented two seminars in June 2023 as part of the Mathematics and Computer Science Division’s CS seminar series. These seminars provide an overview of the state of Mochi and how it can be used in 2023. The first covers high-level motivation, concepts, and key technologies, while the second describes the Mochi methodology for composable data services and highlights success stories in domain-specific data services (HEPnOS) and elastic in situ visualization (Colza).
- Recent tutorials
- Matthieu Dorier, Philip Carns, and Marc-André Vef presented the following tutorial on May 21 at ISC High Performance 2023:
- Tutorial: Developing Custom HPC Data Services Using Mochi
- slides and exercises are available online
- The tutorial includes extensive hands-on exercises that can be done in either C or C++. They begin with simple RPC examples to illustrate concepts and then build up to using templates for complete microservices using Bedrock.
- The tutorial materials also include a docker image to make it easier to get started with a development environment.
- Matthieu Dorier, Philip Carns, and Marc-André Vef presented the following tutorial on May 21 at ISC High Performance 2023:
- New features
- Yokan now enjoys 4 new families of functions: yk_fetch , yk_doc_fetch, yk_iter and yk_doc_iter (each with variants to access multiple key/value pairs or documents at once). These functions are equivalent to yk_get, yk_doc_load, yk_list_keyvals and yk_doc_list, respectively, but take a callback that is invoked on each key/value pair or document, instead of taking a buffer in which the key/value pair or document is copied. These functions allow for fewer memory copies and simpler code (no need for the caller to manage their own buffer or call other functions to query the size of values/documents first). The _iter functions also provide automatic pipelining and batching.
- Mercury 2.3.0 is out now, including several notable performance enhancements for libfabric and CXI:
- new “multi-recv” optimization to improve RPC throughput
- avoid performance degradation in FI_SOURCE
- use WAIT_FD for graceful idling on Slingshot (CXI) transports
- HPE Slingshot status update
- Mercury support for Slingshot (CXI) is feature complete and performing well with Mercury 2.3.0, but there are some important usability issues to be aware of regarding Virtual Network Interfaces (VNIs). VNIs are a mandatory method for Slingshot network access control between compute nodes.
- The default job launcher on HPE systems will automatically provision a VNI for MPI. Mercury will inherit and use this same VNI without any additional action on your part.
- However, this default, launcher-provided VNI is not sufficient for communicating across MPI jobs or among manually-launched processes.
- We are in communication with HPE about this issue, but they are still working on a general solution. If you encounter problems, please alert your facility or vendor contacts and let us know about your experience!