{"id":246,"date":"2018-11-06T14:06:24","date_gmt":"2018-11-06T14:06:24","guid":{"rendered":"http:\/\/press3.mcs.anl.gov\/mochi\/?page_id=246"},"modified":"2025-05-20T18:38:34","modified_gmt":"2025-05-20T18:38:34","slug":"projects-using-mochi","status":"publish","type":"page","link":"https:\/\/wordpress.cels.anl.gov\/mochi\/projects-using-mochi\/","title":{"rendered":"Projects using Mochi"},"content":{"rendered":"<p>The following external projects are using Mochi components:<\/p>\n<ul>\n<li><strong>UnifyFS<\/strong> (LLNL): Distributed burst buffer file system\n<ul>\n<li><a href=\"https:\/\/github.com\/LLNL\/UnifyCR\">https:\/\/github.com\/LLNL\/UnifyCR<\/a><\/li>\n<li>Michael Brim, Adam Moody, Seung-Hwan Lim, Ross Miller, Swen Boehm, Cameron Stanavige, Kathryn Mohror, Sarp Oral, \u201cUnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage,\u201d 37th IEEE International Parallel &amp; Distributed Processing Symposium (IPDPS 2023), St. Petersburg, FL, May 2023.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Proactive Data Containers<\/strong> (LBNL): Novel data abstraction for storing science data in an object-oriented manner\n<ul>\n<li><a href=\"https:\/\/github.com\/hpc-io\/pdc\">https:\/\/github.com\/hpc-io\/pdc\u00a0<\/a><\/li>\n<li>Houjun Tang, Suren Byna, Francois Tessier, Teng Wang, Bin Dong, Jingqing Mu, Quincey Koziol, Jerome Soumagne, Venkatram Vishwanath, Jialin Liu, and Richard Warren, &#8220;Toward Scalable and Asynchronous Object-centric Data Management for HPC&#8221;, 18th IEEE\/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) 2018 <a href=\"https:\/\/sdm.lbl.gov\/pdc\/pubs\/201805_CCGrid2018_PDCsys.pdf\">[pdf]<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>GekkoFS<\/strong> (JGU Mainz): Temporary distributed file system for HPC applications\n<ul>\n<li><a href=\"https:\/\/storage.bsc.es\/gitlab\/hpc\/gekkofs\">https:\/\/storage.bsc.es\/gitlab\/hpc\/gekkofs<\/a><\/li>\n<li>Vef, Marc-Andr\u00e9 &amp; Moti, Nafiseh &amp; S\u00fc\u00df, Tim &amp; Tocci, Tommaso &amp; Nou, Ramon &amp; Miranda, Alberto &amp; Cortes, Toni &amp; Brinkmann, Andr\u00e9. &#8220;GekkoFS \u2013 A temporary distributed file system for HPC applications&#8221;, IEEE Cluster 2018, Belfast<\/li>\n<\/ul>\n<\/li>\n<li><strong>DAOS<\/strong> (Intel): Distributed Asynchronous Object Storage\n<ul>\n<li><a href=\"https:\/\/github.com\/daos-stack\/daos\">https:\/\/github.com\/daos-stack\/daos<\/a><\/li>\n<li><a href=\"https:\/\/docs.daos.io\/\">https:\/\/docs.daos.io\/<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>IOF<\/strong> (Intel): POSIX I\/O forwarding\n<ul>\n<li><a href=\"https:\/\/github.com\/daos-stack\/iof\">https:\/\/github.com\/daos-stack\/iof<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>Hermes<\/strong> (IIT, the HDF Group, and UIUC): management of I\/O storage tiers\n<ul>\n<li><a href=\"https:\/\/github.com\/HDFGroup\/hcl\">https:\/\/github.com\/HDFGroup\/hcl<\/a><\/li>\n<li>H. Devarajan, A. Kougkas, K. Bateman, and X. Sun. &#8220;HCL: Distributing Parallel Data Structures in Extreme Scales.&#8221; In 2020 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 2020.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Seer<\/strong> (LANL): lightweight insitu wrapper library adding insitu capabilities to simulations\n<ul>\n<li><a href=\"https:\/\/github.com\/lanl\/seer\">https:\/\/github.com\/lanl\/seer<\/a><\/li>\n<li>\n<div class=\"presenter-details presenting\">\n<div class=\"presenter-name\">Pascal Grosset,\u00a0Jesus Pulido, and James Ahrens. &#8220;Personalized In Situ steering for Analysis and Visualization.&#8221; In Proceedings of ISAV 2020: In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization.<\/div>\n<\/div>\n<\/li>\n<\/ul>\n<\/li>\n<li><strong>Chimbuko<\/strong> (BNL): in-situ performance analysis for HPC applications\n<ul>\n<li><a href=\"https:\/\/github.com\/CODARcode\/Chimbuko\">https:\/\/github.com\/CODARcode\/Chimbuko<\/a><\/li>\n<li>Christopher Kelly, Sungsoo Ha, Kevin Huck, Hubertus Van Dam, Line Pouchard, Gyorgy Matyasfalvi, Li Tang, Nicholas D\u2019Imperio, Wei Xu, Shinjae Yoo, and Kerstin Van Dam. &#8220;Chimbuko: A Workflow-Level Scalable Performance Trace Analysis Tool&#8221;. In Proceedings of ISAV 2020: In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Dataspaces<\/strong> (Rutgers): shared tuple-space abstraction for use between HPC applications\n<ul>\n<li><a href=\"https:\/\/github.com\/rdi2dspaces\/dspaces\">https:\/\/github.com\/rdi2dspaces\/dspaces<\/a><\/li>\n<li>Zhang, B., Davis, P.E., Morales, N., Zhang, Z., Teranishi, K., Parashar, M. (2023). Optimizing Data Movement for GPU-Based In-Situ Workflow Using GPUDirect RDMA. In: Cano, J., Dikaiakos, M.D., Papadopoulos, G.A., Peric\u00e0s, M., Sakellariou, R. (eds) Euro-Par 2023: Parallel Processing. Euro-Par 2023. Lecture Notes in Computer Science, vol 14100. Springer, Cham.<\/li>\n<\/ul>\n<\/li>\n<li><strong>CHFS<\/strong> (Tsukuba): ad hoc file system for persistent memory based on consistent hashing\n<ul>\n<li><a href=\"https:\/\/github.com\/otatebe\/chfs\">https:\/\/github.com\/otatebe\/chfs\u00a0<\/a><\/li>\n<li>Osamu Tatebe, Kazuki Obata, Kohei Hiraga, Hiroki Ohtsuji, &#8220;CHFS: Parallel Consistent Hashing File System for Node-local Persistent Memory&#8221;, Proceedings of the ACM International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2022), 2022.<\/li>\n<\/ul>\n<\/li>\n<li><strong>SERVIZ<\/strong> (University of Oregon): A Shared In Situ Visualization Service\n<ul>\n<li><a href=\"https:\/\/github.com\/srini009\/serviz\">https:\/\/github.com\/srini009\/serviz\u00a0<\/a><\/li>\n<li>S. Ramesh, H. Childs and A. Malony, &#8220;SERVIZ: A Shared In Situ Visualization Service,&#8221; in 2022 SC22: International Conference for High Performance Computing, Networking, Storage and Analysis (SC) (SC), Dallas, TX, US, 2022 pp. 277-290.<\/li>\n<\/ul>\n<\/li>\n<li><strong>HXHIM<\/strong> (LANL): Hexadimensional hashing indexing middleware\n<ul>\n<li><a href=\"https:\/\/github.com\/hpc\/hxhim\">https:\/\/github.com\/hpc\/hxhim<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>SOMA<\/strong> (University of Oregon): Framework for in situ monitoring and analysis\n<ul>\n<li><a href=\"https:\/\/github.com\/soma-monitoring-toolbox\">https:\/\/github.com\/soma-monitoring-toolbox<\/a><\/li>\n<li><span class=\"author\">Yokelson D<\/span>, <span class=\"author\">Lappi O<\/span>, <span class=\"author\">Ramesh S<\/span>, et al. <span class=\"articleTitle\">SOMA: Observability, monitoring, and in situ analytics for exascale applications<\/span>. <i>Concurrency Computat Pract Exper<\/i>. <span class=\"pubYear\">2024<\/span>; <span class=\"vol\">36<\/span>(<span class=\"citedIssue\">19<\/span>):e8141.<\/li>\n<\/ul>\n<\/li>\n<li><strong>ProxyStore<\/strong> (University of Chicago): wide-area object reference management for distributed applications\n<ul>\n<li><a href=\"https:\/\/github.com\/proxystore\/proxystore\">https:\/\/github.com\/proxystore\/proxystore<\/a><\/li>\n<li>Pauloski, J. Gregory, Valerie Hayot-Sasson, Logan Ward, Nathaniel Hudson, Charlie Sabino, Matt Baughman, Kyle Chard, and Ian Foster. &#8220;Accelerating communications in federated applications with transparent object proxies.&#8221; In <i>Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis<\/i>, pp. 1-15. 2023.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Dstore<\/strong> (Johns Hopkins University and ANL): Distributed deep learning model repository\n<ul>\n<li>Meghana Madhyastha, Robert Underwood, Randal Burns, and Bogdan Nicolae. 2023. DStore: A Lightweight Scalable Learning Model Repository with Fine-Grain Tensor-Level Access. In Proceedings of the 37th ACM International Conference on Supercomputing (ICS &#8217;23). Association for Computing Machinery, New York, NY, USA, 133\u2013143.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Evostore<\/strong> (ANL and Johns Hopkins University): Distributed deep learning model repository with metadata and provenance\n<ul>\n<li class=\"copy__text csl-response\">Robert Underwood, Meghana Madhyastha, Randal Burns, and Bogdan Nicolae. 2024. EvoStore: Towards Scalable Storage of Evolving Learning Models. In Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing (HPDC &#8217;24). Association for Computing Machinery, New York, NY, USA, 148\u2013159.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Neomem <\/strong>(Inria\/Rennes and ANL): Machine learning data loader for continuous learning with rehearsal buffers\n<ul>\n<li><a href=\"https:\/\/github.com\/thomas-bouvier\/neomem\">https:\/\/github.com\/thomas-bouvier\/neomem<\/a><\/li>\n<li>T. Bouvier et al., &#8220;Efficient Data-Parallel Continual Learning with Asynchronous Distributed Rehearsal Buffers,&#8221; in 2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid), Philadelphia, PA, USA, 2024, pp. 245-254.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Chronolog<\/strong> (Illinois Institute of Technology): Distributed tiered log-ordered storage system\n<ul>\n<li><a href=\"https:\/\/github.com\/grc-iit\/ChronoLog\">https:\/\/github.com\/grc-iit\/ChronoLog<\/a><\/li>\n<li>A. Kougkas, H. Devarajan, K. Bateman, J. Cernuda, N. Rajesh, X.-H. Sun. &#8221; ChronoLog: A Distributed Shared Tiered Log Store with Time-based Data Ordering,&#8221; Proceedings of the 36th International Conference on Massive Storage Systems and Technology (MSST 2020).<\/li>\n<\/ul>\n<\/li>\n<li><strong>Copper<\/strong> (Argonne National Laboratory): Cooperative caching service for large scale data loading\n<ul>\n<li><a href=\"https:\/\/github.com\/argonne-lcf\/copper\">https:\/\/github.com\/argonne-lcf\/copper<\/a><\/li>\n<li>Noah Lewis, Kevin Harms, Kaushik Velusamy, and Huihuo Zheng. &#8220;Copper: Cooperative Caching Layer for Scalable Data Loading in Exascale Supercomputers&#8221;, Proceedings of the 9th International Parallel Data Systems Workshop (PDSW 2024).<\/li>\n<\/ul>\n<\/li>\n<li><strong>HVAC<\/strong> (Oak Ridge National Laboratory): a distributed read cache for node-local storage\n<ul>\n<li>https:\/\/code.ornl.gov\/42z\/hvac-high-velocity-ai-cache<\/li>\n<li>A. Khan et al., &#8220;HVAC: Removing I\/O Bottleneck for Large-Scale Deep Learning Applications,&#8221; 2022 IEEE International Conference on Cluster Computing (CLUSTER), Heidelberg, Germany, 2022, pp. 324-335<\/li>\n<\/ul>\n<\/li>\n<li><strong>OpenFAM<\/strong> (Hewlett Packard Enterprise): an API for access to disaggregated memory\n<ul>\n<li><a href=\"https:\/\/github.com\/OpenFAM\/OpenFAM\">https:\/\/github.com\/OpenFAM\/OpenFAM<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>Cargo<\/strong> (Barcelona Supercomputing Center): HPC data staging service\n<ul>\n<li><a href=\"https:\/\/storage.bsc.es\/gitlab\/hpc\/cargo\">https:\/\/storage.bsc.es\/gitlab\/hpc\/cargo<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>DYAD<\/strong> (Lawrence Livermore National Laboratory): File streaming for producer\/consumer and deep learning workloads\n<ul>\n<li><a href=\"https:\/\/github.com\/flux-framework\/dyad\">https:\/\/github.com\/flux-framework\/dyad<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>Diaspora<\/strong> (ANL, University of Chicago, SLAC, ORNL, and Johns Hopkins University): Distributed resilient event fabric\n<ul>\n<li><a href=\"https:\/\/diaspora-project.github.io\/\">https:\/\/diaspora-project.github.io\/<\/a><\/li>\n<li>Bogdan Nicolae, Justin M Wozniak, Tekin Bicer, Hai Nguyen, Parth Patel, Haochen Pan, Amal Gueroudji, Maxime Gonthier, Valerie Hayot-Sasson, Eliu Huerta, Kyle Chard, Ryan Chard, Matthieu Dorier, Nageswara SV Rao, Anees Al-Najjar, Alessandra Corsi, Ian Foster, Diaspora: Resilience-Enabling Services for Real-Time Distributed Workflows, 2024 IEEE 20th International Conference on e-Science (e-Science).<\/li>\n<\/ul>\n<\/li>\n<li><strong>RECUP<\/strong> (SNL, BNL, ANL, and Texas State University): ScaIable Metadata and Provenance for Reproducible Hybrid Workflows\n<ul>\n<li>https:\/\/sites.google.com\/view\/recup-reproducibility\/<\/li>\n<li>Nicolae B, Islam TZ, Ross R, Pouchard LC, et al (2023) Building the I (Interoperability) of FAIR for Performance Reproducibility of Large-Scale Composable Workflows in RECUP. 2023 IEEE 19th International Conference on e-Science (e-Science).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>In addition, the Mochi project itself has also produced the following user-facing data services:<\/p>\n<ul>\n<li><strong>CARP<\/strong>: dynamic indexing of streaming data\n<ul>\n<li>https:\/\/github.com\/pdlfs\/carp<\/li>\n<li>Ankush Jain, Chuck Cranor, Qing Zheng, Brad Settlemyer, George Amvrosiadis, Gary Grider, \u201cCARP: A Streaming Partitioner for Range Queries\u201d, in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis 24 (2024).<\/li>\n<\/ul>\n<\/li>\n<li><strong>DeltaFS<\/strong>: Scalable file system with in situ indexing\n<ul>\n<li><a href=\"https:\/\/github.com\/pdlfs\/deltafs\">https:\/\/github.com\/pdlfs\/deltafs<\/a><\/li>\n<li>Q. Zheng, C. Cranor, A. Jain, G. Ganger, G. Gibson, G. Amvrosiadis, B. Settlemyer, G. Grider. \u201cStreaming Data Reorganization at Scale with DeltaFS\u201d, In ACM Transactions on Storage, Volume 16, Issue 4, No. 23, September 2020.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Mofka<\/strong>: disributed event streaming for HPC platforms\n<ul>\n<li><a href=\"https:\/\/github.com\/mochi-hpc\/mofka\">https:\/\/github.com\/mochi-hpc\/mofka<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>Colza<\/strong>: Elastic in situ visualization service\n<ul>\n<li><a href=\"https:\/\/github.com\/mochi-hpc\/mochi-colza\">https:\/\/github.com\/mochi-hpc\/mochi-colza<\/a><\/li>\n<li>Matthieu Dorier, Zhe Wang, Utkarsh Ayachit, Shane Snyder, Robert Ross, Manish Parashar. \u201cColza: Enabling Elastic In Situ Visualization for High-performance Computing Simulations.\u201d in Proceedings of the 36th IEEE International Parallel &amp; Distributed Processing Symposium (IPDPS 2022).<\/li>\n<\/ul>\n<\/li>\n<li><strong>HEPnOS<\/strong>: High-Energy Physics&#8217;s new Object Store\n<ul>\n<li><a href=\"https:\/\/github.com\/hepnos\/HEPnOS\">https:\/\/github.com\/hepnos\/HEPnOS\u00a0<\/a><\/li>\n<li>Sajid Ali, Steven Calvez, Philip Carns, Matthieu Dorier, Pengfei Ding, James Kowalkowski, Robert Latham, Andrew Norman, Marc Paterno, Robert Ross, Saba Sehrish, Shane Snyder, and Jerome Soumagne \u201cHEPnOS: a Specialized Data Service for High Energy Physics Analysis,\u201d in Proceedings of the 4th Workshop on Extreme-Scale Storage and Analysis (ESSA 2023).<\/li>\n<\/ul>\n<\/li>\n<li><strong>FlameStore<\/strong>: Object-based storage systems for Keras models used in CANDLE cancer research workflows\n<ul>\n<li><a href=\"https:\/\/github.com\/mochi-hpc\/flamestore\">https:\/\/github.com\/mochi-hpc\/flamestore<\/a><\/li>\n<\/ul>\n<\/li>\n<li><strong>Mobject<\/strong>: In-system object store\n<ul>\n<li><a href=\"https:\/\/github.com\/mochi-hpc\/mobject\">https:\/\/github.com\/mochi-hpc\/mobject\u00a0<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>The following external projects are using Mochi components: UnifyFS (LLNL): Distributed burst buffer file system https:\/\/github.com\/LLNL\/UnifyCR Michael Brim, Adam Moody, Seung-Hwan Lim, Ross Miller, Swen Boehm, Cameron Stanavige, Kathryn Mohror, Sarp Oral, \u201cUnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage,\u201d 37th IEEE International Parallel &amp; Distributed Processing Symposium (IPDPS 2023), &#8230; <a title=\"Projects using Mochi\" class=\"read-more\" href=\"https:\/\/wordpress.cels.anl.gov\/mochi\/projects-using-mochi\/\" aria-label=\"Read more about Projects using Mochi\">Read more<\/a><\/p>\n","protected":false},"author":444,"featured_media":0,"parent":0,"menu_order":5,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_members_access_role":[],"_members_access_error":""},"class_list":["post-246","page","type-page","status-publish"],"acf":[],"_links":{"self":[{"href":"https:\/\/wordpress.cels.anl.gov\/mochi\/wp-json\/wp\/v2\/pages\/246","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cels.anl.gov\/mochi\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/wordpress.cels.anl.gov\/mochi\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/mochi\/wp-json\/wp\/v2\/users\/444"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/mochi\/wp-json\/wp\/v2\/comments?post=246"}],"version-history":[{"count":23,"href":"https:\/\/wordpress.cels.anl.gov\/mochi\/wp-json\/wp\/v2\/pages\/246\/revisions"}],"predecessor-version":[{"id":634,"href":"https:\/\/wordpress.cels.anl.gov\/mochi\/wp-json\/wp\/v2\/pages\/246\/revisions\/634"}],"wp:attachment":[{"href":"https:\/\/wordpress.cels.anl.gov\/mochi\/wp-json\/wp\/v2\/media?parent=246"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}