{"id":201,"date":"2015-05-06T15:53:58","date_gmt":"2015-05-06T15:53:58","guid":{"rendered":"http:\/\/press3.mcs.anl.gov\/romio\/?p=201"},"modified":"2015-05-06T15:53:58","modified_gmt":"2015-05-06T15:53:58","slug":"non-blocking-collective-io","status":"publish","type":"post","link":"https:\/\/wordpress.cels.anl.gov\/romio\/2015\/05\/06\/non-blocking-collective-io\/","title":{"rendered":"Non-blocking collective I\/O"},"content":{"rendered":"<p>The MPI standard defines non-blocking communication. It also defines non-blocking (independent) I\/O. When it comes to collective I\/O, the choices are blocking I\/O or the little-developed and little-used &#8220;split collectives&#8221;.<br \/>\nThe HDF Group pushed to add true non-blocking collective I\/O to the MPI standard. MPI-3.1 finally incorporates this feature. The use cases are motivated by things the HDF5 library would like to do in a portable manner at scale:<\/p>\n<ul>\n<li>Modifying metadata of a dataset: Each process has a cache of metadata, so updates done collectively (thus ensuring everyone&#8217;s cache is consistent between memory and file). When evicting members from this cache, HDF5 could issue a non-blocking collective I\/O request for these typically tiny elements, then go do other work.<\/li>\n<li>Backgrounding Data operations: HDF5 knows a bit about the structure of data on disk due to its file format. It also knows a bit about the data a user will want to operate on. A sufficiently clever HDF5 library could issue non-blocking collective I\/O to either read-ahead in anticipation of what a user will need, or to maintain a write-back cache.<\/li>\n<\/ul>\n<p><a title=\"Sangmin Seo\" href=\"http:\/\/www.mcs.anl.gov\/~sseo\/\">Sangmin Seo<\/a> implemented the non-blocking collective I\/O routines for ROMIO. Implementers might find it interesting that he used the extended generalized requests we added to MPICH way back in 2007.<br \/>\nThis feature is available in mpich-master and in the last few pre-releases.\u00a0 If you try out non-blocking collective I\/O, let us know how it worked.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The MPI standard defines non-blocking communication. It also defines non-blocking (independent) I\/O. When it comes to collective I\/O, the choices are blocking I\/O or the little-developed and little-used &#8220;split collectives&#8221;. The HDF Group pushed to add true non-blocking collective I\/O to the MPI standard. MPI-3.1 finally incorporates this feature. The use cases are motivated by &hellip;<\/p>\n","protected":false},"author":362,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[3],"tags":[],"class_list":["post-201","post","type-post","status-publish","format-standard","hentry","category-features"],"acf":[],"_links":{"self":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/201","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/users\/362"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/comments?post=201"}],"version-history":[{"count":0,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/201\/revisions"}],"wp:attachment":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/media?parent=201"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/categories?post=201"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/tags?post=201"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}