{"id":562,"date":"2024-05-17T15:21:24","date_gmt":"2024-05-17T15:21:24","guid":{"rendered":"https:\/\/wordpress.cels.anl.gov\/romio\/?p=562"},"modified":"2024-05-17T15:24:49","modified_gmt":"2024-05-17T15:24:49","slug":"romio-and-large-counts","status":"publish","type":"post","link":"https:\/\/wordpress.cels.anl.gov\/romio\/2024\/05\/17\/romio-and-large-counts\/","title":{"rendered":"ROMIO and &#8220;large counts&#8221;"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Hey remember a long time ago when I went through ROMIO to make it &#8220;big data&#8221; clean? ( <a href=\"https:\/\/wordpress.cels.anl.gov\/romio\/2013\/07\/03\/large-transfers-in-romio\/\" data-type=\"post\" data-id=\"101\">Large transfers in ROMIO<\/a> or this paper about fun with datatypes  <a href=\"https:\/\/ieeexplore.ieee.org\/document\/7018162\">https:\/\/ieeexplore.ieee.org\/document\/7018162<\/a> ) . So I thought I was done when MPI-4 introduced new &#8220;large count&#8221; routines that now take an <code>MPI_Count<\/code> type for counts of items instead of <code>int<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Well it turns out I was *not* done.  A vendor reported problems and we started digging.  Thankfully compilers have gotten more helpful in the last decade.  <code>-fsanitize=undefined<\/code> catches all the runtime overflows, but collaborators kept finding problems.  Clang still has <code>-Wshorten-64-to-32<\/code> , though, and ROMIO can also call the <code>_c<\/code> versions of MPI routines, so it was time to go tackle those lingering warnings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The result is a pretty large and invasive patch set (<a href=\"https:\/\/github.com\/pmodels\/mpich\/pull\/6928\">https:\/\/github.com\/pmodels\/mpich\/pull\/6928<\/a> ) but now you (or more likely your I\/O library like Parallel-NetCDF or HDF5) can pass 2 billion or more items to ROMIO routines.  I think we are going to back-port this to the MPICH-4.1 series as well as the next MPICH-4.2 maintenance update<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hey remember a long time ago when I went through ROMIO to make it &#8220;big data&#8221; clean? ( Large transfers in ROMIO or this paper about fun with datatypes https:\/\/ieeexplore.ieee.org\/document\/7018162 ) . So I thought I was done when MPI-4 introduced new &#8220;large count&#8221; routines that now take an MPI_Count type for counts of items &hellip;<\/p>\n","protected":false},"author":362,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-562","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"acf":[],"_links":{"self":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/562","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/users\/362"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/comments?post=562"}],"version-history":[{"count":2,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/562\/revisions"}],"predecessor-version":[{"id":565,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/562\/revisions\/565"}],"wp:attachment":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/media?parent=562"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/categories?post=562"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/tags?post=562"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}