{"id":75,"date":"2014-06-12T16:51:40","date_gmt":"2014-06-12T16:51:40","guid":{"rendered":"http:\/\/press3.mcs.anl.gov\/romio\/?p=75"},"modified":"2023-04-07T19:16:56","modified_gmt":"2023-04-07T19:16:56","slug":"romio-and-intel-mpi","status":"publish","type":"post","link":"https:\/\/wordpress.cels.anl.gov\/romio\/2014\/06\/12\/romio-and-intel-mpi\/","title":{"rendered":"ROMIO and Intel-MPI"},"content":{"rendered":"<p>ROMIO, in various forms, provides the MPI-IO implementation for just about every MPI implementation out there.\u00a0\u00a0 These implementations incorporate ROMIO&#8217;s hints when they pick up our source code, but they also add additional tuning parameters via environment variables.<br \/>\nThe <a title=\"Intel MPI library\" href=\"https:\/\/software.intel.com\/en-us\/intel-mpi-library\">Intel MPI library<\/a> uses ROMIO, but configures the file-system specific drivers a bit differently.\u00a0\u00a0 in MPICH, we select which file system drivers to support at compile-time with the <code>--with-file-system<\/code> configure flag.\u00a0 These selected drivers are compiled directly into the MPICH library.\u00a0 Intel-MPI builds its\u00a0 file-system drivers as loadable modules, and relies on two environment variables to enable and select the drivers<\/p>\n<ul>\n<li>I_MPI_EXTRA_FILESYSTEM<\/li>\n<li>I_MPI_EXTRA_FILESYSTEM_LIST<\/li>\n<\/ul>\n<p>Let&#8217;s say you had a Lustre file system, like [edit: archives have moved] <a href=\"https:\/\/forum.hdfgroup.org\/t\/phdf-5-independent-vs-collective-io\/3235\">This fellow on the HDF5 mailing list<\/a> Then you would invoke mpiexec like this:<\/p>\n<pre> mpiexec -env I_MPI_EXTRA_FILESYSTEM on \\\n        -env I_MPI_EXTRA_FILESYSTEM_LIST lustre -n 2 .\/test<\/pre>\n<p>I found this information in the <a title=\"Intel MPI Reference Manual\" href=\"http:\/\/software.intel.com\/sites\/products\/documentation\/hpc\/ics\/impi\/41\/lin\/Reference_Manual\/index.htm\">Intel MPI library Reference Manual<\/a>, which contains a ton of other tuning parameters.<br \/>\n(Update 12 May 2015): Intel 5.0.2 and newer have GPFS support.\u00a0 One would enable it the same way with the I_MPI_EXTRA_FILESYSTEM_LIST<\/p>\n<blockquote>\n<pre>mpiexec -env I_MPI_EXTRA_FILESYSTEM on \\ -env\u00a0 I_MPI_EXTRA_FILESYSTEM_LIST gpfs<\/pre>\n<\/blockquote>\n<p>(Update 7 April 2023): Intel MPI has added a few more file systems:\u00a0 panasas (panfs) and DAOS (daos) are supported too.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>ROMIO, in various forms, provides the MPI-IO implementation for just about every MPI implementation out there.\u00a0\u00a0 These implementations incorporate ROMIO&#8217;s hints when they pick up our source code, but they also add additional tuning parameters via environment variables. The Intel MPI library uses ROMIO, but configures the file-system specific drivers a bit differently.\u00a0\u00a0 in MPICH, &hellip;<\/p>\n","protected":false},"author":362,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"footnotes":"","_members_access_role":[],"_members_access_error":""},"categories":[4,6,7,11],"tags":[13],"class_list":["post-75","post","type-post","status-publish","format-standard","hentry","category-gpfs","category-intel-mpi","category-lustre","category-tuning","tag-intel-mpi"],"acf":[],"_links":{"self":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/75","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/users\/362"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/comments?post=75"}],"version-history":[{"count":2,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/75\/revisions"}],"predecessor-version":[{"id":554,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/75\/revisions\/554"}],"wp:attachment":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/media?parent=75"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/categories?post=75"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/tags?post=75"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}