Skip to content

ROMIO

  • ROMIO: A High-Performance, Portable MPI-IO Implementation

Deferred Open

August 5, 2003 by Latham, Robert J.

When I came to Argonne in 2002, my second project was to implement “deferred open”, where we would skip opening the file if certain hints were given.  We never got around to writing a paper about this optimization, though.  There’s a brief mention in the ROMIO users guide , but it wouldn’t hurt to have a bit more documentation about this feature.
First, some background. ROMIO has an optimization for collective I/O called “two-phase collective buffering”.  When writing, ROMIO selects a subset of processes as “I/O aggregators” . These aggregators are the MPI processes that actually write data to the file, after collecting data from all the other processors.   When reading, these I/O aggregators read the data in some file-system friendly way, then scatter the data out to the other MPI processors.  Observe that in two-phase, the non-aggregator processes never touch the file.  We use this observation to implement a deferred open strategy for non-aggregators.
To enable deferred open, two hint conditions must be true

  • romio_cb_write and romio_cb_read must not be “disable”.  That’s the default setting for every file system everywhere, though:  it’s rare to find this condition not met
  • romio_no_indep_rw must be “true”.  With this hint, the user has told ROMIO “I will not do any independent I/O”.   ROMIO will then attempt to avoid opening the file on any non-aggregator processes.
  • optional: The cb_config_list and cb_nodes hints can be given to further control which nodes are aggregators

The “deferred” part comes from the fact that MPI Info tunables are hints, not contracts.   The user might lie to ROMIO, specifying “romio_no_indep_rw” to “true” and then go right ahead and carry out a bunch of independent I/O operations.  In that case, ROMIO will open the file just before the independent I/O operation happens — we say the open has been deferred.
 

Post navigation

Previous Post:

citing ROMIO

Next Post:

ROMIO on Blue Gene /L

Recent Posts

  • ROMIO and MPICH-4.3.0
  • ROMIO and “large counts”
  • Hintdump: a small utility for poking at MPI implementations.
  • Quobyte file system
  • ROMIO at SC 2019

Recent Comments

  • ROMIO » New ROMIO optimizations for Blue Gene /Q on bglockless
  • bglockless | ROMIO on New ROMIO optimizations for Blue Gene /Q

Archives

  • February 2025
  • May 2024
  • April 2023
  • October 2020
  • November 2019
  • February 2019
  • December 2018
  • November 2018
  • September 2018
  • November 2017
  • September 2017
  • March 2017
  • August 2016
  • June 2016
  • January 2016
  • December 2015
  • November 2015
  • June 2015
  • May 2015
  • February 2015
  • October 2014
  • August 2014
  • July 2014
  • June 2014
  • August 2013
  • July 2013
  • February 2012
  • September 2010
  • November 2009
  • November 2008
  • September 2008
  • February 2006
  • August 2003
  • February 2002

Categories

  • development
  • features
  • gpfs
  • intel-mpi
  • lustre
  • presentations
  • publications
  • releases
  • tuning
  • Uncategorized

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org
© 2025 ROMIO | WordPress Theme by Superbthemes