Uncategorized

Darshan version 3.1.7 is now available for release HERE! This version addresses a few bug fixes in the prior Darshan release and also contains a couple of new features:

• Bug fix in handling of DXT module data in the darshan-convert utility
  • Reported by Mahzad Khoshlessan
• Bug fix in darshan-parser backwards compatibility: Darshan logs generated by Darshan versions prior to 3.1.0 may have included STDIO counters that were not properly up-converted
  • Reported by Teng Wang
• Bug fix to MiB reported in I/O performance estimate of darshan-job-summary when both POSIX and STDIO data present
  • Reported/fixed by Glenn Lockwood
• Added Darshan wrapper for ‘__open_2()’ call, needed for properly instrumenting open operations with some versions of gcc/glibc
  • Reported by Cormac Garvey
• Added an instrumentation module for the MDHIM key/val storage system
• Added support for properly handling ‘rename()’, ‘dup()’, ‘fileno()’, and ‘fdopen()’ operations in Darshan

Please report any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.

We are proud to announce that the Darshan team at Argonne National Laboratory has won a 2018 R&D 100 award!  This prestigious award is given to the 100 top technologies of the year as chosen by R&D Magazine.  We would like to sincerely thank the entire Darshan user community for supporting us and helping to make the project so successful!
For more information about the award please see the Darshan R&D 100 award news article at MCS.

Stop by and check out the following events related to Darshan at SC18:

• Glenn Lockwood et al. “A Year in the Life of a Parallel File System”, SC18 technical program paper.
• Robert Latham et al. “Parallel I/O in Practice”, Tutorial.
• Jakob Luettgau et al. “Toward Understanding I/O Behavior in HPC Workflows”, PDSW workshop paper.
• Philip Carns et al. “Analyzing Parallel I/O”, Birds of a Feather session.

The R&D 100 Conference recently announced that Darshan is a 2018 R&D 100 Award finalist.  The award winners will be announced in November 2018.  Thank you everyone for your support!

Update: the normal Darshan download links are now working again.
Our FTP server is down as of 03/29/2018 and we are not yet sure when it will be returned to service.  If you need a tar ball with the latest Darshan release, but our normal download links aren’t working, then you can use this link instead:
https://xgitlab.cels.anl.gov/darshan/darshan/repository/darshan-3.1.6/archive.tar.gz
That will generate a tar ball directly from the git repository tag.

Darshan version 3.1.6 is available for download here. This release addresses two bugs discovered in version 3.1.5:

• Bug fix in mapping of Darshan’s ‘MPI_File_read_all’ wrapper to the underlying MPI symbol. This results in a crash if a dynamically-linked application preloads the Darshan library and calls ‘MPI_File_read_all’.
• Bug fix for linker failures caused when an application links in a library that calls MPI externally (e.g., PETSc).
  • Reported by Bilel Hadri

We highly recommend upgrading from version 3.1.5 to 3.1.6 particularly for deployments on production systems, as these bugs could be disruptive for end users.
Please report any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.

Darshan version 3.1.5 is now available for download here.
The list of new features and bug fixes include:

• Darshan’s MPI instrumentation method now intercepts both MPI and PMPI symbols to workaround MPI library bindings that are calling PMPI routines directly
  • Contributed in part by Chris Zimmer
• darshan-util now includes an analysis tool for visualizing DXT traces: dxt_analyzer
  • ​Contributed by Alex Sim
• Darshan now supports the DARSHAN_EXCLUDE_DIRS environment variable, which specifies a comma-separated list of paths to avoid instrumenting at runtime
  • Contributed by Cristian Simarro
• Darshan’s regression test system now includes test harnesses for Cray systems @ the ALCF (Theta) and NERSC (Cori/Edison)
• Bug fix in Darshan’s shutdown procedure leading to potentially negative timer values
• Bug fix in Darshan’s autoconf scripts causing specified libbz2 locations to be ignored
  • Contributed by Glenn Lockwood
• Bug fix in Darshan’s wrapping of H5get_libversion that was causing linker failures in some cases
  • Reported by Jialin Liu
• Bug fix in darshan-merge utility related to logs containing DXT data
  • Reported by Glenn Lockwood

Please report any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.

We encourage anyone interested in Darshan and related technology to attend the following events at SC17 in Denver:

• Monday’s “Parallel I/O in Practice” tutorial by Rob Latham, Rob Ross, Brent Welch, and Katie Antypas.  It covers a broad range of topics in parallel I/O, including how to characterize and understand I/O behavior:
  • http://sc17.supercomputing.org/presentation/?id=tut145&sess=sess231
• Monday’s PDSW-DISCS workshop will include the presentation of a paper entitled “UMAMI: A Recipe for Generating Meaningful Metrics through Holistic I/O Performance Analysis” by Glenn Lockwood.  It highlights how to integrate I/O monitoring data from many sources into a coherent view of I/O behavior:
  • http://www.pdsw.org/index.shtml
• Thursday’s “Analyzing Parallel I/O” BoF by Phil Carns and Julian Kunkel.  Several guests will share their experiences in analyzing I/O performance and participate in a panel discussion:
  • http://sc17.supercomputing.org/presentation/?id=bof136&sess=sess364
  • https://wr.informatik.uni-hamburg.de/events/2017/bof-analyzing

A new  pre-release version (3.1.5-pre1) of Darshan is available for download here. Please be aware that this pre-release is experimental and not recommended for full-time use in production yet. An official 3.1.5 release will be available in a few weeks.
The list of new features and bug fixes include:

• Darshan’s MPI instrumentation method now intercepts both MPI and PMPI symbols to workaround MPI library bindings that are calling PMPI routines directly
  • Contributed in part by Chris Zimmer
• darshan-util now includes an analysis tool for visualizing DXT traces: dxt_analyzer
  • Contributed by Alex Sim
• Darshan now supports the DARSHAN_EXCLUDE_DIRS environment variable, which specifies a comma-separated list of paths to avoid instrumenting at runtime
  • Contributed by Cristian Simarro
• Darshan’s regression test system now includes test harnesses for Cray systems @ the ALCF (Theta) and NERSC (Cori/Edison)
• Bug fix in Darshan’s shutdown procedure leading to potentially negative timer values
• Bug fix in Darshan’s autoconf scripts causing specified libbz2 locations to be ignored
  • Contributed by Glenn Lockwood

Please report any questions, issues, or concerns using the Darshan-users mailing list, or by opening an issue on the Darshan GitLab page.

A new version of Darshan (3.1.4) is available for download here. This release addresses a couple of minor bugs in the previous Darshan release:

• darshan-parser causing segmentation fault when parsing logs with DXT module data present
• darshan-diff utility not producing output

Documentation for this version of Darshan can be found here.
Please let us know if you have any issues or any feedback you would like to share with us, either using the Darshan-users mailing list or the Darshan GitLab page.