Darshan 2.2.4 release

by

Darshan 2.2.4 is now available on the download page.
If you are upgrading Darshan on a Cray platform, please re-patch your compiler scripts.  If you are upgrading Darshan on a Blue Gene platform (P or Q), please check the BG/P recipe in the installation documentation and note that the –enable-stat-at-open option is now recommended on those platforms.
Change log:
darshan-2.2.4

  • Disable extra stat() of newly opened files by default. This improves performance on shared files for some platforms. Reported by Yushu Yao.
    • prevents Darshan from reporting SIZE_AT_OPEN, FILE_ALIGNENT, and DARSHAN_FILE_NOT_ALIGNED values
    • features can still be enabled via –enable-stat-at-open option
  • Fix missing -lz in post ld flags reported by Yushu Yao. Fixes a link-time error for some corner-case applications.
  • Fix bug in Cray compiler script patches that was setting compiler flags incorrectly. Reported by Yushu Yao.
    • WARNING: if you are using a previous Darshan release (2.2.3 through 2.2.4-pre4) on a Cray platform, please re-patch your compiler scripts.
  • Update darshan-gen-* scripts to support the potential for additonal LDFLAGS or CFLAGS that appear before the object or source file in the compile or link commands. This fixes compatibility with some mvapich2 installations, reported by Dragos Constantin.

Darshan 2.2.3 release

by

Darshan 2.2.3 is now available on the download page.  This release features official support for the Cray XE6 and IBM BG/Q platforms.  Special thanks to Yushu Yao and Katie Antypas of NERSC for their contributions in Cray integration and testing.
Change log:

darshan-2.2.3
  • improved Cray XE6 support
    • support for GNU, PGI, Cray, Pathscale, and Intel compilers
    • patch adding Darshan capability to system compiler scripts
    • software module, including testing and features contributed by Yushu Yao and Katie Antypas of The National Energy Research Scientific Computing Center (NERSC)
    • improved documentation
  • minor bug fixes:
    • properly detect cxx library name when generating BG/Q compiler wrappers
    • improve timing mechanism for cp-shutdown-bench benchmark
    • improve hashing to avoid log file name collisions

Darshan 2.2.2 release

by

Darshan 2.2.2 is now available on the download page.  Change log:

darshan-2.2.2
  • bug fixes:
    • remove debugging message that was inadvertently included in MPI_File_sync() wrapper
    • fix potential hang if the –with-log-path-by-env argument was used at configure time but the environment variable was not set at run time
darshan-2.2.1
  • significant improvements to how counters are handled in multi-threaded applications
  • initial (rough) documentation for using Darshan in Cray programming environments with static linking
  • bug fixes:
    • escape special characters in mount point paths in darshan-job-summary.pl (reported by Mouhamed Gueye)
    • workarounds for various runtime problems with cuserid() and stat() in Cray environments
    • build problems with darshan-utils on some versions of OSX
    • accurate shared file statistics for libraries that use deferred opens

Darshan 2.2.0 release

by

Darshan 2.2.0 is now available for download.   The biggest change is that this release splits the code into separate components for instrumentation and analysis.  There are also several improvements in documentation, portability, and an assortment of bug fixes.
Changelog:

  • split darshan into separate packages:
    • darshan-runtime: for runtime instrumentation
    • darshan-utils: for processing darshan log files
  • changed default output file name for darshan-job-summary.pl to be based on input file name rather than summary.pdf
  • reorganized init and finalize routines so that can be linked separately (to allow for easier integration with other instrumentation tools)
  • add -cc, -cxx, -f77, -f90, and -fc support to compiler scripts generated by the darshan-gen-*.pl scripts
  • bug fixes:
    • potential MAX_BYTE overflow on 32 bit systems
    • incorrect pread and pwrite offset tracking
    • corrections to darshan-job-summary variance table
    • better runtime error handling if bzip or gnuplot tools are insufficient
    • improvements to time range in darshan-job-summary graphs
  • documentation:
    • improved documentation for both the darshan-runtime and darshan-util portions of Darshan can be found in the respective doc/ subdirectory for each

Updated documentation

by

The latest Darshan documentation can be found at the documentation link above.  We’ve made several improvements to the documentation, including “recipes” to help get started on various systems including Blue Gene, Cray, and Linux clusters using MPICH, OpenMPI, or Intel MPI.

Darshan 2.1.2 Release

by

Darshan 2.1.2 is a minor bug fix release to improve error handling in cases where Darshan is unable to write a log file.
Changelog:

  • improved error handling when writing log files.  If a write fails on any process then the log file will be deleted and a warning will be printed to stderr.

Darshan Data Repository now online

by

We are pleased to announce the public release of the Darshan Data Repository.  The Darshan Data Repository is a collection of anonymized I/O characterization data captured from production systems. The first data set to be made available covers three months of activity (as recorded by Darshan) from the Intrepid Blue Gene/P system at the Argonne Leadership Computing Facility.  We hope to add more data in the future.  See the Darshan Publications page for examples of analysis that can be performed with this data.

Darshan 2.1.1 Release

by

This release includes performance and bug fixes.  It also includes a new utility to convert Darshan log files, while also optionally anonymizing them or re-compressing them in bzip2 format.
Changelog:

  • new darshan-convert command line utility for converting existing log files, with optional anonymization and optional bzip2 compression
  • bzip2 support in command line utilities (but not in the darshan library itself)
  • updated log file format that allows for string key/value pairs to be stored in the header
  • added ability to set MPI-IO hints when writing darshan log
    • at configure time: –with-log-hints
    • at run time: DARSHAN_LOGHINTS environment variable
  • bug fix contributed by Sandra Schröder: use case-insensitive search for MPI symbols in Fortran wrapper script
  • performance bug fix: remove unecessary call to MPI_File_set_size when writing log
  • added –with-logpath-by-env configure option to allow absolute log path to be specified via environment variable

Best Paper Award, MSST 2011

by

A paper featuring Darshan (“Understanding and Improving Computational Science Storage Access through Continuous Characterization“) was awarded Best Paper at the 27th IEEE (MSST 2011) Symposium on Massive Storage Systems and Technologies.  The paper outlines a methodology for characterizing a large scale production workload and presents a 2 month study of I/O activity on the Intrepid Blue Gene/P system at Argonne National Laboratory.