{"id":153,"date":"2010-09-30T21:49:58","date_gmt":"2010-09-30T21:49:58","guid":{"rendered":"http:\/\/press3.mcs.anl.gov\/romio\/?p=153"},"modified":"2010-09-30T21:49:58","modified_gmt":"2010-09-30T21:49:58","slug":"lustre-driver-story","status":"publish","type":"post","link":"https:\/\/wordpress.cels.anl.gov\/romio\/2010\/09\/30\/lustre-driver-story\/","title":{"rendered":"Lustre driver story"},"content":{"rendered":"<p>ROMIO has a general-purpose file system driver we call &#8220;UFS&#8221; (for Unix File System). UFS contains no file-system-specific optimizations: just data sieving and two phase collective buffering.<br \/>\nThe generic approach works, in that it gives correct answers, but it has two big problems when writing to Lustre:<\/p>\n<ul>\n<li>When assigning the file domains, UFS simply takes the start, the end, and divides evenly over the I\/O aggregators. We wrote about <a href=\"http:\/\/press3.mcs.anl.gov\/romio\/2008\/11\/20\/tuning-collective-io-strategies-for-gpfs-and-lustre\/\">Wei-keng&#8217;s SC 2008 paper<\/a> in this area earlier.<\/li>\n<li>the collective buffering algorithm will do a read-modify-write if there are any holes or gaps in the request. There is a point (specific to each file system deployment) where data sieving does not win out and e.g. two large writes would be better than a read-modify-write.<\/li>\n<\/ul>\n<p>We rely on the community to contribute many of the fs-specific drivers (e.g. PanFS, XFS), and through 2009 and 2010 the Lustre community did just that. Weikuan Yu did some early work while he was at ORNL. Sun&#8217;s developers contributed more improvements, including an independently-developed version of Wei-keng&#8217;s group-cyclic distribution. End-users Martin Pokorny at NRAO and Pascal Deveze at BULL contributed additional testing and patching. As a result, ROMIO ended up with an optimized Lustre driver incorporating optimizations for the two points discussed above.<br \/>\nLustre users should still let us know how things are going: is collective MPI-IO working well? working poorly? The more community involvement we get, the better we can make things.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>ROMIO has a general-purpose file system driver we call &#8220;UFS&#8221; (for Unix File System). UFS contains no file-system-specific optimizations: just data sieving and two phase collective buffering. The generic approach works, in that it gives correct answers, but it has two big problems when writing to Lustre: When assigning the file domains, UFS simply takes &hellip;<\/p>\n","protected":false},"author":362,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[7],"tags":[],"class_list":["post-153","post","type-post","status-publish","format-standard","hentry","category-lustre"],"acf":[],"_links":{"self":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/153","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/users\/362"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/comments?post=153"}],"version-history":[{"count":0,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/posts\/153\/revisions"}],"wp:attachment":[{"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/media?parent=153"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/categories?post=153"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/romio\/wp-json\/wp\/v2\/tags?post=153"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}