1 / 7

HPC USER FORUM I/O PANEL April 2009 Roanoke, VA

HPC USER FORUM I/O PANEL April 2009 Roanoke, VA. Panel questions: 1 response per question Limit length to 1 slide. Panel Format. <insert panel format here>. Q1. Parallel NFS finally is here!.

koko
Download Presentation

HPC USER FORUM I/O PANEL April 2009 Roanoke, VA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HPC USER FORUM I/O PANELApril 2009Roanoke, VA Panel questions: 1 response per question Limit length to 1 slide

  2. Panel Format • <insert panel format here>

  3. Q1. Parallel NFS finally is here! • With the formalization of Parallel NFS as a standard - what steps are being provided to enable this to be hosted on current (and future) platform choices? • This is really a development question for Garth and Peter • Consumers should require this as part of the RFP process • Remember that pNFS is a protocol and not a file system • POSIX semantics for writing to the same file are not maintained • Though pNFS is a very good thing is does not solve world peace and global warming • The underlying file system might need to be modified to support the new high speed I/O

  4. Q2. Parallel NFS – implementation details… • What tools are available to help optimize this (from application layer level all the way to archival stage)? What is missing and who should provide it? • Actually there are no tools for end-to-end integrated I/O optimization and it would require a major effort to provide an integrated tool kit • There are many obstacles to these types of tools such as understanding the configuration and contention from other processes • Example: A user could make a 30% performance improvement I/O of a job and it might not be recognized given contention from other work • How do you measure the I/O of a job without measurement of the system

  5. Q3. Physical media interconnects … • We all are facing complexity and cost issues. With IB or 10 GbE (40/100 GbE) : where should the HPC community should focus its resources on - for all I/O? • The community should be looking to the future at what will be the commodity technology will be the winner • That in my opinion will be FCoE and GbE • In my opinion sites should begin to plan for this future by planning for this by looking at NICs and cabling plans • The process of change will being in earnest in 2010 and given legacy environments will likely take at least 5 years to complete • Things take longer than expected with storage • Take the 3.5 to 2.5 disk drive change now in progress for an example

  6. Q4. Layer protocols above the interconnects • Too many standards. interconnects, media layers are issues today. iSCSI/FCOE/FcoCEE/FCoIB have all been touted as the solution(s). Is it even relevant in the HPC arena? Is fragmentation the only choice? • In my opinion ISCSI will not make it as an HPC solution • The issue is encapsulation of metadata and small blocks and wasting channel bandwidth • HPC is not all about streaming I/O • Small block random happens more than you think and is the bounding factor for many HPC • Commodity technology will eventually overtake • IB was successful when PCIe became available • And it will limit us again • 40 GbE will win in cost over QDR • Maybe not at first but eventually given volume

  7. Q5. I/O issues not yet addressed? • What do you consider to be the top 3 main (technical or human) issues in HPC I/O? • 1 HPC file systems have historically had a problem opening a single file for all nodes, which means the file counts are growing significantly • 2 There are no really I/O tools for users to tune applications on operational systems especially • 3 User data management (ILM) frameworks are not part of the standard process and are implemented in user space • Part of the complexity of this is a function of #1

More Related