1 / 19

Cornell Theory Center

Cornell Theory Center (CTC) is a Windows-based high-performance computing center at Cornell University. It supports scientific and engineering research in various disciplines and provides resources and services for visualization and software development.

bethanye
Download Presentation

Cornell Theory Center

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cornell Theory Center Cornell Theory Center (CTC) is a high-performance computing and interdisciplinary research center at Cornell University. Scientific and engineering research projects supported by CTC represent a variety of disciplines, including biology, behavioral and social sciences, computer science, engineering, geosciences, mathematics, physical sciences, and business. CTC is unique among HPC centers in that it is a showcase for Windows-based high-performance computing. In 1999, we successfully moved our research community from a traditional, proprietary supercomputer running UNIX to an industry-standard cluster complex. Microsoft and Dell are major supporters of CTC, which is directed by computer scientist Thomas F. Coleman. http://www.tc.cornell.edu/ Cornell Theory Center

  2. Case Studies • Plant breeders and molecular biologists generating detailed information on the genetic makeup of crops e.g. rice and wheat • Management model for manufacturing that helps managers find best balance between capacity, inventory, and subcontracting • New stochastic path approach for modeling protein dynamics in more meaningful time scales • New social science research made possible by developing new statistical methods for analyzing census data • Simulation of crack growth and strength evaluation of airplane components Cornell Theory Center

  3. Compute Power Over 700 Processors Giganet Switch (100 GB/sec) interconnect 100 Mb switched Ethernet 1-4 GB Ram/Node, 27-56 GB Disk/Node Serial, Dual, and Quad processors Dell PowerEdge Servers Pentium processors, 600 Mhz – 2.4Ghz 256 KB - 2 MB Cache/Processor (SMP) Tivoli Backup SystemMicrosoft Distributed File SystemFileservers, 5 Terabytes Storage http://www.tc.cornell.edu/Services/Docs/resources.asp 11/ 2002 256-processor cluster in top 100 618.3 gigaflops over 256 2.4Ghz Linpack benchmark Cornell Theory Center

  4. Visualization Consultation on how to visualize data Creation of animations, videos, publication graphics Independent study opportunities for undergraduate students Training in the use of OpenDX Video production suite Three-wall Windows CAVE: Virtual reality environment (show VRML files) Write your own with OpenGL or OpenInventor Free by appointment with kford@tc.cornell.edu Computer Lab of Visualization Workstations http://www.tc.cornell.edu/services/vis/index.asphttp://www.tc.cornell.edu/services/docs/cave/ http://www.tc.cornell.edu/services/edu/topics/topics.asp?section=opendxhttp://www.tc.cornell.edu/services/Allocations/Account/visform.asp http://www.tc.cornell.edu/Services/Docs/Collab/ Cornell Theory Center

  5. Software • See online list for software available on the login nodes, various types of compute • nodes and the Visualization Collaboratory. This list is updated as new packages are • added. Software is added by researcher's request if feasible. • C/C++ compilers: • Microsoft Visual C++, Intel C/C++, Gnu gcc/g++ • Fortran compilers: • Intel Fortran, PGF77, Compaq Visual Fortran • Libraries: • MPI/PRO parallel programming, Numerical Libraries • Applications software: • Computational chemistry, Data analysis, Finite element modeling, • Fluid dynamics, Visualization tools, . . . • http://www.tc.cornell.edu/Services/Software/http://www.tc.cornell.edu/services/edu/topics/software/ Cornell Theory Center

  6. Allocations • Research allocation: • Submit proposals at any time. • Undergraduate or graduate students must have • their requests submitted by a faculty advisor. • Non-renewable six-month Exploratory: • Allocations of 100 hours. Simplified application. • Charging is calculated using wall-clock hours; i.e. you will be charged for the time nodes are reserved for your use, not for the time you are actually using them. • Wall-clock hours on all of the node types are charged the same flat wall-clock rate, regardless of number of processors. Parallelize! • Charges for compute nodes only, not for login nodes or the collaboratory machines. • http://www.tc.cornell.edu/services/Allocations/http://www.tc.cornell.edu/services/edu/topics/start/ Cornell Theory Center

  7. Access • Connect to a login node (ctclogin1, ctclogin2, …) to compile and submit jobs. • The login nodes are SSH servers, download an SSH client. • Login.bat is run automatically. • For full windows desktop functionality from your machine, use a terminal server, Microsoft Terminal Server Client for Windows, Citrix ICA for non-windows. • For command line access, use telnet: • ssh or telnet ctclogin1.tc.cornell.edu • http://www.tc.cornell.edu/Services/Docs/Access/ Cornell Theory Center

  8. Development Environment Programs can be compiled on your Windows machine, in the Collaboratory in Rhodes Hall, or from one of the login nodes reached via Terminal Server or telnet. Work from the command line or in a GUI environment. GUI Development environment: Start | Programs | Microsoft Visual Studio .NET | Microsoft Visual Studio .NET Command line: Start | Run | cmd (bring up a command line window) setup_intelc32 (set environment variables using setup script **) cl hello.c (compile serial) mpicc hello.c (compile MPI)** To see the full list of setup scripts, issue: call "H:\CTC Tools\setup_list.bat“ Cornell Theory Center

  9. ctclogin1 ctclogin2 Compute Environment BatchScheduler Compute Nodes CCS Login Nodes H: ctclogin… Fileserver Cornell Theory Center

  10. ctclogin1 CCS Login Nodes ctclogin2 H: ctclogin… Serial Batch Job Script • Prepare your executable • Write a batch script file • From one of the login nodes, submit the script to the scheduler • Check the status of the queue, your job, etc The remainder of the steps can be done by hand or script: • When your job has started: copy files to the local T: drive run the program • When your job has ended: Copy any output files back to H: clean up the T: drive end the job Compute Nodes Cornell Theory Center

  11. Login Node Master Node T: H: Worker Node T: Worker Node T: Worker Node T: Parallel Batch Job Script • The main batch script runs on the Master node • Start the script with the four required “REM CCS” keyword commands • Create the “machines” file on T: of the master node • Use mpirun to run setup.bat on all nodes listed in the machines file • Create a working subfolder on the compute node T: drive • Copy the executable (and data files, if necessary) to each node • Copy any files needed only on the Master node • Move the machines file from T: to T:\your_userid • Use mpirun to run the MPI program executable on all nodes listed in the machines file • Your_program.exe runs on all the nodes in the machines file • Use mpirun to run cleanup.bat on all nodes listed in the machines file • Copy the output files from each node to H: • Cleanup the working subdirectory on each node • Release the nodes Cornell Theory Center

  12. Parallel Job on a Single Node (page 1 of 2) REM CCS account = your_userid REM CCS type = batch REM CCS nodes = 1 REM CCS minutes = 10 REM CCS requirements = 1@development T: mkdir %USERNAME% cd %USERNAME% del /Q /S T:\%USERNAME%\*.* call machinemaker Call machinemaker (in CTC Tools, therefore already in your path) to produce a machines list, listing the node(s) your job has been allocated. Note: if you have more than one job running, you can assure that you'll create the machines file for the right jobid by issuing machinemaker jobid Cornell Theory Center

  13. Parallel Job on a Single Node (page 2 of 2) Run the MPI program with mpirun. Set -np to the number of tasks. Set -wd to the working directory in the case where your files are not in your current directory. REM Copy the executable and data files to the node copy H:\users\%USERNAME%\lab\wavesend.exe T:\%USERNAME% copy H:\users\%USERNAME%\lab\wave.in T:\%USERNAME% mpirun -wd T:\%USERNAME% -np 2 wavesend.exe 1>wave.out 2>wave.err REM Copy any output created by the master task back to the H drive. copy T:\%USERNAME%\wave.* H:\users\%USERNAME%\lab REM Cleanup the working subdirectory on each node del /Q /S T:\%USERNAME%\*.* REM Release the nodes ccrelease Cornell Theory Center

  14. Parallel Job on Multiple Nodes (page 1 of 4) REM CCS account = your_userid REM CCS type = batch REM CCS nodes = 4 REM CCS minutes = 10 REM CCS requirements = 4@development REM Move to the T drive T: call machinemaker REM Set the communication mode to TCP or VIA set MPI_COMM=VIA Run machinemaker to create a file called "machines", which contains a list of machines allocated to this job Cornell Theory Center

  15. Parallel Job on Multiple Nodes (page 2 of 4) mpirun -np 4 H:\users\%USERNAME%\lab\ex3setup.bat Use mpirun to call a batch script called setup.bat. Setup.bat to copy the executable, data files, etc onto the nodes listed in the machines file. When calling setup.bat, set -np to the number of nodes, not the number of tasks. REM setup.bat REM Create a working subdirectory on each node T: mkdir your_userid cd your_userid del /Q /S T:\your_userid\*.* REM Copy the executable (and data files, if necessary) to each node copy H:\users\your_userid\lab\wavesend.exe T:\your_userid Cornell Theory Center

  16. Parallel Job on Multiple Nodes (page 3 of 4) REM The input file is only needed on the master node copy H:\users\%USERNAME%\lab\wave.in T:\%USERNAME% cd %USERNAME% move T:\machines T:\%USERNAME% mpirun -wd T:\%USERNAME%\ -np 8 wavesend.exe 1>wave.out 2>wave.err copy T:\%USERNAME%\prog.* H:\users\%USERNAME%\lab Move the machines file from T: to T:\your_userid. It couldn't be put there earlier. Run the MPI program with mpirun. Set -np to the number of tasks. Set -wd to the working directory used by mpirun. Copy any output created by the master task back to H:. Cornell Theory Center

  17. Parallel Job on Multiple Nodes (page 4 of 4) mpirun -np 4 H:\users\%USERNAME%\lab\ex3cleanup.bat REM Release the nodes ccrelease Next another script (cleanup.bat) is run all nodes to both copy any other output files back to the H drive and to clean up the subdirectory. When calling cleanup.bat, set -np to the number of nodes,not the number of tasks. REM In this example, only the master node has output files. copy T:\your_userid\wave.* H:\users\your_userid\lab REM Cleanup the working subdirectory on each node del /Q /S T:\your_userid\*.* Cornell Theory Center

  18. MCP Compute Node Compute Node Fileserver (H:) Compute Node Compute Node Standard copy: Fileserver is accessed N times Copy with mcp Fileserver is accessed 1 time Cornell Theory Center

  19. More Information: • Consulting - Send questions about use of the Velocity Cluster to consult@tc.cornell.edu or call 254-8686 and ask for a consultant. • Can answer specific questions, help with problems, discuss strategies and code planning. • Strategic users (faculty with proven application, suitable for scaling) have access to more in-depth long-term consulting help. • http://www.tc.cornell.edu/http://www.tc.cornell.edu/Services/Docs/UserGuide/http://www.tc.cornell.edu/services/docs/http://www.tc.cornell.edu/services/edu/topics/http://www.tc.cornell.edu/services/edu/events/ • http://www.tc.cornell.edu/Services/Docs/Tips/cygwin.asp • http://www.tc.cornell.edu/Services/edu/topics/writing2/ Cornell Theory Center

More Related