1 / 32

Outline

Outline. Subversion server What is it? Why is it useful? How to use Sandpit cluster What is it? Why is it useful? How to use. Subversion: What is it?. Version control system Successor to CVS. Subversion client 1 C:My Documents. Subversion client 2 D:My Documents.

barrett-hall
Download Presentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Outline • Subversion server • What is it? • Why is it useful? • How to use • Sandpit cluster • What is it? • Why is it useful? • How to use Johan Muskens TU/e Computer Science, System Architecture and Networking

  2. Subversion: What is it? • Version control system • Successor to CVS Subversion client 1 C:\My Documents\ Subversion client 2 D:\My Documents\ Subversion server https://svn.win.tue.nl/repos/san • education • courses • 2IC10 • 2IF20 • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www • courses • 2IC10 • 2IF20 • … • education • courses • 2IC10 • 2IF20 • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www Johan Muskens TU/e Computer Science, System Architecture and Networking

  3. Subversion: Why is it useful? • Sharing knowledge • Now documents, code, etc. are only available on pc of individual group members • These artifacts should be access-able for all • Collaboration • Writing documents together • No need for mailing documents to each other over and over again • Backup Johan Muskens TU/e Computer Science, System Architecture and Networking

  4. Subversion: How to use? Important actions: • Check-out • Update • Add • Remove • Move • Commit Johan Muskens TU/e Computer Science, System Architecture and Networking

  5. Check-out • Create a local working copy of a sub-tree of the repository Subversion client C:\My Documents\ Subversion server https://svn.win.tue.nl/repos/san • education • courses • 2IC10 • 2IF20 • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www • courses • 2IC10 • 2IF20 • … Johan Muskens TU/e Computer Science, System Architecture and Networking

  6. Update • Synchronize updates on the repository with your local copy. Subversion client C:\My Documents\ Subversion server https://svn.win.tue.nl/repos/san • education • courses • 2IC10’ • 2IF20 • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www • courses • 2IC10 ‘ • 2IF20 • … Johan Muskens TU/e Computer Science, System Architecture and Networking

  7. Add • Put a (new) file under version control Subversion client C:\My Documents\ Subversion server https://svn.win.tue.nl/repos/san • education • courses • 2IC10 • 2IF20 • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www • courses • 2IC10 • New file • 2IF20 • … !! File will be added to repository during next commit. Johan Muskens TU/e Computer Science, System Architecture and Networking

  8. Remove • Remove a file from version control (The history will remain !!). Subversion client C:\My Documents\ Subversion server https://svn.win.tue.nl/repos/san • education • courses • 2IC10 • file • 2IF20 • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www • courses • 2IC10 • file • 2IF20 • … !! File will be removed to repository during next commit. Johan Muskens TU/e Computer Science, System Architecture and Networking

  9. Move • Move a file from one location in the repository to a different one. Subversion client C:\My Documents\ Subversion server https://svn.win.tue.nl/repos/san • education • courses • 2IC10 • file • 2IF20 • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www • courses • 2IC10 • file • 2IF20 • file • … !! File will be moved to repository during next commit. Johan Muskens TU/e Computer Science, System Architecture and Networking

  10. Commit • Changes in your local working copy are synchronized with the repository Subversion client C:\My Documents\ Subversion server https://svn.win.tue.nl/repos/san • education • courses • 2IC10 • New file • 2IF20 • File • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www • courses • 2IC10 • New file • 2IF20 • File • … Johan Muskens TU/e Computer Science, System Architecture and Networking

  11. Our Repository https://svn.win.tue.nl/repos/san/ • education • courses • 2IC10 • 2IF20 • … • student_projects • research • personal • jmuskens • mchaudron • … • projects • robocop • space4u • … • www Johan Muskens TU/e Computer Science, System Architecture and Networking

  12. Tools • Windows client • Tortoise SVNClient • Integrated in Windows Explorer • Unix client • Command line (svn add/remove/…) • Graphical client ? Johan Muskens TU/e Computer Science, System Architecture and Networking

  13. Things to remember • Changes are not effectuated at the server until you do a commit • You can only add, remove, … things when you have a local copy of a sub-tree of the repository • You can have multiple local copies of possibly different sub-trees of the repository • You can put files in a local working copy without putting them under version control • History remains • Once you add something to subversion, traces remain (even when you do a remove). Johan Muskens TU/e Computer Science, System Architecture and Networking

  14. Guidelines • Make it available for all • Also old project documentation • Add sources, not the derived artefacts • Don’t use versions in the document names • Subversion does the versioning for you ! • Don’t alter the existing directory structure • How you structure your personal, course or project sub-tree is up to you • Keep it clean Johan Muskens TU/e Computer Science, System Architecture and Networking

  15. SANdpitSAN distributed & parallel integrated terminal Richard Verhoeven to dig up information Johan Muskens TU/e Computer Science, System Architecture and Networking

  16. Content • Hardware • Software • General, OpenPBS, LAM/MPI • Usage • Courses • Projects • Users • Data access Johan Muskens TU/e Computer Science, System Architecture and Networking

  17. Old cluster • One master with 17 nodes (pacluster) • 533 MHz Pentium III, 256 MB, 10 GB, 100 Mbit • RedHat 7.3 with security fixes • Failing power supplies, fans, disks, video cards • Only 13 nodes kept running to some extend Johan Muskens TU/e Computer Science, System Architecture and Networking

  18. New cluster • One master with 17 nodes (sandpit) • 3 GHz Pentium 4 with hyper-threading • 400 GB disk in master, 40 GB disk per node • 1 GB memory • 2 Gigabit NICs • 19” rack mounted Johan Muskens TU/e Computer Science, System Architecture and Networking

  19. Hardware Internet 2x200 GB 40 GB Johan Muskens TU/e Computer Science, System Architecture and Networking

  20. Software • Gentoo Linux distribution • Compile from source • Optimized for platform • Large collection of standard software • Text editors (vi, emacs, nano) • Compilers (gcc, g++, g77, gcj, icc, javac) • Text processing (perl, LaTeX, ps2???) • Graphic processing (netpbm, gstreamer) • GUI apps (gnome, mozilla) • More software is installed upon request Johan Muskens TU/e Computer Science, System Architecture and Networking

  21. /home/user (190 GB) /local/user (190 GB) /wslocal/sb01/user /home/user /wslocal/sandman/user /local/user (30 GB) /wslocal/sb01/user Directory structure sb01 .. sb17 sandpit / sandman 890 GB shared disk space for users Johan Muskens TU/e Computer Science, System Architecture and Networking

  22. OpenPBS • Portable Batch System • Jobs are submitted to the server with qsub • Status of queue is available through qstat • Scheduler allocates nodes to specific jobs. • Four queues for jobs • small (< 20 min, highest priority) • medium (< 2 hours) • large (< 12 hours) • verylarge (< 3 days) • Maui scheduler Johan Muskens TU/e Computer Science, System Architecture and Networking

  23. LAM/MPI • MPI – message passing interface • LAM/MPI – open-source MPI implementation • lamboot – create a multicomputer • mpirun – run parallel program • lamhalt – terminate multicomputer • Under PBS • Create script with lamboot – mpirun – lamhalt • Create multicomputer from allocated nodes Johan Muskens TU/e Computer Science, System Architecture and Networking

  24. Courses • Student accounts • Course specific software • Special PBS queue • Only for students and instructors • Only active queue during instruction hours • Cluster remains accessible for others • Getting started manual Johan Muskens TU/e Computer Science, System Architecture and Networking

  25. Projects • Group of users • Project related directory • Software repository • Documentation • Guest accounts • Faculty, university, national, international Johan Muskens TU/e Computer Science, System Architecture and Networking

  26. Users • SAN members • Copied from UNIX systems • Faculty members • mCRL group • Students of courses • Guest researchers • Use ssh to login and rsh/ssh to move between nodes. Johan Muskens TU/e Computer Science, System Architecture and Networking

  27. Data Access • Central directories are not accessible • UNIX directories not NFS mountable (anymore) • Window G: disk not SAMBA mountable (yet) • Use ssh and scp Johan Muskens TU/e Computer Science, System Architecture and Networking

  28. Data Backup • Cluster not under the backup policy of BCF • Data is often temporary • Too much data for offsite backups • Options: • Copy data to different nodes • Copy data regularly to central network disk • Integrate backup into PBS scripts • Check data into Subversion Johan Muskens TU/e Computer Science, System Architecture and Networking

  29. Usage policy • No enforcement (yet) • By-passing PBS is possible • No disk quota • No accounting of processor time • Fair use • Clean up temporary data regularly • Don’t abuse the resources • Personal projects are allowed as long as they don’t interfere with education or research. • Otherwise • Enforcement will be added or account disabled Johan Muskens TU/e Computer Science, System Architecture and Networking

  30. Possible extension • Use OpenMosix • Extension to move processes between nodes • Add real-time boot configuration • Reboot nodes when job scheduler asks for it • Use second network interface • Small ring • Second connection to the switch • Add cross-compiler tool chains • Familiar Linux for iPAQ • TinyOS for sensor motes • SymbianOS for mobile phones Johan Muskens TU/e Computer Science, System Architecture and Networking

  31. What happened with … • the old cluster? • Already reused in other projects • Several nodes revisited to get them working • Create heterogeneous network setup • Ethernet, wifi, firewire, bluetooth, sensors • Combine or upgrade test environments • Give to family and friends Moet weg tegen elk aannemelijk bod Johan Muskens TU/e Computer Science, System Architecture and Networking

  32. Questions? Johan Muskens TU/e Computer Science, System Architecture and Networking

More Related