1 / 32

The Starlink Project

The Starlink Project. A presentation for the HEPiX conference 15/4/99 by Dave Rawlinson. Before we begin. Me: Joined Starlink in 1988 in software distribution Am now IT manager, responsible for the technical support of our site managers and purchasing all major hardware. What is Starlink?.

tova
Download Presentation

The Starlink Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Starlink Project A presentation for the HEPiX conference 15/4/99 by Dave Rawlinson

  2. Before we begin... Me: • Joined Starlink in 1988 in software distribution • Am now IT manager, responsible for the technical support of our site managers and purchasing all major hardware

  3. What is Starlink? • Computer systems at UK astronomy sites. • A collection of astronomy software. • Support staff (both centrally at RAL and at sites - Site Managers, Programmers) • Funded by PPARC via a SLA to RAL • At RAL, Starlink is based in the Space Science Department

  4. What is Starlink for? (1) • To provide interactive data reduction and analysis facilities for UK astronomers. • To purchase, install and maintain computer system at UK astronomy institutions. • Improve communications both within the UK and with overseas institutes and observatories

  5. What is Starlink for ? (2) • To reduce waste and foster standards through the sharing of software and operational techniques. • To provide and coordinate system administration at astronomy sites. • Provide an integrated computing environment for Astronomers (both in the UK and at observatories)

  6. Starlink - brief history • 1979/80 - Starlink created - 6 VAX 11/780 (RAL, ROE, RGO, CAM, UCL, MAN) - VMS/DECNET • Early 90’s saw UNIX as the way forward, better price/performance. De facto standard in the US • Move completed by mid-1995 (UNIX/TCP/IP) • 1999 >2200 users at 30 sites in the UK • CPUs (Starlink maintained) • ~240 Sun/(Solaris) • ~100 Digital/Compaq Alpha (Compaq Tru64 UNIX) • ~120 PCs (RedHat Linux) • ~280 Xterminals

  7. Starlink - Networks • Starlink was one of the first projects ever to distribute software via the net (DECNET - SPAN) • Sites participated in network discussion forums (VAXNotes) which have been superceded by our own “Forum” conferencing system

  8. What facilities and services does Starlink offer? (1) • Purchase and maintenance of computing equipment (installation where appropriate) • System Management • System managers and their support • General computing advice, e.g. • Programming advice • Advice on spending grant money on hardware • Discussion groups (SLUGs, SSGs)

  9. What facilities and services does Starlink offer? (2) • Software development and distribution • available via WWW and CD-ROM • comprehensive software collection with over 120 software items /packages • “home grown” (UK - Starlink, groups, individuals) • Imported (IRAF, AIPS, FTOOLS) • Commercial software

  10. What facilities and services does Starlink offer? (3) • Software development and distribution (cont.) • Programmers to maintain and extend the software collection (4 at RAL, 6 located at Universities)

  11. Benefits of central system management support (1) • Site managers and programmers feel part of a “team”. • Vast pool of experience for new managers to draw on. • Large savings on bulk purchase of equipment and maintenance contracts. • Easier to limit diversity and hence cost.

  12. Benefits of central system management support (2) • Careful thought given to what equipment should be purchased (no “white elephants” so far) • New operating systems and versions tested at RAL first (e.g. the adoption of Linux and the Solaris 7 release) • New compiler sets tested at RAL to ensure that the Starlink software works (e.g. Workshop V5.0 compilers)

  13. Benefits of central system management support (3) • Share best practice - many University departments trust the decisions Starlink make and follow our example. They see the Starlink Site Manager as an important resource • Non-Starlink groups can “piggy-back” on our bulk savings (buying power)

  14. Benefits of central system management support (4) • Better to expend effort centrally (e.g. investigating SM tools, improving security procedures) than for each SM to do so individually. • Training (both formal and on the job) is provided by the project. • SMs meet 3 times a year to share experiences and knowledge (also to have beer and curry)

  15. Hardware funding • Announcement of Opportunity • Sites put in a bid for hardware • Nos. of users per site (audited by assessors) • Table of current computer resources • List of publications • Peer Review of scientific productivity • Budget allocated to site, all purchasing carried out by RAL (me!)

  16. CPU power • It can be seen from the previous graph that CPU power at any given site is dominated by the workstations purchased in the previous 2-3 years • Starlink has implemented an aggressive policy of removing “older” kit from maintenance in order for the sums saved to be used in purchasing more modern/powerful kit

  17. Benchmarks (1)

  18. Benchmarks (2) • By creating our own benchmark suite, based on our Starlink software collection, we can determine how well systems perform running “real life” applications • Our results tend to show that Linux performs far better, and Solaris less well, than their respective SPECfp95 figures would suggest

  19. Benchmarks (3) • The performance of the Compaq XP1000 workstations would tend to suggest that the IO subsystem is becoming a limiting factor, hence the non-linear fit.

  20. Linux - the way forward? (1) • Starlink adopted Linux at an early stage • Excellent price performance and widens the use of our software • RedHat adopted as the distribution of choice • Kickstart and autorpm advocated as a simple way to install, update and maintain.

  21. Linux - the way forward? (2) • It’s possible that our current Alpha based systems running Compaq Tru64 UNIX may move to RedHat when binary compatibility and compilers (Compaq) are stable. • Linux allows us to keep our options open.

  22. Work for the future • Huge increase in expected data rates • Currently exploring storage strategy • Solutions limited by available funds • A simple, scalable Storage Area Network looks favourite • Security - major concern within Starlink (or any distributed network) • Working towards standard recommendation and policy

  23. Benefits of central software management • Focused strategy for future software projects. • Limited resources can be best distributed for maximum effect. • Development of a standard computing “environment” for all users both at home and abroad.

  24. Software overview(1) • 120 separate items /packages • 5,000,000 lines of code (mostly FORTRAN, C, C++, TCL) • Central “core system” handles data interchange between different formats • Many packages contributed by Astronomers throughout the world

  25. Software overview (2) • Software distributed via RAL • Some QA done by central team • ensure portability across platforms • documentation? • Software placed on central “Software Store” and packaged up for inclusion on biannual CD distributions

  26. Software overview (3) • All Starlink software for each platform is built from a common source code tree • Sites have “control” distributions (full installation is >1GB) • Programmers (at sites) have development distributions, updated from RAL via rsync thus keeping network traffic to a minimum

  27. Software overview (4) • Starlink have negotiated Starlink-wide licenses for some commercial packages (e.g. NAG library and Linux compilers, IDL, Sun compilers, DECCampus) • Vendors see Starlink as “one site” • Some packages hosted on a “central facility” (e.g. Maple)

  28. Interoperability • Most Starlink applications use the same data format, parameter system and graphics. • Hence, it is possible to seamlessly “mix and match” applications from different Starlink applications packages. • Work ongoing to make interoperability possible between different environments (Starlink, IRAF, IDL)

  29. Base set software • Provide a common environment for users: • Document & paper preparation (LaTeX) • Internet tools (web, news etc.) • Compilers (vendor-specific and egcs) • Programming tools • Electronic conferencing (Starlink FORUM) • Editors, mail tools, spelling checkers, etc.

  30. Software store • Developed as a web based “point and click” solution to obtaining (subsets of) the Starlink software. • Users can select the packages they want, any dependencies will be taken care of, and the complete installation will be delivered as a self-extracting shell archive. http://www.starlink.rl.ac.uk/cgi-store/storetop

  31. In summary • Starlink has avoided most of the problems of a large, disparate project through careful consideration of available technology and techniques. • Throughout 6 reviews in 6 years, we have demonstrated our cost-effectiveness and are envied by astronomers overseas.

More Related