The Roadmap to New Releases

1. The Roadmap to New Releases Todd Tannenbaum Department of Computer Sciences University of Wisconsin-Madison http://www.cs.wisc.edu/condor tannenba@cs.wisc.edu

2. Stable vs. Development Series • Much like the Linux kernel, Condor provides two different releases at any time: • Stable series • Development series • Allows Condor to be both a research project and a production-ready system

3. Stable series • Series number in version is even (e.g. 6.2.0) • Releases are heavily tested • Only bug fixes and ports to new platforms are added on a stable series

4. Stable series (cont.) • A given stable release is always compatible with other releases from the same series • Recommended for production pools

5. Development Series • Series number in the version is odd (e.g. 6.1.17, 6.3.0) • New features and new technology are added frequently • Versions from the same development series are not always compatible with each other

6. Development Series (cont.) • Releases are not as heavily tested • Generally not recommended for production pools • … unless new features are required • … unless we recommend otherwise :^)

7. Where is Condor Today? • New stable version 6.2.0 recently released • Previous development series cycle, ver 6.1.*, was far too long • This was our first stable/development series: we're hopefully learning...

8. New Ports in 6.2.0 • Full support (with checkpointing and remote system calls): • All current versions of Linux (x86) • Kernel: 2.2.* and 2.0.* • C Library: glibc-2.[01] and libc-5 • NOTE: libc5 support dropped • Solaris 2.7 and 2.8(Sparc and x86) • Irix 6.5

9. New Ports in 6.2.0 (cont.) • ”Clipped" support (no checkpointing or remote system calls, but all other functionality is available) • Windows NT • Alpha Linux

10. What is New in 6.2.0? • Personal Condor and Grid Support • Flocking • Proof-of-concept support for • Globus Job Universe • Globus Glide-In • Full, integrated support for Symmetric Multi-Processor (SMP) machines

11. What's New in 6.2.0? (cont.) • PVM support • DAGMan (for managing inter-job dependencies) • Use of machine ClassAd attributes in submit file • Example: Executable = foo.$$(ARCH).$$(OPSYS)

12. What's New in 6.2.0? (cont.) • Greatly expanded I/O support for STANDARD universe jobs • Condor can automatically buffer I/O requests from jobs • Compression/decompression on-the-fly! • Users get much more information about the kinds of I/O their jobs are performing • Users can "remap" files to alternate locations

13. What's New in 6.2.0? (cont.) • CondorVersion and CondorPlatform strings included in all binaries and libraries • Helps identify and avoid problems with having the wrong version installed • Different parts of the Condor protocol automatically check for version incompatibilities

14. What's New in 6.2.0? (cont.) • Better accounting • Historical usage logging in collector • Use condor_stats • web interface called CondorView • Accountant stores usage information per user • Better control over user priorities • "Priority factors"

15. What's New in 6.2.0? (cont.) • More powerful administration tools • Setting configuration values remotely • Querying daemons directly for status • Other misc new tools • condor_findhost, condor_hold/release, condor_qedit, condor_userlog, condor_run, … • Lots of performance and bug fixes • A complete list is online in the manual (www.cs.wisc.edu/condor/manual)

16. The 6.3 Development Series • Major focus areas for ver 6.3.x Series • Future-aware scheduling for dedicated resources • MPI support • Security enhancements • Secure network communication • Kerberos, PKI (GSI/OpenSSL) • Limitations on executable, remote syscall sandbox • Condor-G production version • Integrate Bypass technology • Enhanced Windows support

17. What else will be added in 6.3.*? • And also several easy-to-add, user-visible features • ”Master agents" - helper programs spawned by the condor_master to aid in administration • Retrieving remote log, history and/or configuration files • Remote "top", "ps" and other monitoring functions

18. What will be added in 6.3.*? (cont.) • Solution for Condor clusters on • Private network • Behind a firewall • New condor_starter and condor_shadow for the VANILLA universe on Unix • "NT version" will be used for Unix, too • Lots of new features, like transferring files automatically for "vanilla" jobs (no need for a shared filesystem)

19. What will be added in 6.3.*? (cont.) • condor_startd will enforce resource limits dynamically (like RAM usage) • More logic put into the condor_shadow to • Detect temporary problems with a job's execution, put the job on hold, and notify the user • Enforce job submitting user’s policy • Better propagation of error messages

20. The 6.5.* Development Series • 6.5.* will be for adding fundamentally new technology to Condor • Being developed in parallel with 6.3.* • Will ultimately be v7.0 • Will be incompatible with previous versions of Condor

21. New Technology in 6.5.* • New version of ClassAds • Conditionals !! • if/then/else • Aggregates (lists, nested classads) • Built-in functions • String operations, pattern matching, time operators, unit conversions • Clean implementations in C++ and Java • ClassAd collections

22. Other changes for 6.5.* • Re-write of the condor_schedd • Performance enhancements and lowered resource requirements (particularly RAM) • Re-write of the checkpoint server • NEST technology infusion • Enhanced support for multiple servers • Will meta-data along with checkpoint files

23. Planned Future Ports • Full support • Intel Linux 2.4 kernel (RedHat 7.x) • Clipped support • Windows 2000 / Whistler

24. Possible Future Ports • Full Support (I.e. checkpointing, etc) • Windows 2000 • Alpha Linux • DUX 5.x • Clipped-support • {free,open,net}BSD • MacOS X • PowerPC Linux • HPUX 11.0 • AIX 4.2

25. Thank you for coming to Paradyn/Condor Week!