Quantitative Analysis of Developer Productivity in C vs. Real-Time Java Kelvin Nilsen

1. Quantitative Analysis of Developer Productivity in C vs. Real-Time JavaKelvin Nilsen

2. Outline • Description of Experiment • Results • Analysis of Results

3. Description of Experiment • Commercial Development Effort • Management Control Subsystem (MCS) • Command processing and business logic for provisioning of a next-generation telecommunications broadband loop carrier • “Northbound” agents speak common and proprietary protocols (TL1, SNMP, proprietary GUI, others) • “Southbound” communication is directed toward database subsystems residing on various shelves and/or CPUs in the network

4. Description of Device • Low-cost replacement for one or any combination of NGDLCs, DSLAMs, ATM edge switches, SONET ADMs, and Ethernet switches. • Every slot in every chassis supports up to 10 Gbps of user bandwidth • Each shelfsupports480 POTSsubscribers

5. The Experiment • The MCS software was written twice! • First implementation written in C, using an event loop that calls back to various service routines • Second implementation written in “real-time Java”, using individual blocking threads to provide distinct services • Both efforts used similar methodologies: • Functional specs for requirements definition • Design and code reviews • Rational ClearCase and ClearQuest for revision control and defect tracking

6. What do we mean by “real-time Java”? • Standard J2SE-compatible platform • No RTSJ extensions • Real-time garbage collection • Real-time threading and synchronization • Improved timing services • VM Management API and PERC shell

7. Tightened Threading Semantics • A PERC thread honors the priority specified by the developer • Priorities are never automatically “aged” • Every PERC synchronization lock implements priority inheritance • All lock and wait queues are maintained in priority order • Optional use of extended priority range (1-32) • These semantics are offered across all targeted operating systems: ETS, INTEGRITY, Linux, LynxOS, Nucleus, OSE, RTX, VxWorks, WinCE, Windows, QNX

8. Improved Timing Services • Special Timer class implements same services as java.util.Timer with improved semantics: • Not affected by changes in system clock • Uses system “up time”, measured in nanoseconds • PercThread shadows java.lang.Thread; implements sleepUntil(), waitUntil(), and cpuTime() • PERC VM allows configuration of tick period and time slice duration

9. VM Management API • A Java virtual machine represents considerable complexity • Allows transparency to certain VM internals • The PERC VM Management API allows software to: • Query and modify maximum number of heap regions • Query and modify frequency and priority at which increments of garbage collection are performed • Determine which synchronization monitors are locked by which threads, and which threads are waiting • Determine how much time the VM is idle, and how much CPU time is consumed at each priority level • And various other services… • The PERC shell provides a remote command-line interface to the Management API services

10. Real-Time GC • Key attributes of real-time GC: • Preemptive • Incremental • Fully accurate • Defragmenting • Paced • PERC GC is all of the above

11. Mature Development Environment Development Host Target System static link ROMizer Application Java Code Application C/C++ Code compile PERCAccelerator manage PERC Shell 3rd Party Libs Java 1.3 Libs Timer API PERC Shell PERC VM API Direct Memory debug Eclipse PNI/JNI profile Optimizeit Nativecompile/debug PERC Virtual Machine NativeTool Chain RTOS/CPU

12. ROM Size Dynamic Loading Static Linking Large Med Small RAM Size Interpreted Not available from Sun Large Med Small JIT Compiled Not Applicable Start Speed Fast Med Slow AOT Compiled Not available from Sun Not available from Sun Exec Speed Fast Med Slow With Considerable Configuration Flexibility

13. Findings • The C implementation was very brittle: too costly to add new functionality to the framework or to make changes to existing code • This is what motivated the project to be rewritten in Java • The Java system was much more scalable, and more easily maintained • Especially beneficial is the closer association of exception handling with the code it relates to

14. Quantitative Results • Java (PERC) code was one fifth the size of the C code it replaces (mainly because of better logical organization resulting from multi-threaded design) • Developer productivity increased two fold, including the cost of learning OO and Java! • Resulting Java implementation is higher quality, making it more economical to maintain

15. Contributing Factors • Strengths of the Java platform: • Clean object-oriented language was easy to master (unlike C++) • Wealth of libraries both in the J2SE standard and available from third parties (SNMP , TL1) • Especially relevant: high-level multi-threading built in to the language • The multi-threaded Java implementation was much less complex than the single-threaded C implementation!

16. Contributing Factors (continued) • Portability • The exact same platform that is used for Windows, Solaris, and Linux development was also available on single-board computers with specialized real-time operating system (OSE) • Facilitated reuse of existing library code • Eased development (developers could test and debug most of their code on larger, faster, desktop workstations) • Facilitated training and support: outside consultants who were guiding development did not need access to specialized hardware

17. Contributing Factors (continued) • Quality development tools were essential: • Without tools to match the capabilities we had for C development, this effort would have failed • Symbolic cross-debugger was essential (user interface runs on desktop workstation, agent runs on single-board computer) • Remote Optimizeit performance profiler was a life saver – “educated” us regarding performance tradeoffs in object-oriented design

18. Summary • For an application characterized as moderately complex • involving multiple man years of custom software development, • soft real-time constraints measured in tens of ms, • asynchronous interaction with multiple masters, and • responsibility for supervising multiple independent devices Java reduced total development effort by 50% over C

19. Are these results relevant to RTSJ-style development? Not directly

20. RTSJ language semantics (heap referencing integrity) and libraries are very difficult to understand, don’t compose well, lots of implementation dependencies. Contributing Factors • Strengths of the Java platform: • Clean object-oriented language was easy to master (unlike C++) • Wealth of libraries both in the J2SE standard and available from third parties (SNMP , TL1) • Especially relevant: high-level multi-threading built in to the language • The multi-threaded Java implementation was much less complex than the single-threaded C implementation!

21. Standard Java libraries and 3rd party libraries will not run in the RTSJ environment. Each RTSJ implementation will support different libraries. Contributing Factors • Strengths of the Java platform: • Clean object-oriented language was easy to master (unlike C++) • Wealth of libraries both in the J2SE standard and available from third parties (SNMP , TL1) • Especially relevant: high-level multi-threading built in to the language • The multi-threaded Java implementation was much less complex than the single-threaded C implementation!

22. RTSJ threads are not “high level”. Lots of burdensome detail dumped on the feet of the developers (Cost enforcement, deadline overrun handling, memory models, wait-free queues) Contributing Factors • Strengths of the Java platform: • Clean object-oriented language was easy to master (unlike C++) • Wealth of libraries both in the J2SE standard and available from third parties (SNMP , TL1) • Especially relevant: high-level multi-threading built in to the language • The multi-threaded Java implementation was much less complex than the single-threaded C implementation!

23. Only one compliant RTSJ implementation today, for one operating system (Timesys Linux). It is very difficult to write portable RTSJ code. No after-market component libraries. Not a mainstream, well-known technology Contributing Factors (continued) • Portability • The exact same platform that is used for Windows, Solaris, and Linux development was also available on single-board computers with specialized real-time operating system (OSE) • Facilitated reuse of existing library code • Eased development (developers could test and debug most of their code on larger, faster, desktop workstations) • Facilitated training and support: outside consultants who were guiding development did not need access to specialized hardware

24. Not clear if we have commercial strength AOT and JIT compilers, symbolic debuggers, run-time profilers. Contributing Factors (continued) • Quality development tools were essential: • Without tools to match the capabilities we had for C development, this effort would have failed • Symbolic cross-debugger was essential (user interface runs on desktop workstation, agent runs on single-board computer) • Remote Optimizeit performance profiler was a life saver – “educated” us regarding performance tradeoffs in object-oriented design