Outline Introduction Problems of JCA and CAJ JCAL Benchmark tests Summary

1. JCAL (Java Channel Access Light library)H. Ikeda, Visual Information Center, Inc. H. Sako, Japan Atomic Energy Agency (JAEA)EPICS Collaboration MeetingKobeOct 2009 Outline • Introduction • Problems of JCA and CAJ • JCAL • Benchmark tests • Summary

2. Introduction • JCA • “standard” Java channel access libraries • JCA-API • 2 implementations • JCA-JNI • Used in SNS, J-PARC, … • JCA-CAJ (Pure Java version) • Used in J-PARC • CAJ • Pure Java implementation is desirable at J-PARC • Control and commissioning applications are made in Java • Multi-platform (Windows, Linux) • However, unstable behaviors (especially for CAJ-1.1.3) • Connections fail for a few channels out of a few thousands • Sometimes, an application with CAJ can never be finished (CAJContext.dispose() does not end) • JCA-API and CAJ codes were investigated

3. Problems/vulnerability in JCA-API and CAJ • Thread safety is broken • Insufficient or inconsistent synchronizations • Examples of thread safety breaking • Invoking Object.wait method without a conditional loop • Should be avoided according to javadoc of Java.lang.Object • Starting thread in its constructor • Could cause unexpected runtime problems • Vulnerable internal structure • Broken encapsulation • Returning mutable fields without defensive copy • Exposing the reference this in the constructor • Strong inter-dependencies among packages • Hard to maintain or modify • Too concrete implementation in API • API should be more abstract (for maintenance) • Users are forced to use problematic implementations • DBR (fundamental data) is not immutable nor thread safe. • Sub-classes inherit it.

4. Problems/vulnerability in JCA-API and CAJ • Other (minor) problems in JCA-API • Non-standard naming styles • Hardwired numbers for network handling • Other (minor) problems in CAJ • finalizer is used. • It is unclear whether or when it is executed by Java-VM. These problems are hard to repair without major redesign. We tried to develop new API and library.

5. JCAL(Java Access Light library) • Limited functionality and simplified architecture • New API with interfaces • By separating clearly interfaces and implementations, maintenance and improvement become easier • Pure java • Java 5.0 • Apache Commons • Client function only • No Repeater implemented • JCAL does not start up an external Repeater process (as CAJ does). • If a Repeater is running, JCAL uses it. • Convenient adaptor library for JCA-API • JCA-JCAL • Existing apps using JCA-JNI or JCA-CAJ can use JCAL • By setting “jp.go.jaea.jcal.jca.JcalContext" to JCALibrary.createContext

6. Thread safe design of JCAL Thread-safe • Single thread architecture • Immutable fundamental data structure (Dbr) Internal single thread Outer threads (user threads) Manager controls the internal structure ClientManager manager for Clients and Subscriptions Client inner class of Channel Subscription inner class of Monitor RepeaterTransport communicates with Repeater (UDP) BroadcastTransport communicates by broadcast (UDP) ClientTransport communicates with server (TCP) API Dbr fundamental data Context library environment Channel channel Monitor monitor

7. Benchmark test • Process time for connect, get, and put • JCAL • JCA-JCAL • JCA-JNI single-threaded (2.3.2) • JCA-JNI thread-safe (2.3.2) • JCA-CAJ (1.1.5b) • Number of channels • 4~4000 • Environment • A Soft IOC: Dell dimension 4500C CPU: Pentium 4 2.4 GHz • Client: Dell PowerEdge 830 CPU: Pentium D 3 GHz x 2 Memory: 1GB • Conditions • Single thread is used for tests. • Total time to process all channels is measured • 10 sec wait time before each test to avoid effects of a previous test • System.gc() executed before the test (to avoid GC during the test) • Iterate a test 10 times in a VM. Take average time of 2nd to 10th tests. • Not to count class loading time. • Async mode with call-back listeners

8. Connect Test Slower Faster • JCAL: 0.038 msec / ch • JCAL~JNI(single)<JCA-JCAL<JNI(thread-safe)<CAJ • The results may depend on algorithm to send broadcast messages

9. Get Test • JCAL : 0.013 msec / ch • CAJ~JNI(single)<JCAL~JCA-JCAL<JNI(thread-safe)

10. Put Test • JCAL : 0.038 msec / ch • JCA-JCAL<JCAL<CAJ<JNI(single)<JNI(thread-safe)

11. Benchmark Test Summary • JCAL and JCA-JCAL are comparable with JCA-JNI (single-threaded) and CAJ. • JCA-JNI (thread-safe) is slower than JCA-JNI(single-threaded). • Overhead for thread safety

12. Summary • JCA-API and CAJcodes are found to have problems in thread-safety and robustness. • To overcome the problems, a newly designed Java CA client library, JCAL, has been experimentally developed which is thread safe in a single-threaded architecture. • Benchmark tests of JCAL show comparable performance with JCA-JNI and CAJ. • We continue stability and reliability tests of JCAL in J-PARC control system. • So far no problems found with beam monitor applications

13. Sync and Async • Sync and asyn definitions are different between JCA and JCAL • Sync (get) • JCA: pendIO(t) blocks until the transmission is complete within duration t. If not complete, the timeout error is thrown. • JCAL: Similar to JCA. “get” blocks until the transmission has been complete or until timeout occurs. • Sync (put) • JCA: it returns immediately after request has been sent, pendIO(t) does not wait for response from the server. • JCAL: Different from JCA. It blocks until it receives the response from the server. • Async (get/put) • JCA and JCAL (common) : it does not block but returns immediately. Completion of transmission is notified by the registered listener. • We use async mode for JCA-JNI/CAJ/JCAL comparison

14. Fundamental Data Structure (Dbr) • Dbr • unit data of channel access • DbrType • data type • Attribute • a marker interface for attribute • StsAttribute • specific interfaces for each kind of attribute Thread-safe Implementation of Dbr sub-classes

15. How to avoid congestion • CAJ • Reduces frequency to send broadcast messages to search for channels in servers as a function of time. • Similar logic as TCP congestion control • But there is no way to know how many packets are really lost in CA, since we don’t know how many servers exist. • JCAL • Sets small buffer size for UDP sockets • Effectively reduce rate of broadcasts

16. Wait() method without a conditional loop In javadoc of java.lang.Object public final void wait() throws InterruptedException /* Causes current thread to wait until another thread invokes the notify() or notifyAll() method for this object…*/ /* A thread can also wake up without being notified, interrupted, or timing out, a so-called spurious wakeup. While this will rarely occur in practice, applications must guard against it by testing for the condition that should have caused the thread to be awakened, and continuing to wait if the condition is not satisfied. In other words, waits should always occur in loops, like this one: */ synchronized (obj) { while (<condition does not hold>) obj.wait(); ... // Perform action appropriate to condition }

17. Exposing this objectStarting thread in constructer public class Unsafe { public Unsafe(SomeOtherClass someotherclass) { // Unsafe because SomeOtherClass has the // "this“ reference and may change its properties // before the constructor completes. someotherClass.registerObject(this); // Unsafe because the "this" object will be //visible from the new thread before the //constructor completes Thread thread = new Thread(this); thread.start(); } }

18. Defensive copying • A mutable object is simply an object which can change its state after construction. For example, StringBuilder and Date are mutable objects, while String and Integer are immutable objects. A class may have a mutable object as a field. There are two possible cases for how the state of a mutable object field can change : • its state can be changed only by the native class - the native class creates the mutable object field, and is the only class which is directly aware of its existence • its state can be changed both by the native class and by its callers - the native class simply points to a mutable object which was created elsewhere • Both cases are valid design choices, but you must be aware of which one is appropriate for each case. If the mutable object field's state should be changed only by the native class, then a defensive copy of the mutable object must be made any time it is passed into (constructors and set methods) or out of (get methods) the class. If this is not done, then it is simple for the caller to break encapsulation, by changing the state of an object which is simultaneously visible to both the class and its caller. • Example • Planet has a mutable object field fDateOfDiscovery, which is defensively copied in all constructors, and in getDateOfDiscovery. Planet represents an immutable class, and has no set methods for its fields. Note that if the defensive copy of DateOfDiscovery is not made, then Planet is no longer immutable!

19. Returning mutable field w/o defensive copy public final class Planet { /**A mutable object field. In this case, the state of this mutable field * is to be changed only by this class. (In other cases, it makes perfect * sense to allow the state of a field to be changed outside the native * class; this is the case when a field acts as a "pointer" to an object * created elsewhere.)*/ private final Date fDateOfDiscovery; // BAD function // Returns a mutable object without defensive copy // The caller gets a direct reference to the internal field. This is usually dangerous, // since the Date object state can be changed both by this class and its caller. // That is, this class is no longer in complete control of fDate. public Date getDateOfDiscovery() { return fDateOfDiscovery; } /*** GOOD function * Returns a defensive copy of the field. * The caller of this method can do anything they want with the * returned Date object, without affecting the internals of this * class in any */ public Date getDateOfDiscovery() { return new Date(fDateOfDiscovery.getTime()); } }

20. Leader-Followers Pattern Serialize Server Connection node

21. Future Pattern • A method which does a job if it is ready, or waits until it becomes ready if not. • DbrFuture Channel.asyncGet() • CaWaiter Channel.asyncPut()

22. Exposing this object // Interface ExceptionReporter public interface ExceptionReporter { public void setExceptionReporter(ExceptionReporter er); public void report(Throwable exception); } // Class ExceptionReporters public class ExceptionReporters implements ExceptionReporter { public ExceptionReporters(ExceptionReporter er) { /* Carry out initialization */ er.setExceptionReporter(this); // incorrectly publishes the "this" reference } public void report(Throwable exception) { /* default implementation */ } public final void setExceptionReporter(ExceptionReporter er) { /* sets the reporter */ } } // Class MyExceptionReporter derives from ExceptionReporters public class MyExceptionReporter extends ExceptionReporters { private final Logger logger; public MyExceptionReporter(ExceptionReporter er) { super(er); // calls superclass's constructor logger = Logger.getLogger("com.organization.Log"); } public void report(Throwable t) { logger.log(Level.FINEST,"Loggable exception occurred",t); } }

23. Repeater (in server) Reply Server1 • Technical problem to implement it in pure Java. • A packet from client to server via Repeater has Repeater’s IP/port as the source address. In order the server to send a response to the client, Repeater must replace the source address to that of the client, but it is not possible in Java. • No parameter for the source IP address and port in the CA protocol. Source:client Repeater broadcast Client Server2 Source:Repeater (w/o packet modif.) Source:client (w/packet modif.)

24. Sync and Async • Sync and asyn definitions are different between JCA and JCAL • Sync (get) • JCA: pendIO(t) blocks until the transmission has been complete within duration t. If not complete, the timeout error is thrown. • JCAL: “get” blocks until the transmission has been complete or until timeout occurs. • Sync (put) • JCA: it returns immediately after request has been sent, pendIO(t) does not wait for response from the server. • JCAL: it blocks until it receives the response from the server. • Async (get/put) • JCA: it does not block but returns immediately. Completion of transmission is notified by the registered listener. • preemptive_callback=true • The listener’s callback is only while event-loop of Context is running. • JCAL: similar to JCA • (no flag preemptive_callback)

25. Ideal A Soft IOC: Dell dimension 4500C Pentium 4 2.4 GHz Client:Dell PowerEdge 830 CPU: Pentium D 3 GHz x 2 Memory: 1GB realistic Beam monitor IOCs: VME Advme7501AR Client: IBM ThinkPad Lenovo R60 CPU: Genuine Intel T2300 1.7GHz x 2 Memory: 1GB Connect Test Preliminary Factor 7 longer Soft IOC : 0.038 msec / ch (JCAL) JCAL=JCA(single)<JCA-JCAL<JCA(multi)<CAJ Monitor IOCs : 0.28 msec / ch (JCAL) JCA-JCAL=JCAL<JCA<CAJ

26. Get Test A Soft IOC Beam monitor IOCs Preliminary Similar soft IOC : 0.013 msec / ch CAJ=JCA(single)<JCAL=JCA-JCA<JCA(multi) Monitor IOCs: 0.040 msec /ch (JCAL)

27. Put Test A Soft IOC Beam monitor IOCs Factor 4 longer Preliminary • Note:CAJ, JCA(“sync”) are not really synchronous. They do not wait for server response. • Soft IOC : 0.038 msec / ch (JCAL) • Beam monitor IOCs : 0.16 msec / ch (JCAL) • async : JCA-JCAL < JCAL < CAJ < JCA(single) < JCA(multi)

28. Benchmark test Ideal A Soft IOC: Dell dimension 4500C Pentium 4 2.4 GHz Client:Dell PowerEdge 830 CPU: Pentium D 3 GHz x 2 Memory: 1GB • Process time for connect, get, and put • JCAL • JCA-JCAL • JCA-JNI single-threaded (2.3.2) • JCA-JNI multi-threaded (2.3.2) • JCA-CAJ (1.1.5b) • Number of channels • 4~4000 • Conditions • 10 sec wait time before each test to avoid effects of a previous test • System.gc() executed before the test (to avoid GC during the test) • Iterate a test 10 times in a VM. Take average time of 2nd to 10th tests. • Not to count class loading time. • Async mode with call-back listeners Realistic Beam monitor IOCs: VME Advme7501AR Client: IBM ThinkPad Lenovo R60 CPU: Genuine Intel T2300 1.7GHz x 2 Memory: 1GB

29. Benchmark Test Summary Summary • JCAL is comparable with JCA-JNI (single-threaded) and CAJ. • JCA-JNI (single-threaded) is fastest. • Why it takes much longer time in real control VME IOCs? • CPU of client/servers? Network conditions? To be investigated. • JCAL is slightly slower than JCA-JNI. • JCAL is faster than CAJ in connection, but slightly slower in get/put. • JCA-JCAL is slightly slower than JCAL. • JCA-JNI (multi-threaded) is slow.