IRMIS3 Data Service and Application Layer

1. IRMIS3 Data Service and Application Layer Gabriele Carcassi Oct 14 2008

2. Disclaimer • The specifications are not final • parts are missing and there will be a certain amount of evolution on the part that exist • good time to participate • The overall architecture and the guidelines, though, are stable • we will concentrate on those

3. IRMIS3 architecture Browser(Firefox, IE, …) Web server(Glassfish) Javaapplets IRMIS3 RDB (MySQL) IRMIS3Data service AJAX components XML protocol Scripts and CLI(perl, Python, …)

4. Data service • The architecture is your typical Service Oriented Architecture (SOA) and service is a Representational State Transfer Web Service (REST-WS) • Allows to capture the business logic (all the rules that separate valid transactions from invalid transactions) in one place • Clients can be “stupid” • Exposes the data in a well defined xml format • Allows the internals of the database to change • Allows to use any available xml tools

5. Data service: guidelines • The service is responsible that all data modifications are valid and to always return valid information • It should fail to run invalid changes (i.e. installing a building in a room) • If it doesn’t, it’s a bug on the server • Note that valid != correct • Service is stateless

6. IRMIS3 software stack 3rd party Java apps Jython scripts 3rd party Perl/Pyton scripts Integration with external tools (i.e. physcs) Few database utilities: backup, consistency check, etc… Web applications Client JavaScript bridge Applets and Widgets Java Client API XML protocol (REST style WS) Data Service layer Server Database layer

7. How you can interact with IRMIS3 Just use the GUI Depending on your requirement and your expertise, you can interact with IRMIS at different levels of “sophistications” Run scripts that others develop Develop your own GUI Write Jython script using the API Write phyton/perl scripts that consume the XML We expect people at the training are interest in the bottom part Read the XML directly

8. XML protocol 3rd party Java apps Jython scripts 3rd party Perl/Pyton scripts Integration with external tools (i.e. physcs) Few database utilities: backup, consistency check, etc… Web applications Client JavaScript bridge Applets and Widgets Java Client API XML protocol (REST style WS) Data Service layer Server Database layer

9. XML protocol • An XML schema defines the elements and attribute used by the service. • To read: each different query corresponds to a different web address. HTML parameters are used for query parameters. • To write: send an XML which represent a transaction. Each element in the transaction is processed independently.

10. XML protocol: an Oscilloscope <componentTypes> <componentType id="90" description="Oscilloscope: 4 Channel @ 500MS/s; 500MHz BW" name="HP54540A"> <manufacturer id="5" name="Agilent (HP)"/> <formFactor id="5" description="Freestanding"/> <functions> <function id="46" description="CCMS"/> <function id="8" description="Instrument"/> </functions> <requires> <interface id="143" description="GPIB_Slave" relType="control"/> <interface id="5" description="Freestanding" relType="housing"/> <interface id="78" description="120VAC" relType="power"/> </requires> <presents> <interface id="234" description="Port" relType="control"/> </presents> <ports> <port name="Chnl 1"> <portType id="13" name="BNC-F" group="RF Connectors"> <pinDesignator id="303" name="1"/> </portType> <pin usage="Chnl 1" pinDesignatorId="303"> <signalType id="1" direction="IN"/> </pin> </port> <port name="Chnl 2">...</port> <port name="Chnl 3">...</port> <port name="Chnl 4">...</port> <port name="RS232">...</port> ... </ports> </componentType> ... </componentTypes> Accessing /IRMISDataService/data/componentTypes returns the full list of manufacturers

11. XML protocol: a transaction <transaction> <create> <manufacturer id="0" name="New function"/> </create> <delete> <interface id="2" description="Building" relType="housing"/> </delete> ... </transaction> A POST on /IRMISDataService/data/ executes the transaction.

12. XML protocol guidelines • Queries are designed to return “chunks” with related information • minimize roundtrips at the cost of repetition • a componentType returns all the details of the ports, interfaces, manufacturers, etc… • port information for a component returns all the ports, pins, portType, cables and conductors • more or less the opposite of a database query • Transactions are executed in a single DB transaction • No guarantee that the data you read wasn’t modified when you write (no “FOR UPDATE”) • Will provide the timestamp for the last modification

13. Testing the protocol Server URL At /IRMIS/testService.html you can find a simple client of the data service XML to send Server reply Write to server Read from server

14. Demo • Let’s have a look at the Data Service

15. Exercise • Using the test interface • List the manufacturers(at /IRMISDataService/data/manufacturers) • Create a new manufacturers called “ACME”(schema documentation at /IRMISDataService) • List the manufacturers again • Delete manufacturer “ACME”

16. IRMIS software stack 3rd party Java apps Jython scripts 3rd party Perl/Pyton scripts Integration with external tools (i.e. physcs) Few database utilities: backup, consistency check, etc… Web applications Client JavaScript bridge Applets and Widgets Java Client API XML protocol (REST style WS) Data Service layer Server Database layer

17. Java API: design principles • The Java API implement the XML protocol and provides objects that represent the data • All reads are cached: no need to keep caches in your code. • All reads are synchronized • you can have reads on multiple threads on the same data and will not trigger multiple reads on the server (extremely useful for GUIs)

18. Java API design principles • The read access is object oriented • Data is exposed through interfaces • Component, ComponentType, Manufacturer • Some classes function as “finders” for the classes containing the data • Components.getById(123), FormFactors.getByName(“Rack”) • You can follow references from one object to the other • component.getComponentType().getManufacturer()

19. Java API design principles • The write access is protocol oriented • You create a transaction • You call methods to add elements in the transaction • Manufacturers.createManufacturer(transaction, “ACME”) • You save the transaction, which may fail • The read access is refreshed only after the data is saved

20. Importing components from Excel (using Jython) This is an example of a script we might write for you from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful"

21. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Load excel file

22. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Connect to the service

23. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Loop over the rows

24. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Get data outCreate components

25. Demo • Let’s browse the API documentation

26. Exercise • Write a Java (or Jython, JRuby, Groovy, …) script that exports the components (id, Type name, Serial, FieldName) to an Excel file • API and documentation at /IRMISDataService • Use JXL from http://jexcelapi.sourceforge.net/

27. Applets and javascript 3rd party Java apps Jython scripts 3rd party Perl/Pyton scripts Integration with external tools (i.e. physcs) Few database utilities: backup, consistency check, etc… Web applications Client JavaScript bridge Applets and Widgets Java Client API XML protocol (REST style WS) Data Service layer Server Database layer

28. Why applets?!? • Especially given the bad history of applets, and the number of Web 2.0 javascript framework (jMaki, GWT, DOJO, DWR, …)? • The code generated by these framework cannot be directly modified: you have to know the framework • By providing you a Java API and applets, you can more easily develop on top: either by writing a couple of lines of javascript, or by integrating in a javascript framework

29. What we provide • A set of Swing models to access the data • These models will be able to populate themselves asynchronously (some already can), so that no logic needs to be put in the GUI • A set of Swing components • You are free to take these components and put them in your application • A way to wrap these components in applets • So you can script on top of them with web pages

30. Scripting applets • We had a summer student (Vikram Aravamudhan) write a general purpose library that allows to register javascript to Swing events Example at /IRMIS/componentTypeList.jsp

31. Scripting applets registerEvent(); function componentTypeChanged(res1) { document.getElementById('manufacturer').value = res1.getSource().getSelectedValue().getManufacturer().toString(); document.getElementById('description').value = res1.getSource().getSelectedValue().getDescription().toString(); document.getElementById('ffactor').value = res1.getSource().getSelectedValue().getFormFactor().toString(); document.getElementById('name').value = res1.getSource().getSelectedValue().getName(); } function registerEvent() { // The applet might not be ready when this function is called the first time. // If it fails, it retries after a second. try { document.applet1.registerEvent("valueChanged","componentTypeChanged"); } catch (ex) { setTimeout('registerEvent()', 1000); } } Executed when the list selection is changed Execute the JS function componentTypeChanged as a ListSelectionListener.valueChanged(ListSelectionEvent) The idea is that each IRMIS applet wraps around a JComponent, and you can register to the events that that component exposes.

32. Demo • Let’s have a look at the documentation

33. Exercise • Run the scripting example • Modify the example to use the ComponentTreeApplet • Applet documentation at /IRMIS/ • Glassfish unpacks the application in glassfish/domains/domain1/applications/j2ee-modules/IRMISWebUI(This is a hack to get stuff done quickly here)