ALMA Integrated Computing Team Coordination & Planning Meeting #3 Socorro, 17-19 June 2014

1. ALMA Integrated Computing TeamCoordination & Planning Meeting #3 Socorro, 17-19 June 2014 Review of ALMA User Interfaces Erich Schmid ESO

2. Background • Many of our GUI tools have been used for Cycle 0, 1 and 2; Cycle 3 is coming soon enough • Design and architecture goes back well over a decade • Technologies have evolved significantly – and will continue to do so • We can’t just sit and wait – maintenance is our task, obsolescence management, adaptive and perfective maintenance are part of it. • … and “people don't know what they want until you show it to them.”SteveJobs

3. What happened so far? • Not much: Alan, Maurizio and Erich had the idea of a brainstorming session nick-named Observatory Interfaces 2020 • Also sparked by ASAC comments that “… the OT is stale” and other feedback, e.g. user survey • A few weeks ago we met in Garching, also including Alisdair, Alessandro, Craige, Marcus, Robert K., Rodrigo T. • So here we are reporting our ideas, and suggestions for action. • Note that Decisions and Actions will only become reality with your endorsement

4. Topics covered • Database access • Duplication checking algorithm (a bit off topic) • Desktop GUI vs Web App • Technologies review • Ideas and Concerns In no particular order of importance!

5. Database Access (1) • Databases (the archive) are used by GUI tools everywhere, including many tool proprietary ones • An attempt to visualize this in a drawing has resulted in a big mess of arrows • A better attempt is here in a Google Spreadsheet • Decision: Proprietary DBs should be writable only by the owner • Requires “tool-level” Oracle user • Action: Cross-tool use must be approved and tracked (how?) • Action: Better communication/central registry needed. E.g. we don’t know the status of ERMA, Scheduler APDM

6. Database Access (2) • Decision: XML is and will remain the data exchange format for ALMA software. • Decision: Oracle XML DB is not helping us and we should move away from it. • Action: Migrating ASDM to relational is well underway and should be completed. Additional help from LERMA • Action: Migrating APDM to relational needs further investigation before going ahead

7. Proposal/Project Duplication Detection • Duplication algorithm available in Ph1M. • Used for Cycle 2 review with good results • No clear message for other tools (OT, AQ) • Decision: Proposal duplication checking against (unsubmitted) proposals will never be done • Action: SciOps must provide detailed and all-encompassing requirements for all tools. • Decision: We will go ahead making the current Ph1M implementation an Archive feature that is also available for others, e.g. OT, or AQ.

8. Desktop GUI vs Web App • Web Apps have many pros over traditional Desktop GUIS: • No deployment on client, online bug fixes, … • … but also cons • Server must always be up, validation burden on server, … • Action: investigate if/how the OT, or some of its functions, can be converted into a web app. • Wider scope, i.e. other apps? • Establish working group? Who? How? When? • Is this Maintenance or ALMA Development?

9. Technologies Review (1) • Decision: Java will remain the predominant language. • It’s the most widely used programming language, hence people are easy to find, skills are in-house, and most importantly most of our existing code is written in Java • Decision: Scripting support shall be done in Python/Jython • predominant language used and known by our users. • Decision: No alternative for Java Swing (yet) • Swing declared deprecated by Oracle. Despite this, there is no clear alternative. Moving to Java FX seems premature, because it has not gained much momentum and may itself be shorter lived than Swing itself. • Action:Evaluate FX with Startup Checklist and Request Handler

10. Technologies Review (2) • Decision: New GUIs or ones receiving a major update or refactoring should opt for Web GUI where feasible • Decision: Where Web GUIs are not possible, Netbeans should be used. • Eclipse RCP is a possibility (currently used in TMCDB explorer, Event GUI, Alarm profiler), but should be avoided for future projects, unless there’s a very compelling reason • Decision: Spring and Hibernate are suitable technologies • No reason looking for alternatives, they should be used wherever possible • Action: Spring and Hibernate provide new and better features in later versions. Existing apps should be scheduled to be updated to take advantage of them when possible

11. Technologies Review (3) • Decision: ZK is a suitable choice, but requires lots of server resources • It’s also a niche technology. • Angular + JQuery are better alternatives and should be used for new developments or major updates. • ZK can stay where no problems exist • Decision: Django is not a technology we want to engage further. • Dashboard will continue with some limited exposure to it, but should be refactored when there’s a good opportunity • Decision: Plone stays for the user portal until there’s a compelling reason to change. • Plone shall not be used for anything else. D15: D16: D17: Deviations from the above recommendations must be approved by ICT management

12. Technologies Review (4) • Decision: Deviations from the above recommendations must be approved by ICT management • Action: Establish Architecture Board • e.g. one member from each regional ICT • Reports to ICT management • Reviews and recommends new technologies for approval • ….

13. Ideas & Concerns (1) • Idea: Provide advanced scripting capabilities for debugging purposes • i.e. “injecting” additional code into running system without patches. • Not clear how benefits vs risks are. Think about it in more detail • Idea: Considering the use of Java RMI instead of ACS for certain communications • Requires more thinking to judge benefits vs risks/additional services, etc.. • Idea: Submission service uses a REST server (SPRING) • Decision: Agreement that JAX-WS should be deprecated in favor of REST servers (SPRING/RMS)

14. Ideas & Concerns (2) • Concern: We note that plugins are taken out of the OMC (e.g. Quicklook, what else?) • The danger is that the variety of disjoint tools grows and duplication of effort increases. • Action: Marcus to initiate an OMC architecture review, involving all OMC “customers” to find a suitable path forward. Plugins must not interfere with each other but keep the ability to communicate. • See Maurizio’s presentation • Concern: Archive user interfaces are very disjoint. • This is for both, the L&F and the underlying technologies. (not just the ones delivered by the Archive subsystem, this is true for many others) • Action: L&F recommendation and standardization required. As a first step Erich should get a mandate from ICT management and stakeholders

15. Ideas & Concerns (3) • Idea: Moving away from Oracle. • Pros are that we’ll save a substantial amount of money every year on support licenses (~ 10s of K$) • Cons are Oracle expertise at JAO, working replication, XML DB, and quite a few Oracle specific queries. • Effort (cost) to move away from Oracle seems to far outweigh annual licenses for many years, plus risks of period of instability following a switch. • Decision: No compelling reason to ditch Oracle at this point. • Decision: Do not engage in further use of Oracle specifics, if the above gets overruled one day. • Idea: Replacement of NGAS with “something better” • Decision: NGAS replacement not urgent, watch what ESO DOE does. • SKA initiative to combine efforts not yet considered

16. Ideas & Concerns (4) • Concern:Software written at the OSF/SCO outside the ICT processes is still a major concern. • Probably too long lead time for ICT software is the main reason • This can be avoided with more careful planning • Action: Someone from SciOps IPT must pull the brake and get this under control. It’s clear that we can’t swap overnight, but we cannot let this drag on forever • Concern: Not all Subsystem Scientist/Lead relations work perfectly. • This relationship is all important for an effective and efficient planning and delivery of software. • Action: suggest a (regular) review of subsystem scientist assignments between ICT and SciOps. How often?

17. Ideas & Concerns (5) • Concern:OT has grown for many years, which has clearly impacted its usability and maintainability • Some very old technologies are used, e.g. JSky • The spectral visualizer, an in-house tool, considered to be in need for an update (referring more to the code than the presentation) should be refactored • Action: Get a mandate from stakeholders for performing a review involving both ICT internal, ALMA scientists and external user interface expertise. • Idea: (re-)implement OSSas a service that is available from the OT • To the best of our knowledge an unsupported prototype from Control. • Requires the Control/ACS infrastructure to run and can therefore only be executed at the OSF.

18. Ideas & Concerns (final) • Idea: Collaborate with other projects • We have already some collaborations with other projects in place (CTA approved, LLAMA in the process of being approved) • Members of the ICT also being engaged in EVLA, VLT, E-ELT, SKA, CCAT, … at various levels. • Should we ask for a mandate to strengthen such collaborations? • Possible candidates could e.g. be in the revamp of the OT with SKA, or joint Phase One system for ALMA and VLT, since ESO is going to re-do theirs • At what level should such relationships start, be approved, etc.?

19. The End • Maybe we haven’t quite achieved the 2020 target, but it’s important to keep doing such exercises “…he not busy being born, is busy dying”Bob Dylan

20. Duplication Detection(from ASAC/ESAC 2013) • Currently available in the Ph1M for Cycle 2 • Limited to duplications within the same Cycle only • Suggestion only, needs to be confirmed (or declined) • Duplications can be added manually • Duplication confirmation and recommendation is required • Duplication detection is a future OT feature – not yet scheduled • Potential archive feature – not yet discussed

21. Duplication Detection Algorithm • The proposals belong to the same scientific category. • The proposals contain at least one Science Goal in common, i.e. all of the following conditions are fulfilled: • They have the same target (coordinates coincide within a certain radius) • The difference in the highest spatial resolution is less than or equal to a factor of 2. • The difference in the requested rms is less than or equal to a factor of 2. • Each baseband of the two considered Science Goals contains at least one spectral setup that duplicates a spectral setup of the corresponding baseband of the other Science Goal, as per the following criteria: • The spectral windows overlap by more than 50% of the narrower one. • The difference in spectral resolution between these overlapping spectral windows is less than or equal to a factor of 2.