Abandoning APEX in BE-CO - Why and How

Database Tutorials 2017, 20-11-2017 https://indico.cern.ch/event/672720/ Abandoning APEX in BE-CO - Why and How Anti Asko, Lukasz Burdzanowski, Chris Roderick BE-CO-DS

Abandoning APEX–Why (part 1) The not so technical reasons… Chris Roderick

A deep long standing dependency on Oracle Abandoning APEX – Why (part 1), Chris Roderick BE-CO-DS We have a long history with Oracle for Accelerator Controls Databases in use since ~1985 – still heavily used today (see slides from Lukasz)

Freedom of when to upgrade and controlled impact is crucial Abandoning APEX – Why (part 1), Chris Roderick BE-CO-DS Need GUI applications for users to configure the control system Looking for: stability & good integration with the database Oracle Forms 4, 5, 6i (on the Web, breaking changes), 9i (more breaking changes) Oracle HTMLDB 1.6, 2.0 → APEX 2.x, 3.x (breaking changes),4.x (breaking changes), 5.x (breaking changes) Oracle ADF 10g, 11g (breaking changes) Notice a pattern here?

We decide…. Abandoning APEX – Why (part 1), Chris Roderick BE-CO-DS Reason 1 We want to master what we change and when, aligning to the needs and schedules of the accelerators – not Oracle’s.

Great people are CERN’s greatest asset Abandoning APEX – Why (part 1), Chris Roderick BE-CO-DS We rely heavily on Students and Fellows to extend and improve the tools used to interact with the Control system The market for young engineers is competitive! Oracle seems to be generally un-cool to the latest generations of software engineers 😢 APEX is even less cool, and it is a very niche product Newcomers to CERN usually have to learn APEX while they are here. When they leave, they take little transferrable skills with them…

APEX doesn’t have much interest in the marketplace Abandoning APEX – Why (part 1), Chris Roderick BE-CO-DS APEX 5.1 APEX 5.1.3

APEX doesn’t have much interest in the marketplace Abandoning APEX – Why (part 1), Chris Roderick BE-CO-DS

We should recruit excellence & offer relevant development opportunities Abandoning APEX – Why (part 1), Chris Roderick BE-CO-DS Reason 2 We want to attract talented engineers to CERN, allowing them to help build and maintain controls applications that satisfy the needs, whilst giving them an opportunity to gain relevant experience for the marketplace.

Abandoning APEX–Why (part 2) Use cases from Controls Configuration Service Lukasz Burdzanowski

The Controls Configuration Service Abandoning APEX – Why (part 2), Lukasz Burdzanowski BE-CO-DS The main purpose of the CCS: centralize all the information relevant to the Control systems Configuration and reference for high-level applications and services HIGH-LEVEL Accelerators settings management (LSA), data logging (CALS), monitoring and alarming (LASER, Diamon) Meta-data, configuration and reference data of middleware services MIDDLEWARE Directory services, front-end software meta-models, local and central timing, authorization schemes, beam interlocks • The challenge: • >30 years of history • 1500 DB session per day • 80 calls/sec on Java API(s) • 700 domain tables • over 40k DML operations per day • 500 distinct users of GUIs • variety of users • operations-critical components Low-level configuration of front-end computers and hardware modules LOW-LEVEL Directory of hardware equipment types, FECs startup configuration, HW drivers configuration, WhiteRabbit networks Configuration of the machines, from OP and Equipment Group perspectives (RF, BI, ABT, EPC and more), is not possible without CCDB and often CCS GUIs.(though not critical to machine operations or safe shutdown of the machine) CCS in nutshell: large and complex domain(s), very diversified group of users, critical for CO, OP, EG, limited in man-power, having long history and large tech-debt. * Production instance of CCDB is hosted on ACCCON cluster (ABC account + satellite accounts), test environments: ACCINT, INTDB11, dev on DEVDB11 Abandoning APEX in BE-CO, Lukasz Burdzanowski BE-CO

APEX in CCS Abandoning APEX – Why (part 2), Lukasz Burdzanowski BE-CO-DS CCS provides >10 data editing tools, including multipage editors with complex user-interface requirements. Main data browsing tool (read-only) delivers over 150 report pages. The scope of usage varies from expert only (controls middle-ware, interlocks, central timing), equipment groups (FESA, Devices) to sector wide tools (Hardware, RBAC, Browser) and operations in particular (RBAC, Browser). Abandoning APEX in BE-CO, Lukasz Burdzanowski BE-CO

APEX – problems and motivators for phase-out Abandoning APEX – Why (part 2), Lukasz Burdzanowski BE-CO-DS • Based on the role of CCS in the Control System, its requirements and scope – the following issues arise when using APEX as the prime technology for GUIs: • constraining development model (no version control, no IDE, necessity to mix of JavaScript/SQL/PL/SQL, etc.) • black-box technology with poor feature-set (limited UI components, possibilities to customize UI (CSS), need to generate HTML in DB) • niche technology (comparing to the world of HTML5 or desktop with JavaFX or C#, recruitment is an issue) • infrastructure not without issues (ORDS, firewalls, disrupting interventions – but thanks to all for always effective help) • not reassuring strategy from the sole provider: THE Oracle(APEX 5 has backwards incompatibilities, lacks documentation, unclear future in general) • not seen as reliable and stable for end-users(frequent issues with AJAX errors, no multi-browser-tab support, heavy and slow at times, not supported across all browsers) In short: APEX is unsuitable for a service like CCS and management of critical accelerator data at CERN. Abandoning APEX in BE-CO, Lukasz Burdzanowski BE-CO

Controls Configuration Data Editor Abandoning APEX – Why (part 2), Lukasz Burdzanowski BE-CO-DS • The CCDE is a new generation user interface meant to replace all APEX applications used in CCS, characterized by : • use of de-facto standard software industry development practices(GitLab, Jenkins with integration/GUI testing, automatic deployments and versioning) • well-established technologies for Web applications(Spring Boot + embedded Tomcat, TypeScript + Angular, HTML5, excellent availability of know-how resources, attractive and prospective for new young software engineers) • feature-rich web technology addressing known problems(multi-tab support, possibly to make responsive design, no constraints with respect to UI design or look and feel, rich collection of free and commercial extensions, i.e.: HighCharts) • infrastructure tailored to our needs(rolling upgrades with load-balancing, deployment in standard CO model as a Java process, full control over interventions falling into CO/OP/EG upgrades cycle) Foremost, the CCDE is re-designed and implemented from scratch following user needs. It is not yet another iteration of Oracle stack: Oracle Forms > ADF > APEX. Abandoning APEX in BE-CO, Lukasz Burdzanowski BE-CO

CCS APEX Use case: Hardware Editor (1) Abandoning APEX – Why (part 2), Lukasz Burdzanowski BE-CO-DS CCS APEX Hardware Editor – Edition of Computer details. Abandoning APEX in BE-CO, Lukasz Burdzanowski BE-CO

CCS APEX Use case: Hardware Editor (2) Abandoning APEX – Why (part 2), Lukasz Burdzanowski BE-CO-DS CCS APEX Hardware Editor – Edition of Computer physical configuration – crate and modules Abandoning APEX in BE-CO, Lukasz Burdzanowski BE-CO

CCDE: Computer entity view Abandoning APEX – Why (part 2), Lukasz Burdzanowski BE-CO-DS CCDE Computer entity view – all related information are on a single screen, separated logically. Abandoning APEX in BE-CO, Lukasz Burdzanowski BE-CO

Abandoning APEX–How? → ACW What we are doing instead of APEX…. Anti Asko

ACW Objectives Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS To avoid development chaos, duplications, mixture of libraries, different DevOPs strategies... ... in January 2016 we started Accsoft Commons Web as a collaboration amongst concerned BE-CO-DS services (CCS, Layout, AFT, …). Development of ACW is founded on: • Use of the same technologies and common architecture • Less maintenance thanks to cautious selection of third party components and libraries • Development and deployment based on the same tools Accsoft Commons Web, Anti Asko BE-CO-DS

ACW Seed stack Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS The ACW Seed is implemented with a modern and de-facto standard stack for HTML5 web applications with a Java/Spring back-end Server-side: Spring boot and Spring data rest Spring Boot 1.5 (web, data-jpa, data-rest, logging, security etc.) Client-side: TypeScript 2.1, AngularJS 1.6, Bootstrap 3.3 (Font-Awesome, ui-router, ui-grid, lodash, moment, Highcharts etc.) The core artifacts are released as: • accsoft-web-server (jar) • accsoft-web-components (npm) Accsoft Commons Web, Anti Asko BE-CO-DS

ACW Seed Core features Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS • Integration with CERN Single Sign-On based on Spring Security SAML • Transparent integration with RBAC*for authorization and authentication based on Spring Security* RBAC (Role-Based Access Control) is a standard accelerators controls authorisation and authentication service provided by BE-CO. • Built-in LDAP authentication fallback, for when SSO is unavailable • Application configuration based on properties • ACW server-side is designed to work "out-of-the-box" ... and of course user interface aspects and components. Accsoft Commons Web, Anti Asko BE-CO-DS

ACW Components 1/2 Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS ACW includes a set of components which help us to build modern web applications without duplicating code. • A main frame integrated with NavigationService • Authorization tools to customize UI, based on user roles • Form validation utilities • SettingsService to facilitate handling of user settings • Simple UI components input, dropdown, panel, phonebook link etc. Accsoft Commons Web, Anti Asko BE-CO-DS

ACW Components 2/2 Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS Some ACW Components are also full-stack modules, which allow us to create advanced UI features based on a simple configuration in the database. • Reporting grid with Search • Topology Visualization Component (TVC) • Database stored settings

Development environment Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS • Nodejs/npm • Industry standard for building JavaScript/HTML5 applications • Webpack • JavaScript application module bundler • Gradle • Industrial standard building and release tool • GitLab • Streamline the development process in a “feature branch workflow” • IntelliJ IDEA • A modern powerful IDE (refactoring, debugging, VCS support etc.)

Continuous Integration - Deployment Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS

ACWSeed stack – an example Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS

Summary Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS We are still heavily dependent on APEX • Steadily moving away → aim to be APEX-free by end-LS2 Newcomers quickly fit-in and take away a lot • Arrive usually being familiar with the technology • Leave with much better skills and relevant experience Developers are productive and happy • Interesting technologies with large global communities Users are more satisfied • We are delivering better tools ...We are very happy with our choices

Thank you Questions? ACW wiki

ACW Common UI Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS A common stack does not necessarily guarantee a unified look and feel and a coherent user experience (UX). When designing and developing ACW user interface components we took into account the fact that common styles without consistent behaviour can be troublesome for users. To address this, we aim to streamline the UX for ACW by: • Creating a common style guide • Holding regular meetings and open discussions • Doing active follow-ups • Creating interactive documentation

ACW-Future plans • Increase the community around ACW at CERN • On-going discussion about collaboration with BE-ICS • Interest from BE-OP for a new E-Logbook • Upgrade from AngularJS to Angular • Make implementation not CERN specific • Database agnostic, RBAC, SSO etc. … going Open Source in the long term Abandoning APEX – How? → ACW, Anti Asko BE-CO-DS

Abandoning APEX in BE-CO - Why and How