WMF Holding Point Application Upgrade

1. WMF Holding Point Application Upgrade Natalia Emelianenko 7/7/2005

2. Why We Need to Upgrade • Imposed migration to J2EE Public Service File upload on the server will not be possible • Problems revealed during the code analysis: • Measurement identification out of control • Database design fits to the workflow as it was imagined in the beginning of the project • Lack of organizing routings for the engineers • Poor security on the Web

3. Solutions 1. Upload Upload files directly into the database Benefits: • Detailed upload logging and traceability. • File replacement can be blocked after the files are downloaded for the analysis. • Upload information is accessible via the database, and the files can be managed with SQL.

4. WMF Workflow • Upload – send mail to MAS-MA engineer • Download • Analysis in Excel • Loading of the information extracted from the files into the database tables • Similarity check • Transfer files to the DB – delete files from the server • Mail to PE • Mail to firm

5. Workflow with New Version • Upload – send mail to MAS-MA engineer • Download – mark as “analysis started” • Analysis in Excel • Loading of the information extracted from the files into the database tables • Similarity check • Mark measurement as “analyzed” • Mail to PE • Mail to firm

6. Solutions 2.1. Identification Measurement identification: Current scheme: attempts to find the unique combination date + magnet + time + time of ap2 + … Proposed: let the firm control what is a measurement, i.e. decide whether the data files being uploaded are • corrected version of previously uploaded files • results of a new measurement of the same magnet performed on the same day

7. Identification Scheme Explained • A couple of data files for the magnet XXXX tested on DD-MON-YYYY is being loaded for the first time with the measurement ID DD-MON-YYYY_XXXX_1 • Next uploads will be at firm’s discretion • replacement,ID not changed, or • new measurement, and its ID will be DD-MON-YYYY_XXXX_2 • Measurement ID is assigned on upload and never more guessed from the data.

8. Solutions 2.2. Database Design New database design fits the real data model. New objects: • Table for uploads (replaces WMF_DATA_SOURCE): • Store both source and info files in one record • Log & manage upload, download and analysis • Table with the list of magnets to manage: • Final measurement for each magnet/stage. • Renaming of magnets (dead measurements).

9. Solutions 2.3. Validation • Analyzer should have the possibility to choose the final measurement for a given magnet at any moment (or until the mail to firm was sent) – easy and safe with the new database design. • Analysis and similarity check can be performed inside the database with PL/SQL or Java.

10. Solutions 2.3. PDA Organizer New functionality: • List of not yet analyzed downloaded files. • List of measurements to send a message to PE on. • List of measurements to send a message to firm. • List of analyzed measurements for which messages were not sent for some reason. • Mailing lists stored in the database

11. Solutions 2.4. Security • Make the session timeout really working (available for test on the J2EE prototype Report Maker). • Choose another type of authentication instead of unsafe basic and /or enforce the use the SSL • Create own password for each firm.

12. New Features Examples Next 6 slides show all changes to the PDA holding point Web applications for the users in MAS.

13. Download 1

14. Download 2

15. Mail to PE 1

16. Mail to PE 2

17. Mail to firm 1

18. Mail to firm 2

19. Upload Illustrations…

20. Upload 1

21. Upload2 – at attempt to load the second measurement for the day

22. Upload3 – uploaded as a new measurement

23. Upload4 – an attempt to load 3rd measurement for the day

24. Upload5 – check all measurements uploaded for the magnet and day

25. Upload6 – the 1st measurement data files have been replaced

26. Download1

27. Download2 – downloaded files disappear from the “waiting list”

28. Upload7 – an attempt to upload files after previous ones have been downloaded

29. Upload8 – replacement of downloaded files is blocked

30. Summary of What Was Done New features: • Safe measurement identification • File upload managed by the database • Mailing lists managed by the database • Organizing features: • List of measurements for the analysis • List of mails to be sent • Less manual input

31. Summary for the Discussion Analysis in Excel • List of new files for the analysis • Skip download – get files directly from the database • Similarity check inside the database • Data loading without conformity check Workflow management • Do we need to mark a measurement as analyzed without mail to PE ? • Should there be a possibility to replace analyzed measurements ? Access rights managed in the database Are we ready to migrate to the new database design ?

32. Database Loading Guidelines The following is proposed by Elena: • Data from any file should be always loaded. • The analyzer should decide the nature of the data “invalidity” (i.e. should the magnet be re-measured). • New valid measurements may automatically invalidate the previous ones. • Tolerances should be stored in the database (to be taken from there for the analysis).

33. Migration Plan • While the safe database migration is being prepared: • Fill in mailing lists in the database, define responsibilities. • Install the new version of the upload (now available for tests on PDA). Excel macro will not take the measurement ID from the file name but construct it as it is done now. • Test the new functionality with the current database design on the prototype and then on PDA. • Migration to the new database structures: • Choose a day when there will be no hanging measurements. • Transfer the data to the new tables. • Install the new web applications on PDA and on the prototype.