Quality – Some ideas

Quality – Some ideas Linda Cornwall WP3 meeting 15th January 2004

What is quality? • Dictionary defines quality as `degree of excellence’ • Lots on software engineering quality, it’s a profession in itself • Books, Journals etc. • Lots of different ideas on what constitutes a quality system, and how to develop one

What I don’t think is quality • Complying with standards • If you write the correct documentation you get a nice tick and comply with the standard • Laying out code in a certain way to comply with a standard • Use of certain development tools • But standards may help develop a quality system • Successfully running a series of tests • A certain set of tests run successfully under a certain set of conditions • But testing is necessary to develop a quality system

R-GMA development • R-GMA has been developed with the focus on getting a lot done quickly and trying out ideas • This is recognised generally in the EGEE exec summary “The current state-of-the-art in Grid computing is dominated by research Grid projects that aim to deliver test Grid infrastructures providing proofs of concept and opening opportunities for new ideas, developments and further research” • A lot has been achieved in a short time • Procedures for code management and testing on the developers machine well developed (CVS, Cruise Control, rgma tests)

No criticism • R-GMA developers are not being criticized, but we should recognize that we now need to move on from maximizing functionality and trying out ideas to ensuring we have a quality system that many people want to use.

What is Quality for R-GMA? • Maybe we should define for ourselves what constitutes quality for R-GMA • Reliability • Does R-GMA run reliably? • Robustness • Is R-GMA robust? Does it cope neatly regardless of circumstances? • Do the resilience tests pass? Has enough been tried? • Maintainability • Is the design well documented? • Is it easy for a developer to find their way around, are the classes neat, and is it easy to follow? Or is there too much linkage between different classes? • Is the code written in a way that is readable by a future developer?

What is Quality for R-GMA -cont • Ease of Use • Is R-GMA easy to use? • Is R-GMA easy to install and configure? • Is the documentation up to date, easy to follow, and accurate? • Does it clearly state what R-GMA does? • Flexibility • Can it cope with different types of usage? (including what we haven’t yet envisaged) • Compliance with requirements • Does R-GMA comply with requirements? • Security • Is R-GMA secure?

Reliability and Robustness • Some things from WP3 support and Bugzilla during the last week “workload management starting to go shonky” “There are a few NullPointerExceptions in catalina.out on tbn08, which think would stop stream producers being deregistered if they are closed by the GRRPThread.” “edg-rgma tool hangs” “looks like the machine has been running out of memory all day - we've got a bug fix for this.” • Does not appear very reliable and robust.

Maintainability • Steve Hicks has produced UML diagrams – indicating a lot of connections between classes. • Hard to maintain • Difficult to avoid problems with robustness • Java code is laid out well using Jalopy. • Some class descriptions need improvement • Other code (e.g. scripts) are better than average (in my experience) in layout. • Should look through and improve, ensure variables are defined, add comments where appropriate • Architecture document was last updated 9 months ago

Usability • Documentation needs improving. • Already gone a long way to improving User Manual • Installation Guide needs much improvement • R-gma web page needs improvement • Need to document clearly what R-GMA does • Not convinced it is all that easy for a programmer to use R-GMA for information management without help from the WP3 team. • Need to assess this further – maybe not immediate priority

Flexibility • Can cope with a wide variety of uses • Especially if we have the user defined schema (Canonical producer) • Very flexible concept

Compliance with requirements • WP3 has listened to other WPs a lot to ascertain their functional requirements • A lot of functional requirements are there, in fact it’s in this area that R-GMA is most advanced • Probably as stated in the usability – we need to ensure what R-GMA does is well documented, and we can better clarify how well it suits the needs of users

Security • There are known security holes • E.g. storing the MySQL user name and password in cvs • Suspect there are plenty of other holes, due to the multiple connections between servlets, between classes. • Getting away with Security through obscurity! • We will need good secure authorization in place if R-GMA is to be used widely

Re-Engineering • The UML diagrams indicate that some re-engineering is necessary to make R-GMA much more robust, reliable and maintainable • One option would be to treat the current software as a prototype, and start again • But I hope we don’t need to do this • I’d suggest taking the UML diagrams, and working out how to modify them so that the structure and connections are simpler. Improve how we treat the classes • Possibly write down detailed requirements, and ensure our re-engineered R-GMA complies • Re-design either using UML diagrams, or some other design tool, to clean up the design. • Don’t rush it - get it right

Analysing the design • We should analyse the design before changing it in the code to ensure that whatever the circumstances • It can’t hang • It can’t run out of memory • We cope with all situations • There are no security holes • Could look at the UML diagrams, discuss the design in groups of 2 or 3, discuss various scenarios • Another possibility is to look at formal methods • But does anyone have the expertise? • Take some time, make sure we get it right

Re-coding and testing • When we are happy with our re-working of the design, then we should modify the code • In parallel, we should update the architecture document and include the new UML diagrams (at least at a high level) • We should thoroughly test this new code • Include resilience tests on a multitude of machines • Include extensive tests of abnormal situations • As well as the sort of tests we carry out each time we check in • We should do this, before we add any further functionality • Then we should have a robust and reliable basis for adding further functionality

Stability • When I say stability, I mean the code base for basic R-GMA functionality should be stable • Then it can be well documented • Other functionality should be added later in a carefully controlled way

Recommendation • Re-engineer to make all the classes and connections in R-GMA as clear and straight forward as possible • Analyse the design and interconnections to ensure that all situations are dealt with, not just the `working as it should’. • This should Improve resilience • No hanging/out of memory etc. • Code and test • If we don’t re-engineer towards a robust, reliable and secure R-GMA I suspect • We will spend forever fixing problems, and introducing new ones • R-GMA will get used for a year or two in the particle physics community and then die when something more robust and secure comes along

Quality – Some ideas