MR_Ti: MAGE-RAD Translator’s importer

1. MR_Ti:MAGE-RAD Translator’s importer Junmin Liu John Brestelli

2. So much to learn as programmer • Programming languages: Basic, Fortran, C, C++, Perl, PHP, Java, D, C#, XML, SQL, Ruby on Rail • Web programming: cgi, servlet, jsp, jsf, web service, SOAP • Framework: MVC programming, struts, WebWork, Spring, PhpMVC, Symphony • Database: Oracle, MySQL, Sybase, SQL Server, DBI, Java Persistent API, Hibernate, Oracle TopLink, • Client programming: Javascript, Ajax, Ajaxan, backbase ajax, data grid technology • Unit test, extreme programming, aspect programming, AspectJ, Agile methodology, model-driven programming, AndroMDAframework, JSP unit testing, catus container • XML technology: JAXB, HyperJaxb, Cocoon, DTD, xml schema, XSLT, XQuery, XLinq in .net • code management: ant, maven, cruise control • IDE: visual C++, visual J, Eclipse, JBuilder, JDeveloper • GUS Object layer, WDK classic, WDK, WSF, build system

3. There are a very few good things to learn • Design patterns • Best practices for software development • iterative development cycles • code to improvement not to perfectness • refactor: continuous integration • unit test: quality control

4. Design Patterns • Programming to interface or abstract class not to concrete class • GoF design pattern, Head first design pattern • Anti-patterns • J2EE core design pattern • Service pattern, iterator pattern, MVC inside WDK • "I see patterns". • "I see dead people, ..., All the time. They're everywhere." --- Cole Sear, The sixth sense

5. Design Pattern in JUnit Further readings: 1. IBM’s DeveloperWorks, AOP series “Design with pointcuts to avoid pattern density” 2. Junit: a cook’s tour by Kent Beck and Erich Gammer

6. Best practices for software development • Refactor: improve the existing code by Martin Fowler • j2ee design and development by Rod Johnson

7. Introduction to MAGE doc • MIAME compliant: MAGE-ML v1, MAGE-ML v2, MAGE-tab, SOFT, MINiML …. • Microarray meta data information • Contacts • Protocols • Array Design • Experiment Design • ….

8. Examples of MAGE doc

9. The goal of MR_T RAD rad.fv.value: 5 <fv> value=5 </fv> ArrayExpress ae.fv.value: 5 • A translator, or object mapper, which will loyally translate MAGE object into RAD, or vice verse • Its data manipulation ability should be thin • Angel wrote the MR_T exporter part back to 2003 MR_T

10. MR_Ti overview: a framework not a tool MagemlReader Reader use MageTabReader SoftReader VO’ BaseProcessor Module 1 use Processor Module 2 ProcessModule Module 3 MR_TiException VO BaseValidator Rule 1 use Validator ValidationRule Rule 2 Rule 3 VoToGusTranslator VO use VoToMageTab Translator make GusObj ServiceFactroy read-in Config.xml MR_Tilogger Study->submit use SqlReporter Reporter SqlTester use GUS Commit Tester

11. The One Constant In Software Development CHANGE No matter where you work, what you’re building, or what language you are programming

12. Problem #1 • There are different formats of documents which are MIAME compliant: MAGE-ML v1, MAGE-ML v2, MAGE-tab, SOFT, MINiML …. • Quick answer: write one loader/plugin for each data format.

13. Problem #2 • We need to validate the MAGE docs before we submit to db, we will have a list of validation rules, the rules can be in different levels of strictness. • We need to be able to pick up the rules if we want. • Different project managers have different flavor on how strict the validation check is, it may end up with different set of rules. • Quick answer: write different types of validators for each project or each project manger.

14. Problem #3 • Every microarray databases: array express, Stanford microarray database, RAD produce different mage doc with different flavors, different policies. We need follow RAD policies when loading them into RAD. • Quick answer: write single or multiple processors to accommodate those differences

15. Problem #4 • How to test it? How to test against database? We don’t want to pollute cbilbld • Unit test • Toyota Production System • What about dumping data into cbiltest? • What about the space shuttle testing? we can’t launch shuttle to test it

16. Problem #5 • “Sometimes bug fixes and tweaks can have unexpected consequences elsewhere. Can we lock down the MR_T code after release?” • What if later on we add new reader, new translator, will there be changes to the code? if (magedoc is MAGE-ML v1) { create reader for mage-ml v1, read the file } elseif (magedoc is MAGE-Tab) { create reader for mage-tab, read the file } elseif (magedoc is SOFT) { create reader for SOFT, read the file } elseif () { …. } elseif () { …. } • The code must be able to easily evolved, should be closed for change, but open to extension • Methods: Inversion of Control, programming to Interfaces, Factory classes

17. Problem #6 • Provide object-orientated exception handling: • SQLException by Spring Framework • “The JDBC API is an object lesson in how not to use exceptions. As well as ensuring that we release resources even if we encounter errors, our abstraction layer should provide a better exception handling model for application code than JDBC does.” • public interface SQLExceptionTranslater { DataAccessException translate (String task, String sql, SQLException sqlex); }

18. Pair Programming

19. Data Transfer Object / Value Object

20. MR_Ti Service Classes

21. MR_Ti Exception Package

22. MR_Ti Sequence Diagram

23. MR_Ti process modules

24. MR_Ti process modules --- continue process() { //any pre-processing code goes here aProcessor->process(); //delegate to the decoree //any post-processing code goes here $self->myProcess(); } My $processor = BaseProcessor->new(); $processor = NamingBioSource->new($processor); $processor = NamingAcqAndQuan->new($processor); $processor->process($docRoot);

25. MR_Ti process modules --- continue call process() process() process() return BaseProcessor NamingBioSource NamingAcqAndQuan

26. MR_Ti process modules --- continue

27. Install and Run

28. Acknowledgement • John • Chris • Elisabetta

29. Software Development Triangle XP/Refactor Design Code evolve

30. Project release cycles • gather list of use cases, order them by priorities • establish first coding cycle and goal, put the most important lists into the cycle * have a 5-pound bag, not a 10-pound bag • go straight for the goal, don't refactor! even the code smells bad, keep a note of it • refactor, better pairing with someone * know when to stop! • release code, find users "dare" to try them, and get feedback and new use cases, re-order all of the use cases * don't work overtime, don't stress yourself • go to step 2, establish another coding cycle until you meet all use cases • Code to improvement not perfectness!!!

31. The Project Card ProjectTag: MAGE Importer Release: v0.8 on 11/15/2006 Priority: Highest Authors: Junmin L., John B. on: 10/02/2006 Accepted: Estimate: 5.5 weeks Description: load MIAME compliant documents into RAD Consideration: 1. there are several types of MAGE doc around: MAGE-ML v1 MAGE-TAB, MAGE-ML v2, SOFT, MINiML…, 2. we need some validations, and processing of these documents before we loaded