Hiding legacy software using Perl and SOAP as glue

1. Hiding legacy softwareusing Perl and SOAP as glue Abstract The worst nightmare a software developer can face is a major re-engineering of a mature system. We'd all much rather sit down and write new code than carry out software archaeology on a half understood system written by a dozen different people. This is where Perl can come in, wrapping legacy code in XS and SOAP means that you can hide the horror a mature system behind a clean interface. Perl makes an excellent glue language, and implementing new systems on top of your legacy code no longer means that you have to know the exotic internals of the legacy system. By using web services as building blocks new functionality can be written quickly and efficiently. Old code never dies, it just gets hidden. Alasdair Allan University of Exeter, Exeter, U.K.

2. Encapsulating knowledge • Your code base is the fossil record of your organisation • The worst mistake you can make is to try and re-implement your existing code base from scratch London Perl Workshop

3. Throwing code away… • A very painful example from my own experience is “Chunking” • This wasn’t even because we thought we could do it better, we simply thought it wasn’t relevant to the modern world • We should have modified, not discarded London Perl Workshop

4. Second system effect • Re-implementing things can be a lot more costly than you would expect Second System Effectn. (sometimes, more euphoniously, `second-system syndrome') When one is designing the successor to a relatively small, elegant, and successful system, there is a tendency to become grandiose in one's success and design an elephantine feature-laden monstrosity. London Perl Workshop

5. Software archaeology • Software naturally forms layers • Underneath the convenience layer you’ll normally find a more powerful layer which is usually much harder to understand London Perl Workshop

6. Code reuse There are two main approaches, • Wrapping code • Software as services Both you and the code will benefit from, • Refactoring London Perl Workshop

7. Refactoring Why should you refactor? • To fix architectural problems • To remove inefficiency Just plain ugly is not a good reason. London Perl Workshop

8. Just plain ugly • Code is often ugly for a reason • Every bug fix makes the code harder to understand and much uglier • Refactoring just to make the code pretty could remove added goodness London Perl Workshop

9. Wrapping code • The traditional way to hide legacy code is to wrap it up behind a convenience layer • In Perl this is usually done using XS and lately the Inline::* modules London Perl Workshop

10. Refactoring by stages • Once you wrap components you can remove them in a piece meal fashion • This is especially true if you use the “software as services” paradigm London Perl Workshop

11. The Book… Extending and Embedding Perl Tim Jenness & Simon Cozens Manning, ISBN 1930110820 Costs £21.17 at Amazon.co.uk “…the canonical book for this type of programming” -- Alasdair Allan London Perl Workshop

12. Perl XS • It looks like a couple of talks on the Advanced track will be discussion XS in depth, so I’m not going to bother • Thank goodness for that… London Perl Workshop

13. Inline Module • Inline lets you write Perl subroutines in other programming languages like C, C++, Java, Python, Tcl and even Assembly. • You don't need to compile anything. All the details are handled transparently so you can just run your Perl program like normal. London Perl Workshop

14. What’s going on? • Your Perl module reads the code from the appropriate place usually below the DATA handle • An MD5 checksum is calculated for the code in question • This checksum and other information are compared with the XS modules previously generated by Inline. London Perl Workshop

15. What’s going on? • If the checksum does not match, an XS module is generated based on the functions and arguments in the inlined code. • The module is built and and installed into a local directory • If the checksum matched, the relevant module is loaded London Perl Workshop

16. Inline::C use Inline C; hello_world(’Your Name Here'); __END__ __C__ void hello_world(char* name) { printf("Hello %s!\n", name); } London Perl Workshop

17. Inline::Python use Inline Python; print "9 + 16 = ", add(9, 16), "\n"; __END__ __Python__ def add(x,y): return x + y London Perl Workshop

18. Inline::Python and Objects use Inline Python; my $obj = new Myclass(); __END__ __Python__ from mylibrary import myclass as Myclass London Perl Workshop

19. Inline::Java use Inline Java; my $obj = new Example( ‘some data’ ); print $obj->get_data() . "\n”; __END__ __Java__ public class Example { private String data = null; public Example( String s){ data = s; } public String get_data(){ return data ; } } London Perl Workshop

20. Inline::Java and STUDY use Inline ( Java => 'STUDY', STUDY => ['java.util.HashMap'] ); my $hm = new java::util::HashMap(); $hm->put("key", "value"); my $val = $hm->get("key"); print $val . "\n"; London Perl Workshop

21. Inline::* • Inline::C • Inline::Java • Inline::Python • Inline::Tcl • and others… London Perl Workshop

22. Software as services • A different approach to hiding legacy code is the “software as services” paradigm • Add yet another layer on top of the wrapped legacy code so that the interface to the outside world becomes language neutral • Perl is good at this… London Perl Workshop

23. SOAP::Lite See www.soaplite.com for details The latest release is V0.65 Beta 2, although this is not yet on CPAN. Adds, amongst other things, • MIME and DIME support London Perl Workshop

24. Google Web Services See www.google.com/apis/, use SOAP::Lite; my $key = "000000000000000000000000"; my $wsdl = "http://api.google.com/GoogleSearch.wsdl"; my $query = "foo"; my $google = SOAP::Lite->service( $wsdl ); my $result = $google->doGoogleSearch( $key, $query, 0, 10, "false", "", "false", "", "latin1", "latin1"); my $results = $result->{'estimatedTotalResultsCount’}; print “About $results returned\n”; London Perl Workshop

25. The Book… Programming Web Services with Perl Randy Ray & Pavel Kulchenko O’Reilly, ISBN 0596002068 Costs £19.95 at Amazon.co.uk Now slightly out of date, but still a good source of information about the SOAP::Lite modules London Perl Workshop

26. The Mailing Lists… See www.soaplite.com for details, • Main list is on Yahoo! Groups, see groups.yahoo.com/group/soaplite/ • But there is also soaplite-announce and soaplite-devel lists hosted at SourceForge London Perl Workshop

27. Amazon Web Services • A good example of the software as services paradigm London Perl Workshop

28. ItemSearch Service sub make_request { my $keywords = shift; my $request_type = \SOAP::Data->value( SOAP::Data->name('Keywords')->value($keywords), SOAP::Data->name('SearchIndex')->value('Books')); my $itemsearch_request = SOAP::Data->value( SOAP::Data->name('SubscriptionId’) ->value($subs_id), SOAP::Data->name('Request')->value($request_type)); my $aws_handle = SOAP::Lite->service("$aws_wsdl"); $aws_handle->ItemSearch($itemsearch_request); my $som = $aws_handle->call(); return $som; } London Perl Workshop

29. Building our Complex Type my $request_type = \SOAP::Data->value( SOAP::Data->name('Keywords')->value($keywords), SOAP::Data->name('SearchIndex')->value('Books')); my $itemsearch_request = SOAP::Data->value( SOAP::Data->name('SubscriptionId’) ->value($subs_id), SOAP::Data->name('Request')->value($request_type)); Builds the following complex type <SubscriptionId>$subs_id</SubscriptionId> <Request> <Keywords>$keywords</Keywords> <SearchIndex>Books</SearchIndex> </Request> London Perl Workshop

30. Calling the Service my $aws_handle = SOAP::Lite->service("$aws_wsdl"); $aws_handle->ItemSearch($itemsearch_request); my $som = $aws_handle->call(); return $som; Creates a SOAP object calls the Amazon service using the WSDL found at the URL Returns a SOAP::SOM object containing the results of the query London Perl Workshop

31. A simple SOAP Server The simplest way is to use an existing web server, the following would end up as cgi-bin/service.cgi #!/usr/bin/perl use SOAP::Lite; use SOAP::Transport::HTTP; use ANY_OTHER_MODULE_NEEDED; SOAP::Transport::HTTP::CGI ->dispatch_to(‘/path/to/perl/modules/’) ->handle; London Perl Workshop

32. dispatch_to( ) A collection of Perl modules, dastardly{aa}: ls modules/ drwxr-xr-x 2 nobody users 4.0K Sep 16 2003 ./ drwxr-xr-x 6 root root 4.0K Oct 11 21:56 ../ -rw-rw-r-- 1 nobody users 118 Sep 16 2003 Echo.pm -rw-rw-r-- 1 nobody users 85 Sep 16 2003 Ping.pm package Ping; sub ping { $class = shift; return “ACK”; } London Perl Workshop

33. Easily consumed… You don’t need to write WSDL to use the service, use SOAP::Lite; my $soap = new SOAP::Lite(); $soap->uri(‘http://www.company.com/Ping/); $soap->proxy(‘http://www.company.com/cgi-bin/service.cgi’); my $result; eval { $result = $soap->ping(); }; if ( $@ ) { print $result->faultstring(); exit; } print $result->result(); London Perl Workshop

34. Asynchronous web services • SOAP::Lite does not support .NET asynchronous callbacks • But you can implement asynchronous web services using the module • Use contextual web services and persistent state to keep track of things by hand • This isn’t as hard as it sounds… London Perl Workshop

35. Authentication • See WebService::TicketAuth for a full blown solution • Alternatively you could do a light weight implement of authentication using HTTP cookies by sub-classing the relevant SOAP::Transport module. • Look at www.astro.ex.ac.uk/people/aa/ for an implementation of the later London Perl Workshop

36. No more language neutrality • If you can serialise it you can send it over a wire. Data::Dumper can be a powerful tool my $dumper = new Data::Dumper([$object], [qw($object)]); my $serialised = $dumper->Dump();\ and at the far end, my $object = eval $serialised; if ( $@ ) { print “Warning: Cannot de-serialise object\n”; } London Perl Workshop

37. SOAP::Lite has problems • Interoperability - Although have a look at the SOAP::WSDL module if you need .NET interoperability • Complex types - You’ll find that the SOAP::Data::Builder should simplify things • WSDL - Perl is loosely typed, this means no automatic WSDL generation London Perl Workshop

38. So why did I talk about Inline? • Because of the lack of “proper” WSDL support I’m currently using a mixture of Java and Perl services to hide legacy Fortran (and C) • Perl serves as an excellent glue language for this task London Perl Workshop

39. Grid Services? See www.sve.man.ac.uk/Research/AtoZ/ILCT WSRF::Lite is the follow on work from OGSI::Lite, it implements the Web Service Resource Framework which has effectively replaced OGSI, for details on WSRF visit www.globus.org/wsrf London Perl Workshop

40. Lots of code, broken compiler? If you have a large mature code base but the limits of the underlying infrastructure are starting to cause problems, then perhaps you should • Re-implement the compiler, not your code • No, seriously… London Perl Workshop

41. Re-implement the system? The alternative is to rewrite the entire system in a new language but, • Experience shows that such projects often (almost always?) fail • You loose years (decades?) of customised business logic • You make a lot of people very unhappy London Perl Workshop

42. Use Parrot? • Implementing functional compilers for 4GL languages on top of Parrot has been done before (err, once) • See “Building a Parrot Compiler” by Dan Sugalski at the O’Reilly’s OnLamp.com • For simpler languages, or problems, you could target the compiler for Perl 5 as Dan did initially London Perl Workshop

43. Problems with Legacy Code • Layering - the higher level routines will believe the output of the lower level ones. • Bugs - You still have all the old bugs, although on the bright side you, you don’t have any new ones. • Backdoors - The possibility of backdoors in the lower level code. London Perl Workshop

44. Conclusions • Perl is an excellent glue language • You can use it to hold together the unlikeliest collection of different pieces of code • It may be ugly but this is better than loosing decades of “smartness” • Your code has evolved, don’t lose that London Perl Workshop