Perl

1. Perl CISC/QCSE 810

2. Dirty Secret • In computation-based science, a shockingly large number of person-hours are spent on bookkeeping • Common worries: • have I run all the data through the filter? • have I run all four algorithms on my new test case? • in the graph in my paper generated using the latest optimization algorithm, or the one before? • how can I track the progress of this data through pre-processing, search, filtering, and graphing? • Modularity is most effective when combined with automation

3. History of Perl In the mid 1980's, Larry Wall was working as a sysadmin and found that he needed to do a number of common, yet oddball functions over and over again. And he didn't like any of the scripting languages that were around at the time, so he invented Perl. Version 1 was released circa 1987. A few changes have occurred between then and now. The current version of Perl has exceeded 5.8.0 and is a highly recommended upgrade.

4. What Is Perl? • Practical Extraction and Report Language • Originally focused on text parsing, input and output • Since has generalized to a "glue" language, as well as having libraries dedicated to particular scientific and computer science areas

5. Where is it used? • Anywhere you parse or manipulate text • Web servers (CGI scripts) • Database access and processing • Anywhere you want to automate file handling and organization • Generally not the first choice for numeric calculations • not compiled, not type-safe

6. Type of Language • Semi-interpreted • Uses byte-code representation like Java • Doesn't store byte code for later re-use • In desktop usage, most like BASIC • write program, then run it • Advantages – no compiling, no makefiles • Disadvantages – can be slower running, not type-safe, can encourage poor style

7. Example #!/usr/bin/perl -w if (@ARGV != 1) { die "Usage: prog1 <filename>\n"; } $filename = shift @ARGV; open(FILE, $filename) or die "Unable to open $filename\n"; %word_count = (); while ($line = <FILE>) { chomp $line; @words = split('\s+', $line); foreach $word (@words) { $word_count{$word} += 1; } } close(FILE);

8. Example continued # *** Print words in alphabetic order *** foreach $word (sort (keys(%word_count))) { print "$word: " . $word_count{$word} . "\n"; } # *** Print words in descending count order *** sub hashValueDescendingNum { $word_count{$b} <=> $word_count{$a}; } print ("-" x 80) . "\n"; @sorted_words = sort hashValueDescendingNum keys(%word_count); foreach $word (@sorted_words) { print "$word: " . $word_count{$word} . "\n"; }

9. Basic Datatypes • Scalar --- $ • any single value • can (and often is) converted at will between integer, double, string • Array --- @ • an array of scalars • Hash --- % • like the STL map, an easy way to construct fast lookup tables

10. Perl as File Walker #!/usr/bin/perl -w @ppt_files = <*.ppt>; print "PPT files in the directory are ". join(", ", @ppt_files) . "\n"; @all_files = <*>; foreach $file (@all_files) { if (-f $file) { print "file: $file\n"; } elsif (-d $file) { print "dir: $file\n"; } }

11. Ways to access shell • Backticks • $response = `<command>` • system • system("<command>") • exec • exec("<command>")

12. Example #!/usr/bin/perl -w @all_files = <*>; FILE: foreach $file (@all_files) { if (! -f $file) { next FILE; } $wc_resp = `wc -l $file`; chomp $wc_resp; @wc_resp_arr = split /\s+/, $wc_resp; $lines_in_file = $wc_resp_arr[0]; print "$file: $lines_in_file lines\n"; }

13. Perl as a Web Browser • with LWP Perl library, Perl can • act as a web browser • download web pages • identify and follow links • download non-HTML files

14. Web Browser Example # Create a user agent object use LWP::UserAgent; $ua = LWP::UserAgent->new; $ua->agent("MyApp/0.1 "); # Create a request my $req = HTTP::Request->new(POST => 'http://search.cpan.org/search'); $req->content_type('application/x-www-form-urlencoded'); $req->content('query=libwww-perl&mode=dist'); # Pass request to the user agent and get a response back my $res = $ua->request($req); # Check the outcome of the response if ($res->is_success) { print $res->content; } else { print $res->status_line, "\n"; }

15. Perl as a Controller • Perl can communicate via ports • can set supervisor running on one computer • can set delegates running on set of others • delegates can talk to server, get run parameters, execute code, and report back • automates algorithms with "embarassing parallelism"

16. Diagram

17. Personal and Professional Uses • Automated the uploading of slides, via perl FTP package • Downloaded all the web-based version of a textbook to a local copy • Batch run data analysis across multiple machines • Manage process of updating graphs of research results • Load TA assignments from a flat file into a database, using DBD package • Generate HTML reports from database • Check directories for redundant files

18. Perl isn't unique • Python and Ruby are two other languages that can play similar roles • not compiled • can be included on web servers • similar intent of easy file and text manipulations • Using Perl is a personal preference for me • learned first, haven't felt anything missing • included in almost every Unix distribution

19. EXTERIOR: DAGOBAH -- DAY With Yoda strapped to his back, Luke climbs up one of the many thick vines that grow in the swamp until he reaches the Dagobah statistics lab. Panting heavily, he continues his exercises -- grepping, installing new packages, logging in as root, and writing replacements for two-year-old shell scripts in Python. YODA: Code! Yes. A programmer's strength flows from code maintainability. But beware of Perl. Terse syntax... more than one way to do it... default variables. The dark side of code maintainability are they. Easily they flow, quick to join you when code you write. If once you start down the dark path, forever will it dominate your destiny, consume you it will. LUKE: Is Perl better than Python? YODA: No... no... no. Quicker, easier, more seductive. LUKE: But how will I know why Python is better than Perl? YODA: You will know. When your code you try to read six months from now.

20. Today's Exercises • Modify the word counting script so it returns only the top 10 words in a file • Write a program that outputs the list of files in a directory, but listed in decreasing order of size • Same, but in order of "last modified" (search for "perl file test operators") • Search for pairs of files (e.g. "Song01" and "Song01.f". For each, report whether the second file is newer than the first

21. Resources • Learning Perl by O'Reilly • http://proquestcombo.safaribooksonline.com/0596101058 • Perl books on-line • http://www.perl.org/books/library.html • CPAN – Comprehensive Perl Archive Network • http://www.cpan.org/ • Perl Monks • http://www.perlmonks.org/