Chapter 9: Perl Modules and Automated E-mail

1. Chapter 9: Perl Modules and Automated E-mail • As Perl became a popular programming language many programmers developed libraries of Perl functions to handle commonly needed tasks. • Perl function libraries are called modules. • There are hundreds (if not thousands) of them available for free at the open source CPAN (Comprehensive Perl Archive Network, www.cpan.org). • Most of them are well documented and easy to use.

2. Some Perl modules can come in handy when developing Web applications. • Among others, this book utilizes and discusses: • CGI(for common Web programming tasks) • Mail::Sendmail(for sending e-mail from within a program) • DBI(for database interface) • XML::DOM(for parsing XML files) • The :: notation means that the module on the right was derived from (extends the functionality of) the module on the left. For example, there are several derivations of the Mail module, with Sendmail being one used for sending an e-mail from within a Web application.

3. The CGI module, written by Lincoln Stein, and its derivations have been widely used by Web programmers to handle things like printing HTML form elements, generating long session IDs, and state file operations. • Since the term CGI is used in contexts other than referring to a Perl module, the CGI module is usually referred to as CGI.pm. Here .pm is the file extension used by text files containing Perl modules. • Installing Perl modules on a given computer is covered in Appendix C. • However, most standard Perl distributions come with CGI.pm (and common derivations) already installed.

4. Assuming a module is installed on a Web server, all it takes to load it into a Perl program is a use statement: • use CGI; • use CGI::SecureState; • use DBI; • We have used the require statement to import files containing ordinary functions. In contrast, Perl modules contain objects so the use statement is needed to import them. Aside from that difference, the effect is the same -- to import external function into a program for use there.

5. Object syntax in Perl • A base class for some modules is instantiated by calling the new constructor on the module name: • $cgiObject = new CGI; • The scalar variable is a reference to the CGI object. • Since Perl was developed before objects became the rage, and the period (.) was used for concatenation, Perl chose to use "arrow" syntax to call methods of objects and to access properties. • Example: a call to a return method and a void method • $thevalue = $cgiObject -> param("textFieldName"); • $cgiObject -> save(STATEFILE);

6. There are slight differences among modules. Some modules require an object to be instantiated calling a special method on the module name: • $dbHandle = DBI -> connect(parameters); • To learn to use a particular module, the ample documentation provided on the CPAN Web site is usually adequate. • It is worth noting that some modules don't even require the object-oriented syntax in favor of just importing stand-alone functions to call. However, we use objects for all module use in this text.

7. Using CGI.pm for submitted data • The ensuing discussion assumes that the following two lines of code have been executed: • use CGI; • $cgiObject = new CGI; • When the CGI object is constructed, any incoming URL-encoded data string (GET or POST) is automatically decoded and stored in the object in some internal format.

8. We don't really care about the format the object uses to store that data internally, but simply access submitted data using the param() method. (The name=value pairs are sometimes called the parameters passed around in Web transactions.) • The following examples store the data submitted from form elements which submit only one value: • $theText = $cgiObject->param("textFieldName"); • $theText = $cgiObject->param("textAreaName"); • $theChoice = $cgiObject->param("RadioGroupName"); • $theChoice = $cgiObject->param("PopUpMenuName");

9. Form elements which submit multiple values are returned by param() as an array. • @choices= $cgiObject->param("multipleMenuName"); • @choices= $cgiObject->param("checkboxGroupName"); • For this to work in the case of a group of checkboxes, all of the checkboxes must have the name name. • If only one value is submitted under a given name, param() simply returns an array containing only one element. • If checkboxes in a group each have different names, then one would have to loop over the names and see for which ones param() returns a value.

10. When param() is called with no parameters, • @names = $cgiObject -> param; • it returns a list of the names (not values) of all submitted form elements. • See source code for paramEcho.cgi example. Play with the program by sending data something like ?x=a&x=b&y=c.

11. Note: CGI.pm versus our decoding routine • CGI.pm does not deal with requests other than GET or POST. Thus, your program can't automatically detect a HEAD request or offline execution. • param() basically works like our %formHash but: • param() is not a data structure with operations such as exists, keys, and values. • param() returns an array when multiple values are submitted under the same name. • In %formHash, the multiple values are concatenated together and have to be split out into an array.

12. Using CGI.pm for printing HTML forms • There are methods for printing all of the various types of form elements, but we give only one example. • Example: using the radio_group() method • print $cgiObject->radio_group ( • -name=>"groupName", • -values=>\@values, • -default=>"defaultSelectedValue", • -labels=>\%hash ); • Result:

13. The parameters are sent to radio_group()as a hash, where the keys are special tokens beginning with a dash (-). • The values for the radio buttons have to be sent as (reference to) an array. • The labels appearing next to the radio buttons have to be sent as (reference to) a hash whose whose keys are the valuesfor the radio buttons and whose values are the labels. • Using these methods in CGI.pm is usually more cumbersome than simply printing form elements manually manually. • Thus, we will not use these methods in our examples.

14. Using CGI.pm for managing state data and files • A common practice is to simply dump all submitted data into a state file, even if not all of it needs to be saved between HTTP transactions. • Once a file is open for writing, CGI.pm has a method to write all submitted name=value pairs to the file. • open (STATEFILE, ">fileName.state"); • $cgiObject -> save(STATEFILE); • close (STATEFILE); • Of course, the name of the state file is usually a long session ID.

15. The state data is extracted back out of the state file (presumably during some later transaction) by opening the state file and creating a new CGI object sending the file handle to the constructor. • open (STATEFILE, "fileName.state"); • $cgiObject_from_file = new CGI(STATEFILE); • close (STATEFILE); • The new CGI object works just like any other, except that the name=value pairs are from a file (and a past transaction), rather than a new HTTP transaction. • Thus, you would typically have two CGI objects in a program, one from the incoming query string and one from the state file.

16. We often want to customize what goes into the state file, not just dump all submitted data in there. Given a CGI object (presumably of constructed from a state file) you can: • Add a new name=value pair • $cgiObject -> param( -name => "some_name", • -value => "some_value" ); • Add multiple name=value pairs with the same names • $cgiObject -> param( -name => "some_name", • -value => ["val1","val2"]); • Delete a name=value pairs with the same names • $cgiObject -> delete("some_name"); • Delete ALL name=value pairs in the object • $cgiObject -> delete_all();

17. Notes on using CGI.pm for state management: • You have to create the long IDs. • You have to propagate the ID (hide in page or use a cookie) to enable a session. • You can choose whatever directory you wish for the state files or use multiple directories. • The state file cache is not policed automatically -- it can grow without bound. • You would have to implement your own features such as session timeouts.

18. Using CGI::SecureStatefor managing state data and files • Since this module is derived from CGI, all methods such as param() are inherited. • However, the constructor used to create a new CGI::SecureState object is different. • use CGI::SecureState; • $cgiObject=new CGI::SecureState ("/states_dir"); • If you call the constructor without specifying a directory for the state files, the module will dump them into the same directory as the CGI program. • $cgiObject = new CGI::SecureState;

19. You will deal with name=value pairs in the same way as with CGI.pm. • CGI::SecureState does offer some built-in session tools: • If a name=value pair with name ".id" is not detected as part of the incoming data string, a new session ID is automatically created and placed into the CGI object in a name=value pair of the form ".id=sessionID". Moreover, a corresponding state file is created and ALL the CGI object's name=value pairs are saved to the file. • If an incoming session ID is detected ( in an .id=sessionID pair), then the name=value pairs from the corresponding state file are automatically read into the CGI object.

20. Thus, in a Web application with a sequence of HTTP transactions, the first transaction creates a session ID and state file. • Subsequent transactions automatically read the state file into the CGI object. • But note that name=value pairs from an incoming data string have precedence in the CGI object over ones from the file in the sense that a pair in the file will not replace an incoming pair with the same name. • That is, an incoming pair will effectively overwrite one with the same name in the state file. • This may not be a desired behavior. In some cases, it is advantageous to keep state data totally separate from incoming data.

21. CGI::SecureState does have methods to aid in propagating the session ID. • The method call • $cgiObject -> state_url("URL") • returns the URL with the session ID (which is stored somewhere in the object) attached in the proper format (.id=sessionID). • The method call • $cgiObject -> state_field() • returns the a hidden form element containing the sessionID in the proper format. • <input type=hidden name=.id value=sessionID>

22. CGI::SecureState does automatically save manually added (with the param() method) data to the state file. • The method call • $cgiObject -> add() • overwrites the state file with all name=value pairs currently in the object. • The delete() and delete_all() methods work the same as in CGI.pm, but make the corresponding changes to the state file. • Additional methods delete_session() deletes the state file and age() returns it's age in days.

23. Notes on using CGI::SecureState for state management: • Automatic creation of the sessionID and automatic read/ write to state file is nice. • If you do augment the data with a new name=value pair you create in the program, you still have to do an add() (write state). • It's kind of a hassle to have to use methods to propagate the sessionID. On the other hand, you don't otherwise have to deal with the ID directly. • The state file cache is not policed automatically -- it can grow without bound. • You would have to implement your own features such as session timeouts. The age() method would help.

24. Using CGI.pm (or CGI::SecureState) to handle cookies: • Retrieve the value of an incoming cookie using its name. • $cookieValue= $cgiObject -> cookie("cookieName"); • Create a new cookie object. • $cookie = $cgiObject • -> cookie( -name => "some_name", • -value =>"some_value"', • -expires => "+1y", #1 year • -path => "/some/path", • -domain = ".uweb.edu", • -secure => 1); • Of course, all but the name and value fields are optional.

25. Print a $cookie object using the header() method. • print $cgiObject -> • header(-cookie => $cookie); • Print multiple cookie objects • print $cgiObject -> • header(-cookie=>[$cookie1,$cookie2]); • The Set-Cookie line(s) are first printed , followed by the Content_type line, then a blank line.

26. Notes on using CGI.pm for cookies: • The syntax for creating and printing a cookie is much more cumbersome than simply printing them manually. • On the other hand, the incoming cookies are automatically split apart and decoded for you.

27. Sending e-mail from CGI programs. • The program has to get the e-mail message to an SMTP server. SMTP servers are like post offices in the sense that they move mail around the Internet. • Many existing CGI programs send mail using an external program (like the Unix sendmail system command). • Rather, we will explore the platform independent Mail::Sendmail Perl module.

28. Common header fields used in e-mail messages

29. The purposes of the header fields are straight-forward, but there is subtlety regarding "bounced" e-mail messages -- ones which are not deliverable for some reason like invalid address. • An SMTP server uses the envelope sender address for returning undeliverable messages. • This address is NOT communicated to the SMTP server as one of the header fields of the actual mail message. Rather, it is up to a given e-mail utility to send that to the SMTP server separately. • Some e-mail sending programs will automatically use the From: address as the envelope sender address. Others may allow a default envelope sender address to be set.

30. There are numerous Perl modules which can send messages directly to an SMTP server (Mail::Sendmail, Mail::Mailer, Net::SMTP, etc.). • The most easy to use, and hence the best one to illustrate the main concepts here, is Mail::Sendmail. • The ensuing discussion assumes that the Mail::Sendmail module is installed on the server. • The easiest way to illustrate how to send an e-mail from within a CGI program using Mail::Sendmail is to break down the process into 3 steps.

31. Step 1. In the Perl program, load the module. use Mail::Sendmail; Step 2. Prepare a hash containing the headers and e-mail message. %mail = ( "To" => $toEmail, "From" => $fromEmail, "Cc" => $ccEmail, "Subject" => $subject, "message" => $message, "smtp" => $smtp_server ); Step 3. Send the message to the SMTP server . sendmail(%mail);

32. Things to Note: • The message field is a string which contains the entire body of the e-mail message, including any new line characters. • If an SMTP server is also running on the Web server, then • "smtp" => "localhost" • is sufficient to send the message. Otherwise, the address of some SMTP server would be used, and that server would need to be configured to accept messages from the Web server. (i.e. the SMTP server would serve as an open relay for the Web server.) • Conveniently, the Mail::Sendmail module uses the From: field as the envelope sender address.

33. Example: A sign-up utility. • To get an account, a signup form has to be filled out. • The user is then e-mailed a randomly generated temporary password. • Also, an account file is created for the user. • Presumably, the first logon using the new account would require the new user to choose a permanent password.

34. Using such a sign-up strategy is sometimes desirable because it verifies that the user has supplied a viable e-mail address. • Forms which allow the user to choose a password upon initial signup are vulnerable to bogus accounts being created. See source file newuser.cgi