Design Patterns in Tcl/Tk

1. Design Patterns in Tcl/Tk Brent Welch <welch@scriptics.com> http://www.scriptics.com/people/brent.welch

2. Themes • Tcl Architecture Background • Scripting vs. Systems Programming • Design Patterns and Sample Apps • “Micro” patterns for coding techniques • “Macro” patterns for application structure

3. Tcl Design Patterns • “Bricks and Glue” • Script callbacks • “Glue and GUI” for UNIX programs • Cooperating applications and Safe-Tcl • Control logic for data processing channels • Device control and test environments • Middle tier applications • Web-based applications

4. Tcl is two things: • Tcl is a scripting language • Interpreted for rapid turnaround • Variables, procedures, error handlers • exec programs, file I/O, network sockets,... • Tcl is a C library • Easy to add to applications • Applications define Tcl commands that are implemented in C,C++, or Java

5. When to Script? • Systems Programming (C, C++, Java) • Strong types, Static interfaces • Scripting (Tcl, Perl, Python, Basic, ...) • Dynamic types, Fluid interfaces • Scripting Architecture • The art of defining the “sweet spot” between scripts and components

6. The First Pattern • Components are Commands • Implemented in C, C++, Java, or Tcl • Take string arguments, return a string* • Language constructs such as if, proc, while, and foreach are commands • Command-base approach eliminates a lot of syntax found in other languages

7. Variables Control-Flow Procedures Namespaces Strings Pattern Matching Lists and Arrays Introspection Exec Programs File System Input / Output Clock and Timers Network Sockets Safe-Tcl Tk GUI Toolkit Various Extensions... Built-in Tcl Commands

8. “Bricks and Glue” • Commands are like (fancy) Bricks • They are specialized and efficient • They have configuration knobs and dials • Scripts are Glue that assembles bricks • Scripts set configuration parameters • Scripts define behaviors (callbacks) • Scripts sequence actions

9. Application Structure “glue” Scripts Tcl interpreter Callbacks “bricks” Components

10. Command Callbacks • Building blocks call out to Tcl scripts at important times to determine behavior • I/O channel read and write handlers • Server socket accept handlers • Variable traces • Tk buttons • SQL Iterator Scripts Tcl interpreter

11. Pure Script Pattern • 100% Pure Tcl Applications • Cross platform • Rapid Development • Not the original design center for Tcl! • Namespaces, multiple interpreters, threading added for structure • Sample Applications: • HTML editor (webtk), web server (tclhttpd)

12. “Glue and GUI” • Layer Tcl scripts and Tk GUI on top of existing applications • System administration tools, but with nice GUI • exec command launches program, returns the standard output as value • open pipelines with file redirection • Not as efficient as built-in commands

13. Expect • Automatic control of interactive applications. • exp_send - generate input • expect - pattern match on output. expect { login: {exp_send “welch\r\n”; continue} *assword: {exp_send “a secret\r\n”} timeout {puts “Remote system is down”} }

14. Expect Applications • Automate FTP, Telnet, S/Key • Set password on 150 remote systems • “Screen scrape” mainframe (3270) apps • Automatic test frameworks • Play two copies of chess against each other

15. “Glue and GUI” (exmh) • MH applications, Tcl/Tk user interface • www.beedub.com/exmh Unread Mail Msg Filters URL scan User Procs Tcl/Tk GUI MIME Bounces Glimpse Search C prog Ispell MH email PGP Faces

16. Extensible Applications • Library facility makes it easy to add user code to an application • Introspection makes hook points easy to implement • Choosing hook points is an art • X resource database trick for extensible buttons and menus

17. Introspection • Example: hooks for extensibility. • Just define extension procedures: proc EditHook1 {widget file} {...} proc EditHook2 {widget file} {...} • In application, look for and invoke procedures. foreach proc [info procs EditHook*] { $proc $widget $file }

18. User-Defined Buttons, Menus • X resources store widget attributes, but have limitations (i.e., we have to hack a bit) • Non-standard resources record buttons, menu buttons, and menu entries • Standard resources define button comands, text, variables • Non-standard resource define menu entry commands, text, cascades

19. Exmh • Core User Interface • MH Mail programs • Hook points and user script library • Community Contributions • Glimpse full text search • PGP authentication • Faces glyph database • Ispell spelling checker • Url Scanning

20. Model Sim • Leading CAD Simulation Tool from Model Technologies • www.model.com • Application GUI built in Tcl/Tk • Customers create custom control panels for their simulations

21. Control for Data Processing • Control Logic vs. Data Processing • Tcl for setup and configuration • C, C++, Java for data processing modules • Application-specific framework for processing modules • Audio / Video / Image Processing • Simulation Frameworks

22. Data Source Data Sink Image / Audio Processing Tcl Control Logic, TK GUI Data Filters • CD compression - try many parameter • permutations each night

23. Network Servers • Tcl’s Socket and I/O subsystem make this easy • Callbacks to handle connections • Event-driven I/O to multiplex the server • Script-based servers are robust • Tcl doesn’t leak memory • Script errors do not “dump core”

24. Socket Setup • Client socket set s [socket $host $port] • Server socket proc Accept {newsock ipaddr port} { # New connection in newsock fileevent $newsock readable \ [list Read $newsock] } socket -server Accept $port vwait forever

25. Event Driven I/O • Wait for I/O with fileevent fileevent $sock readable [list Read $sock] • Read one line or block per event proc Read {sock} { if {[eof $sock]} { close $sock ;# clears fileevent } else { gets $sock line } }

26. Desktop Agent • Shamancorp.com desktop agent • Tcl for Windows, Mac portability • Small footprint compared to competitors • Agent performs software inventory, allows distribution of updates, including update of the agent itself

27. Cooperating Applications • Network Managemnt Applications • NationsBank • Agent implementations • What about trust? • Safe-Tcl limits capabilities

28. Safe-Tcl • Dual interpreters (like kernel/user space). • Unrestricted interpreter (i.e., kernel) for user and receiving application. • Incoming Tcl script executes in restricted interpreter (i.e., user space): no dangerous commands. • Aliasesfrom restricted interpreter back to unrestricted interpreter (i.e., kernel calls). • Plug-in for Netscape and Explorer Unrestricted Interpreter Restricted Interpreter(s) Aliases

29. Custom Control Systems • High Energy Physics, Oil Drill Rig • One of a kind machinery • Device drivers • Database • Control logic • Tk GUI • 1-200 KLines, 50% C, 50% Tcl

30. Tcl Test Suite Table-Driven Tk GUI Device Under Test Serial Interface Family of Telecom Line Card Controll Chips ASIC V1 ASIC V2 ASIC V3

31. Device Under Test (AMD) • Version 1, completly custom DOS testing application • Version 2, with more on the way • Tcl-based table-driven approach: • Describe serial communication interface • Describe processor instructions • Auto-gen Tk GUI for test engineer • Tcl script test suite

32. Middle-Tier Application Vignette CNET AOL Scriptics Web Browser App Server Tcl SQL Database

33. TclHttpd Web Server • 250-line “mini server” • 7000 line full featured server • Points of extensibility • URL subtree • Application direct URL • Document handlers • Tcl+HTML Templates • Authentication schemes

34. Application Direct URL • Introspection for URL hook • Tcl Procedure name corresponds to URL • Form data mapped automatically onto Tcl procedure arguments • Return value of procedure is the HTML content returned to the browser

35. Application Direct URL Direct_URL Debug/debug proc Debug/echo {args} { set html <dl> foreach {name val} $args { append html “<dt>$name<dd>$val” } append html </dl> return $html } http://www.com/debug/echo?a=b&c=d

36. Template Page HTML Page Web Page Generation HTTP Request form data cache Application Code

37. Tcl+HTML Templates • Tcl embedded in HTML expanded by the server (i.e., the subst command) • Shared files in each directory contain code, global settings • Server can cache result in HTML file, only re-gen when template or shared files change

38. Site-Wide Look [PageHead “The title” “some keywords”] [MainMenu Support FAQ] HTML content [PageFooter] • A few lines of Tcl expand into hundreds of lines of HTML to create navigational structure around page content

39. XML “Templates” <Page title=“Foo” keywords=“Bar”/> <MainMenu lev1=“Support” lev2=“FAQ”/> HTML content </Page> Convert XML to HTML as needed

40. Scriptics Web Site • Global Look and feel through templates • Tcl Resource Center. Annotated index of 500+ Tcl-related URLs • On-line visitor database • Web store front. Credit card service, fulfillment service • Email pipe into Scriptics database

41. Appendix • The remaining slides show • Where to get Tcl/Tk • Some suggested books • Some low-level coding patterns and examples

42. Getting Started Downloading Tcl and Tk http://www.scriptics.com/software/choose.html Guide to which release is best for you. ftp://ftp.scriptics.com/pub/tcl Primary FTP site Tcl Shell Programs wish, Tcl and Tk GUI toolkit tclsh, Tcl only

43. Documentation Practical Programming in Tcl and Tk, 2nd Ed. http://www.beedub.com/book/ ~20 books on the market On-line manual pages http://www.scriptics.com/man/ Unix "man" command, TkMan Windows help files Macintosh HTML folder

44. Tcl Books http://www.scriptics.com/resource/doc/books/ Practical Programming in Tcl and Tk, Welch Graphical Applications in Tcl/Tk, Johnson Tcl/Tk Tools, Harrison et. al. Effective Tcl/Tk Programming, McLennan, Harrison Tcl/Tk for Programmers, Zimmer Tcl/Tk for Real Programmers, Flynt Tcl and the Tk Toolkit, Ousterhout Tcl for Dummies, Webster, Francis Exploring Expect, Libes

45. Saving Data As Tcl Scripts foreach name [info globals] { upvar #0 $name var if [array exists var] { puts [list array set $name \ [array get var]] } else { puts [list set $name $var] } }

46. Efficient String Processing • Convert input into a Tcl program • Protect Tcl special characters • regsub -all to re-write data into a Tcl program • eval or subst to apply commands to the data • Example: decoding %xx in a URL regsub -all {%([0-9a-fA-F][0-9a-fA-F])} $url \ {[format %c 0x\1]} url set url [subst $url]

47. Subst vs. Eval after the Regsub • subst: only process data matched by regsub • HtmlDecodeEntity example in paper &lt; &copy; &169; • quote $, [, and \ • rewrite into the following general form: unmatched data [command matched data] • eval: process both matched and unmatched data • Html_Parse example in paper • convert braces and backslashes into HTML entities • rewrite into the following general form: } command matched data { unmatched data

48. HTML_Parse example HTML input <title>Tcl { syntax }</title> This is an image. <img src=foo.gif> Tcl program func {start} {} {} {} func {title} {} {} {Tcl &ob; syntax &cb; } func {title} {/} {} {This is an image.} func {img} {} {src=foo.gif} {} func {start} {/} {} {}

49. Multiple Interpreters • Interpeters have path names (lists) • {} is self. {a b} is a child of {a} interp create ?-safe? path path eval command args interp delete path • Safe interpreters start with a limited set of commands. E.g., no open or exec.

50. Tcl as a C library interp = Tcl_CreateInterp(); Tcl_CreateCommand(interp, “foo”, Foo_Cmd, fooData); Tcl_SetVar(interp, “foo_version”, “1.0”); Tcl_Eval(interp, “source myfile.tcl”); Tcl_SetResult(interp, “the answer”); Tcl_LinkVar(interp, “x”, &value, TCL_LINK_INT);