Unicon from Application to Evolution

1. Uniconfrom Application to Evolution Clint Jeffery University of Idaho jeffery@uidaho.edu

2. In this talk... • A “cookie talk” presented at AT&T Labs Research in Florham Park NJ, 1/27/10 • nperf, an important application • Progress, plans, and the grand vision for the Unicon programming language • research opportunities • evolving the language and its tools

3. Implementing a Network Analysis Tool Using a Very High Level Language Kostas Oikonomou Clint Jeffery AT&T Labs Research University of Idaho ko@research.att.com jeffery@uidaho.edu

7. Outline (for the cookie talk) • Motivation - nperf • Background - Unicon • Performance discussion • Results of AT&T – U Idaho collaboration • Next steps • Conclusions

8. nperf: software block diagram

9. nperf – the program • Kostas Oikonomou, Rakesh Sinha • A literate program written in Icon • migrating gradually to Unicon for OO • C libraries for key components • Successful experimental capabilities • Now, how to scale it up • Speed, even with tuned C critical parts • Belady-Lehman pain curve

10. nperf Implementation • “small” • 21 KLOC of noweb files • 14 KLOC Icon + 700 LOC C • Numeric-intensive • Not surprising, but not Icon's forté • Not just matrices • Matrices of structs (or objects)

11. From Snobol to Icon to Unicon • “Very high level” programming languages enable and accelerate experimental research • Successive domain generalizations • Snobol, AT&T, 1960- • pattern matching language • Icon, U. of Arizona, 1980- • goal-directed imperative • Unicon, Unicon.org, 2000- • OO and IO

12. Unicon vs. Perl vs. Python adapted from a comp.lang.python posting • class MyClass:AnotherClass(name) • method greet() • write(“Hi, I'm “, name) • end • end • procedure main() • m := MyClass(“Bob”) • m.greet() • end • class MyClass(AnotherClass): • def __init__(self, name): • self.name = name • def greet(self): • print “Hi, I'm “, self.name • m = MyClass('Bob') • m.greet() package MyClass; @ISA = ('AnotherClass'); sub new { ($class, $name) = @_; $self = {}; bless $self, $class; $self->{'name'} = $name; return $self; } sub greet { ($self) = @_; Print “Hi, I'm “, $self->{'name'}, “\n”; } $a = MyClass->new('Bob'); $a->greet();

13. Unicon vs. Perl vs. Python selected underlying issues • a better Icon • goal-directed evaluation ! • classes and packages are optional • compile-step • a better ABC • Lisp, Icon • OO • highly interactive • reference count • Indent-syntax ! • a better sh, awk • regex'es++ • readability • caters to lone wolves • scripting !

14. Some other Unicon Applications • The SSEUS system at NLM/NIH • Database peer review management • MKR (Knowledge Representation Lang) • Richard McCullough, AT&T (ret) • CVE (Collaborative Virtual Environment) • 3D, online, social IDE • Unicon translator • Merr (Error table generator for Yacc) • Ivib (GUI builder) • Udb (Unicon debugger) • Framework for automatic debugging agents

15. AT&T – UIdaho Collaboration • Started late 08 • K.O. + Jeffery + J. Al-Gharaibeh • Improve nperf's two scalability issues • Performance • Will it run faster on diamond? • (not until Unicon is concurrent) • How bad is the Icon VM bottleneck? • Code maintainability • Will OO refactoring help?

16. Nperf08 Performance • Carefully instrumented by K.O. T1=25.63 T3=2.15 T4=19.78 T5=5.60 T6=54.61 T7=4.80 T8=14.67 sec • Most time spent reformatting matrices to get in and out of C • Idea: modify Unicon to use native C arrays • Idea: extend and streamline Unicon-C interface

17. IntArray and RealArray • n-dimensional, C-compatible • Initially, new data types prototyped for Unicon in summer 09 (Jeffery) • minimal nperf tweaks: ~10% speedup • could modify nperf's matrix-of-structs representation for more speed • Melted implicitly into the list type in Fall 09 (Al-Gharaibeh)

18. Unicon Optimizing Compiler • “unicon -C” based on iconc (K. Walker) • Scalable type inferencing (M. Wilder) • Able to handle nperf? • Yes! ... after numerous workarounds • 2X-2.5X overall speedup for nperf • despite critical sections already in C • Complements array speedups

19. Next Steps • Fix nperf-related compiler bugs • Standardize array representations within the list data type • Modify nperf to use array representations more aggressively where needed • Radically enhance the Unicon-C interface • Inline C • type-rich autowrapper • Review nperf code for OO opportunities to increase its flexibility • Longer term: concurrent Unicon

20. Conclusions • We have improved nperf performance without extensive rewrites • We can improve it more with “modest” changes that leave research flexibility • Improving the underlying language is sometimes better than overtuning • Key: better Unicon – C collaboration

21. Unicon Evolution Clinton Jeffery Jafar Al Gharaibeh Hani Bani Salameh your name may be here someday

22. Changing Plans? • The bosses at AT&T ignored our proposed plans, and jumped all over the two lines mentioning “concurrent Unicon” • Maybe this is because their machine diamond is a 128 processor shared memory machine with, like, 256GB of shared memory. • Maybe it is because nperf runs many hours on current simulations, after heavy tuning • Round 2 of the AT&T collaboration is mostly about concurrent Unicon • Actually, this was “bait” in my cookie talk, because I wanted to go after it.

23. Unicon Evolution: the Big Picture • Languages that don't evolve are dead • The past 10 years of evolution have been about Input/Output (3D, DB, IP) • Multicore support is now essential • C/C++ interoperation is also vital • Research goal: get there without “ruining” Unicon!

24. Concurrency in high level languages • Scripting languages are late to the party • Explicit • Threads • Forks and joins • Communication and synchronization • Mostly considered “too hard” for mainstream • Implicit • “Free” concurrency • Vectorizing compilers • Lisp and Prolog • Difficult to get much benefit for mainstream languages & applications

25. Concurrency in Unicon • Co-expressions are synchronous threads • Under what circumstances is it safe to turn them loose? • How do we make the VM thread-safe? • Native co-exprs vs. pthread coexprs • Local co-exprs vs. loaded programs

26. The C Calling Interface • Completely uninteresting to many customers; completely essential to some • Icon's C Calling Interface • Write a wrapper wrapperfunc() to convert data values and call func() • P := load(“foo.so”, “wrapperfunc”) • Idea #1: wrappers from prototypes • Idea #2: in-line C • Idea #3: Unicon classes wrap C++ classes

27. SWIG • David Beazely; LANL, UChicago, and ?? • C/C++ → HLL • External interface file, external tool • Great for applications developers, suboptimal for a language vendor

28. SWIG Example • %module example %{ • /* Put header files here or function declarations like below */ %} • extern double My_variable; • extern int fib(int n); • extern int my_mod(int x, int y); extern char *get_time(); • … • unix % swig -perl5 example.i • unix % gcc -c example.c example_wrap.c \ • -I/usr/lib/perl/solaris/5.003/CORE • unix % ld -G example.o example_wrap.o -o example.so

29. Unicon, Batchu example • procedure main() • write(“Fibonacci of 4 is “, fib(4)) • end • $c • fibonacci.o { int fib(int) } • $endc