An Overview of the SUIF2 Base Infrastructure

1. An Overview of the SUIF2 Base Infrastructure David Heine Stanford University http://suif.stanford.edu/

2. SUIF Development Overview I. Demo: Interactive Compiler II. SUIF IR and Extensions III. Building a Pass IV. Annotations and IR Extensions V. Getting Started

3. I. Demo Interactive Compiler > cat b.c int main() { if (5 == 1 * 7) return(0); return(1); } > ls $NCIHOME/bin # executables > c2suif b.c # generate a .suif file > sbrowser b.suif # browse the .suif file > ls $NCIHOME/solib/lib*.so # DLLs

4. Running the Suifdriver • SuifDriver - a simple standalone program that loads DLLs and executes modules > suifdriver -i suif> list_modules suif> require usefulpasses suif> list_modules suif> list_interfaces suif> load b.suif suif> check_types suif> require transforms suif> fold_statements suif> require s2c suif> s2c -D ARCH=Linux b.out.c > cat b.out.c • Shell/perl/tcl scripts that compose passes • c2suif, suif2c, f2suif, pass_driver

5. TclSuif examples • TclSUIF - using a TCL shell to load DLLs and execute modules > cat ~/.wishrc source $env(NCIHOME)/suif/suif2b/tclsuif/tclsuif/utils.tk source $env(NCIHOME)/suif/suif2b/tclsuif/tclsuif/tclsuif.tk suif_init > wish8.0 % suif require transforms % suif load b.suif % show_object [suif_fsb]

6. II. The SUIF IR • SUIF IR - An ownership tree embedded in a labelled directed graph • Nodes are sufficient to handle semantics for C and Fortran • Nodes are generic enough to be extended to handle Object semantics • Object Hierarchy • e.g. BranchStatement inherits from Statement • Metaclass system provides for multiple independent extensions • Ownership Tree • e.g. An IfStatement owns a condition, then part, and else part. • e.g. A LoadVariableExpression references its source variable.

7. SUIF IR Hierarchy Object SuifObject SuifBrick IntegerBrick StringBrick SuifObjectBrick OwnedSuifObjectBrick AnnotableObject Annote BrickAnnote GeneralAnnote SymbolTableObject FileSetBlock GlobalInformationBlock CInformationBlock TargetInformationBlock SymbolTable BasicSymbolTable GroupSymbolTable ScopedObject FileBlock DefinitionBlock ProcedureDefinition VariableDefinition ValueBlock ExpressionValueBlock MultiValueBlock RepeatValueBlock UndefinedValueBlock ExecutionObject Statement Expression SymbolTableObject Type DataType ArrayType MultiDimArrayType GroupType StructType UnionType NumericType BooleanType FloatingPointType IntegerType EnumeratedType PointerType VoidType LabelType ProcedureType CProcedureType QualifiedType Symbol VariableSymbol FieldSymbol NestingVariableSymbol ParameterSymbol CodeLabelSymbol ProcedureSymbol Statement StatementList ScopeStatement WhileStatement DoWhileStatement ForStatement IfStatement BranchStatement MultiWayBranchStatement JumpIndirectStatement JumpStatement ReturnStatement StoreStatement StoreVariableStatement CallStatement EvalStatement VaEndStatement VaStartOldStatement VaStartStatement LabelLocationStatement MarkStatement Expression Constant IntConstant FloatConstant BinaryExpression UnaryExpression SelectExpression SymbolAddressExpression LoadExpression LoadValueBlockExpression LoadVariableExpression ArrayReferenceExpression MultiDimArrayExpression FieldAccessExpression VaArgExpression

8. Program Representation owns ProcedureDefinition references a = b; SymbolTable destination StoreVariableStatement “a” source “b” LoadVariableExpression

9. The Top of the SUIF Ownership Tree • SuifEnv contains a FileSetBlock • FileSetBlock contains FileBlocks • FileBlocks contain ProcedureDefinitions and VariableDefinitions • ProcedureDefinitions contain ExecutionObjects • ExecutionObjects • Statements • Expressions • Machsuif instructions • Machsuif instruction lists • SymbolTables (Scopes) at all levels implicitly linked for lookup

10. Ownership Relationships FileSetBlock external_symbol_table symbol_table_objects file_set_symbol_table file_blocks symbol_table definition_block variable_definitions initialization procedure_definitions body source_ops symbol_table definition_block* information_blocks BasicSymbolTable, GroupSymbolTable symbol_table_objects ExecutionObject source_ops Statement child_statements StatementList statement (child_statements) ScopeStatement body (child_statements) symbol_table definition_block WhileStatement, DoWhileStatement condition (source_ops) body (child_statements) ForStatement lower_bound (source_ops) upper_bound (source_ops) step (source_ops) body (child_statements) pre_pad (child_statements) IfStatement condition (source_ops) then_part (child_statements) else_part (child_statements) BranchStatement, MultiBranchStatement decision_operand (source_ops) JumpIndirectStatement target (source_ops) ReturnStatement return_value (source_ops) StoreStatement value (source_ops) destination_address (source_ops) StoreVariableStatement value (source_ops) CallStatement callee_address ( source_ops) EvalStatement expressions (source_ops) VaStartStatement, VaStartOldStatement, VaEndStatement ap_address (source_ops) Expression BinaryExpression source1 (source_ops) source2 (source_ops) UnaryExpression source (source_ops) SelectExpression selector (source_ops) selection1 (source_ops) selection2 (source_ops) LoadExpression source_address (source_ops) LoadValueBlockExpression value_block (source_ops) ArrayReferenceExpression base_array_address (source_ops) index (source_ops) MultiDimArrayExpression base_array_address (source_ops) offset (source_ops) index (source_ops) elements (source_ops) FieldAccessExpression base_group_address (source_ops) VaArgExpression ap_address (source_ops) SuifBrick OwnedSuifObjectBrick object AnnotableObject annotes BrickAnnote bricks Type ArrayType lower_bound upper_bound MultiDimArrayType lower_bounds upper_bounds GroupType group_symbol_table Symbol FieldSymbol bit_offset NestingVariableSymbol bit_offset ValueBlock ExpressionValueBlock expression (source_ops) MultiValueBlock sub_block RepeatValueBlock sub_block

11. III. Writing SUIF passes • About Passes • Building the EmptyPass • Tools for writing non-empty passes • Traversing the IR • Changing code • A selection of useful primative functions • Cloning code • Available data structures • Building interesting passes • Debugging Hints

12. What Passes Do • Compose other passes • Analyze the IR • Annotate the IR • Transform the IR • Enhance the IR semantics

13. Before You Write a Pass • Check if it already exists! • Leverage existing tools to reduce the complexity of your pass • Use available passes to simplify the SUIF input • Use clean up passes after your pass • Making it extensible • Publish interfaces in the SuifEnv for extensions • Use CascadingMap<> for SuifObject specific actions

14. SUIF Pass Classes • FrontendPass • Produces a FileSetBlock • implement FileSetBlock *build_file_set_block(void); • Pass • Uses a FileSetBlock • implement void do_file_set_block(FileSetBlock *); • PipelinablePass • Subclass of Pass that uses procedures of a FileSetBlock • implement void do_procedure_definition(ProcedureDefinition *); • BackendPass • consumes a FileSetBlock • implement void do_file_set_block(FileSetBlock *);

15. Writing the EmptyPass • Building the EmptyPass with command line options #include <suifpasses/passes.h>#include <suifkernel/command_line_parsing.h”#include <suifkernel/suif_env.h>#include <stdio.h>class EmptyPass : public Pass {public:EmptyPass(SuifEnv *env, LString name = “empty_pass”) : Pass(env, name), _verbose(false) {}virtual void initialize() { Pass::initialize(); _command_line->set_description(“Empty pass that does nothing”); _flags->add(new OptionLiteral(“-v”, “Do nothing verbosely”, &_verbose, true));}virtual Module *clone() const {return(Module*) this; }virtual void do_file_set_block(FileSetBlock *) { if(_verbose) printf(“Verbose\n”);}protected: bool _verbose;};extern “C” void init_empty(SuifEnv *) {...}

16. Building the EmptyPass • Standard Makefile for a library SRCS=empty.cpp OBJ_FILES=empty.o TARGET_LIB = empty LIBS += -lcommon -liokernel -lsuifkernel -lsuifpasses include $(NCIHOME)/Makefile.std > source NCIHOME/nci_setup.csh > gmake > ls $(NCIHOME)/solib/libempty.so > suifdriver -e “require empty; empty --help” > suifdriver -e “require empty; load b.suif; empty_pass; save b.copy.suif”

17. Simplifying Your Pass • Use available passes to put the code into the form you want • Dismantle complex IR nodes: • IfStatements, WhileStatements, DoWhileStatements, ForStatements, ScopeStatements • Add for guards, folding • MultiDimArray transforms • Clean up passes • fold_statements; copyprop, deadcode_pass • rename_colliding_symbols - Use after creating new variables • recycle_trash - Use after removing or replacing objects • gc_symbol_table - Use after replacing variables

18. Tools for Traversing the IR • Iterators • imperative iteration over the ownership tree • local iteration over an object’s referenced objects • local iteration over an object’s named fields • Walkers • closure applied to IR objects • Object Collections • build a list of the IR objects

19. Tools for changing code (the utils directories) • The Utils and cfeutils libraries have lots of useful interfaces for common functionality • IR builders, folding • Creating nodes • Typebuilder - maintain some invariants (typebuilder) • Nodebuilder - simplified object creation • Substitution • replace_expression, replace_statement • Evaluation - folding • Duplication - inlining and cloning

20. Useful Primitives • Creating a type Object IntegerType *create_integer_type(suif_env, bit_size, bit_alignment, is_signed); • Using the typebuilder IntegerType *TypeBuilder::get_integer_type(bit_size, bi_alignment, is_signed); • Replacing Expressions/Statements Expression *replace_expression(Expression *, Expression *) Statement *replace_statement(Statement *, Statement *) • Removing Suif Objects SuifObject *remove_suif_object(SuifObject *obj); • Inserting Statements Statement *insert_statement_after (Statement *loc, Statement *) Statement *insert_statement_before (Statement *loc, Statement *)

21. Cloning the Representation owns ProcedureDefinition references original cloned y = clone_statement(x); SymbolTable y x StoreVariableStatement “a” StoreVariableStatement LoadVariableExpression “b” LoadVariableExpression

22. Available Data Structures • Templatized structures • list, vector • indexed_list • maps, hash tables • splay tree • Segmented strings, string dictionary • Lattice value classes • Infinite precision integers, rationals • Compiler structures • bitvector, sparse bitvector • directed graphs - adjacency list and dense matrix • Directed graph algorithms • iterated dominance frontiers • SCCs

23. A Useful? Pass Example • Count all direct calls class CountDirectCallPass : public PipelinablePass {void do_procedure_definition(ProcedureDefinition * proc_def) { int count;for (Iter<CallStatement> iter = object_iterator<CallStatement>(proc_def); iter.is_valid(); iter.next()) { CallStatement *call = &iter.current();if (get_procedure_target_from_call(call) != NULL) count++; };printf(“Procedure %s: %d direct calls\n”, proc_def->get_procedure_symbol()-> get_name().c_str(), count); }};

24. An Instrumentation Pass • instrument all normal procedure entries and exits class InstrumentProcedureStack : public PipelinablePass { void do_procedure_definition(ProcedureDefinition *proc_def) { Statement *pre_ret = build_proc_push_statement(proc_def); Statement *enter_proc = build_proc_pop_statement(proc_def); suif_assert_message (is_kind_of<Statement> (proc_def->get_body()), (“Expected Statement”)); /* insert instrumentation code on procedure entry */insert_statement_before(to<Statement>( proc_def->get_body()), enter_proc); list<ReturnStatement*> *l = collect_objects<ReturnStatement>(proc_def);for (list<ReturnStatement*>::iterator iter = l->begin(); iter != l->end(); iter++) { ReturnStatement *ret = *iter;insert_statement_before(ret, deep_suif_clone<Statement>(pre_ret)); }delete pre_ret; delete l; }};

25. IV. Annotations and IR Extensions • Annotating the IR - LString-indexed information • Flat Annotations - list of “Bricks” • Structured annotations • Extending the IR- adding semantic information • Adding new Expression or Statement Types • What is the difference? • Annotations may be safely removed

26. When to Create New IR nodes • Building a Frontend Pass • To capture high-level semantics for analysis (OSUIF) • To capture AST (abstract syntax tree) for software engineering reasons • Build new IR nodes to match the parse tree (e.g. cfor) • Build dismantlers to standard representation • High level semantic operations • Succinct representation - easy to read, easy to transform further • e.g. SPMD nodes • Persistent interfaces between passes • e.g. region graph, ecr nodes, interprocedural SSA

27. Extending the RepresentationAdding Annotations • Unstructured Annotation void mark_direct_target(SuifEnv *s, CallStatement *call, ProcedureSymbol *ps) { SuifObjectBrick *br = create_suif_object_brick(s, ps); BrickAnnote *an = create_brick_annote(s, “call_targets”); an->append_brick(br); call->append_annote(an); } • Structured Annotation concrete CallTargetAnnote : GeneralAnnote { list<ProcedureSymbol * reference> targets; }; void mark_direct_target(SuifEnv *s, CallStatement *call, ProcedureSymbol *ps) { CallTargetAnnote *an = create_call_target_annote(s, “call_targets”); an->append_target(ps); call->append_annote(an); }

28. Extending the RepresentationAdding IR Nodes - SPMDStatement An Interface for SPMD code transformation concrete SPMDStatement : Statement { ## interface to spmd dismantler, a lower level ## code generator SymbolTable *owner spmd_symbol_table build { create_basic_symbol_table( _suif_env ); }; ParameterSymbol *reference spmd_pid; VariableSymbol * reference spmd_nproc; Statement * owner spmd_body in child_statements; Statement * owner serial_body in child_statements; list<SPMDAccess *owner> spmd_externals; list<SPMDTransform * owner> spmd_transforms; ## for spmd dismantler implementation Statement * owner serial_init default {0}; Expression * owner serial_par_test default {0}; };

29. Debugging Hints • Use existing passes to check the consistency of the IR • check_types - type checker • validate_suif - object ownership validation • Generate the problem with a smaller input program • Debugging with dynamically loaded libraries • Build a standalone pass • Dealing with GDB and DLLs • break main; run; break dll_init_hook; continue; break where_you_need_to_break • set overload-resolution off

30. Visit Web page: The SUIF2 Infrastructure Guide (http://suif.stanford.edu/suif/suif2/doc/) Overview of the SUIF2 system (postscript) Download and run the compiler Available programs and compiler modules Writing your own SUIF pass SUIF program representation guide (postscript) Organization of the SUIF source directories SUIF programmer guide (postscript) SUIF interfaces guide Examples of a pass and an IR module using hoof SUIF System Programmer’s Reference All the high-level interfaces documented (example) Generated from the .h files by Doxygen All the cross references, e.g. class hierarchy information V. Getting Started