Milestone 1 and SVN

1. I shall follow the submission instructions. Note the team registration deadline, Friday, January 15th. If we don’t hear from you, we’ll assign you a team. Milestone 1 and SVN Hans-Arno Jacobsen Take note of TA Schedule et al. (Tech Talks in Lab.) http://ece297.msrg.utoronto.ca/T2011s/TeachingAssistants Labs Tue & Wed 3-6 PM: GB243, GB251, SF2102 Tech talks 3-4 PM: GB404 (Tue) & GB405 (Wed)

2. Agenda • Storage server Milestone 1 overview • Subversion (svn) • Code versioning • Concurrent development

3. Storage server Milestone 1 Limited API Application Client get(…), set(…), … S Config. Table Table auth(…) Network Table Table One client at a time Storage server Tables are memory resident only Data schema is strings only “TorontoOntario-7.0“

4. Tables and records “95 Aplus“ • Storage server manages tables • A table is a collection of records • Each record is uniquely identified by a key • Records comprise one or morevalues • A record is a string with a maximum number of characters (Milestone 1!) 5

5. Keys are unique Course is the table’s key field. The key must be unique. 6 6

6. Storage server API void *storage_connect(char *hostname, int port) int storage_disconnect(void *conn) int storage_auth(char *username, char *passwd, void *conn) int storage_get(char *table, char *key, struct storage_record *record, void *conn) int storage_set(char *table, char *key, struct storage_record *record, void *conn)

7. What do I need to implement?

8. What do I need to implement? • Client-side library functions (get, set etc.) • Server-side functions (get, set etc.) • Communication between client and server • How does the server know what function is called? • How does the server know what the input to the function is? • Table management • How can the server find a table, a record in a table? Determine the table/record does not exist?

9. What do I need to implement? • Client-server authentication Authentication on the server side & associated error handling is missing … storage_connect(…) … storage_auth(“admin”, “dog4sale”, …) … // gets & sets … storage_disconnect(…) Client server_hostlocalhost server_port 1111 server_username admin server_password_encxxxnq.BMCifhU table marks S Config. Table Table Table Table • > cd storage/src • > make encrypt_passwd • > ./encrypt_passwd dog4sale • xxxnq.BMCifhU Storage server

10. What don’t I need to implement? • Read the skeleton code! ECE 297

11. get & set “MyCourses” • int storage_get( • const char *table, • const char *key, • struct storage_record *record, • void *conn • ); • int storage_set( • const char *table, const char *key, • struct storage_record *record, • void *conn • ); ??? struct storage_record { char value[MAX_VALUE_LEN]; uintptr_t metadata[8]; }; (given, see handout) ??? ECE 297 12

12. What else do we need? • Besides retrieving (get) and inserting (set) a record?

13. Functionality • Retrieve an existing record • storage_get(…) function • Insert a new record • storage_set(…) with a key that does not already exist in the table • Update an existing record • storage_set(..) with a key that already exists in the table • Delete a record • storage_set() with a key that already exists in the table and a NULL value as record

14. Error conditions • ERR_INVALID_PARAM (all 5 functions) • parameters do not conform to the specification. • ERR_CONNECTION_FAIL (all 5 functions) • connection problems to server • ERR_AUTHENTICATION_FAILED • client-server authentication problems • ERR_TABLE_NOT_FOUND (storage_get()/_set) • specified table does not exist • ERR_KEY_NOT_FOUND (storage_get()) • server indicates that the specified key does not exist in the specified table. • ERR_UNKNOWN • flag any other errors (out of memory, file not found, …)

15. Questions?

16. BPMeduNet • Exchange program with • Free U. Berlin, Germany • U. of Applied Sciences, Graz, Austria • U. of Coimbra, Portugal • Spend a semester at one of these places • Do a summer research internship there • Scholarships available

17. Subversion (svn) Note the quiz in the lab later in January

18. Subversion (client: svn) Main tasks Version control “I just fixed something and nothing works anymore” Shared, concurrent development Development in a team with 2 to 2000 developers Also helpful, if you work alone Also applicable to concurrent development on text, web pages, … Concurrent development is the reality (cvs, svn, git & github, ClearCase, …) ECE 297 26

19. Basic idea Like a file server, but tracks ALL modifications to files and to directory structure Where is yesterday’s version? Where is Tuesday’s version from 1:03 AM? Who modified yesterday’s version? What were the changes? What’s the difference to the current copy? svn repository Harry’s working copy Sally’s working copy ECE 297 27

20. How to proceed from here? • All: • Go through the examples in our Assignment 1 handout • Read at least Chapter 2 of the svn-book (pp. 15 – 37) • You should master a handful of commands • svn help <command>!! • svn add, svn delete, svn checkout, svn update, svn commit • Helpful are also: svn info, svn help, … • Also, see Tech talk in labs on svn !

21. svn’s copy-modify-merge model • Each user connects to repository to retrieve working copy of source tree • Users work simultaneously and independently and modify private copies (i.e., their working copy) • Ultimately, working copies are merged • svn assists in merging • User may have to resolve conflicts manually!

22. 2 1 Copied from http://svnbook.red-bean.com/ Both edit files svn checkout 4 3 svn commit svn commit

23. 2 1 Copied from http://svnbook.red-bean.com/ svn update 4 3 svn update svn commit

24. Copied from http://svnbook.red-bean.com/ C – conflict (there may be a conflict) svn update svn update svn commit

25. Conflict example $ svn updateConflict discovered in 'sandwich.txt'. Select: (p) postpone, (df) diff-full, (e) edit, (h)elp for more options : p C sandwich.txt Updated to revision 2. $ ls sandwich.txt sandwich.txt.mine sandwich.txt.r1 sandwich.txt.r2 E.g., Sally changes sandwich.txt, but does not yet commit. Meanwhile, Harry commits changes to sandwich.txt. Sally has to update her working copy before she can commit, and she gets a conflict.

26. Conflict example $ svn updateConflict discovered in 'sandwich.txt'. Select: (p) postpone, (df) diff-full, (e) edit, (h)elp for more options : p C sandwich.txt Updated to revision 2. $ ls sandwich.txt sandwich.txt.mine sandwich.txt.r1 sandwich.txt.r2 Sally’s file prior to svn update

27. Conflict example $ svn update Conflict discovered in 'sandwich.txt'. Select: (p) postpone, (df) diff-full, (e) edit, (h)elp for more options : p C sandwich.txt Updated to revision 2. $ ls sandwich.txt sandwich.txt.mine sandwich.txt.r1 sandwich.txt.r2 Sally’s file prior to any of her changes. Her last svn checkout. (a.k.a., BASE revision)

28. Conflict example $ svn update Conflict discovered in 'sandwich.txt'. Select: (p) postpone, (df) diff-full, (e) edit, (h)elp for more options : p C sandwich.txt Updated to revision 2. $ ls sandwich.txt sandwich.txt.mine sandwich.txt.r1 sandwich.txt.r2 File just received from repository due to svn update. (a.k.a., HEAD revision)

29. Top piece of bread Mayonnaise Lettuce Tomato Provolone <<<<<<< .mine Salami Mortadella Prosciutto ======= Sauerkraut Grilled Chicken >>>>>>> .r2 Creole Mustard Bottom piece of bread $ cat sandwich.txt Changes by user to conflicting area in file Changes received from repository within conflicting area in file. Conflict markers

30. Conflict example $ svn resolve --accept working sandwich.txt Resolved conflicted state of 'sandwich.txt' $ svn commit -m “<message text>." Others: $ svn resolve –accept base … $ svn resolve –accept mine-full … $ svn resolve –accept theirs-full …

31. Conflict example Say, we had disregard the conflict created by svn update and go ahead and commit $ svn commit -m "Add a few more things" svn: Commit failed (details follow): svn: Aborting commit: '/home/sally/svn-work/sandwich.txt' remains in conflict

32. Questions

33. Effective use of svn • Commit often, but don’t break the build • Comment your commits • Retrieve an older version • See who modified what • …

34. Pass/Fail quiz to bank points • Optional quizto bank a few points that can be traded in for failing tests later on (redemption would happen automatically) • Close to the code submission deadline for Milestone 1, we’ll quiz you in the lab on • the effective use of svn and • the proper understanding of the submit script • Detailed modus operandi see our web site

36. Recommended Development Cycle Update sources from repository Develop the feature, fix the bug, … Add new test cases to test new functionality Test system with existing and new test cases Fix any problems and regressions Test again until all tests pass Commit sources including the new test cases to the repository ECE 297 44

37. Questions ECE 297 45

38. The storage server vis-à-vis a fully fledged DBMS • In addition to the functionality provided by our storage server, a database management system (DBMS) offers among others: • A declarative query language, an expressive data definition language (e.g., SQL) • Transaction support • Recovery from failure • Security and access control • … • Examples of DBMSs are IBM DB2, Microsoft SQL Server, Oracle … ECE 297 46 46

39. Why then a storage server at all? • DBMSs are developed to be most flexible and general • Broadest possible applicability • Flexibility and generality comes at a price • $$$$, maintenance, administration, performance • Many applications do not require bells & whistles of a DBMS • Each DBMS contains a storage server • Storage server as basic building block • Management of configuration data … ECE 297 47

40. ECE 297 48

41. errno • Functions return values to indicate failure • typically -1, null pointer, or constants, e.g.,EOF • But, what exactly is the error? • Need to consult error code stored in the variable errno declared in errno.h • You can’t use errno to determine failure • If a call failed, then examine errno

42. ECE 297 50

43. Design considerations • How should the data managed by the storage server be stored on disk? • What is the communication protocol between the client library and storage server?

44. Design considerations • How is the connection state between client and server represented? • Is the connection reused for multiple get and set requests from the client? • What are the implications of your design when multiple clients interact with the server at the same time? • How do the server and client library handle failures? • For example, what happens if there's an error in the configuration file? • What if the server is not running? • What if an invalid table name is given?

45. ECE 297 53

46. High-level SVN Architecture ECE 297 54 Copied from http://svnbook.red-bean.com