cs4411 – Operating Systems Practicum

1. cs4411 – Operating Systems Practicum Supplementary lecture 3 October 14, 2011 Zhiyuan Teo

2. Today’s lecture • Administrative Information • Machine endianness • Network header generation • General implementation hints • Discussion

3. Administrative Information • Project 3 updated on Sunday, 9 October. - Please re-download if you used the initial version we released.- Project 3 slides and the instructions have been amended. • Groups may be reshuffled. • Test cases for project 2 are in the project 3 bundle. - Test your implementation to make sure preemption and alarms work.- We will not be grading you on preemption and alarms, but it will be used later on.

4. Machine endianness • Memory is organized as an array of bytes. • Multibyte data items such as integers and shorts take up more than 1 byte of storage. - 4 bytes for an integer.- 2 bytes for a short. • Machine endianness refers to the physical ordering of bytes within the memory for the multibyte data item. - Actual ordering of bytes in memory does not change the value stored.- Ordering of bytes does not change final result of arithmetic shift operations.

5. Machine endianness Big-endian (SPARC, DLX, etc): Big-endian (SPARC, DLX, etc): Big-endian (SPARC, DLX, etc): Big-endian (SPARC, DLX, etc): Big-endian (SPARC, DLX, etc): Big-endian (SPARC, DLX, etc): 32 bitvalue: 0x12345678 32 bitvalue: 0x12345678 32 bitvalue: 0x12345678 32 bitvalue: 0x12345678 32 bitvalue: 0x12345678 32 bitvalue: 0x12345678 16 bitvalue: 0xdead 16 bitvalue: 0xdead 16 bitvalue: 0xdead 16 bitvalue: 0xdead 16 bitvalue: 0xdead 12 34 56 78 de ad Little-endian (Intel, VAX, etc): 32 bitvalue: 0x12345678 32 bitvalue: 0x12345678 32 bitvalue: 0x12345678 16 bitvalue: 0xdead 16 bitvalue: 0xdead 16 bitvalue: 0xdead 16 bitvalue: 0xdead 78 56 34 12 ad de • Humans write in big-endian form; it feels more natural to read.

6. Which endianness is better? • Short answer: nobody knows. • Advantages of big-endian representation: - Faster when determining an approximation to the stored value.- Sometimes faster for checking if one value is bigger than another.- Sign (+/-) can be checked very quickly since it is contained in the 1st byte. • Advantages of little-endian representation: - Simpler processor hardware implementation: first byte is 2560, second byte is 2561, third byte is 2562 and so on.- Addition of two little-endian values is faster.

7. Endianness problems • A copies his integer value into a network packet, byte for byte. - first byte in memory = first byte into packet. • B copies the network packet into memory byte for byte. • What happens if A and B are of different endianness?

8. Endianness: Solution • Big-endian format is the standard for networking. - Frequently referred to as network byte order.- Host byte order is the native representation for individual machines; it could be big or little endian. • Berkeley sockets API has functions for converting between host and network byte order. - htonl(), htons() : host to network long/short.- ntohl(), nthos() : network to host long/short. • But we will not be using Berkeley sockets functions! - Use our own pack and unpack functions.

9. Network header generation • Having a common header format will be fun later on. • Use the pack and unpack functions that we provide. • Do not use a simple byte copy of the network address, this is incorrect.- Port numbers may be stored as ints in your program but must be converted to unsigned shorts when packing.- Reminder: do not pack port numbers as ints! • The protocol field will be useful in the next project. • Set the protocol char field to PROTOCOL_MINIDATAGRAM for this project.

10. Network header generation – the bad way • Given the header specs: • 1 byte protocol type- 8 bytes source address- 2 bytes source port- 8 bytes destination address- 2 bytes source port • Allocate a (1+8+2+8+2) byte buffer and manually fill in the contents. char* header = (char*) malloc(sizeof(protocol_type) + sizeof(source_address) + sizeof(source_port) + ...);memcpy(header, &protocol_type, sizeof(protocol_type));memcpy(header + sizeof(protocol_type), &source_address, sizeof(source_address));

11. Why is this a bad idea? • Tedious to code. • Lots of memcpy() and sizeof() operations.- Code looks plain ugly. • What if the header specs change later? • Must manually change all offsets.

12. Iteration #2 • Idea: use a struct to store all fields so they are arranged correctly in memory. • compiler arranges a contiguous block of memory for the struct.- memory layout of the struct follows the order declared by the struct. struct header{ char protocol_type; network_address_t source_address; unsigned short source_port; network_address_t destination_address; unsigned short destination_port;};struct header hdr;network_send_pkt((char*) &hdr, sizeof(hdr), ...);

13. Iteration #2: close but no cigar… • Padding! • computers usually load in units of words.- if a multibyte variable spans 2 words, then 2 loads are needed.- align the variable to some word boundary so it requires exactly 1 load. “Variables are aligned to certain power-of-2 numbered boundaries for faster access.” - (Getting ready for) Project 1 slides • Padding is unpredictable and is a waste of resources to transmit.

14. Third time’s the charm • Idea: use a struct that cannot possibly have padding. • chars require exactly 1 load no matter where they are located.- Therefore consecutive char fields in a struct are not padded.- Works regardless of compiler options for padding. struct header{ char protocol_type; char source_address[8]; char source_port[2]; char destination_address[8]; char destination_port[2];};struct header hdr;network_send_pkt((char*) &hdr, sizeof(hdr), ...); • Use packing functions to convert and populate the char arrays.

15. Implementation Hints • Use an array for your ports. - O(1) time when using unbound ports (since user specifies the port he wants).- O(1) time when creating bound ports before a wraparound; O(n) time afterwards us acceptable (since you need time to check each port). • Use semaphore_P and semaphore_V for blocking and unblocking threads. • Remember how we did this in project 2; consider places where you need to disable interrupts. • Reuse your queue implementation. • This is useful for storing data in FIFO order.

16. More Implementation Hints • Perform sanity checks. - Is the protocol type correct?- Are you sure the received packet is meant for you?- Is the packet malformed (header too short, invalid port numbers, etc)? • Consider semantics for unused ports. • Data sent to unused ports should not actually be transmitted.- Data received on an unused port should not be queued. • Consider reuse semantics for unbound ports. • When an unbound port is destroyed and later re-created, any prior queued data should no longer be there.- Don’t forget to reset the counting semaphore too.

17. Project 3 FAQ • Dynamic memory responsibilities. • network interrupt handler passes you an network_interrupt_arg_t, which you have to eventually free.- The user-supplied buffer for both minimsg_send and minimsg_receive should not be freed by you. • Occasional lost packets across machines. • This is normal.- Try re-executing your program again. • Unable to communicate between two machines. • Make sure both machines can ping each other.- Try running on two machines in the CSUG lab.- Redrover is known to have problems with machine visibility.

18. Project 3 FAQ • Mutexes and semaphores for unbound ports. • You will need a counting semaphore.- But technically you won’t need a mutex. (Why?) struct miniport { char port_type; int port_number; union { struct { queue_t incoming_data; semaphore_t datagrams_ready; } unbound; struct { network_address_t remote_address; int remote_unbound_port; } bound;};

19. Q & A