390 likes | 482 Views
A Smart Port Card Tutorial - The Exercises John DeHart Washington University jdd@arl.wustl.edu http://www.arl.wustl.edu/~jdd. Question?. How many of you have never built a NetBSD kernel?. Outline. Exercises Download a kernel Build a kernel for the SPC Modify a file system
E N D
A Smart Port Card Tutorial - The Exercises John DeHart Washington University jdd@arl.wustl.edu http://www.arl.wustl.edu/~jdd
Question? How many of you have never built a NetBSD kernel?
Outline • Exercises • Download a kernel • Build a kernel for the SPC • Modify a file system • Build a file system from scratch • Configure an SPC for IP connectivity • Install the SPC hardware in a Switch • Use completely separate set of switches from other exercises • Do this as you have free time throughout SATURDAY
If time permits... • Open Exercises: • things we have not done yet • things we can discuss and/or start working on. • Building a kernel - you add the SPC mods • Download program using AAL5 frames • Investigate Linux kernel for SPC
WUGS Switch Port 1 Port 0 Port 7 Port 2 Port 3 Port 6 SPC/ Port 4 Port 5 2 Serial Cables Setup • We need to divide into 7 Groups: • Each group gets its own: • login (spc1, spc2, spc3, … spc7 passwords: Spc1Tut, Spc2Tut, …) • switch with SPC already installed on Port 4 • PC with • APIC connected to Port 3 • /usr/SPC directory tree PC APIC Fiber Pair
Directory Tree for each Group: /usr/SPC/NetBSD NetBSD source directories /usr/SPC /bin WUGS and SPC utilities binaries /usr/SPC /boot_loader Source for SPC boot loader /usr/SPC /configuration GBNSC config files and Jammer scripts /usr/SPC /download Directory from which to download kernels /usr/SPC /root_wdir Directory for portion of kernel build done as root /usr/SPC /spc_utilities Source code for SPC utilities
Directory Tree for each Group: • WUGS Utilities: • GBNSC • Jammer • GBNSC config file • Jammer script for configuring switch • SPC Utilities: • resetSPC • download_APIC • download_ENI • mknl: make symbol-only netbsd • mkimg: make BSS image • download directory • pre-built bootloader • boot.out • bit file • Pre-built kernel • spcKernel/netbsd.bss.ORIG • SPC kernel source tree • non-SPC kernel source tree • Scripts for building kernel • SPC Filesystem
Exercise 1 - Download and boot a Kernel • Reset switch • Run GBNSC and Jammer to configure switch: • as root: • cd /usr/SPC/configuration • sh config.apic.connections • as regular user: • cd /usr/SPC/configuration • ../bin/GBNSC -r -a config.switch.APIC & • ../bin/Jammer 0.1 wooster 3551 SPC_Control.js
Exercise 1 - continued • Configure the PC (/etc/remote) so you can use tip: spc1:dv=/dev/tty01:dc:br#9600:pa=none spc0:dv=/dev/tty00:dc:br#9600:pa=none • Start the tip processes in two separate windows as root: • tip spc0 • tip spc1
Exercise 1 - continued • Download bootloader and kernel • cd /usr/SPC/download • ../bin/download_APIC -a 0 41 0 42 • download will take 4-5 minutes… • make sure you see this fairly quickly: boot loader started, sleep(2) Sending entire kernel...sendfile(filename=netbsd.bss, load_addr = 0x100000, start = 0x0, sendlen = 0x40000000) • while you’re waiting look at the scripts and source code: • /usr/SPC/configuratoin/SPC_Control.js • Look at the WUGS connections to get to the SPC • /usr/SPC/spc_utilities/download_APIC/main.cc
Exercise 1 - continued • Boot to multi-user mode • When you see this: “Enter pathname of shell or RETURN for sh:” • hit <return> • Then at ‘#’ prompt, type ‘exit’ • Log in as root (no password) on the serial console • Use Terminal Type: vt100 • Notice which port is the console port. tty00 or tty01? • Check the size of the netbsd kernel in the file system: • ls -l /netbsd • du -s /netbsd • Check the size of the file system: • df -k / • While looking at your watch, try this: > sleep 20 • what is the effect of these SPCs running at 25 MHz? Why?
Exercise 1 - continued • Send a reset control cell • /usr/SPC/bin/resetSPC -a 0 41 0 42 • See that the serial console is now dead • Why does resetSPC hang? Look at code... • Kill it with a <ctrl-C> • Download bootloader and kernel again • See boot messages in tip window…
Exercise 2A - Building a kernel • Do these steps as regular user (i.e. not as root) • Make sure the kern and pci source code is configured: • cd /usr/SPC/NetBSD/usr/src/sys/kern • make init_sysent.c • cd ../dev/pci • make -f Makefile.pcidevs • Kernel config file provided: • /usr/SPC/NetBSD/usr/src/sys/arch/i386/conf/SPC_24MB • cd /usr/SPC/NetBSD/usr/src/sys/arch/i386/conf • config SPC_24MB • cd ../compile/SPC_24MB • make depend
Exercise 2A - Building a kernel (con’t) • Three shell scripts are provided: mk.1, mk.2, mk.3 • Its easier for this part if you have two windows open • root window in directory /usr/SPC/root_wdir • regular user window in directory: /usr/SPC/NetBSD/usr/src/sys/arch/i386/compile/SPC_24MB • Stage 1: mk.1: • run as regular user • performs make to build kernel • may take several minutes: • take a look at mk.1, mk.2, mk.3 while you are waiting • where does the kernel file end up when it is ready for download? • cd /usr/SPC/NetBSD/usr/src/sys/arch/i386/compile/SPC_24MB/ • ../mk.1
Exercise 2A - Building a kernel (con’t) • Stage 2: mk.2 • run as root • inserts symbol-only copy of /netbsd into filesystem • cd /usr/SPC/root_wdir • ./mk.2 • Stage 3: mk.3 • run as regular user • copies filesystem into netbsd kernel file • takes a couple of minutes... • cd /usr/SPC/NetBSD/usr/src/sys/arch/i386/compile/SPC_24MB/ • ../mk.3 • Download bootloader and kernel again • don’t forget to reset SPC • and if you use resetSPC, don’t forget to kill it with <ctrl-C>
Exercise 2B - Modifying a kernel • Modify the kernel • Change kernel so it boots to multi-user mode • look in sys/arch/i386/i386/md_root.c (search for SINGLE) • Change timing of clock in system FPGA • look at: sys/arch/i386/isa/timerreg.h (#define TIMER_FREQ) • add a kernel config file option to change TIMER_FREQ • If needed, configure switch as in Exercise 1 • Rebuild your kernel • You will need to do a ‘> make clean ‘ in the compile/SPC_24MB directory • If needed, configure switch as in Exercise 1 • If needed, send reset control cell to SPC • Download bootloader and kernel • Log in via serial port • Check the file system and verify that it is what you built
Exercise 2C - Modify a file system • Modify the SPC Filesystem (as root, be careful...) • cd /usr/SPC/root_wdir • vnconfig -t spc24MB -v -c /dev/vnd0d SPC24MB.fs • mount /dev/vnd0d /mnt • cd /mnt/etc • Put something in it that will tell you that it is yours • e.g. a file in /mnt/etc/<your_name> • Change console to other TTY port • modify /mnt/etc/ttys (in SPC filesystem) • also change default terminal type for non-console tty to vt100 • also need to change the kernel config file and re-run config • umount /mnt • vnconfig -u /dev/vnd0d
Exercise 2C - Modify a file system (con’t) • Rebuild your kernel • If needed, configure switch as in Exercise 1 • If needed, send reset control cell to SPC • Download bootloader and kernel • Log in via serial port • Check the file system and verify that it is what you built
Exercise 2D - Build a file system • Build your own Filesystem from scratch • Edit /etc/disktab to create a new sized disk: 18MB. spc18MB|spc18MBkernSPC Kernel FileSystem Disk:\ :ty=simulated:se#512:nt#16:rm#300:ns#27:nc#80:\ :pa#34560:oa#0:ba#4096:fa#512:ta=4.2BSD:\ :pb#34560:ob#0:\ :pc#34560:oc#0: • cd /usr/SPC/root_wdir • Use dd(1) to create a file • dd if=/dev/zero of=<filename> count=<numBlocks> • Use vnconfig(1) to configure the file as a vnode disk • vnconfig -t <disktab entry> -v -c /dev/vnd0d <filename> • Use disklabel (1) to give it a label • disklabel -rw /dev/vnd0d <disktab entry> <label> • disklabel -r /dev/vnd0d • Use newfs(1) to put a filesystem on the disk • newfs -m 0 -o space -i 5120 -c 54 -T <disktab entry> /dev/rvnd0a
Exercise 2D - Build a file system (con’t) • Mount it. • mount /dev/vnd0a /mnt • Populate the disk: • Use mknod(1) to make devices in /dev directory • see next page for details! • Mount the original SPC filesystem we used before on /mnt1 • vnconfig -t spc24MB -v -c /dev/vnd1d <original filesystem filename> • mount /dev/vnd1d /mnt1 • if /mnt1 does not exists you may have to create it with ‘mkdir’ • Look through /mnt1/* and populate your filesystem from there • Useful command, this does a tar(1) from directory to directory: • cd <dir1>; tar cf - . | (cd <dir2>; tar xfBp -) • Unmount • umount /mnt • vnconfig -u /dev/vnd0d • umount /mnt1 • vnconfig -u /dev/vnd1d
Exercise 2D - (con’t) : mknod(1) Devices • mknod rmd0a c 24 0 • mknod rmd0b c 24 1 • mknod rmd0c c 24 2 • mknod stderr c 22 2 • mknod stdin c 22 0 • mknod stdout c 22 1 • mknod tty c 1 0 • mknod tty00 c 8 0 • mknod tty01 c 8 1 • mknod ttyp0 c 5 0 • mknod ttyp1 c 5 1 • mknod ttyp2 c 5 2 • mknod ttyp3 c 5 3 • mknod zero c 2 12 • mknod md0a b 17 0 • mknod md0b b 17 1 • mknod md0c b 17 2 • chmod 600 console • chmod 640 drum • chmod 640 io • chmod 600 klog • chmod 640 kmem • chmod 640 mem • chmod 666 null • chmod 640 md0a • chmod 640 md0b • chmod 640 md0c • chmod 640 rmd0a • chmod 640 rmd0b • chmod 640 rmd0c • chmod 666 stderr • chmod 666 stdin • chmod 666 stdout • chmod 666 tty • chmod 644 tty00 • chmod 644 tty01 • chmod 666 zero • cd /mnt • mkdir dev • cd dev • mknod bpf0 c 23 0 • mknod bpf1 c 23 1 • mknod bpf2 c 23 2 • mknod bpf3 c 23 3 • mknod console c 0 0 • mknod drum c 4 0 • mknod io c 2 14 • mknod klog c 7 0 • mknod kmem c 2 1 • mknod lkm c 28 0 • mknod mem c 2 0 • mknod null c 2 2 • mknod ptyp0 c 6 0 • mknod ptyp1 c 6 1 • mknod ptyp2 c 6 2 • mknod ptyp3 c 6 3
Exercise 2D - (con’t) : mknod(1) Devices • mknod rmd0a c 24 0 • mknod rmd0b c 24 1 • mknod rmd0c c 24 2 • mknod stderr c 22 2 • mknod stdin c 22 0 • mknod stdout c 22 1 • mknod tty c 1 0 • mknod tty00 c 8 0 • mknod tty01 c 8 1 • mknod ttyp0 c 5 0 • mknod ttyp1 c 5 1 • mknod ttyp2 c 5 2 • mknod ttyp3 c 5 3 • mknod zero c 2 12 • mknod md0a b 17 0 • mknod md0b b 17 1 • mknod md0c b 17 2 • chmod 600 console • chmod 640 drum • chmod 640 io • chmod 600 klog • chmod 640 kmem • chmod 640 mem • chmod 666 null • chmod 640 md0a • chmod 640 md0b • chmod 640 md0c • chmod 640 rmd0a • chmod 640 rmd0b • chmod 640 rmd0c • chmod 666 stderr • chmod 666 stdin • chmod 666 stdout • chmod 666 tty • chmod 644 tty00 • chmod 644 tty01 • chmod 666 zero • cd /mnt • mkdir dev • cd dev • mknod bpf0 c 23 0 • mknod bpf1 c 23 1 • mknod bpf2 c 23 2 • mknod bpf3 c 23 3 • mknod console c 0 0 • mknod drum c 4 0 • mknod io c 2 14 • mknod klog c 7 0 • mknod kmem c 2 1 • mknod lkm c 28 0 • mknod mem c 2 0 • mknod null c 2 2 • mknod ptyp0 c 6 0 • mknod ptyp1 c 6 1 • mknod ptyp2 c 6 2 • mknod ptyp3 c 6 3 OK, OK. There is a script for this. /usr/SPC/bin/mknods
Exercise 2D - Build a file system (con’t) • Rebuild your kernel • Change the kernel config file • Look through the make scripts to change filesystem • Re-config and re-make the kernel • If needed, configure switch as in Exercise 1 • If needed, send reset control cell to SPC • Download bootloader and kernel • Log in via serial port • Check the file system and verify that it is what you built • Keep it running while you go on to next exercise ...
Exercise 3A - IP Between SPC and PC • Save any changes you make for the SPC in your filesystem • Configure an IP address for the SPC’s APIC • ifconfig • Configure an IP address for the PC’s APIC • Configure an APIC connection on each • atm_ifconfig • Add routes on PC and SPC: • route add ... • Configure the necessary WUGS connection(s)
Exercise 3A - IP Address Assignments > cat filename | telnet postscript.cs 9100
Exercise 3A - IP Between SPC and PC • Try to ping • Iterate until you can ping from PC to SPC • Tools for debugging the IP connection: • netstat • APIC readCounts utility • /usr/local/bin/readCounts on both PC and SPC • WUGS cell counters (Jammer) • Turn on telnetd • edit /etc/inetd.conf on SPC • kill -HUP <inetd pid> • iterate until you can telnet from PC to SPC • Be patient • on some machines telnet seems to take a while to connect. • Not sure why yet...
Exercise 3A - IP Between SPC and PC • Make sure you have saved all your changes • Reset switch • Rebuild kernel with your newly modified filesystem and re-download. Does it still work...
Exercise 3B - IP Between Neighbor SPCs • Use SPCs as routers: 192.168.10.10 192.168.20.10 PC PC 192.168.20 Domain 192.168.10 Domain . . . 192.168.10.1 SPC SPC 192.168.20.1
Exercise 3B - IP Between Neighbor SPCs • Save any changes you make for the SPC in your filesystem • Configure an IP address alias for the SPC’s APIC to put it in the neighboring SPC’s domain • ifconfig …alias • Remove the network route that the above caused: • route delete -net 192.168.20 • Configure an APIC connection on each SPC • atm_ifconfig • Add routes on PC and SPC: • route add ... • Configure the necessary WUGS connection(s)
Exercise 3B - IP Between Neighbor SPCs • Try to ping • Iterate until you can ping from PC to neighbor PC • Tools for debugging the IP connection: • netstat • APIC readCounts utility (/usr/local/bin/readCounts on SPC) • WUGS cell counters • Make sure you have saved all your changes • Reset switch • Rebuild kernel with your newly modified filesystem and re-download. Does it still work...
Exercise 4 - Installing an SPC in a Switch • Power Off • Open Switch • Removing Glink Line Card from Port 4 or 5 • Install SPC • Install Serial Cables, feed in through front panel • Re-Install Glink Line Card on top of SPC • Close Switch • Connect serial cables to PC • Power On • Download and Boot Kernel Tom will provide a Slide or two
Open Exercises: SPC Mods for NetBSD 1.4.1 • Do this first if you are interested in SPC/Linux • Compare kernel config files: • /usr/SPC/NetBSD/usr/src/sys/arch/i386/conf/SPC_24MB • /usr/SPC/NetBSD/usr/src/sys/arch/i386/conf/APIC_1_4_1 • Note the major differences • Find the network devices in both. Anything interesting? • Compare source files: • /usr/SPC/NetBSD/usr.ORIG • /usr/SPC/NetBSD/usr • Directories where there are differences: sys/arch/i386/i386 sys/... sys/arch/i386/isa sys/dev/ic sys/dev/pci
Open Exercises: AAL5 Download Program • currently we download via APIC control cells • 16 bytes of data per cell • send one cell, wait for response cell before sending next one • slow: 4-5 minutes for a 24MB kernel • Preferred method would be AAL5 • Use APIC Control cells to configure AAL5 descriptors • Send Kernel in AAL5 frames to SPC • Read it back to verify?
Open Exercises: Linux Kernel for the SPC • ONLY if you are already familiar with NetBSD/SPC mods • ONLY if you are already VERY familiar with Linux • Kernel Changes: • Memory Disk • Serial port console • APIC Configuration by APIC driver • System FPGA impact: • Timer configuration • RTC requirements • Serial port configuration • Static IRQ assignment • Download and Boot • Boot loader • Other things that Linux expects from the BIOS or bootloader?