MSR Tutorial: MSR_Config and the AAL5_Download Utilitiy

MSR Tutorial:MSR_Configand theAAL5_Download Utilitiy John DeHartWashington University, Applied Research Labjdd@arl.wustl.eduhttp://www.arl.wustl.edu/arl/

MSR_Config The MSR Configuration module and the beginnings of the MSR Control Processor Software

MSR Control Processor (CP) • PC connected to MSR Port 0 • CP Software runs on Linux or NetBSD • Will use threads in the future • Eventually it will probably be only Linux • newGBNSC and Jammer • newGBNSC: WUGS Switch Controller • Replaces old GBNSC • Jammer: WUGS scripting language interpreter • Not part of the CP • Can still be used for monitoring • CP takes care of its own WUGS connections • it formulates and sends WUGS control cells directly

Current CP: MSR_Config • Phases of Operation • Switch Reset (resets WUGS, SPCs and FPXs) • Switch Configuration • Build connections needed by MSR • Discovery • Determine what is on each port (SPC, FPX, Glink, …) • SPC • Configure connections for Download • Download • system(3) and shell script to invoke SPC utility: AAL5_Download • AAL5_Download to be discussed separately • MSR Kernel Initialization • uses sendcmd API library • Run • Currently: exit() • Eventually: thread invocation

Source and Executables • source location: wu_arl/msr/cp/configuration • executable location: • wu_arl/msr/bin/$OSTYPE/MSR_Config • Actually, probably doesn’t work from here right now because it is dependent on a few scripts which are kept in the source directory. • For now, run it from the source directory: • cd wu_arl/msr/cp/configuration • Linux/MSR_Config –f config.MSR • This is how we will do it in the exercises • Scripts used (located in wu_arl/msr/cp/configuration) • msrDownload.sh (invokes AAL5_download…) • INITIALIZATION: uses libcmd APIC: • wu_arl/msr/utils/command/cmdAPI.[ch] • When you get idle time during exercises, look through the various scripts and source code.

MSR_Config Usage • Typical usage: • cd wu_arl/msr/cp/configuration • ./$OSTYPE/MSR_Config –f <configuration file> • other options –drr: turn on DRR –dq: turn on Distributed Queueing -fipl: Turn on FIPL -d: turn on debug info -p: turn on WUGS read port test

AAL5_Download • Replaces old download_APIC utility • Can “shotgun” download to <= 8 SPCs at once • Puts exactly same kernel on all 8 • This works fine for MSR where we want same kernel • May not be good for all uses • “Shotgun” number controlled from command line • Can download kernel and file system separately • no need to load file system into kernel before download • still need to load symbol version of kernel into file system before downloading • Takes about 1 minute for the whole download and boot process.

AAL5_Download • Implemented and runs in Stages: • Stage1 uses APIC Control cells to: • reset the SPC(s) • download Stage2 to run on the SPC(s) • Stage2 (boot.out) runs on the SPC • runs SPC boot code to configure CPU and Northbridge • configures APIC on SPC to use AAL5 descriptors • sends pings back to CP, waiting for Stage3 • receives AAL5 frames from Stage3 and sends ACK/NACK • Stage3 runs on PC • syncs with Stage2 • downloads the kernel or FS via 36KB AAL5 frames • retransmits if any errors

Simple View Stage1 PC SPC Stage1 VCI=0x321 APIC VCI=21 VCI=21 VCI=25 Stage3 VCI=23 Stage2 VCI=24

Control cells Reset cell AAL5 Frames Sequential View SPC Memory PC SPC 0x0 Boot and Stage2 Stage1 APIC RESET 0x0FFE00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 Stage3 Stage2 Kernel and Filesystem 0x0100000 0xFFFFFFF0 Boot ROM

Source and Executables • source location: wu_arl/utilities/SPC/AAL5Download • binary location: • wu_arl/utilities/bin/$OSTYPE/Stage1 • wu_arl/utilities/bin/$OSTYPE/Stage3 • wu_arl/utilities/bin/$OSTYPE/AAL5_download.sh • wu_arl/utilities/bin/$OSTYPE/boot.out • wu_arl/utilities/bin/$OSTYPE/FindNM

Usage • Individual command line commands • ./Stage1 –shotgun 8 –svci 21 –rvci 21 –dvci 23 –uvci 24 –cvci 25 • ./Stage3 –shotgun 8 –native netbsd.MSR –both 24 • ./Stage3 –shotgun 8 –bss SPC24MB.fs –address 0x0196434 –both 24 -go • AAL5download.sh script • ./AAL5download 8 netbsd.MSR SCP24MB.fs 21 21 23 24 25 • Use from MSR Configuration Module • system(msrDownload.sh);

Special Usage Note on Linux • AAL5download.sh script: • needs to be able to determine the start location for the File System • NetBSD uses nm(1) • Linux nm(1) won’t work on a NetBSD binary • We have written a version of nm specifically for AAL5_download.sh: FindNM • FindNM is used automatically when AAL5_download.sh is run on Linux

AAL5_download.sh Usage • > ./AAL5_download.sh • Usage: ./AAL5_download.sh <netbsd_file> <filesystem_file> <start_vci> [<second_vci>] [<third_vci>][<fourth_vci>] [<fifth_vci>]

Stage1 Usage • > ./Stage1 • ./Stage1: [vpi/vci data] [load info] • ./Stage1: [-svpi #] (VPI for local sending of AAL5 control cells (0)) • ./Stage1: [-svci #] (VCI for local sending of AAL5 control cells) • ./Stage1: [-rvpi #] (VPI for local reading of AAL5 control cells (0)) • ./Stage1: [-rvci #] (VCI for local reading of AAL5 control cells) • ./Stage1: [-cvpi #] (VPI for remote CC writes of AAL5 control cells (0)) • ./Stage1: [-cvci #] (VCI for remote CC writes of AAL5 control cells) • ./Stage1: [-dvpi #] (VPI for local write of AAL5 data frames (0)) • ./Stage1: [-dvci #] (VCI for local write of AAL5 data frames) • ./Stage1: [-uvpi #] (VPI for local read of AAL5 data frames (0)) • ./Stage1: [-uvci #] (VCI for local read of AAL5 data frames) • ./Stage1: [-resp #] (Response port of AAL5 control cells) • ./Stage1: [-start #] (Initial VCI sequence (41, 42, 43, 44)) • ./Stage1: [-watch 0x#] (Monitor location for changes) • ./Stage1: [-apic #] (Linux: Apic Id (0 <= X <= 3)) • ./Stage1: [-abit #] (Alternating bit setting) • ./Stage1: [-file <path>] (Name of boot loader (boot.out)) • ./Stage1: [-swap] (use htonl() on output) • ./Stage1: [-shotgun #] (multicast loading 8X) • ./Stage1: [-msr #] (VCI for MSR ONLY AAL5 data frames) • ./Stage1: [-address #] (Physical data write address (0)) • ./Stage1: [-noreset] (do not write reset register)

Stage3 Usage • > ./Stage3 -help • [-switches] (-help) args • -bss <file> Load .bss format file • -native <file> Load native uncompressed file • -strip Remove Symbols from kernel • -debug print lots of stuff • -verbose Dump Elf headers • -elf Elf format execution • -rvpi <VPI> Load VPI number • -rvci <VCI> Load VCI number • -svpi <VPI> Load VCI number • -svci <VCI> Load VCI number • -both <VCI> Set both VCI numbers • -address <PhysAddr> Load file at <addr> • -start <PhysAddr> Start file at <addr> • -go Start kernel • -shotgun <count> Multicast loading 8X

MSR Tutorial: MSR_Config and the AAL5_Download Utilitiy