GLAST Large Area Telescope Instrument Flight Software Flight Unit Peer Review 16 September 2004

1. Gamma-ray Large Area Space Telescope GLAST Large Area Telescope Instrument Flight Software Flight Unit Peer Review 16 September 2004 LAT Communications Board Driver Ed Bacho Stanford Linear Accelerator Center ebacho@slac.stanford.edu 650-926-5324

2. LCB: Requirements • Flight Software General Requirements • EPU-SIU Interface (5.2.1.1) • The EPU FSW shall communicate with the SIU via a custom CPU-to-CPU serial message protocol described in the TEM, AEM, CRU, EBM, GEM, and PDU Programming ICDs. • Science Data Interface to the Event Builder (5.2.1.3) • Science Data Rate (5.2.1.3.1) • The FSW shall accommodate LAT science data interface data transfer rates up to 66 Mbps to SC storage via the Event Builder (TBR. Pending CCR to change to 40 Mbps). • Science Data Volume (5.2.1.3.2) • FSW shall send to the SC, via the Event Builder, a maximum of 104 Gbits of high-rate science data in any given 24 hour period. • Science Data Packet Format (5.2.1.3.3) • FSW shall format data into CCSDS telemetry packets tagged with application IDs (APIDs) in the same manner as the CTDB telemetry as specified in GLAST 1553 Bus Protocol Interface Control Document. • Science Data Packet Synchronization Word (5.2.1.3.4) • Each CCSDS source packet shall be preceded with a synchronization word to aid in data recovery. Flight Unit Peer Review - LAT Communications Board Driver

3. FSW Layer Model Applications Layer Charge Injection Calibration (LCI) Spacecraft Messages(LSM) Instrument Physics (LPA) Diagnostics Framework(LDF) Spacecraft Attitude/Time Thermal Control(LTC) Instrument Manager(LIM) Watchdog(LSW) Housekeeping(LHK) File/Object(LFS) Abstract Communication Layer Process Basic Services/VxWorks (PBS) Inter-task Communications System (ITC) Service Layer Command and Telemetry/1553 Service (CTS) LCB Service(LCS) Driver Layer 1553 Driver (CTDB) LCB Driver (LCBD) Flight Unit Peer Review - LAT Communications Board Driver

4. LCB Driver Overview • The LAT Communications Board (LCB) is a cPCI module residing in every cPCI crate within the LAT • 2 SIU and 3 EPU crates • The LCB provides communication within the LAT using a custom LATp communications protocol. • The LCB hardware design calls for two FPGAs that communicate via intermediate FIFOs. • The PCI FPGA controls the PCI bus and provides the interface to PCI registers. • The LATp FPGA controls LATp specific I/O features, e.g. sending commands, receiving response and receiving event data. • The LATp FPGA also contains a small number of 32-bit registers. Flight Unit Peer Review - LAT Communications Board Driver

5. LAT Internal Communications Fabrics • An LCB can communicate directly with other LCBs (in other crates) on one of two communications fabrics: • An Event Data fabric (described in detail earlier by JJ Russell) • A Command/Response fabric. • On the Event Data fabric: • Data arrives unsolicited, i.e. whenever it is ready. A handler is notified and processes the unsolicited data in a timely manner. • On the Command/Response fabric: • A node can issue a command to another LATp node and receive the solicited response. For example, this is used to read a register on a remote electronics module. Flight Unit Peer Review - LAT Communications Board Driver

6. SIB LCB SIU1 Communications on theCommand/Response Fabric • The command/response fabric is the “control” fabric • Provides write and read access to registers throughout the instrument • Messages routed by the Command/Response Unit (CRU) Legend ACD: Anti-coincidence detector GEM: Global Trigger Electronics Module TEM: Tower Electronics Module SIU: Spacecraft Interface Unit EPU: Event Processing Unit LCB: LAT Comm. Board SIB: Spacecraft Interface Board ACD0,1 GEM0,1 PDU0,1 TEM0 TEM15 LCB EPU0 CRU SIB LCB SIU0 LCB EPU1 LCB EPU2 Flight Unit Peer Review - LAT Communications Board Driver

7. Communications on Event Data Fabric(Receive Side) • As its name implies, this is the “data” fabric • Data is written to memory inside the EPUs • Data is routed via the Event Builder Module (EBM) ACD0,1 GEM0,1 PDU0,1 TEM0 TEM15 LCB EPU0 8 EBM 8 LCB EPU1 8 LCB EPU2 Flight Unit Peer Review - LAT Communications Board Driver

8. Communications on the Event Data Fabric(Send Side) • The major CPU to CPU communications path • The second SIU, additional EPUs, and associated paths omitted for clarity 8 SIB EBM LCB 8 SIU LCB EPU0 8 (5 MHz) Solid State Recorder 40 Mbps Flight Unit Peer Review - LAT Communications Board Driver

9. LCB Driver: System Architecture LCB PCI BUS CPU SSR LCB Service Task Result FIFO (4KB) CommandResponseHandler Result In Synch Data LCBInterruptHandler Dispatch Interrupt Dispatch Input Services Event Handler CPU-To-CPU Protocol 2 Protocol 3 Input Logic Event Descriptor FIFO (4KB) Asynch Descriptor Descriptor In Decoupling FIFO (4KB) Ring Buffer (.5MB) Data In Asynch Data Data Out Output Services Data Free Asynch Data Free Data Flight Unit Peer Review - LAT Communications Board Driver

10. Command/Response Items and Lists • Command and responses can be issued/received individually. • However, to speed up configuration and other commanding operations, multiple Command/Response items can be bundled into DMA transfer operations between a flight CPU (e.g., an SIU) and the LAT Communications Board (LCB): • Multiple command items in a “command list” are DMA’d to the LCB. • There the command items are sent one by one to LAT subsystems. • The command list can be up to 4 KB long, enough space to hold 150 register write commands. • Multiple responses are assembled into a “results list.” • All result items have a timestamp and a status word. • For command items that expect a response, the result item also contains the data read back from the instrument subsystem. • The results list can also be up to 4 KB long. • A one-to-one relationship between items on the command and results lists is maintained. • The results list is DMA’d back to the flight CPU, and LCB raises an interrupt to signal the transfer is complete. Flight Unit Peer Review - LAT Communications Board Driver

11. The Special Case of Event Data:Writing to the Solid State Recorder • Again, Event Data delivery via the LCB Receive path was covered earlier in today’s session. • The LCB can put data on the event fabric for 3 possible uses: • LCB to LCB communications • Boot message from SIU to EPU • SSR data through Event Builder Module • On the SSR pathway: • EPUs send data to EBM serially at 20 Mbps • EBM outputs on SSR: 8 bit @ 5 MHz for a total of 40 Mbps Flight Unit Peer Review - LAT Communications Board Driver

12. The LCB Test Package • The LCB driver is critical software that affecting all aspects of CPU-to-CPU communications and CPU-to-LAT communications. • All Test Stand and Testbed development depends on having a reliable LCB driver. • A special test package, LCBT, has been created for hardware/software testing: • Independent testing of driver functions • Testing of individual functions • “Stress” testing • Special case testing: • e.g., fill the LCB queues and verify correct behavior • Introduce events of random size, set a range of data patterns, etc. Flight Unit Peer Review - LAT Communications Board Driver

13. Gamma-ray Large Area Space Telescope GLAST Large Area Telescope Backup

14. RAD750 SIB LCB Data Paths via the LCB SIU GASU CRU Command / Response Unit 4 x 32 bit registers RAD750 PDU’s TEM0 cPCI SIB TEM1 TEM2 . . EBM LCB TEM15 31 x 32 bit registers EPU 0 Event Builder Module SIU In RAD750 SIU GEM EPU 0 In EPU 0 cPCI SIB 22 x 32 bit registers 17 x 64 bit registers 19 x 96 bit registers 1 x 112 bit register EPU 1 In EPU 1 LCB Merge SSR EPU 1 AEM ACD Electronics Module 6 x 32 bit registers cPCI GLT Electronics Module TEM0 CombinatoricLogic TEM1 Event Data TEM2 Tower Event Fragments . Cmd/Response . Trigger Data TEM15 Trigger Data Trigger Accept Flight Unit Peer Review - LAT Communications Board Driver