Fast A/D sampler FINAL presentation

1. Presented By: Tal Goihman, Irit Kaufman Instructor: MonyOrbach Winter 2012 Fast A/D sampler FINAL presentation

2. Goals • Project Goal: • Design and implement an A/D system using Xilinx Virtex6 development board for sampling at highest possible rate. • Sample to virtex6 development board DDR memory. • Transfer the sampled data to PC memory through PCIe and save the data to disk.

3. ML605 development board FMC125 Fast A/D FMC Conn H/W Block Diagram DDR3 A/D IC Virtex6 FPGA PCI-E Connector PC

4. ML605 development board

5. FMC125 Fast A/D

6. A/D Sampler The FMC125 is a Quad-Channel ADC that provides four 8-bit ADC channels enabling simultaneous sampling of 1, 2, or 4 channels @ 5 , 2.5 , 1.25Gsps respectively. • Problems: • 4DSP provides free of charge a reference design only for 4ch @ 1.25Gsps. A reference design for 1ch @ 5Gsps priced at 2300 EU.

7. 4DSP reference design

8. MicroBlaze UART ADC Control Implemented In XPS Timer Aggregator FMC125 AXI Master DDR3 Memory Controller (MIG) AXI Slave PCI Express AXI Slave CDMA Implemented In ISE project Navigator AXI Master Mailbox “wormhole” AXI4 AXI4-lite Design Block Diagram

9. Block Diagram: Main Data Channel Aggregator AXI Master DDR3 FMC125 Memory Controller (MIG) AXI Slave • FMC125 delivers data using 4 128-bit lanes of a proprietary bus running at 125Mhz into the aggregator. • Aggregator unites and synchronizes the different channels into a 512-bit bus. • AXI Master sends the data over a 256-bit wide AXI bus running at 200Mhz to the AXI Slave interface of the Memory Controller. • Memory controller handles Writes to DDR3. • FIFO’s and H/W Flow control in every component’s input (and some components output) throughout the channel to achieve highest possible bandwidth. • Throughput is measured using a timer from the start of the write operation until assertion of a write done signal from the AXI master

10. AXI Bus background • Xilinx has adopted AXI bus, which is a standard bus protocol from ARM used in modern ARM SoC. • Characteristics • Memory mapped, 32-bit addresses • Write Address, write data, write response, read address, read data • Variable width, clock & burst length over a single bus.

11. AXI write example

12. Memory & Memory controller • 512MB DDR3 64-bit @ 400Mhz (800MT/s) • Theoretical bandwidth of 800MT/s * 64bit / 8 = 6.25GB/s • Main channel matched to this theoretical bandwidth (200Mhz * 256bit /8 = 6.25GB/s) • Memory & controller isn’t perfect, has a utilization factor. • We achieved 5.13GB/s Throughput (82% utilization!)

13. MicroBlaze UART ADC Control Implemented In XPS Timer Aggregator FMC125 AXI Master DDR3 Memory Controller (MIG) AXI Slave PCI Express AXI Slave CDMA Implemented In ISE project Navigator AXI Master Mailbox “wormhole” AXI4 AXI4-lite Design Block Diagram

14. Block Diagram: memory to PCIe • After the data is sampled to memory the system transfers the first chunk to the Host PC’s DMA buffer. • The subsequent chunks are transferred upon receiving a command from the Host PC. • Data is read from memory and transferred to PCIe by the DMA engine • Throughput was matched to PCIe • Transferred in 16MB chunks (the chosen DMA buffer size) • Address translation occurs in the PCIe core

15. Block Diagram: PCIe • Xilinx Available solutions include: • A PCIe integrated block with support for up to x8 gen2. Hard to work with: • Need to know inner workings of PCIe • Need to implement several proprietary interfaces with many rules and signals • A wrapper for memory mapped AXI with support for up to x4 gen1 / x2 gen 2 • Connects to a standard AXI4 bus • Chosen in our design due to high ROI • We chose to use the wrapper due to high ROI • x4 gen1 configuration, up to 1GB/s • One 64bit AXI BAR for sample data transfer with Configurable address translation, one PCIe BAR for mailbox communication.

16. Block Diagram: MicroBlaze • MicroBlaze is a soft-core processor by Xilinx. • Runs firmware written in C from a dedicated 64KB BRAM. • Firmware communicates with Host PC’s software through a PCIe mailbox • Supports a predefined set of commands to accomplish the functional use case and provide debug capabilities • Initializes and configures the entire system per to the Host PC’s instructions. • Provide visual status and information to the user through UART massages.

17. Block Diagram: ADC ctrl • ADC ctrl is a custom core deigned to enable MicroBlaze to communicate with other custom cores • Developed using the Create peripharelwizrd in XPS and implements mamory mapped register access through AXI4-lite • Provides the following capabilities: • Translate the wormhole transactions to / from the FMC125 core. • Provide AXI Master with burst count, write start, test signals • Provides the aggregator with enabled ADC channels

18. Software • Written in C# .NET 4 in VS2012, GUI in WPF • Enables simple interaction with the system • User adjustable burst size, burst count, active channels, output file location and external trigger mode • compatible with Windows 7/8 x64.

19. Software: Jungo WinDriver • Easy driver creation • Simple interface for DMA buffer allocation, read and write operations • Slightly problematic from a .NET environment since it lacks proper documentation, and the wizard generates a non working solution • Reference for .NET and DMA was found in a bundled example for PLX chips

20. Test Environment • Verification of sampled data on DDR3 is accomplished by firmware on MicroBlaze comparing read data with predetermined written pattern • FMC125 incoming data is observable through ChipScope. • Bandwidth was calculated by measuring the time to write a chunk of data using a Timer. • Data transfer from memory to PCIe is validated through observing the memory on system and comparing with the data received from PCIe manually.

21. Questions?Thanks

22. Backup • Documantaion