Department of Electrical Engineering Electronics Computers Communications

1. High Speed Digital Systems Lab. Dynamic Hardware Reconfiguration Controlled by ANDROID OS On ZYNQ Performed By: ItamarNiddam and LiorMotorin Instructor: Inna Rivkin Bi-Semesterial. Winter 2012/2013 Department of Electrical Engineering Electronics Computers Communications Technion Israel Institute of Technology

2. SOPC on ZYNQ running Android Application performance acceleration Dynamic Hardware Programmable Logic Core 0 : A9 ARM Peripherals Controllers Hardware accelerator Core 1 : A9 ARM Software application AXI4 BUS Processing System running Android • Standard SOPC approach • The Hardware is constant • The user can only switch between application • Task specific - Software

3. SOPC on ZYNQ running Android Performance Acceleration of applications Using Dynamic Partial Hardware Reconfiguration Dynamic Hardware Programmable Logic Core 0 : A9 ARM Peripherals Controllers Core 1 : A9 ARM AXI4 BUS Processing System running Android New approach The user controls the software and the hardware Task specific - Software & Hardware • Partial dynamic hardware reconfiguration by OS (Android) • Hardware system dynamically changed & adapted to a specific application. • The hardware change is done at runtime by application & OS Custom IP Hardware Acceleratior#1 Custom IP Hardware Acceleratior #2 Application #1 Application #2

4. Sobel Edge Detection Filter Example for(i = 0; i < height, i++){ for(j=0; j < width; j++){ x_dir= 0; y_dir= 0; if((i > 0 && (i < (height-1)) && (j > 0) && (j < (width-1))){ for(rowOffset= -1; rowOffset <= 1; rowOffset++){ for(colOffset= -1; colOffset <=1; colOffset++){ x_dir= x_dir + input_image[i+rowOffset][j+colOffset]* Gx[1+rowOffset][1+colOffset]; y_dir= y_dir + input_image[i+rowOffset][j+colOffset]* Gy[1+rowOffset][1+colOffset]; } } edge_weight= ABS(x_dir) + ABS(y_dir); output_image[i][j] = edge_weight; }

5. SobelFilter Software Implementation Sobel Filter

6. Sobel Filter Software Implementation

7. Sobel Filter Software & Hardware Specific Accelerator Dynamic Hardware Programmable Logic Core 0 : A9 ARM Peripherals Controllers Sobel Hardware module Core 1 : A9 ARM Sobel Software application AXI4 BUS Processing System running Android Sobel Filter

8. Sobel Filter Using Hardware Specific Accelerator

9. Integrating android WITH ZYNQ

10. SYSTEM COMPONENTS HDMI Processing System Core 0 : A9 ARM Core 1 : A9 ARM FMC AXI4 Programmable Logic LogicBricks HDMI Controller Custom IP UART USB 0

11. Processing System • The Xilinx ZC-702 Board for Zynq-7000 have the ability to run Android OS on it. • Android porting supplied by Iviea • FMC Touch screen controller supplied by Xylon. • Android relies on Linux kernel 2.6. • Hardware accelerator drivers are implemented by loadable kernel modules. • Drivers interact with hardware by memory-mapped IO. • Android apps communicates with hardware by System services. • System service represents an Hardware abstraction layer between the App and the kernel driver Linux Kernel Xylon GPU Driver Programming Logic Zynq ZC-702 ARM CPU0 Xylon Hardware

12. Processing System Partial Re-Configuration Service Linux Kernel Partial Configuration Driver Custom IP Module Driver Xylon GPU Driver Programming Logic Zynq ZC-702 Custom IP ARM CPU0 Xylon Hardware

13. Processing System App Code Bitstream Partial Re-Configuration Service Linux Kernel Partial Configuration Driver Custom IP Module Driver Xylon GPU Driver Programming Logic Zynq ZC-702 Custom IP ARM CPU0 Xylon Hardware

14. APP How does the application access the hardware? System service API • Make our system service available through the SDK. • Supplies a basic interface between the APP and the custom service that has been created. • The end user application provided by the client. • The custom service run within the System server. • Enables the system server to reach the HAL. • Bridging methods called by the user in JAVA to native code in order to use the HAL. • The Hardware Abstraction Layer API. • Supplies basic interface between the service and the custom new hardware. • Essential for bridging the native and Java code. • Supplies basic functions on the given hardware type, using the driver. • Provided by the driver developer. • A shared library located in /system/lib... The custom linux device driver. .ko file located in /dev/… Binder System Server Custom Service JAVA JNI C HAL API HAL module Driver module

15. The development environment

16. Xilinx Platform Studio • Xilinx SDK • VIVADO HLS • XPS & SDK - Setup and configure the base system (which can run Android OS). • Xilinx Vivado HLS - implement a custom IP module using a native programming language (C). • XPS & SDK - Integrate the custom IP with the system.

17. Developing the custom device driver in C. • Developing the HAL (Hardware Abstraction Layer) which supplies a simple interface between the user App and the custom hardware. • Customizing the Android Kernel in order to provide the partial reconfiguration OS support. • C/C++ for Android Kernel

18. Developing a custom android application in java, which can use the HAL in order to get the services provided by the custom IP we implemented. • Developing the system service which is a part of the HAL. • Java Eclipse

19. So.. What have we done so far? • Setting up a development environment to modify and compile Android OS & Linux kernel. • Configuring a Fully working Android OS system on ZYNQ with a touch screen. • Custom IP development & Hardware integration using Xilinx tools. • Developing a simple Linux device driver (char device) which can be accessed (Read / Write) by the Android on Zynq. • Solving Android privileges limitations by writing a service which can communicate with our char device. • Enabling a smooth access to the custom hardware from any typical Android JAVA code.

20. The system

21. Next steps … • Learning the partial reconfiguration technology deeply in order to get the ability to operate it by the OS. • Developing the Partial reconfiguration driver and service in order to support Dynamic Hardware. • Writing more complicated Android drivers to support AXI-4 Streaming for video processing • Writing efficient hardware with Vivado HLS and integrate it with the system.

22. Thank you !