1 / 29

The OSKAR simulator (version 2!)

The OSKAR simulator (version 2!). 3GC-II Workshop, 21 st September 2011 Benjamin Mort, Fred Dulwich, Stef Salvini http://www.oerc.ox.ac.uk/research/oskar. What is OSKAR?. Interferometer and beamforming simulator package. End-to-end s imulations of the phase 1 SKA.

roland
Download Presentation

The OSKAR simulator (version 2!)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The OSKAR simulator (version 2!) 3GC-II Workshop, 21st September 2011 Benjamin Mort, Fred Dulwich, StefSalvini http://www.oerc.ox.ac.uk/research/oskar

  2. What is OSKAR? • Interferometer and beamforming simulator package. • End-to-end simulations of the phase 1 SKA. • Based on a full sky Measurement equation formalism. • High performance library based on NVIDIA CUDA.

  3. Motivation for writing OSKAR? • Understanding system noise limits on SKA like AA interferometers. • Dynamic range limits • All sky simulation. • High performance required for large AA interferometer end-to-end simulation. • e.g. ~1e6 sources, ~25 stations of ~10,000 dual polarisation antennas. • Interferometer configuration studies with aperture arrays.

  4. What I’m going to talk about! • The OSKAR ME implementation. • OSKAR version 2.0 • Some example and results.

  5. Measurement Equation • The ME as implemented by OSKAR • Scalar version currently being tested, polarised version under development. • K – interferometer phase. • E – Station beam. • G – Antenna element field pattern. • P – Propagation term. • B – Source brightness. • V – Complex visibility. Baseline p, q for all visible sources, s.

  6. Sky Model • The global sky model • Equatorial coordinate point source model. • A local sky model is generated for each snapshot • Remove sources below the horizon • Transforming the source Stokes parameters to the horizontal frame. • Emission from bright, extended objects can be included as large collections of point sources. Source Brightness (B) The global sky model is a list of point sources in equatorial coordinates. The local sky at each snapshot is produced by removing sources below the horizon, and transforming the source Stokes parameters to the horizontal frame. Emission from bright, extended objects can be included as large collections of point sources (Fig 1).

  7. G – Antenna field pattern matrix • The average (fully polarized) embedded element pattern for antennas within a station • Factored out if the antennas are sufficiently similar. • otherwise, it is absorbed into the calculation of the E-Jones term. • Requires input from EM simulations (University of Cambridge). • Depends on the station geometry (cross coupling) and element design. • Simple functional responses also possible. Embedded element patterns

  8. Station Beams (E-matrix) • Rotate phase tracking centre (=beam direction) and all sources from equatorial (RA, Dec) to horizontal (azimuth, elevation). • Evaluate station beam response for every source and station, for the direction of interest. • Sources far from the phase centre will be suppressed by station “primary” beams. • (And sources far from thezenith will be suppressedfurther by antenna elementpattern.) • Obtain complex matrix

  9. Station Phases (K-matrix) • K-matrix effectively “phases-up” the array of stations. • Compute phase of each source s at every station a. • Determine station (u,v,w) coordinates by rotating (x,y,z) onto a plane perpendicular to direction of phase centre.

  10. “Correlator” • Multiplies appropriate Jones matrices with the source brightness to obtain a complex visibility per source and per baseline, and then collapses the source dimension. • Time-average smearing: each visibility point can be averaged over time. • K is recomputed to include motion of baseline during integration period. • E is allowed to vary throughout the integration at a slower rate than K. • Bandwidth smearing: multiply each visibility by fs,i,j before collapsing the source dimension.

  11. The OSKAR Package • OSKAR simulation function library based written in C making extensive use of CUDA • Multiple libraries with simple dependencies • Liboskar • Core CUDA function library. • Liboskar_ms • Interface to casacore for writing simple measurement sets. • Liboskar_apps • Utility library for using OSKAR to write C/C++ applications. • Liboskar_widgets • Set of utility widgets written in Qt4/Qwt5. Plotting, gui components. • Liboskar_imaging • FFT imager (CUDA based imager in development) with w-projection. • Liboskar_fits • Interface to cfitsio for writing UVFITS and image fits files.

  12. The OSKAR Package • Simple C like interface to the main simulation library using intrinsic types where possible. • Aims to make it possible to quickly construct new C/C++ simulation applications. • Designed to interface easily with other languages • MATLAB either with loadlibrary() or though a MEX interface. • (Python) • MATLAB and C/C++ applications • Beamforming simulator • AA Interferometer simulator • DFT imaging • Simple image post processing MATLAB scripts e.g. CLEAN.

  13. Why CUDA?

  14. Why CUDA? • What is CUDA? • CUDA (Compute Unified Device Architecture) is NVIDIA’s • Program development environment based on C/C++ with some extensions. • Compatible with other multi threaded code. • Multiple GPUs can be used to work together for very large problems. • Cost and power effective desktop supercomputing. • SIMT parallelism model. • Requires tens of thousands of threads to be efficient. • Multiple GPUs work together for very large problems

  15. Summary of current Features / assumptions

  16. Some examples of using OSKAR

  17. Source distribution, 2-degree “hole” at phase centre (49993 sources)

  18. Field of view 2 deg across, nearest source is 1.32 degrees from centre (Telescope at 40 degrees, 480 snapshots, 49993 sources elsewhere) Peak @ ~3E-3

  19. Imperfect source subtraction • Bright interfering source on the flank of the station beam at position X. • A number of other sources scattered over the sky. • Because the source has effectively become highly time-variable, a simple subtraction of its clean-component model leaves large residuals. • limiting the dynamic range of the image.

  20. Source removal • Solving for differential gains in MeqTrees (Ian Heywood) is far more effective.

  21. Simulation Example: Observation setup. • Telescope at Faro • Pointing around Cassiopeia. • 24h observation.

  22. Simulation Example: AA station setup • Offset grid geometry. • ~80m diameter. • ~2600 antennas.

  23. Simulation Example: Beam pattern.

  24. Simulation Example: Sky model (Simulation with E-Jones disabled)

  25. Simulation Example: Telescope setup.

  26. Simulation Example: results

  27. Simulation Example: Dirty image snapshots

  28. Simulation Example: Frequency time source brightness profiles (1) (3) (2)

  29. Next Steps • Currently working on • Polarisation. • Efficient modelling of system noise. • Antenna gain and phase errors. • Limited precision numerics (currently floating point). • Scaling up to very large simulations using multiple GPUs • Any suggestions?

More Related