LOFAR Operations and Schedule

1. LOFAR Operations and Schedule Antonis Polatidis ASTRON is part of the Netherlands Organisation for Scientific Research (NWO)

2. Rollout Status Can observe with 14 core, 6 remote and 3 international stations Up to 34 (NL) + 2 (DE) HBA antenna fields

3. Towards integrated operations:1st Control and Metadata Busy Week • Test the flow of an observation starting with an astronomer defining a project, interaction with the Radio Observatory, data taking and the flow of processed data through the online and offline systems up to the data archive. • Simulated an astronomer using MoM to describe an observation (LEA128 as an example), interaction with RO support scientists and observers, and observations taking place under SAS/MAC (multiple observations). FIRST RESULTS (15-23 Feb and beyond) • Many issues identified; some solved, others entered in CRAFT for further work. • Definition of observations through MoM and SAS functioning OK - Overhaul to add Beam-forming modes. • Interaction between scientist, observers, friend & scheduler going far more smoothly. • Now able to schedule HBA through MoM & SAS • Data management on the agenda • Archiving of LOFAR datasets again in progress after a long period

4. Towards integrated operations:2nd Control and Metadata Busy Week • Put more emphasis to the flow of processed data through the online and offline systems up to the data archive. • Simulate planning, scheduling and executing of multiple observations interfacing the LOFAR SCHEDULER to MoM and SAS. • Reduce the setup time between individual observations (from min to sec). • Test multiple beam functionality. • Integrate SAS with the pipeline and process data all the way to the archive. • Develop data scrutiny protocols. • Observations: • Many, short observations alternating in frequency and mode (LBA, HBA) • Long, multiple beam observation • Parallel interferometric and beam formed observations • Parallel interferometric and TBB dumps

6. MoM - Defining details of observation

7. Current Issues • Problems with Blue Gene rack allocation - Became obvious when recording data at high rates ( >30 antenna fields with HBAs) • A failed process causes the whole subrack to freeze and hence the observations can not recover. • Stations continue streaming data and overflow the network switches • Solution seeked: • Hardware problem: Node replaced (8 March) • Operating System: IBM notified • Arrange network to avoid data overflows • Check if process hung and re-allocate the sub-rack switching off only the affected part. •  Seems that Pulsar observations are not affected by this problem • Amplitude ripples • Ripples in HBA XX polarization in core stations (when splitter is on)

8. Coordination of CEP offline resource usage With increased number of stations, and large data ratesrequire to write to large number of storage nodes simultaneously. Reshuffling of data is highly inefficient. Simultaneous use of storage nodes for observing and processing expected to severely degrade the performance of the systems (will test this further). Current strategy of appointing subclusters to user groups is no longer sufficient Solutions under consideration • Keep /data1 in each storage node free for observations - Process observations and move data out as soon as possible. • Stick with the “dedicated” subcluster allocation for usual work and • Allocate additional resources to larger observations for limited time (ie spare subclusters, all /data1 partitions): • User groups should assign a “spokeperson” for faster communication. • Assign to user groups permission to access the additional resources for a limited time (and clear up the additional disk space). • Assign a “friend” of resources (support scientist) Will consult with the user groups and implement a policy soon.

9. Commissioning Activities March - April 2010

10. Proposal Review Feedback from the Programm Committee sent Feb. 2nd LEA proposals are incorporated in the general commissioning plan. This plan is FLEXIBLE so that it can accommodate the needs and requests of commissioners as well as the available resources of the Radio Observatory. RO will contact proposers to arrange when they are available to schedule their observations and presence in Busy Weeks. Reviews of Long Term Proposals will start reaching Pis by end of March. This will be the first step of an iteration that will lead to guidelines for writting observing proposals Feedback with the Programme Committee: T. Van der Hulst vdhulst_AT_astro.rug.nl Discuss with the observatory regarding observing schedule: lofarscheduler_AT_astron.nl

11. Proposal Rating: CLASS • Class A: proposals judged to be those projects with high commissioning value or scientific impact and will be scheduled early in the term. • Class B: proposals judged to be those projects with significant commissioning value or scientific impact, but requiring either adjustment or system capabilities not available till later in the observing period. These projects will be scheduled on a best effort basis. • Class C: Projects that were not recommended for observations because the LOFAR Programme Committee found them of low commissioning value. Where applicable, some of the aims of these projects can be achieved by getting access rights and processing resources for existing or previously scheduled data sets.

12. Proposal Rating: GROUPA Scheduling Indication • Group 1: Projects that can be scheduled in the next few months with the hardware and software available before MSSS • Group 2: Projects that can be scheduled concurrently or use data taken during MSSS • Group 3: Projects that require hardware and functionality that will be available after the completion of MSSS. • Group 4: Projects judged to require functionality that will be available well after MSSS is completed or that wish to test modes that will be under development later than the end of 2010. These projects will not be scheduled in this round of commissioning.

14. LOFAR Schedule Viewer • The current schedule of LOFAR can be seen online: “www.astron.nl/lofar-schedule/schedule/schedule.php” LOFAR Management of Measurements “http://lofar.astron.nl/mom3/”