Aqua/Aura Automated On-Board Normal Operations (Auto-Ops)

1. Aqua/Aura Automated On-Board Normal Operations (Auto-Ops) Aqua/Aura Flight Operations Team ITAR CONTROLLED TECHNICAL DATA This document contains technical data as defined in ITAR Part 120.10, and thus is controlled by the U.S. Department of State and is being exported in accordance with 22 CFR 125.4(b)(2) TA# 3107-04. Export/re-export of this document requires prior written authorization from the Department of State. Diversion contrary to U.S. International Traffic In Arms Regulations is prohibited.

2. Background • Goal • Remove ground commanding for normal operations • SSR Playbacks & Routine Flight Software Controller Log Dumps • Benefits • Eliminates the dependence on a good command link to dump science data and logs • SSR Playback can occur prior to establishment of Command Link • More efficient use of contact duration; time not wasted waiting for command lock • SSR Playback can still occur if ground station has problems with S-Band • Allows online staff more time to focus on other functions • More time to monitor S/C Health and Safety • Earlier dump start allows for full playback every orbit (less latency) • Allows capability to capture data when a ground communications problem prohibits spacecraft connectivity • T-1 line outage to Norway or Alaska • EOC outage (BEOC usage)

3. Background • Aqua/Aura FOTs had data loss on DOY 158 & 186 (‘06) Data Loss Data Loss % of Total Mission DOY 158 DOY 186 Data Lost Aura 1H11M47S 4H22M07S 79.8% Aqua 3H39M24S 6H43M26S 63.7% • Known Limitations • No automated replays of any data, including: • Data played back after LOS of X-Band signal • Data not captured by EDOS or the station • Data corrupted during the playback • Packet loss at end of SSR dump due to no overlap (Data packet split over 2 consecutive passes is currently not put back together by EDOS)

4. Current Aqua/Aura Commanding • Current Aqua/Aura commanding during EPGN contacts • Command Test (Tic-Toc Command) • Done at completion of successful lock on telemetry • Start X-Band Playback • Cecil PROC moves Read Pointer back 1000 SSR Words (30000 for Aqua) & begins playback • Start FSW Log Dump (Now done via Auto_Log) • Cecil PROC enables Format Table 15’s across all controllers • End Playback • Cecil PROC records Read Pointer locations at completion of dump for playback stat comparison with EDOS • Loading of Master Command Load (once per day) • Loading of Ephemeris (once per day)

5. EPGN / Nominal X-Band PB StatsAqua 2006/002 thru 2006/016 Avg 06:24 PB Duration Shortest 05:28 Longest 09:15 Min 7:07 Avg 8:29 Max 11:27 PB Stop (Time since AOS) PB STOP Min 1:14 Avg 2:04 Max 4:59 PB Start (Time since AOS) Command PB START Min 0:43 Avg 1:31 Max 4:40 TICTOC (Time since AOS) Command TICTOC Contact Duration Shortest 08:37 Average 12:10 13:51 1 2 3 4 5 6 7 8 9 10 11 13 12 0 14

6. Auto-Ops Concept • Auto-Ops consists of three SCS’s on the Command & Tlm Controller (CTC) • Master SCS contains logic to detect GNs (via X-Band Modulator Rate) • X-Band Playback SCS contains commanding sequence to start a Playback • Log Dump SCS contains commanding sequence to start a Log Dump • Master SCS active on the CTC-Online monitors the X-Band Modulator Rate • Master SCS activated via ground command • SCS loops back to beginning and repeats continuously until the ground or OBFM terminates it • SCS(s) are reliant on MCL commanding (X-Band Modulator -> 150 Mbps), but do not require any changes to MCL, PAS, or any MMS activities • Pass Plans will require removal of ground commanding • Activates the Playback SCS when Modulator changes to 150Mbps & iff SCS-057 Enabled • Waits 60 Seconds • Activates the Log Dump SCS iff SCS-060 Enabled • Playback SCS executes an X-Band playback & commands Direct Broadcast • Log Dump SCS commands the Log Dump

7. Master SCS (CTC-055)(Enhanced) • “Master” SCS is activated via ground command • There are no restrictions on when SCS can be activated • SCS will wait for X-Band modulator to be at 15 Mbps, then 150 Mbps (indicating an X-Band ground station pass) • After the X-Band modulator goes to 150 Mbps, the Master SCS will activate the PB SCS (CTC-57), if enabled • After Master SCS starts PB SCS, it waits 60 seconds and activates the log dump SCS (CTC-60), if enabled • The Master SCS then waits for the X-Band Modulator to go back to 15 Mbps and loops back to wait for 150 Mbps on the X-Band modulator

8. Playback SCS (CTC-057) • Playback SCS is activated via Master SCS • PB SCS can be activated from ground if desired • SCS waits 8 seconds to be recorded in Telemetry • SCS then sends command to disable serial output to the TIE (From S-Band PB) • SCS then sets PB partition sequence and initiates the playback • After confirming PB has started, SCS waits for PB to complete and then sends command to set X-Band modulator back to 15 Mbps • After that command, SCS ends

9. FSW Log Dump SCS (CTC-060) • Log Dump SCS is activated via Master SCS • Log Dump SCS can be activated from ground if desired • SCS sends 5 commands to enable Format Table 15 on each controller (Memory Dump Format Tables) • SCS then waits 8 seconds to assure Active status one (1) update in telemetry • After that command, SCS ends

10. Auto-Ops Nominal Contact Design + 5s PB END Comm +120s – 30s AOS +60s LOS S-Band DB Mode (15 Mbps) DB Mode (15 Mbps) X-Band PB Mode (150Mbps) GN Contact • GN SCS 21 • X-Band Mod to 150 Mbps • GN SCS 20 • X-Band Mod to 15 Mbps • GN SCS 3/4 • Set Coherency • Call SCS 109 • GN SCS 173 • Call PCS 22: TIE to EPGN mode • Call PCS 36: Select Packet List 1 • Disables Transmitter A I&Q PN. • TX-B LN & Sub-carrier modulation Off- Selects NOCO mode. • Mod to 15 Mbps • Call SCS 6: Receiver Rate 125 Bps • Call SCS 2: Turn OFF TX-A&B • SCS 57 - Auto X-Band PB • - Wait 8 seconds • Disables the serial output to the TIE • Enables the SSR to Read Select • Defines Partition Playback Seq. • Initiate Playback • Waits for completion • Activates SCS 20 (Mod to 15 Mbps) • Utility SCS 109 • Linear Mod On • Sub-Carrier Mod On • Turn ON TX-B Auto-Ops SCS’s • SCS 60 • Cmd Controller Log Dumps • SCS 55 - Automation Master • Call SCS 56: Auto Playback • Call SCS 60: Log Dumps

11. Operational Use • There is a Cecil PROC to transition between the various States. SYS_AUTO_CONTROL (’New State’) Where: New State is one of the following valid states: MANUAL_OPS, IDLE, AUTO_OPS, AUTO_PB, AUTO_LOG Examples:SYS_AUTO_CONTROL (’MANUAL_OPS’) • The design of the utility PROC performs the following functions: • Range Check • Verify the input string is valid • Initial Prerequisite Check • Verify Inhibit ID #88 on the CTCN is not set • Verify the X-Band modulator is on • Determine the current state of Auto-Operations, echo to Central Cecil window • Perform required commanding to achieve the desired state • Verify each command with telemetry • Verify the desired state is achieved • Echo new state to Central Cecil

12. Auto Playback 55 57 60 Manual Ops Idle Auto Ops 55 57 60 55 57 60 55 57 60 Auto Log Dump 55 57 60 Auto-Ops State Change Diagram Terminated Disabled Enabled Running Legend SYS_AUTO-OPS_CONTROL SCS-055 (Auto-Master) SCS-057 (Play Back) SCS-060 (Log Dump)

13. Operational Testing • Loaded the Auto-Ops SCS’s • Loaded 55, 57 & 60 • “Unit” Tested the Auto-Ops SCS’s • Tested 60 - Ran FSW proc to activate SCS 60 • Tested 57 - Ran FSW proc to activate SCS 57 • Option 1: Run & Replay (Handover required for Aqua) • Option 2: Run in place of a ground commanded Playback • Option 3: Run on TDRSS, move pointers • Test 55 - Used utility proc to activate SCS 55 • SCS 57 & 60 are Enabled & Inhibit ID 88 Set • Verified execution 1-2 times and leave running overnight • Verified SCS 55 execution times in SCILR (from previous day) • “System” Tested the Auto-Ops Process • Used utility proc to set mode to AUTO_OPS • Verified successful execution over 1 EPGN (handover for Aqua) • Manually ran Replay • Used utility proc to set mode to AUTO_LOG • Verified successful execution over 1-2 orbits • Remained in AUTO-LOG for Normal Operations

14. Contingency Operations • Auto-Ops does not allow interruption of the current PB or pre-empting Auto-Ops during the same contact • Therefore, all action would require an emergency blind acquisition contact to preempt • All contingency X-Band scenarios work as follows • Place Auto-Ops into Manual Mode (or Auto-Log) • Execute Manual X-Band contingency as defined in SOP #035 • Place Auto-Ops into Auto_Ops mode • Proposed Implementation: First three (3) months of execution of Auto-Ops will not be interrupted for routine playback contingencies • Manual Ops always be available if needed

15. Documentation Aqua Aura • Online SOP #100 In CM In CM • “Automated On-Board Normal Operations Concept” • Describes the philosophy behind operating with Auto-Ops • Described the methodology of design of the three SCS’s • Contains flowcharts & command tables of the SCS’s • Online SOP #101 In CMIn CM • “Automated On-Board Normal Operations Process” • Describes the Auto-Ops States • Describes the flow of the Auto-Ops utility PROC & Operational Use • Describes how/when to execute contingency action • Online SOP #035 In CM In CM • “X-Band Nominal Playback Procedure” (Master) • Describes how to execute manual SSR X-Band contingency actions • Scratch LRV’s CompletedCompleted • Text LRV’s display automation mode via configuration Cecil PROC

16. Work Remaining • Refresher Course/Online Training • Get EDOS to fix overlap issue • Future plans • Finalize Auto-DB Procedure (draft available) • Finalize Auto Event Setup Procedure (draft available) • Schedule:  FOT can be ready in 1-2 weeks after “GO” is received from ESMO and Project Scientist • If contingency responses are needed, FOT would need an additional 1-2 weeks (2-4 weeks total)  

17. Aqua/Aura Automated On-Board Normal Operations (Auto-Ops) One Year Analysis Noah O’Connor Command & Data Handling Engineer Aqua/Aura Flight Operations Team phone 301•614•5017fax 301•614•5009 noconnor@eoc.ecs.nasa.gov ITAR CONTROLLED TECHNICAL DATA This document contains technical data as defined in ITAR Part 120.10, and thus is controlled by the U.S. Department of State and is being exported in accordance with 22 CFR 125.4(b)(2) TA# 820-00C. Export/re-export of this document requires prior written authorization from the Department of State.

18. One Year MIR Analysis(10/1/05 – 9/30/06) • “If Auto-Ops had been running continuously for one year, how would the overall science data capture rate for Aqua and Aura been affected?” • When would Auto-Ops have captured more data than Manual-Operations? • Data lines issues • Ground station S-Band problems • When would Auto-Ops lose data as compared to Manual-operations? • Ground station X-Band problems

19. Events Causing Data Latency • Events Causing Data Latency (Manual Operations) • Short Contact or Delayed Commanding • Playback must be killed by operator, 5 – 50% of the orbit data is late • No commanding • Playback never started, 100% of orbit data is late • For both of the above, Auto-Ops improves (reduces) data latency, since it does not rely on command link to dump science data

20. Events Causing Data Loss • Events Causing Data Loss: Manual Operations • Data lines to Ground Station down • No command ability for several orbits (S-Band Issues) • Operator Error • Events Causing Data Loss: Auto-Ops • X-Band tracking problems at ground station • High Uncorrectables • GN Station RED • GSIF & Line-Outage-Recorder anomalies

21. Frequency of Events Causing Auto-Ops Loss Over one year (5,316 Orbits / 23 ground track repeat cycles):

22. Results of One-Year Analysis: Data Loss *Percentage Net loss is in addition to loss from Manual-Operations

23. Aqua Monthly Data Capture Rate

24. Aura Monthly Data Capture Rate

25. Manual Operations & Auto-Ops Total Data Capture Rate If Auto-Ops had been running continuously from for one year, Oct 1, 2005 – Sept 30, 2006, how would overall data capture rate been affected?

26. Results of Analysis: Data Latency • Auto-Ops would improve (reduce) data latency since it does not rely on the S-Band command link. Aqua routinely must kill the playback due to short contact times or delayed commanding. • The number of latency incidents and the percentage of that orbit’s data that was one orbit late is shown below:

27. Auto-Ops Analysis: Aqua • Aqua analysis from October 1, 2005 – September 30, 2006 (5316 Orbits). Each MIR, data-loss incident and contingency operation was taken into account. • Total S/C Dumped = 37,381,963,240 CADU • Auto Ops Loss = 197,024,212 CADU (0.527% of total) • Auto Ops Gain = 49,721,457 CADU (0.133% of total) • Total Auto-Ops loss for analysis time period would have been: • 147,302,755 CADU • 0.394% of total CADUs dumped • Approximately 21 orbits of data • Data Latency improvement: • There were 249 CDH_SSR_PBKILL proc executions, of which 166 would have had improved (reduced) latency under Auto-Ops. 137 were due to nominally scheduled short contacts.

28. Auto-Ops Analysis: Aura • Aura analysis from October 1, 2005 – September 30, 2006 (5315 Orbits) + 1st orbit of Oct 1, 2006. Each MIR, data-loss incident and contingency operation was taken into account. • Total S/C Dumped = 6,553,803,316 CADU • Auto Ops Loss = 29,939,057 CADU (0.457% of total) • Auto Ops Gain = 2,040,326 CADU (0.031% of total) • Total Auto-Ops loss for analysis time period would have been: • 27,898,731 CADU • 0.426% of total CADUs dumped • Approximately 29 orbits of data • Data Latency improvement: • 19 orbits of data that had a data latency of one orbit would have no latency with auto-ops in place

29. Missing Science PacketsProblem Summary and Options James H. PawloskiAqua FOT Flight System Engineer phone 301•614•5077fax 301•614•5009James.Pawloski@gsfc.nasa.gov

30. Summary of the problem • Aqua/Aura FMU/SSR sends down complete CADUs with a 1-15 CADU overlap. • Some instrument packets can span multiple CADUs (1-8) • EDOS has collected every single science CADU recorded • FOT C&DH has also recorded this performance • EDOS processes packets from a single SSR dump • EDOS does not put together packets that span two SSR dumps • Therefore, EDOS sees "broken" packets over this time • Currently X-band procedures manually back up each partition 1000 SSR words (30000 for Aqua) so broken packet is picked up on next playback • Auto-ops does not have this overlap capability

31. FMU/SSR Diagram

32. FMU/SSR Background • SSR Data structure: • 1 SSR word = 960 bits • 1 X-Band CADU = 8192 bits • 15 X-Band CADUs = 128 SSR words • The SSR nominally stores the CADU boundary offset in firmware and automatically adjusts the read pointer for the next playback • When the read pointer is commanded to a value, the firmware deletes the offset, so the read pointer must be placed on a CADU boundary in order to not loose frame sync

33. Data Loss due to Broken Packets NOTE: HSB, AMSU-A & HIRDLS are not affected by this issue

34. EDOS Updates and Future Changes See other presentation

35. Back-Up Slides X-Band/S-Band Antenna Gain Charts Auto-Ops SCS Command Tables Possible Solutions for Missing Packet Issue

36. X-Band Antenna Gain

37. S-Band Antenna Gain

38. Aqua/Aura GN Geometry 5° a Horizon 95° 6378km + 705km 6378km

39. Automation Master SCS(CTCN-055 – enhanced)

40. Playback SCS(CTCN-057)

41. Log Dump SCS(CTCN-060)

42. Potential Solutions for Missing Packet Issue • Have EDOS process the X-Band playback with some or all of the previous dump • Modify FMU logic to move the READ POINTER back x number of SSR WORDS • Flight Software Patch / PVVF • Live with it • Get the data through Direct Broadcast

43. Solution #1: EDOS Fix 1) Have EDOS process the X-Band playback with some or all of the previous dump. • This would involve processing two Spacecraft Contact Sessions (SCSs - an X-Band playback) together which would be a time hit on processing, or processing a piece of the last dump with the new dump. • This would involve an EDOS software modification to break off some portion of the last SCS and "tape" it to the new dump before processing to give EDOS the needed overlap to not miss packets.

44. Solution #2: FMU Fix 2) Modify FMU logic to move the READ POINTER back x number of SSR WORDS. • This would involve a patch to the FMU logic to account for this. • The FMU firmware cannot be programmed on-orbit • This was not done for either Aqua or Aura pre-launch, so this solution is not possible.

45. Solution #3: Software Patch 3) Flight Software Patch / IVVF • This option is to have the Aqua/Aura Flight Software Maintenance (AFM) group design a routine to move the READ POINTERs after every playback back x number of SSR WORDS to provide the needed overlap for EDOS. This would require a lengthy design and testing effort prior to utilization, although this option would least affect operational processes like FOT PROCs and EDOS software.

46. Solution #4: Do Nothing 4) Live with it • Only some of the instruments are having this problem • AMSR-E, CERES, AIRS, MODIS, OMI, MLS and TES • Limited to those instruments that have larger packets • Limited also to those that do not use fill to complete a CADU and therefore split packets over multiple CADUs • Also limited to 1 packet per dump or 15 packets per day per instrument.

47. Solution #5: Direct Broadcast 5) Get the data through Direct Broadcast • This option is not feasible. The data lost would not be in the timeframe that the Direct Broadcast is on.

48. Possible Solutions Summary