System Accuracy Status (SAS) Monthly Assessment Procedures

1. System Accuracy Status (SAS) Monthly AssessmentProcedures January 4th, 2010

2. System Data Accuracy Level (SAS) versus Unit Readiness. • SAS Level is only a measure of how accurate unit M-SHARP data is maintained, which requires routine attention. • SAS Level is NOT a measure of unit readiness, but M-SHARP may present skewed readiness metrics if data accuracy is NOT maintained. • Units must pay close attention to their M-SHARP data accuracy by appreciating that T&R changes and aircrew changes can and do affect SAS Level. • Many units have incorrectly assumed that their initial SAS Level 1 “Green” would endure indefinitely, requiring no further action. As such, they have not established effective procedures for maintaining SAS Level 1 to ensure accurate portrayal of unit T&R metrics. • The following assessment process will ensure M-SHARP System Accuracy Status across the MAG if formally adopted as part of the MAG SOP.

3. Squadron SBTP Execution Report • The “Crew” page of this report is designed to indicate CSP, MSP, & Combat Leadership status for the unit. • The T&R Program Manual, NAVMC 3500.14B, defines the business rules for the calculated values displayed. • The report is reliant upon two inputs from the Marine users: • Logged flight execution via the M-SHARP NAVFLIR to indicate the ongoing T&R execution by the squadron’s aircrew. • Baseline entries to account for all the archived flight execution data for every aviator in the squadron who does not have a complete flight history in the M-SHARP database.

4. Baseline ? • Baseline entries are required for every single metric that is missing from the M-SHARP database - for every single aviator, for every single calculation of core / mission skill, T&R designation / qualification, individual T&R event proficiency, flight hours, etc. • The baseline status of a unit is defined by the collective baseline status of all aircrew in the unit. • Every single change in assigned permanent, TAD, or augmenting squadron aircrew can change the baseline status of the unit. (turn over of aircrew happens daily in Marine Aviation) • Only the Squadron Ops O and his training officers, as the squadron’s T&R subject matter experts, can effectively manage the status of the squadron’s baseline data accuracy by carefully auditing the individual flight history of every existing and potential aircrew in the unit’s M-SHARP database.

5. MCO 3125.1B ExcerptM-SHARP Data accuracy & Level 1/Green Assessment • This excerpt of the MCO highlights the requirement for an objective assessment of each unit by the MCAS M-SHARP Support Representative in coordination with the Commander’s own assessment of his unit to achieve Level 1/Green System Accuracy Status.

6. System Accuracy Status Level (SAS) https://msharpsupport.com/fly_usagestatus.asp • This tool was designed to indicate a unit’s progression toward achieving initial data accuracy in M-SHARP. • A unit’s initial SAS level is a snap shot in time on day the CO first assessed his unit as Level 1/Green with the Support Rep. • The flaw in SAS Level understanding is that it changes with each new change in aircrew or in T&R. • Ops O & Training O must routinely conduct assessments to ensure SAS remains at Level 1.

7. SAS Monthly Assessment Process • Monthly assessments are required to maintain SAS Level 1, enabling automated reporting of accurate T&R data as defined by MCO 3125.1B (SBTP/FHP order). • As the SBTP Execution Report is the primary output tool for objective T&R execution, a monthly assessments of this report will identify perceived discrepancies in data accuracy. • The T&R Designation, MSP, & CSP discreet reports in M-SHARP “feed” the SBTP Execution Report Crew page, and are indicators of potential individual aircrew data inaccuracies. • This assessment will be a cooperative effort between the Squadron Ops O and the MCAS M-SHARP Support Representative. • The air station M-SHARP Support Representative will provide guidance on T&R business rules, and teach the mechanics of editing unit M-SHARP data to achieve data accuracy.

8. Monthly Assessment – Step One • The 5th working day before the end of each reporting month, all ACC reporting units will conduct an assessment of their M-SHARP SBTP Execution Report to determine if the forecasted CSP/MSP/Combat Leadership values match their expectations toward CMMR goals. • This will provide the “Big Picture” view of the pre-audit values that will be pulled by Current Readiness personnel and passed to the NAE monthly Aircrew Core Competency brief. • Each unit will review their actual (“ACT”) execution, compared to their forecasted (“FORE”) numbers to determine how they are tracking toward CMMR goals.

9. Monthly Assessment – Step One Example of unit assessing October “actual” values for CSP & Combat Leadership... …versus “forecasted” CMMR goals. Unit is 7 short of EW goal in this case.

10. Monthly Assessment – Step Two • Each unit will review their discreet M-SHARP CSP/MSP/ T&R Designation Reports to determine which individual aircrew they expect “should be” CSP/MSP/Combat Leaders, and record any perceived discrepancies. • These reports provide a detailed picture of aircrew T&R status. • Commander's Intent for who “should be” CSP/MSP/Combat leaders will be assessed by the Ops O & Training Officers. • After the assessment of their T&R data is complete for CSP/MSP/Combat leadership, the unit will record & identify to the M-SHARP Support Representative where perceived discrepancies exist using a simple M-SHARP export tool (MS Excel) portrayed on the next slide.

11. Monthly Assessment – Step Two(MS Excel Export from M-SHARP of Unit CSP Report) Time stamp: 25NOV09 1635 The Ops O can identify apparent discrepancies with aircrew who the unit believes “should be” CSP by exporting the CSP report to MS Excel. By using the non-standard M-SHARP color , the Ops O can highlight to the M-SHARP Rep where they need assistance in determining apparent baseline discrepancies. PURPLE

12. Monthly Assessment – Step Three • By COB on the 5th working day before the end of the reporting month, the units will provide the MS Excel exported assessments of the CSP/MSP/ T&R Designation pages to the MCAS M-SHARP Support Representative . • By the last working day of the month, the M-SHARP Support Rep will provide feedback to the unit in written form (example provided). • The units will work with the M-SHARP Support Rep to help them interpret the T&R business rules that must be achieved to enable Smith, Jones, etc. to appear as CSP / MSP / Combat Leaders on the SBTP Execution Report.  • SAS level will be reevaluated by M-SHARP Support Representative based upon the audit process, and communicated to each squadron and their MAG HQ. A tentative update to each unit’s SAS level will be determined.

13. Monthly Assessment – Step Three(Example of how Support Rep would asses unit data in M-SHARP) Unit identified aircrew that “should be” CSP for EW. Support Rep would assess that aviator is missing baseline entry for the 0301 code of the EW Attain Table, for example.

14. Monthly Assessment – Step Three(Example Support Rep MS Excel export feedback to the Squadron) Time stamp: 30NOV09 1500

15. Monthly Assessment – Step Four • Once this cooperative audit is complete, each unit will have from the end of the month until the 5th of the following month to properly baseline the individual aircrew CSP/MSP/Combat leadership values to provide the most accurate SBTP Execution Report output, in accordance with MCO 3125.1B. • By COB on the 5th day of the new month, the updated SAS level is communicated to ATB and promulgated in an update to the ATB managed SAS Level report at http://msharpsupport.com/fly_usagestatus.asp. • Units that complete the actions required to achieve baseline will receive a SAS Level 1 if the command has assessed and validated his data accuracy. • Units that have achieved baseline, but no command assessment is completed will be SAS Level 2. • Units that have not achieved baseline requirements revealed by the audit process which have not been corrected will be reported SAS Level 3.

16. Monthly Assessment – Step Five • On the 6th of each new month, Current Readiness personnel will pull the final SBTP Execution Reports for every ACC reporting unit from M-SHARP, and the data displayed in the ACC brief will represent the information as presented by each unit via this audit process. • Along with the SBTP Execution Reports, the ACC brief will also indicate the current M-SHARP SAS level resulting from the monthly audit. • Unit readiness that appears in ACC brief accompanied by a SAS Level 1 or 2, should indicate true objective readiness data. • Unit readiness that appears in the ACC brief accompanied by a SAS Level 3 or less, should indicate units that may have M-SHARP data accuracy issues affecting readiness reporting.

17. Summary • To ensure the accuracy of automated reporting of objective T&R data, a routine audit of M-SHARP output reports must occur in every aviation unit. • The existing Current Readiness / NAE ACC reporting cycle provides the perfect structure to support a thorough audit process. • M-SHARP System Accuracy Status (SAS) can be routinely evaluated, updated, and promulgated in synchronization with the Current Readiness / NAE ACC reporting cycle. • Acting as agents for their commanders, unit Operations Officers can achieve unit data accuracy in reporting through M-SHARP via this recommended auditing procedure. • ATB provides on site support for this procedure in the form of the MCAS M-SHARP Support Representative.