1 / 42

TMDE AND FLEET SYSTEM RISK

TMDE AND FLEET SYSTEM RISK. Dennis Jackson NSWC Corona Division (MS-20) 7 May 2009. Overview. System Specifications System Testing and Risk System Adjustment and Specifications. Prime System Traceability.

lance
Download Presentation

TMDE AND FLEET SYSTEM RISK

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TMDE ANDFLEET SYSTEM RISK Dennis Jackson NSWC Corona Division (MS-20) 7 May 2009

  2. Overview • System Specifications • System Testing and Risk • System Adjustment and Specifications

  3. Prime System Traceability • Program / acquisition managers will provide measurement requirements that are in support of prime systems to the TMDE EA via MIL-STD-1839C CMRS data submitted. -- NAVSEAINST 4734.1B • Recommended TMDE shall be capable of measuring or generating to a higher accuracy than the measurement parameters being supported. Unless otherwise specified, a minimum Test Uncertainty Ratio (TUR) of 4 to 1 is desired. The actual TUR shall be documented. -- MIL-STD-1839C • Development procedures shall result in MRCs that ensure system or equipment operation is within performance standards and established readiness criteria. -- MIL-P-24534A

  4. Test Accuracy Ratio (TAR) Example: (Expressed as 10:1) • TAR = The ratio of the system specification to the TMDE tolerance • TAR is a key driver for test decision risk and cost

  5. Test Uncertainty Ratio (TUR) • TUR = The ratio of the system specification to the TMDE uncertainty • Since the TMDE tolerance is often assumed to be the uncertainty, TAR and TUR are often used as equivalent. • The new ANSI/NCSLI Z540.3 standard defines TUR as the ratio of the TMDE tolerance to the 95% Calibration Process uncertainty. • The Z540.3 provides a less ambiguous definition. • The Z540.3 definition applies to calibration rather than system testing.

  6. Example System Mk 99 Missile Fire Control System • Controls the loading and arming of the selected weapon • Launches the weapon • Provides terminal guidance for AAW (Anti-Air Warfare) missiles • Controls the target illumination for the terminal guidance of SM-2

  7. Specification Closeup

  8. Specifications • An MRC directs testing with a TMDE to ensure a system parameter is within a specification. • For example: • Ensure voltage measured is 27.0 V (± 2 V) • Ensure voltage measured is 28.0 (26.6 to 29.4) VDC

  9. Specification Choice • What does ± 2.0 V mean? • At 29.0 V, is performance degraded? • At 25.0 V, is performance degraded? • Specifications should not be determined solely on TMDE accuracy

  10. Utility Curve Acceptable performance Degraded performance Loss of Utility

  11. Testing • Navy systems are tested using test equipment (TMDE) Test Equipment Navy System Measures

  12. SCAT 4245

  13. TMDE Index How do TMDE tolerances relate to testing systems?

  14. TMDE Measurement Error Test Equipment Navy System Measures • All Measurements have some error • TMDE errors need to be small • Large TMDE errors cause bad test decisions • If, for example the test equipment measurement above was off by 1 volt, the system would fail incorrectly • This would cause unnecessary maintenance

  15. Utility Curve Uncertainty for TMDE Measurements tested inside the specification could actually be in the degraded performance region Acceptable performance Degraded performance Loss of Utility

  16. MeasurementRisk Events • False Accept occurs when a parameter is observed through testing by a TMDE to be acceptable, but is actually outside specifications • False Reject occurs when a parameter is rejected through testing by a TMDE , but is actually inside specifications

  17. Good Test Decisions

  18. Risk Consequences • False Accepts directly harm the Fleet • Degradation of mission capability • Failure of mission • Injury • Loss of life • False Rejects indirectly harm the Fleet • Increased maintenance cost • Decreased availability

  19. TMDE Measurement Model TMDE SYSTEM PARAMETER Measures TMDE Measurement = Parameter Value + Test Error • The parameter value is the true output from the system • The TMDE measurement estimates the parameter value • The test error is due to inaccuracy in the TMDE as well as in the test setup

  20. True In Tolerance A parameter is truly in tolerance if: Lower Spec < Parameter Value < Upper Spec Parameter Value Lower Spec (-L) Nominal Upper Spec (L)

  21. Observed In Tolerance (Acceptance) A parameter is observed in tolerance if: Lower Spec < TMDE Measurement < Upper Spec TMDE Measurement Lower Spec (-L) 0 Upper Spec (L)

  22. TMDE Measurement Test Error Parameter Value - L L Nominal False Accepts False Accept (FA): • The TMDE Meas is observed in tolerance [ -L < Meas < L ] • The Par Value is out of tolerance [ Value > L or Value < -L ] • The decision to accept the Parameter is incorrect

  23. TMDE Measurement Test Error Parameter Value - L L Nominal Probability of False Accept Probability of False Accept (PFA): PFA = Pr( [Observed In Tolerance] and [True Out Of Tolerance] ) = Pr( [-L < Meas < L] and [Value > L or Value < -L] ) PFA is the probability of making an incorrect acceptance decision

  24. True Out Of Tolerance A parameter is truly out of tolerance if: Parameter Value < Lower Spec orParameter Value > Upper Spec Parameter Value Lower Spec (-L) Nominal Upper Spec (L)

  25. Observed Out Of Tolerance (Rejection) A parameter is observed out of tolerance if: TMDE Measurement < Lower Spec or TMDE Measurement > Upper Spec TMDE Measurement Lower Spec (-L) Nominal Upper Spec (L)

  26. Parameter Value Test Error TMDE Measurement - L L Nominal False Rejects False Reject (FR): • The Par Value is in tolerance [ -L < Value < L ] • The TMDE Meas is observed out of tolerance [ Meas > L or Meas < -L ] • The decision to reject the Parameter is incorrect

  27. Parameter Value Test Error TMDE Measurement - L L Nominal Probability of False Reject Probability of False Reject (PFR): PFR = Pr( [Observed Out OfTolerance] and [True In Tolerance] ) = Pr( [Meas > L or Meas < -L] and [-L < Value < L] ) PFR is the probability of making an incorrect reject decision

  28. Inputs Needed to Calculate Risk Probability • The Specification Limits (-L, L) • The MRC Card • The Measurement Uncertainty for the Test Process • Generally, this can be considered the TMDE uncertainty • Should also include test set-up uncertainty (cables, etc.) • The Observed Test Parameter Reliability • 3M Data

  29. Risk Tool

  30. Risk Examples

  31. Not Calibrating Costs Money • Inaccurate test equipment cause bad test decisions • Wrong test decisions mean unnecessary maintenance • Maintenance cost is driven by TMDE uncertainty, system spec’s, cal periodicity, and maintenance periodicity

  32. The Key Is to Minimize Cost • More calibration increases calibration budget • Less calibration increases maintenance budget, but allows extended deployment and fewer personnel • Minimize the total budget (calibration + maintenance)

  33. Testing vs Adjusting • For some parameters, the TMDE is used to adjust the parameter value rather than testing it • Often, no specification is given for this situation • With no specification, there is no basis for choosing the TMDE since TAR is unknown • The following shows an example.

  34. Utility Curve Uncertainty for TMDE TMDE adjustment accuracy should be 4 times better than the system requirement Acceptable performance Degraded performance Loss of Utility

  35. Adjustment and TUR • Assume TUR is the ratio of the 95% TMDE uncertainty to the system specification • With a 4:1 TUR, an adjusted parameter has very close to a 0% chance of being out of tolerance after adjustment. • Assuming a stable system parameter (good repeatability) • With a 1:1 TUR, an adjusted parameter has about a 5% chance of being out of tolerance after adjustment. • A 1:1 TUR occurs when the system specification is set to the TMDE tolerance

  36. Conclusions • Specifications are needed for every measurement event requiring a TMDE • Specifications should be directly related to system performance • Specifications should answer the question: “Does system performance degrade when the parameter is outside the specification?”

  37. Conclusions • TMDE tests are used to make decisions about Fleet systems. • Wrong decisions cause consequences to the Fleet: • Degradation of mission capability • Loss of mission • Injury • Loss of life • Unnecessary maintenance cost • System unavailability • The probability of wrong decisions can be calculated using risk methods

  38. Conclusions • To assess the probability of wrong decisions (PFA and PFR), you need: • System specifications • TMDE tolerances/uncertainty • System reliabilities • If the system specifications are related to system performance, the risk measures directly relate to system reliability • Allows assessment of impact of TMDE testing on system performance • Allows assessment of impact of calibration of TMDE on system performance

More Related