1 / 27

Unlocking Precision with Formal Methods: Improving Software Correctness and Efficiency

Explore the benefits and challenges of formal methods in achieving precise software correctness and efficiency. Discover the importance of mathematical models, specifications, and implementations in ensuring reliable software development. This workshop covers statistical software, formal methods, and the concept of code correctness. Join us to enhance your understanding of formal methods and uncover potential overlooked opportunities.

cworsley
Download Presentation

Unlocking Precision with Formal Methods: Improving Software Correctness and Efficiency

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Welcome

  2. Certain trade names and company products are mentioned in the text or identified. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology (NIST), nor does it imply that the products are necessarily the best available for the purpose.

  3. Workshop Thesis Formal methods can deliver tremendous increases in correctness, etc. … at some adoption cost and modest ongoing cost.

  4. Statistical Software • The software that Census is interested in. • Software involving • randomized algorithms, • Monte Carlo simulation, sampling, etc. • differential privacy, and • summary algorithms. • standard deviation, F statistics, histogram, etc.

  5. Formal Methods • “… a set of techniques based on mathematical foundations and analysis.”† • Other approaches, such as testing, are empirical in nature. †Black, Hall, Jones, Larson, and Windley, “A Brief Introduction to Formal Methods,” IEEE CICC 96, pp. 377-380

  6. Romans and medieval Europeans built great structures, … but expertise passed haphazardly from master to apprentice.

  7. Formal Methods Work! • Formal methods are an approach relying on mathematical models and reasoning. • Chief benefit: 100% coverage of design space • Chief drawback: difficulty building models

  8. Formal Methods Use Three Parts • a mathematical model of the design's intended behavior or properties • the specification, • a mathematical model of the design's structure, • the implementation, • mathematical rules for analysis to show that relations between the models hold.

  9. Example: Code Correctness • Does this code swap x and y? x = y – x; y = y – x; x = y + x; x y . X Y Y-X Y-(Y-X)→X X+(Y-X)→Y

  10. The Specification • Clear, unambiguous statements of desired behaviors, properties, etc. • May be comprehensive or may be just a few critical requirements • Choose level of abstraction • more abstract specs are more concise and easier to have confidence in. • more concrete specs are easier to relate to the implementation

  11. Model Your Implementation • Top down • Add detail • Bottom up • Organize pieces • Primitives or “atoms” • Compose • Adapt existing models

  12. Relating and Reasoning • Some methods (“logics”) are • model checking • theorem proving • equivalence checking • stress analysis • Some methods are automatic. • Some methods are interactive.

  13. = ? Tricky to get the reasoning right 10 L 5 L

  14. = ? Tricky to get the reasoning right 10 L 5 L 15 ºC 21 ºC

  15. = ? Tricky to get the reasoning right 10 L 5 L 15 ºC 21 ºC Water Alcohol

  16. Self Introductions Name, Affiliation, Expectation

  17. Keeling and Pavur, “Statistical Accuracy …” • Compared accuracy of eight packages • using NIST Statistical Reference Datasets and Wilkinson’s Tests • for calculations such as summary statistics, F statistics, regression analysis, rounding, univariate statistics, scatterplots, and statistical distributions. Kellie B. Keeling and Robert J. Pavur (2011) Statistical Accuracy of Spreadsheet Software. The American Statistician 65(4):265–273, DOI: 10.1198/tas.2011.09076

  18. Reliability Measures • Log Relative Error (LRE) LRE = • Manual procedure to find autocorrelation • Pass/fail percentage on Wilkinson’s tests • Overall percentage of “exact” results for 116 distributions

  19. “Software testing studies are important and have resulted in improved software.” • [one package] “has seen significant improvement from a 47% accuracy rate to 99% accuracy.”

  20. Schedule 1005 A Trustworthy Mechanized Formalization of R, Martin Bodin 1030break 1100 Formalizing Statistical Computation in HOL4, Jared Yeager 1120 Trustworthy Data Wrangling and Analysis, Eric Davis 1140 Using SAST for Validation and Verification, Arthur Hicken 1200lunch

  21. Schedule 1330 Introduction to Differential Privacy, Simson L. Garfinkel 1340 Assuring Real-World Differential Privacy, José Calderón 1405 Detecting Violations of Differential Privacy, Zeyu Ding 1445 break 1500 A Formal Methods Tools Rodeo, Paul Black Static Analysis Challenges, Arthur Hicken 1600 workshop ends

  22. Workshop Thesis Formal methods can deliver tremendous increases in correctness, etc. … at some adoption cost and modest ongoing cost.

  23. Charge to Attendees • What formal methods are we overlooking?

More Related