1 / 30

Formal Verification – Is It Real Enough?

Formal Verification – Is It Real Enough?. Yaron Wolfsthal Haifa Research Lab. Rebecca Gott Systems and Technology Group. Assertion-Based Verification. “How can one check a large routine in the sense that it's right? … make a number of definite assertions which can be

peigi
Download Presentation

Formal Verification – Is It Real Enough?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Formal Verification – Is It Real Enough? Yaron Wolfsthal Haifa Research Lab Rebecca Gott Systems and Technology Group

  2. Assertion-Based Verification “How can one check a large routine in the sense that it's right? … make a number of definite assertions which can be checkedindividually, and from which the correctness of the whole program easily follows.” Alan Turing, 24 June 1950 "Checking a Large Routine” 2005: IEEE P1850 PSL -- standard assertion language

  3. What Are the Options? • Dynamic assertion checking • On top of NC-Sim, VCS, ModelSim… • Relatively easy to do – and to argue for • Static assertion checking – Formal Verification

  4. Position • Formal Verification is absolutely for real • here and now • Carries unique, significant benefits and ROI • (without PhDs) • Insights based on many projects, ~10 years • IBM servers, game processors, ASICs • Customer projects

  5. Switch Chip I, IBM High-End System

  6. Switch Chip II, IBM High-End System

  7. IBM Ethernet Core Data collected from IBM Ethernet Core Project

  8. Benefits of Formal Verification • Early Availability • High Coverage • Enabling effective integration • Quality, productivity/schedule and cost gains

  9. Some Usage Numbers • Unit size • Number of assertions • Amount of resources • Extent of logic covered

  10. FV of a Communication Core

  11. IBM eServer™ p690 (Power4™) "We applied FV to some extent on approximately 40 design components throughout the processor and found more than 200 design flaws at various stages and of varying complexity. At least one bug was found by almost every application of FV. In most cases, FV began significantly later than verification. It is estimated that 15% of these bugs were of extreme complexity and would have been difficult for traditional verification.In some cases, a late bug found in verification or in the laboratory was recreated and its correction verified efficiently with FV." Ludden et al.,IBM Journal of R&D 46(1), 2002

  12. IBM Engineering Services UnitExperience Report Source: DAC’04 / PSL Consortium Meeting • Formal Verification of an average logic module • Requires up to a month • Involves the development / debugging of 50-70 assertions • Consumes 20% of designer’s time for supporting the work • Designer can realistically support 3-5 modules • Involves running 5 – 10 assertions concurrently • Consumes few CPUs • Scales up with additional CPUs • In an average project, some 40% of the logic modules are formally verified with RuleBase

  13. Formal Verification of Gigabit Ethernet Core, 2002-2003 • 400,000 gates • 40% of logic went through Formal ABV • Formal ABV practiced by 3 engineers out of a team of 10 • Formal ABV found 33% of documented design bugs • Zero bugs found in logic that went through Formal ABV • Late Formal ABV found bugs in areas that were heavily simulated IBM Microelectronics, Haifa Design Center http://www/pslsugar.org/papers/ABV-in-IBM-Haifa.pdf

  14. Bug Classification • Bugs found due to schedule advantage • Holes in simulation coverage • True corner cases • Performance bugs • Impossible bugs • Deadlocks

  15. Position cont. • Formal Verification is for real, here and now, and it carries unique, significant benefits and ROI (without PhDs) BUT There can be some “Hindering Factors” • The state-space explosion • Soft considerations

  16. “Hindering Factors” • The state-space explosion • Soft considerations • Perception of cost and difficulty • “What we have is Good Enough”

  17. Hindering Factors : State-Space Explosion • Need to have proper methodology in place • Divide and conquer • Falsify vs Verify • Must have battle-hardened technology and tools • Different designs require different search strategies, and the tool must be able to transparently support it • Strength alongside versatility • Ballpark: 1000s (formal)  10000s (semiformal)

  18. Hindering Factors cont : Perception • Assertions, and Formal Verification is perceived as a “difficult” and “costly” technique • However, concrete data suggests otherwise • Undergraduates routinely employed in FV projects • Cf. DAC’04 / PSL Consortium Meeting http://www.pslsugar.org/papers/pm2_EyalGonenDAC04.pdf • DeepChip survey on assertions • FV learning curve survey by IBM

  19. Proliferation of Assertions

  20. User Assessment of “The Formal Verification Learning Curve”

  21. Hindering Factor cont : “Principle of Good Enough” • Satisficing - People will tend to make choices based on their most important current needs rather than through a rational process. • Engineers typically look at various constraints and find trade-offs to try to meet all requirements “well enough” to allow the product to be built • A new way of thinking, which would require some upfront investment, requires some activation energy Herbert SimonTuring Award, 1975Bank of Sweden Prize in Economic Sciences in Memory of Alfred Nobel, 1978

  22. Recommendations for Deployment • Good engineering practices (see proceedings)

  23. Position Summary • Formal Verification is for real, here and now, and it carries unique, significant benefits and ROI (without PhDs) BUT There can no shortcuts • Need a solid technology foundation • Same for methodology • Management commitment

  24. Complex chips, challenging verification problems? Welcome to the club of Formal Verification Thank You

  25. Backup Slides

  26. Formal Verification:Myths and Facts • Doing Formal Verification requires PhDs • Only small designs can be subjected to FV • FV is a push-button technology • FV is predictable • FV is beneficial, helps increase quality, improve TTM • Introducing FV into the design flow is a strategic decision that needs investment as and commitment from management • Once you have found some good corner-case bugs, the designers become your captive audience!

  27. Technology Foundation • Formal Verification is about analyzing very large state spaces – tools must be battle-hardened • Algorithms (optimization of FSMs, fast search) • Parallel computations • Assuring horsepower as well as versatility • Corporate CAD, focused at solving the real-life formal verification problems, is a winning strategy

  28. Methodology Foundation • Design for FV • Involve FV engineers in spec process from early stages • Define the verification targets – unit function and size • Determine verification strategy – assurance vs bug hunting • FV Reviews • Develop properties systematically • Etc..

  29. Management Commitment • A critical factor of success • Invest-first, harvest later • Identify focal point • Leverage existing interest, experience • Initiation and education

More Related