Tracey Clavell Head of Estimating and Pricing – Corporate

1. Tracey ClavellHead of Estimating and Pricing – Corporate Defence Cost Estimating Conference Melbourne Oct 2012 Commercial in Confidence

2. Estimation Approaches, Methods &Techniques Structure/Approach Method Technique Parametric Analogy Expert Judgement Delphi Forums Algorithmic Build up Historical Single Point Two Point Three Point Top Down Bottom Up Commercial in Confidence

3. Accuracy V’s Technique V’s Requirement Commercial in Confidence

4. Case Study 1 - Background • Software product only (integration onto the IT infrastructure was not in scope) • Intelligence data gathering and presentation system • High visibility product • Some COTS software, some developmental • Two iterations: • Initial parametric estimate to verify team’s ROM • Formal response, short response period, parametric estimate to verify bottom up Commercial in Confidence

5. Engagement Process “We should brief the team in Canberra on the current state of parametric modelling and bid support” “Thanks for the briefing. Now that you are here, I have a job you can help us with. Are you free to pop upstairs to meet the team?” Commercial in Confidence

6. Initial Model Development End User Requirements Document Extract Transform and Load Mine data to extract meaning Store data and meaning Construct specialised interactive presentations Present developed materials Replay presentations Package presentations in a canned non interactive format • Bid Support Engineer came up with the initial PBS based on the general user statements • Nature of each component modelled from Engineer’s understanding of solution, and from probing the team • DBMS added – as we would need such underlying technology • System Admin added – user rights management etc Commercial in Confidence

7. Next Iteration • Team saw the first cut • Yes it matches the user needs • There are a couple of things that don’t quite match how I think I will build this • Team had a draft system architecture document • Broke up the product into 3 separate products • Had a little more clarity around what each component did • So version 2 of the PBS was created Commercial in Confidence

8. Version 2 of the PBS • Three separate products; • Components will be COTS; • Admin required for more than just users Commercial in Confidence

9. Engagement Process – Part 2 “Here’s your parametric estimate report. Let us know if you have any questions, or if you want us to do more for you.” “Thanks. That stacks up really well to our ROM. That’s all we need for now, but we’ll be back in touch later.” “Guys, are you able to update your parametric model for us? We have just received the formal requirements, and our response time is really short.” Commercial in Confidence

10. Team Had Developed a Product Architecture Commercial in Confidence

11. Mapping Architecture to Our PBS Not in our scope Commercial in Confidence

12. This Wasn’t Quite Right • And also … • They changed the delivery strategy • Some COTS products had now been identified and chosen • And not everything was going to be COTS Commercial in Confidence

13. A New PBS Emerged • Three Phases (where Phase 1 was system analysis and design) • Team structure was solidified • The team’s requirements of us changed • They needed resource loading as well • We got a little more clarity on the COTS products • This slightly changed the PBS, as some items needed to be divided up or merged Commercial in Confidence

14. Now How Does the Mapping Look? • Compared to architecture • “HMI Views” was split – 2 products used • Presentation creator and delivery merged • Otherwise maps pretty well – in fact better than my first attempts Commercial in Confidence

15. Summary Case Study 1 • It was iterative – which was expected • The first cut at understanding the end product, while not exactly accurate, prompted discussion, and in fact influenced design decisions • Or at least made the team make a decision • The final PBS reflected not just the end product but: • The delivery strategy of 3 phases • The team structure that they had now decided upon • The division of functionality among COTS products, which had previously not been chosen • Not shown here is how the parameters of each item were developed • Bid Support Engineer made a start, from the architecture description, and the team’s explanation of what each component did Commercial in Confidence

16. Background Case Study 2 • Distributed systems, loose integration in most cases • COTS software throughout • Request for Proposal • Customer asked for a ROM to ± 30% accuracy • Team planned to use parametric estimate as the submitted ROM • Early parametric model used to guide a Design to Cost workshop • Design to Cost Workshop identified 4 separate trade offs to perform, each requiring parametric modelling to quantify the differences Commercial in Confidence

17. Engagement Sequence Initial Request for ROM Initial Solution ROM Submission Baseline Parametric Model Initial Parametric Model Option 1 Parametric Model Decisions Updated Solution Option 2 Parametric Model Design to Cost Workshop Four Key Trade Studies Option 3 Parametric Model Option 4 Parametric Model 17 Commercial in Confidence

18. Initial Model Development – One Page Architecture Commercial in Confidence 18

19. Generated Model Structure – Compressed 19 Commercial in Confidence

20. Structure Details • Clearly the PBS required a bit more than the one page diagram • We built an initial PBS, and it refined over a short discussion into the one on the previous slide • The RFP documentation also arrived just after the initial PBS, and helped expand and/or change the initial structure • All through this period, the Bid Support Team had weekly video conferences with the technical team 20 Commercial in Confidence

21. Engagement Process – Design to Cost Workshop “Now that we have an initial model, we believe that the Engineering Function’s Design to Cost workshop would be of use” “Sounds like a good idea. We can also help refine the details of the model while you are here” 21 Commercial in Confidence

22. Engagement Process – Design to Cost Workshop “Now that we have an initial model, we believe that the Engineering Function’s Design to Cost workshop would be of use” “Sounds like a good idea. We can also help refine the details of the model while you are here” “Hello, we’re from the Engineering Function, and we’re here to help!” 22 Commercial in Confidence

23. DTC Worksheet – How the Parametric Model Was Used Generate some pie charts to understand % breakdown of costs Commercial in Confidence 23

24. All yellow cells in this column are from pie charts or model data Need a cost target to start everything All yellow cells in this column are from pie charts or model data Percentages are used to derive the targets All green cells in this column are from model data – shows where target < estimate All green cells in this column are from model data – shows where target < estimate Percentages are used to derive the targets DTC Worksheet – How the Parametric Model Was Used (2) And this data is not from the parametric model, but from company past project data and rules of thumb Commercial in Confidence 24

25. What Did the DTC Workshop Uncover • Our estimated price was higher than the target (but not by much) • We identified where the greatest “bang for buck” would be – this is the major point of the DTC workshop • This led to 4 specific technical trade studies • Each trade study option meant a separate model • Each new model was to be compared to the baseline solution, giving information to inform decisions Commercial in Confidence 25

26. Summary Case Study 2 • Parametric estimation was pivotal to this RFP in two ways: • It was the primary estimating technique for the submission • It was central to the Design to Cost process • This particular solution was quite volatile, so weekly video conferences were helpful, BUT … • Being able to visit (for the DTC workshop) was even more useful 26 Commercial in Confidence

27. Summary • Left Shift • Engage face to face initially • Design to Cost Workshops • Lever Analysis • What if Analysis • Ensure Independence • Ensure Requirements / Scope is correctly modelled • Analyse, question, reiterate • Quick Turn around Times Commercial in Confidence

28. Accuracy V’s Technique V’s Requirement • Decision support tool which works well if supported by a sound process • Don’t be afraid to utilise Parametric Analysis • Imagine how powerful this technique could be when combined with Uncertainty Cost Analysis ? • Imagine how powerful this technique could be to compare to Bottom Up? • Determine the Accuracy needed at what point in the process you are • This may change your view on what technique or techniques are needed • Ensure you are modelling the Requirement / Design • Utilise Design to Cost Commercial in Confidence