1 / 14

How To Benchmark Applications Development or Maintenance: Theory and Practice David G. Rogers

How To Benchmark Applications Development or Maintenance: Theory and Practice David G. Rogers. How to benchmark Apps, in a nutshell. Sponsor the benchmark at a senior level Understand the risks, costs and timescales Be actively involved in the benchmark

preston-house
Download Presentation

How To Benchmark Applications Development or Maintenance: Theory and Practice David G. Rogers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How To Benchmark Applications Development or Maintenance:Theory and PracticeDavid G. Rogers

  2. How to benchmark Apps, in a nutshell • Sponsor the benchmark at a senior level • Understand the risks, costs and timescales • Be actively involved in the benchmark • Passive benchmarking (“Speak when you’re spoken to”) is bad for your health • Cater for the measurement problems unique to Applications • Watch the UKSMA website – launching initiative to solve major AM measurement problem • Plan round the comparison problems 30 mins 10 mins David G. Rogers

  3. Senior sponsorship • In-house – • IT director • Outsourced – • Customer’s IT director (not just contract manager) • Supplier’s relationship and delivery managers (not just contract manager) • Give the benchmark the level of management commitment warranted by the risks David G. Rogers

  4. The Risks • The result might ruin formerly win-win relationships, damage careers, cost many jobs • Relationship of the Applications service supplier with its customers is at risk, whether in-house or outsourced • What will senior management do if a benchmark result says: “Your AM costs 3 times the market average”? Or “ … 1/3 the market average”? • First, they will decide whether they believe it. • If they don’t, reputations are damaged • If they do … David G. Rogers

  5. The Risks • The result might be wrong • Mistakes abound • Not necessarily (but possibly) by the benchmarker • The quality of the benchmark is your responsibility. Don’t delegate all responsibility for quality to the benchmarker • Watch the detail • Check all data going into the process • Ensure all services and all costs are reported(this may seem obvious, but …) • Ensure in writing that you have the right to check for possible arithmetic errors by the benchmarker (they are only human) • Build in cross-checks where possible David G. Rogers

  6. The Costs • Major cash costs in benchmarking Applications: • Benchmarker’s fee • FP counting costs could easily be higher • Staffing • One senior (reporting to the Sponsor) manager responsible • Full-time benchmark manager • System experts when required David G. Rogers

  7. Timescales • Only passive benchmarks keep to the benchmarker’s schedule • In a “Passive benchmark” you: • Do only what the benchmarker tells you • Supply only the information you are asked for • Sit back and wait for The Answer • Passive benchmarking is bad for your health! David G. Rogers

  8. Some other stuff you MUST get right • Objectives • Crucial but usually easy if outsourced • Primary reason for outsourcing: 48% say “Reduce cost” • (that explains a lot … imagine recruiting senior executives on the same principle …) • Crucial but slippery if in-house • Like-for-like comparisons • Very hard to achieve … you have to help the benchmarkers • Releases • Very hard to match output to input … don’t leave it all to the benchmarker David G. Rogers

  9. Application Maintenance • The key metric: £ / FP maintained • Commercially crucial measurement • (see Risks above!) David G. Rogers

  10. How do you obtain £ / FP? • £ : the price to you of running AM • FP : the size of the maintained portfolio How is FP obtained? • Count the FPs: +/- 7.5%, but usually much too expensive • “Fast counts” etc: less accurate (+/- 20% or more) • Too inaccurate if results are commercially important • Much used: BACKFIRINGCount Source Lines Of Code (SLOC), and “backfire” to FPs using average ratios David G. Rogers

  11. How accurate is backfiring? • In one recent benchmark, benchmarker claimed +/- 10% • Most experts say +/- 100% to 400% • The experts differ – but if the latter,benchmarking represents a HUGE commercial risk … • … so it is financially important to find out David G. Rogers

  12. Initiative launched to accumulate proof • Nothing even remotely confidential:only size matters • No dates, times, costs, prices, regions • No names except the verifying UKSMA member • The result will bear the imprimatur of UKSMA, and will be in the public domain David G. Rogers

  13. The desired outcome • The usefulness of SLOC as a metric for use in benchmarks will be finally and permanently quantified • This will benefit EVERYONE WHO EVER BENCHMARKS APPLICATIONS David G. Rogers

  14. Primary contact for this UKSMA initiative:David@RogersTeam.co.uk+44 7812 189 672Any questions, suggestions or offers of data?David Rogers is an EDS employee, but in this initiative is acting solely on UKSMA’s behalf. David G. Rogers

More Related