Experimental run time analysis

1. Experimental run time analysis Martin Jagersand Figure slides to support whiteboard explanations and derivations

2. Why experimental runtime analysis? • You may use routines/programs you didn’t write. • They may be long and complex to parse • Or you only have a binary. • Sometimes supplied information (often a research paper) and actual implementation differs. • It can give more precise runtime estimates for your program than an analysis based on reading the code and counting statements.

3. Ways to measure time • Detailed “profiling”: Insert special statements in code (by hand or automatically) which count how many times a particular segment is executed. • Global run time: Just run whole routine or program and measure time to completion for different input sizes.

4. Steps • Run routines/programs to collect data. • Graph data for initial guess. • Loglog graph to either: • Just extrapolate T(n) for larger n • Compute constant k in n^k • If needed, numerically fit detailed mathematical model to experimental T(n) data

5. Three unknown algorithms tested. Run time T(n_k) for small input sizes Times not dramatically different. Example graph 1

6. Only for larger input sizes are times due to different O(f(n)) clearly distinct. Can say which is better. But not which one is what f(n) in O(f(n)) Graph 2

7. Turns n^k into linear with different slope k Calc. slope on big n’s (to avoid constant and low order k) Log-log plot

8. Just read T(n) for n=10,000 Extrapolate in log-log N = 10,000

9. Different fitted functions

10. Graphs similar. Numerical residual analysis shows: C*n: r=25 C*nlogn r=10 Hence O(nlogn) Detail of n vs nlogn