1 / 25

Performance Analysis O f Generics I n Scientific Computing

Performance Analysis O f Generics I n Scientific Computing. Laurentiu Dragan Stephen M. Watt Ontario Research Centre for Computer Algebra University of Western Ontario SYNASC 2005. Overview. Motivation Parametric Polymorphism Implementation Generalizing A Numeric Benchmark

brant
Download Presentation

Performance Analysis O f Generics I n Scientific Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Performance Analysis Of Generics In Scientific Computing Laurentiu Dragan Stephen M. Watt Ontario Research Centre for Computer Algebra University of Western Ontario SYNASC 2005

  2. Overview • Motivation • Parametric Polymorphism Implementation • Generalizing A Numeric Benchmark • Language Issues • Results • Potential Optimizations • Conclusion

  3. Motivation • Increasing demand for generic code • Scientific code requires high-performance making optimizations very important • Generic code – not as fast as specialized code • No tools to measure performance of generic code • Benchmarks – tool to measure the performance • SciGMark – benchmark for generic code • Compilers – optimize the generic code – performance close to hand specialized code

  4. Parametric Polymophism Implementation • Some languages with support for Generics • Aldor, C++ • Java, C# • Some types can be given as parameters • Implementations • Homogeneous: Java, C# • Share the generic code • Example: Vector<Integer> → Vector with elements of type Object • Heterogeneous: C++, C# • Specialize the generic code • Example: std::vector<int> → new specialized class

  5. Generalizing A Numeric Benchmark • SciMark 2 • Polynomial Multiplication • Implemented in Aldor, C++, C#, Java

  6. SciMark 2 • Fast Fourier transform – 1024 • Complex arithmetic, shuffling, non-constant memory reference, trigonometric functions • Jacobi successive over-relaxation – 100x100 • Typical access patterns in finite difference applications • Monte Carlo integration • Random number generator, function inlining • Sparse matrix multiplication – 1000, 5000 non-zero • Indirection addressing, non-regular memory references • Dense LU factorization – 100x100 • Dense matrix operations

  7. Class SOR { double[] array; } R R a(R o) double Class SOR < R extends IRing<R> > { R [ ] array; } + DoubleRing void ae(R o) From SciMark 2 to SciGMark • SciMark – double hardcoded • Arrays are of type double • Any change – extensive modifications to the code • SciGMark – classes are parametric • Change representation – minimal code changes • Double becomes parameter R

  8. Basic Generic Types • IRing • Provides operations for addition, subtraction, multiplication, division – mutable, non-mutable • Conversions to and from int and double • Factories to produce new elements of these type • DoubleRing – wrapper for double • Implements IRing • Complex • Implements IComplex (simple extension to IRing) • Complex<R extends IRing<R>> implements IComplex<Complex<R>,R>

  9. Generic Tests • GenFFT • Uses R: Complex<DoubleRing> • Complex numbers – two consecutive entries in the array • Depending on the application – different representation (e.g. Hermitian matrix) • GenMat, GenLU • Use R: DoubleRing • The classes contain more methods – the whole class contains a type parameter • GenSOR, GenMonteCarlo • Use R: DoubleRing • Have single static method with a type parameter

  10. Polynomial Multiplication • 40 coefficients • Dense representation unidimensional array • Regular memory access, temporary objects creation (memory allocation) • Implementation • DensePolynomial • DensePolynomialG <E extends IRing<E> > implements IRing<DensePolynomialG<E> > • SmallPrimeField • Represented by an int • SmallPrimeFieldG implements IRing<SmallPrimeFieldG>

  11. Specializing Polynomial Multiplication • The code was initially implemented using generics • Inlined all the calls to SmallPrimeField • Replaced all the instances of SmallPrimeField with int • Essentially the inverse of the operation performed to “generalize” the SciMark • No changes to the algorithm – all changes could be performed automatically

  12. Language Issues • Java • No operator overloading • Homogeneous – erasure technique – subclassing • Implemented at language level – no virtual machine support – limitations – require object factory • Type inference for generics is invariant – Pass the type as argument • Complex <R extends IRing<R>> implements IComplex<Complex<R>,R> • C# • Reference types (homogeneous) – Java; primitive types (heterogeneous) – C++ • Structures instead of classes – structures in collections are boxed

  13. Language Issues • C++ • Heterogeneous • Parametric polymorphism (templates) macro processor • No bounded polymorphism • No way to test the generic class until is instantiate • Aldor • Homogeneous • Supports dependent types • Polymorphic types constructed using domain constructing functions

  14. SciGMark Results • Results in MFlops • Testing environment: • Pentium IV – 3.2GHz (1MB cache), 2 GB RAM • Windows XP SP2 • Cygwin/GCC 3.4.4 • Sun JDK 1.5.0_04 • Microsoft .NET v2.0.50215 • Aldor 1.0.2

  15. Test C++ Java C# Aldor Size Gen Spe Gen Spe Gen Spe Gen Spe FFT 59 365 23 321 7 242 1 340 1024 SOR 71 419 66 681 22 417 15 417 100x100 MC 46 65 22 26 28 62 90 203 N/A MM 87 739 111 410 39 477 4 485 1000, 5000 LU 103 780 74 982 18 403 5 553 100x100 PM 62 365 48 227 28 321 6 156 40 Comp. 71 434 57 441 24 320 20 359 N/A SciGMark Results

  16. Test C++ Java C# Aldor Size Gen Spe Gen Spe Gen Spe Gen Spe FFT 59 365 23 321 7 242 1 340 1024 SOR 71 419 66 681 22 417 15 417 100x100 MC 46 65 22 26 28 62 90 203 N/A MM 87 739 111 410 39 477 4 485 1000, 5000 LU 103 780 74 982 18 403 5 553 100x100 PM 62 365 48 227 28 321 6 156 40 Comp. 71 434 57 441 24 320 20 359 N/A SciGMark Results

  17. Test C++ Java C# Aldor Size Gen Spe Gen Spe Gen Spe Gen Spe FFT 59 365 23 321 7 242 1 340 1024 SOR 71 419 66 681 22 417 15 417 100x100 MC 46 65 22 26 28 62 90 203 N/A MM 87 739 111 410 39 477 4 485 1000, 5000 LU 103 780 74 982 18 403 5 553 100x100 PM 62 365 48 227 28 321 6 156 40 Comp. 71 434 57 441 24 320 20 359 N/A SciGMark Results

  18. Aldor Results • Testing environment: • Pentium IV – 3.2GHz (1MB cache), 2 GB RAM • Linux Fedora Core 3 • Aldor 1.0.2 • Stanford benchmark • Aldor’s performance can be almost as good as C++

  19. Test Aldor C++ Time Iterations Time Iterations Permutations 0.43 23400 0.37 26901 Towers 0.58 17297 0.21 46924 8-Queen 0.52 19700 0.45 21987 Mat Mult 0.65 15386 0.20 49155 Puzzle 2.89 3484 2.16 4626 Quick Sort 0.79 12538 0.66 15214 Bubble Sort 0.74 13526 0.53 19089 Tree Sort 1.00 10 2.00 10 FP Mat Mult 0.69 14342 0.49 20355 Oscar FFT 0.38 26838 0.24 40719 Comp FP 1.07 1.05 Comp int 1.43 1.29 Aldor Results

  20. Potential Optimizations • 6-18 times performance improvement • Specialized code • Same algorithm • Generic types replaced by specialized types • Eliminate generic wrapper objects – primitive types

  21. Test Case Aldor • Domain producing function: PolynomialVect(C: Ring) == add { Rep == Vector Polynomial C; (f: %) + (g: %): % == { res := new(#f); rf := rep f; rg := rep g; for k in 1..#f for i in rf for j in rg repeat res(k) := i + j; per res }}PC == PolynomialVect(Complex DoubleFloat);PQ == PolynomialVect(Rational);

  22. Test Case Aldor Domain producing function: PolynomialVect(C: Ring) == add { Rep == Vector Polynomial C; (f: %) + (g: %): % == { res := new(#f); rf := rep f; rg := rep g; for k in 1..#f for i in rf for j in rg repeat res(k) := i + j; per res }}PC == PolynomialVect(Complex DoubleFloat);PQ == PolynomialVect(Rational);

  23. Test Case Aldor • Specialize the domain producing function PC == add { Rep == Vector Polynomial Complex DoubleFloat; (f: %) + (g: %): % == { res := new(#f); rf := rep f; rg := rep g; for k in 1..#f for i in rf for j in rg repeat res(k) := i + j; -- ‘+’ from Complex per res }}

  24. Optimize Data Representation • Scalar product of vector of complex numbers dot(u: Vector Complex R, v: Vector Complex R): Complex R == {s: Complex R := 0; for i in 1..n repeat s := s + u.i*v.i; return s;} dot(u: Vector Complex R, v: Vector Complex R): Complex R == {x: R := 0; y: R := 0; for i in 1..n repeat {x := x + real(u.i)*real(v.i) - imag(u.i)*imag(v.i);y := y + real(u.i)*imag(v.i) + imag(u.i)*real(v.i); } return complex(x,y);}

  25. Conclusion • Generics important for scientific computing – rich mathematical models – easy to implement with generic code • Need a tool to measure the compiler ability to produce efficient code • We have seen difference of 6-18 times between generic and specialized code – room for improvement in compilers capabilities • Presented some optimizations ideas • http://www.orrca.on.ca/benchmarks/scigmark/1.0/

More Related