1 / 8

Expanding Software Collaboration beyond HEP

Expanding Software Collaboration beyond HEP. Fons Rademakers. Main Problem. Each science/profession/field has their own computing culture (from Univ up to professional life). HEP Case. HEP community is quite small but has unique use cases: E xtremely big data Very large code bases

dylan-potter
Download Presentation

Expanding Software Collaboration beyond HEP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Expanding Software Collaboration beyond HEP Fons Rademakers

  2. Main Problem • Each science/profession/field has their own computing culture (from Univ up to professional life)

  3. HEP Case • HEP community is quite small but has unique use cases: • Extremely big data • Very large code bases • Very long experiment life times • A lot of low level programming to maximize usage of resources • Try to use everything new we can… bleeding edge • Some stuff we can share: ROOT, G4, XROOT, EOS, cernvmfs …

  4. Others • Biologists • Minimal low level computing experience • SQL queries • One button interfaces • Typically a large usage impedance mismatches

  5. R-e-Inventing the Wheel… • R started as a statistical package for engineering students with a simple, slow custom language for small in-core data sets • After 22 years these limitation are still there • Typically many social science, engineering and biology (pre genome) data sets are much smaller than ours, and perfectly fine for R • For ROOT joining R at the time was not an option as R was not addressing any of our issues and vice versa for R

  6. Why do others not profit from HEP • We disseminate our work badly, only: CHEP, ACAT, ... • Typically not allowed (or very difficult) to visit conferences in other fields (at least in PH-SFT) like VLDB, Super Computer, Google I/O, WWDC, etc, etc

  7. Results of our Self Imposed Autism • Google reinventing the wheel by making Dremmel and Map/Reduce • CERN reinventing the wheel by not not learning enough from external sources • Not rewarded to collaborate on projects outside HEP (there is also a fear that we will end up having to provide the support due to our long experiment life time) • At most Our stuff typically trickles down in other fields by HEP people moving on to other fields (e.g. finance)

  8. What do we need to Collaborate • Spend time and effort to adapt our solution to other communities environment • money and effort not spend on HEP (but having a larger community pays back in a more robust system) • Spend time and effort to adapt their solution to our environment • typically the other community is larger and has less interest to cater to HEP • Be the ones willing to maintain the stuff as we probably will be using it the longest

More Related