ST + Tracking in 2008: What did we learn ? M. Needham EPFL

1. ST + Tracking in 2008: What did we learn ? M. Needham EPFL ‘Had we succeeded well, We had been reckoned ‘mongst the wise: our minds Are so disposed to judge from the event’ Euripides

2. Outline TT in 2008 IT in 2008 • The detector • Lessons learned • The team Cosmics • Studies performed • Lessons ST group extremely active on software Many generic tools being developed that are being used by wider LHCb community Evidenced by large number of commits of fixes + improvements to cvs See also ST analysis meetings in December 2008 + February 2009 TED running • Studies performed • To do • Lessons General Lessons • Hopes for the future

3. J. Van Tilburg TT Status Broken bonds Dig board show problems recently. Broken bonds: glued Broken bonds: modules removed HV problem 97 % of the detector working Last week start of repair work on low power optical links 71 VXCELs beings exchanged Start of run aim to have 98 - 99 % working detector

4. TT Timing from TED Calculated timing for backward events using fibre length: BTW, getting the signs right in this table is not so easy. … agrees nicely with timing from Landau MPV fit

5. C. Salzmann TT alignment with TED • Extrapolated Velo tracks to TT → VeloTT tracks • Alignment in Tx for the full station. • Unbiased residuals ~ 500 micron. • → extrapolated from Velo • Survey Database • Module position → from survey in pit. • Sensor position → from metrology in lab. • Main effect comes from survey of modules • Next: align also in Ry (long lever arm).

6. C. Salzmann TT alignment with TED • A-C side asymmetry about ~62% • Christophe had a look at the possible causes: • Timing A-C side • Clear effect when summing tracks in different spills. • Still to do: merge spills in TT to avoid double counting. • Data quality • Clear effect when excluding runs with error banks • Still to do: recover partial banks. • Geometrical acceptance • Clear effect when seen when taking beam pipe into account. • Use hit expectation tool to calculate efficiency (works) • Did not look at pattern recognition, but probably above effects are enough to explain asymmetry. Rough estimate of effects: Timing: 62% → 41% Error banks: 41% → 26% Acceptance: 26% → 22%

7. IT Status IT Status now, few more problems developed since end of run 2.5 % dead 2 % noisy Known this small lost leads to 1 % loss in tracking efficiency

8. Fixes for 2009 • Backplane problems + weakest optical links will be replaced • Precise # still to be worked out, likely to be ~ 83 • We start with the worst service boxes and see how it goes • Always a risk that in attempting to repair we cause more damage • N.B. repair criteria is a matter of judgement of time/effort/parts versus time • ~ 100 more links in TT , ~ 100 more links in IT could be candidates for future repair • During run careful monitoring to check if more link problems develop • In parallel new Tell1 firmware developed • Can recover from loss of synchronization caused by ‘link’ loss • Data for that event on effected link is still lost • Error bank is sent and recoverary mode needed

9. IT Cosmic Running Maybe 500 k events ? but IT/OT opened, alignment changed….around 18th August (according to emails, hints in logbook). Older firmware etc, Not considered July- 21st August ? OT closed 2000 k good IT events at least ! Care: OT not present in all runs Runs with RICH TAE settings removed Runs with only Central spill TAE window changes run to run Runs with calibration events in data stream Not all data copied to castor 5 th September 400 k good IT events OT in for all runs Only cosmic runs good for TT 1 Tell1 board in IT1A missing Six bad Beetles not masked [rec mode needed] 19 th - 21st September

10. IT Cosmic dataset 19/20th ~ 180k events 21/21st ~ 200 k events run 32406 50 k events 5th September run 32385 120 k events 5th September run 32197 2 k events 4th September (no events) 32181 39k events 5th September run 31959 5k events 1st Sept run 31732 50 k events 30th August 2 bx run 31731 30 k events 30 August 2 bx run 31729 51 events 2 bx run 31720 37k events 2 bx run 31715 8 k events 2 bx run 31714 20 k events 30th August run 31557 180 k events 29th August run 31026 20 k events 26th August run 30664 40k events 21st August run 30662 20k 21st August run 30654 90k 21st August run 30653 8k 21st August run 30648 23k 21st August run 30636 11k 21st run 30639 13k 21st run 30640 1.3k 21st run 30641 15k 21st run 30645 6 k 21st run 30659 7k 22nd run 30660 11k 22nd run 30661 17k 22nd run 30666 41k 22nd run 30669 22k 22nd run 31300 50k 27th run 31537 5k events run 32432 10 k events 5th Sept run 31542 200 events 29th run 31723 76k event, 30th run 31716 1k events, 30th run 31719 19 k event 30th run 31725 10k event 30th run 31727 10 k events 30th run 31626 60 events 30th run 31733 10k events 30th run 31734 37k events 30th run 31736 4k events 30th run 31737 62k events 30th run 31739 32k events 30th run 31744 59k events 30th run 31747 9k events 30th run 31760 1.5 kevents 30th run 31761 0.2k events 30th run 31789 [no OT] 21 k run 31791 0.2 k 31th run 31792 83 k 31th run 31793 11k 31th run 31794 [calib] 85k 31th run 31795 40 events 31st run 31797 120 k event 31st run 31817 10k events 31st run 31821 77k events 31st run 31838 31k 31st run 31841 85 k 1st Sept run 31842 1k 1st Sept run 31843 14k 1st Sept run 31844 8k 1st Sept run 31845 1.8k 1st Sept run 31846 66 k 1st Sept run 31848 5k 1st Sept run 31849 24k 1st September run 31856 40 k 1st September run 31860 0.1k 1st September run 31874 150 k 1st September run 32381 15 k events th September • 2.55 million events in 15 days at 10 Hz • Could have collected 13 million • Few days not running on cosmics • IT missing/giving bad data [10 %] • Room for improvement ! • Filter to set of 27 k IT events • Filtering was a lot of time and effort • Really need to understand the logbook • Changing run conditions • Calibration data in the datastream • OT opening/closing around August 22nd

11. Cosmic Results Unbiased residual 70 candidates, Calo confirmed through two boxes (300 MeV 'energy’) Mainly on A or C side due to geometry 2 pass through TT ty

12. Cosmic Alignment Align for Tx: Fix X1U in T1, T2 X2V in T3 Require 10 hits in layers being aligned survey ~270 m more entries ideal ~390 m

13. Cosmic Lessons • Many cosmic tools developed: filtering, spill merging, ITGenericTracking • Important to have long runs with stable conditions • Fighting the noise is important. Aim to reduce this year • Good time alignment: reduce spills needed to 3 • Remove hotspots (working header correction ?) • Important to have all detectors working to confirm tracks • plots of track parameters important checks of tracking quality To collect ~ 500 IT cosmics + perform cosmic alignment of IT next year, Need: • 2 - 3 weeks continuous cosmics running, stable detector collect 10 million events • Working OT + Calo for confirmation • Working book-keeping + Data Quality • HLT alley or offline express stream to reduce to ~ 10,0000 or less • Full Cosmic detector reconstruction CRAB In the ST mini-review management urged us to concentrate on TED [~August]. Still possibility of Detector Open TED in June. No effort on ST cosmic alignment from now on

14. TED: First Steps First alignment in x using histograms Stereo is messy, ladders not surveyed Verified correctness of survey Position ladders + boxes to accuracy 20 microns Work to appear soon in LHCb note

15. TED: First Steps Width tells us about ladder misalignments [If we know the momentum] If dominated by primaries from TED p ~ 20 GeV TED-like MC particle gun, 500 muons/event Vary p and apply same procedure as data Either: p ~ 5 GeV and no ladder misalignments p > 20 GeV + 100 micron ladder Misalignments Or something in between

16. Tracking Understanding • Looked to be a problem in T/B box survey • In fact tracking problem • Two competing conventions for x in x search • x = x(yMid) [LHCb 2007-037] • x = x(y = 0) [LHCb 2008-042] Surveyed, T2, core 200 microns y box y = 0 cm Survey indicates ladder rotations ~ 1mrad • Reverting to xMid convention better results • x=x(y=0) relies on incorrect assumption • Problem still exists in standard tracking code Default, T2 core 130 microns

17. TED Tracking • Assumptions: • x ladder position known to ~ 150 micron • y ladders worse [no survey], guess ~ 250 micron • Search windows: • x +/- 0.6 mm • y +/- 10 mm [4  0.25  sin 5o ] • Some tracks come from the TED: • tx < 0.02 mrad, ty < 0.05 mrad • - 3 < x(TED) < 7 m, |y(TED)| < 5 m • Ensure uniqueness: • Many outliers in search window from other tracks • Many incompatible candidates, choice needed • Two hits from a sector in window, split candidate in two • Delayed decision: • Follow all valid candidates to the end [ie full Kalman fit] • Many incompatible candidates, choice needed • Winner takes all algorithm [TrackCompetition] • Rank tracks by # hits, then fit c2

18. TED Tracking Unbiased residuals C side A side Convincing for Top/Bottom Side boxes: longer ladders Worse combinatorics ? More suspectible to misalignment ?

19. TED MC TED occupancies: T/B boxes Ghost rate ~ 0.8 % L/R boxes Ghost rate ~ 6.2 % 20 GeV muons TED occupancies: T/B boxes efficiency ~ 98 % L/R boxes efficiency ~ 81 %

20. Matching to TT MC Top-IT TTb Clear signals seen Can be used for understanding alignment Imply p in the 10 - 20 GeV range Bottom-IT TTa

21. TED Alignment L. Nicolas Stereo layers worst Aligned ? Survey +first alignment in x y box alignment

22. TED Alignment L. Nicolas TT validation TTaU, Top IT box Align

23. TED Alignment L. Nicolas In fact bugs in my hand-made TED alignment, where I had moved stereo Layer in pair wrongly with respect to x When this is corrected both methods agree level 20 microns Still some subtle weak modes ? Things improve, e.g x ladder residuals ~ 150 to ~ 100 microns Side boxes no convincing results obtained

24. To Do • Speed improvements in generic tracking • Documentation: alignment note, ITGenericTracking tracking note • Alignment studies: • ~ finished • to be documented in Louis thesis • A/C-side sensor survey: will be added by Geraldine • Probably small effect, but every little helps • Still few tools/studies to be done: • Matching to Velo • Use TT as confirmation device • Not done yet due to manpower constraints

25. Lessons Care needed with assumptions in Pattern Recognition • Projecting to xMid assumes that ladder rotations small + they are not • Problem will never be fixed by better alignment • Finding this problem took a lot of time Hotspots + memory management • High occupancies necessary to make full fit of all valid candidates • High densities: memory management the issue • Only have ~100 events, so spending processing 10 minutes on hot event is ok, and I could fix this anyway • Some events ~ 100 - 200 k tracks made • No obvious hot spots [everything is hot…] • Less a problem in MC…. • Solution: only use memory really need • Kill obvious clones early. Use TrackCompetition algorithm • Standard clone killer tried but nested double loop means slow • Track competition: two single loops, fast, generic

26. Lessons Monte Carlo useful • Code developed + debugged without Monte Carlo • Proved very useful for understanding to have Monte Carlo samples • A lot of work to generate [tune particle gun, run Gauss, tune Boole,…] • Easy to do this for ST • Other detectors: need to understand simulation/timing etc Go generic… • Code looks for straight lines + is totally generic • Developed for TED, tested on cosmic data, tested on beam gas MC • Will work for beam gas, magnet off, collisions @ ~ 10 m • Easy to tune for different conditions with few options Can go along way with simple residual plots • But time consuming, a lot of work by hand • Takes ~ day or two to make layer/box alignment • A lot of room for error: editing XML by hand +need XML not SQL • …XML is not so nice if trying to combine with Velo • Try to automize ?

27. General Lessons • Real data is a lot tougher than Monte Carlo • Its easy to get bogged down trying to understand problems • That makes it hard to work on several things in parallel • Be well prepared [a lot of work preparing ST software for 2009] • Fixes have to be made quickly • Issue of projecting to y=0 reported in December, but not yet fixed • Find it very suprising will take 1 month for other subdetectors to deal with delta-rays • For track confirmation/quality studies need inter-detector interaction • Make sure technical problems solved quickly • Make sure improvements fed in quickly • Tools still missing to measure the detector/tracking efficiency + resolutions

28. ST Development • Have a lot of generic tools developed this year • STEventMerge, ITGenericTracking, STCosmicFilter • Shutdown work: Make code development easier • New base class, helper functions/short-cut, print functions, monitoring • Cluster selectors, clusters to lite clusters , …. • Optimize for ease of use, avoid hacking ‘speed’ • Make it easy to tune MC • Cluster killers in Brunel, changes to turn in delta-rays in Gauss • Documentation: Updated and improved • Aim: make code development faster and simple, ready for everything • Always aim to develop generic code that can be used elsewhere Higher level tools: efficiency calculation, resolution model, event merging • Discussed in tracking meetings since 2007, no progress on common tools • ST group will develop own tools building on the flexible framework we have developed over last months

29. Summary • The tracking must be prepared to react quickly to an imperfect detector • Now is the time to ensure we have the tools to understand the detector • Flexibility is crucial: react to the detector as it is, not as you would like it to be. Don’t make assume anything. Being fast but locked into one set of assumptions is a recipe for disaster. • 2008 the disaster that never was, since we didn’t run: • Understanding link loss problems/developing new firmware took ~ 1 month • Understanding desychronization problems related to TTC fibers took time • Always would be new problems [new bad links, broken bonds, failing Tell1s …] • Debugging would require access/removing IT from data taking • Hard to imagine how detector eff would be more than 90 % • Fold in imperfect alignment, imperfect Velo, IT, ….. • Tracking must be able to respond quickly to possible problems during long run • We hope no more links develop optical power problems, but it could happen • We hope TT broken bond problem will not develop further • Once we close the detector, no more hardware fixes possible

30. Backup

31. ITCosmic dataset 1st round of stripping, present in T-Rec October 15th 1.1 million events: • Second round of stripping: • Better understanding of when OT closed on the 21st August • Better understanding of the logbook, recover many runs around 31700-31800 • Recover many of the runs with calibration events mixed in the data • Take all runs, even those with less than 10k events Found another 1.45 million events, now have 2.55 in total • 70 - 80 tracks depending on the cuts • Took 3 days of studying logbook, finding data, understanding… • The largest reprocessing of real data performed so far on LHCb ? • N.B, OT [A or C] not available in many of these runs

32. IT Cosmic dataset 2 tracks going through 3 IT boxes Too few, to do anything with apart from making nice plots Any two boxes, mainly A or C side Tried to use for layer alignment Statistics too poor, TED better ~70 tracks going through 2 IT box ~1000 tracks going through 1 IT box Not looked in detail OT geometry problems, lack of manpower Look at residuals try to align IT to OT

33. IT Cosmic Alley Count clusters with > 12 ADC Reject n = 0, n > 1000 > 3 clusters (15 ADC) In 1 spill +/- 3 mm window > 7 clusters 3 in one box + spill > 50 clusters Accept Reduce 2.6 million events to ~27, 000

34. X search: Box Alignment Using survey [box residuals]  ~ 0.23 mm  ~ 0.13 mm  ~ 0.27 mm  ~ 0.43 mm

35. MC Expectation Running ITGeneric tracking on DC06 beam gas MC… MC find 0.08 mm [1.4 * binary resolution] If survey accurate to 50 micron expect 0.09 mm 100 micron expect 0.13 mm Hmm…we are far from understanding IT alignment

36. X search (return) I only move the boxes in T2, not the layers, lets try that… Top, move 0.1, 0.3 mm  ~ 0.43 --> 0.33 mm A-side, move 0.14, 0.03 mm  ~ 0.23 --> 0.16 mm C-side, move 0.02, 0.03 mm  ~ 0.13 --> 0.125 mm Bottom, move 0.14, 0.9 mm  ~ 0.27 --> 0.25 mm ITT2TopX1, ITT2BottomX2 are the ‘bad’ guys

37. Data available Going through the logbook is like an episode of CSI • Pages of ‘conditions same as the last run…’ • Which detectors are in the run ? • What are the conditions ? TAE ? How many spills ? RICH TAE ? • What is the run number ? When did it start/stop ? Some things not clear in logbook or not in logbook • I know (from email) that OT opened/closed ~18th - 21st August • Hard to know this from the logbook • Calibration data in the physics data at start of September • I know this, reported on mailing lists at time, … • Where is the data ? Copied to castor or not ? • Whats the directory ? • How does the non-expert know all this ?

38. IT Cosmic dataset 10 k events, in POOL_ROOT format

39. Merging spills • Timing changes run to run • Forward + backward cosmics have different timings • Solution: merge spills using the algorithm: STEventMerge • In case two clusters have same key: take one with highest charge • In case of overlapping clusters: take one with highest charge • STCluster class extended to have information on spill • Typically ~ 3-4 noise clusters/spill • Merging means we have ~ 15- 20 clusters in five spills • 5 spills are needed: lose ~ 10 % tracks if take only 3 spills

40. The Golden Track All clusters > 10 ADC shown A-side Run 31720 30th August Very clean event, track with 10 hits Have one hit just fail to include on track…..

41. TED MC Biased residual Unbiased residual

42. IT Overlaps Look for overlaps between boxes Take Top/Bottom tracks Residuals to Side boxes Early on no signal seen With final survey geometry overlaps ! Bad combinatorics Convincing signal for A-side/Bottom A-side with Bottom C-side with Top