1 / 41

K5/VSSP data processing system of small cluster computing at Tsukuba VLBI Correlator

K5/VSSP data processing system of small cluster computing at Tsukuba VLBI Correlator. M. Machida , M. Ishimoto, K. Takashima Geographical Survey Institute, Japan T. Kondo, Y. Koyama National Institute of Information and Communications Technology, Japan. contents.

pepin
Download Presentation

K5/VSSP data processing system of small cluster computing at Tsukuba VLBI Correlator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. K5/VSSP data processing system of small cluster computing at Tsukuba VLBI Correlator M. Machida, M. Ishimoto, K. Takashima Geographical Survey Institute, Japan T. Kondo, Y. Koyama National Institute of Information and Communications Technology, Japan contents • major issues at Tsukuba Correlator in 2005 • architecture of K5 correlation system • distributed computing • plan for future

  2. Major issues of Tsukuba VLBI Correlator in 2005 • upgraded correlator system from K4 to K5 • got more efficient with distributed computing by K5 • has operated K5 so far as a software correlator in IVS

  3. outlook of K5/VSSP correlation system at Tsukuba Correlator correlation servers data servers management computer

  4. system upgrade at Tsukuba Correlator K5 stations K4 K5 correlator K4 current position 1997 2005 Mar. 2006 Jan.

  5. system upgrade at Tsukuba Correlator K4 K5 • hardware correlator • on site computing • sequential process • software correlator • distributed computing • parallel process

  6. K5/VSSP at station side hard disk data file 8 ch reflector 7 ch d 6 ch maser P-cal 5 ch S-band LNA 4 ch 3 ch c 2 ch Band-pass filter 1 ch 1st local down converter 8 ch 7 ch distributor b 6 ch 5 ch 2nd local video converter X-band 4 ch 3 ch Time tag A to D conversion a 2 ch 1 ch Differential counter GPS

  7. media for storing data ; from MT to disk K4 K5 magnetic tape cartridge disk cartridge 250GB×~8 units 100GB×~15units 10cm

  8. K5/VSSP is enough technology to join with eVLBI ONSALA WETTZELL internet Tsukuba correlator Tsukuba 32-m

  9. Manegement computer Current form of K5/VSSP at correlator side hard disk Correlation servers (with 3GHz Intel Xeon dual CPUs) data servers correlator Data server correlator correlator Data server correlator Data server correlator correlator Data server correlator correlator

  10. Data disk mounting Data server / disk Data disk sd1 mount JADE0512 JADE0512 sd2

  11. Disk mounting Data disk Port 1 Data disk Port 2

  12. Disk mounting Data servers (8) Drive unit insertion Disk cartridge (16 units)

  13. Management computer Correlator server File sharing logic / / home home vlbi vlbi ipvlbi ipvlbi work work auxiliary files schedule schedule log log Data server NFS apri apri cout cout data data / disk disk disk serv?? serv?? serv?? Raw data sd1 sd1 sd1 NFS sd2 sd2 sd2

  14. System comparison on operating/re-engineering issues K4 K5 high cost low cost maintainability extensiveness poor easily achievable improvability poor easily achievable efficiency(1/PF) 0.6 1.21 (intensive) e-VLBI ----- compatible Mark5 ----- convertible

  15. Demand for intensive and domestic sessions at Tsukuba Correlator session year

  16. efficiency comparison with PF from intensive session total length of correlation processing factor = total length of a session 4.0 K4 K5 3.0 PF 2.0 1.0 0.0 Apr Jun Sep Nov Feb May Jul Oct 2004 2005 Month

  17. components of K5 correlation system in current form K5 correlation system equipment management computer correlation servers data servers software kernel program intelligent application “PARNASSUS”

  18. improvement of K5 correlation system preliminary form current form • single task control • manually input • multi tasks control • graphical user interface kernel program kernel program + intelligent application

  19. Data processing flow Data (Y) Data (X) cor aprior delay & rate komb Correlator output makedb apri_calc sdelay delay residual Schedule file Mark III database

  20. Data processing flow Data (Y) Data (X) cor aprior delay & rate komb Correlator output Distributed computing makedb apri_calc sdelay delay residual Schedule file Mark III database

  21. Developing PARNASSUS current position building databases bandwidth synthesis correlation execution apriori calculation Mar., 2005 Sep., 2005 Jan., 2006 Mar., 2006

  22. data handling 8 7 6 ( in preliminary form ) 5 Waiting for task operator Correlator unit 4 1 tasks 2 3

  23. in processing task data handling 8 7 ( in preliminary form ) 1 6 Waiting for task 5 2 3 4

  24. in processing task data handling 8 ( in preliminary form ) 1 2 7 Waiting for task 6 3 4 5

  25. in processing task data handling 1 ( in preliminary form ) 2 3 8 Waiting for task 7 4 5 6

  26. data handling 8 7 6 ( in current form ) Intelligent application operator Waiting for task 5 request tasks 4 1 2 3

  27. data handling 8 7 6 ( in current form ) Intelligent application operator 5 tasks 4 1 2 3

  28. data handling ( in current form ) 8 7 Intelligent application operator 6 in processing task tasks 5 3 4 1 2

  29. Correlation servers Management computer

  30. Management computer in processing task 1 in processing task 2 in processing task 3 ・ ・ ・ in processing task 8

  31. Management computer in processing task 1 in processing task 2 in processing task 3 finishes task 4 ・ ・ ・ finishes task 7 in processing task 8

  32. in processing task 1 Management computer in processing task 2 in processing task 3 task 9 In process task 9 ・ ・ ・ task 10 in processing task 8

  33. distributed computing in searching fringe ; lag window size in subsidiary range [usec] A , B ; end points in whole search range A 0 B

  34. distributed computing in searching fringe A 0 B

  35. Main frame of intelligent application menu project Disk mount Parameter setting apriori delay&rate correlation File conversion Log out PARNASSUS (P)rocessing (A)pplication in (R)eference to (N)ICT's (A)dvanced (S)et of (S)oftwares (U)suable for (S)ynchronization

  36. Parameters setting window

  37. Fringe search window

  38. Plan for future • system extension to fasten

  39. System extension terminal PARNASSUS result request operator high expectation now ! Management computer With more correlation units, Data processing would get faster than ever. Data servers Correlation units

  40. conclusion • It got efficient to process data with K5 correlation system, for it enabled us to handle data in distributed computing. • With more correlation units, data processing will get faster than ever.

  41. private advisory group from VLBI team in GSI developer K. Takashima operation manager (GSI) M. Machida technical staff (GSI) K. Nozawa (AES Co.,Ltd) M. Ishimoto technical staff (GSI) K. Takano system engineer (AES Co.,Ltd) • planning • primary design (rough sketch) • requirements definition • general and functional specification • requirements analysis • structural design • coding and programming • test and debug • installation • instruction

More Related