1 / 23

HEP Data GRID in Japan

KEK High Energy Accelerator Research Organization. HEP Data GRID in Japan. Yoshiyuki Watase Computing Research Center KEK. HEP Data Grid Workshop Nov. 8-9 2002 at CHEP KNU Korea. Outline. Network Infrastructure HEP Grid Activities in Japan ATLAS Belle R&D of GFarm

birch
Download Presentation

HEP Data GRID in Japan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. KEK High Energy Accelerator Research Organization HEP Data GRID in Japan Yoshiyuki Watase Computing Research Center KEK HEP Data Grid Workshop Nov. 8-9 2002 at CHEP KNU Korea

  2. Outline • Network Infrastructure • HEP Grid Activities in Japan • ATLAS • Belle • R&D of GFarm • Related “Virtual Laboratory” activities • Collaboratory • GAN: Global Accelerator Network • Funding status for Grid • Conclusion HEP Data Grid Workshop @ KNU

  3. Network Infrastructure 1 • SuperSINET (Science Network) • Jan. 2002, fiber optic network was provided by NII for research in Univ.’s and Institutes ( ~ up to 40 nodes). • 10 Gbps IP backbone and many p-to-p GbE links for research groups: HEP, Astronomy, Bioinformatics, Nano-tech, GRID computing • HEP • MPLS-VPN of HEPnet-J for ~25 university groups by sharing the 10 Gbps IP backbone • GbE links from KEK to Univ. ‘sTohoku, Tokyo, Nagoya, Kyoto, Osaka, Tsukuba, Titech(Tokyo Inst. of Tech), ICRR(Inst. Cosmic Ray Research) NII: National Institute for Informatics HEP Data Grid Workshop @ KNU

  4. Tohoku U OXC NII Chiba IP 10 Gbps WDM path NIFS IMS IP router Nagoya U NIG Nagoya Hub Hokkaido Osaka Hub Tokyo Hub KEK Osaka U Tsukuba US，EU Kyoto U Korea NII KyushuU ISAS U Tokyo ICR Kyoto-U Internet TITEC NAO Waseda IMS U-Tokyo Network Topology of SuperSINET HEP Data Grid Workshop @ KNU

  5. Network Infrastructure 2 • East bound: to US, EU • NII will implements 2.4 G x 2 terminated at NY on Jan. 2003. • We need transit line between NY and StarLight for connection to CERN with end-to-end GbE. • We need peering to ESnet at NY. • West bound: to Korea • Genkai project will provide ~ GbE connection between CHEP(KNU) and KEK. • Being demonstrated at APAN Fukuoka Meeting Jan. 22, 2003 HEP Data Grid Workshop @ KNU

  6. Network Infrastructure TEIN NII- NY TRANSPAC Taiwan-US Japan (NII) -NY : 2.4G x 2 Jan. 2003 Japan – US: 622M x 2 (TRANSPAC) Korea – US: 45 M Korea – Japan: 2.4G Jan. 2003 Korea – Europe: 2 M (TEIN) China(IHEP) – Japan(KEK): 128 kbps (HEP) China – US: 10 M Taiwan – Japan: 155 M Taiwan – US: 622 M (Dec. 2002) HEP Data Grid Workshop @ KNU

  7. LHC/Atlas-1 • Atlas Regional Center • Being set up at ICEPP (Int’l Center for Particle Physics, U. Tokyo) • Installation of PC farms and Storage • PC farm: 78 x P-III 1.4GHz + New PC farm :214 x Xeon 2.8 GHz (early 2003) • Tape TLO 7TB ( CASTOR) + 20-30 TB Disk data server (early 2003) • Data Challenge DC1 • Phase-1 (~ Sep. ): Monte Carlo production 5x107 events • Phase-2 ( Nov. ~ ) : Generate events with event-overlapping HEP Data Grid Workshop @ KNU

  8. LHC/Atlas -2 • Globus installed in PC’s at KEK and ICEPP • Testing Globus environments • Grid environments for LHC exp. : LCG – 1 • Start test on Nov. 2002 • Testing between ICEPP and KEK • Testing between ICEPP and other sites in EU/US ( 2003 ) • Remote data access via GbE • Atlas simulation data at ICEPP being stored in HPSS storage at KEK through GbE link • Test Resources: ICEPP PC farm 78 cpu’s KEK PC farm 100cpu’s HPSSS ~ 20 TB HEP Data Grid Workshop @ KNU

  9. LHC/Atlas - 3 Data Transfer Throughput Test KEK ICEPP, Tokyo GbE (~60 km) IBM RS6000/SP HPSS PC Farm PC Farms Single stream netperf: ~70 MB/s ftp : ~ 30MB/s simulation data ~ 10 TB netperf test over 2 streams Data Transfer Throughput Test over GbE 1. netperf test 2. ftp 3. HPSS access over GbE Local access ~ 30 MB/sec WAN access ~ being measured HEP Data Grid Workshop @ KNU

  10. KEKB/Belle • B Factory experiment • Luminosity reached to 8 x 1033 /cm2/s in Oct. 2002: updating the world record and still improving • Data storage rate = ~ 500 GB/day • Data analysis • Resources at KEK: • 1000 PC’s for production and simulation • Storage ~ 630 TB • Resources at Univ.’s • Nagoya Tokyo Tohoku • Smaller resources in the collaboration institutes compared to BaBar exp. • BaBar has partners in IN2P3, RAL. • Need more funding for universities • planning to extend their computing power in Data Grid paradigm HEP Data Grid Workshop @ KNU

  11. KEKB/Belle: Data Moving for Data Analysis over SuperSINET GbE e+e- Bo Bo Neutrino Center Tohoku U. 400 GB/day ~45 Mbps ~ 100GB/day NFS Osaka U. KEK 170 GB/day 1TB/day ~100Mbps • U. Tokyo Nagoya U. HEP Data Grid Workshop @ KNU

  12. R&D of Gfarm(Grid data farm) • Gfarm http://datafarm.apgrid.org/ • Development by AIST, Titech, KEK since 2000 • Architecture • PC farm with large local disk/node as a GRID fabric • Large data file is divided into fragments and stored in the disks by read-in • Data file integrity is managed by the Gfarm metadata DB • Data I/O by parallel file system • Affinity scheduling of process and storage for data residence • Service daemon process: gfsd is running at each node • Authentication by gfarm key and/or GSI AIST :Advanced Inst. for Science and Technology HEP Data Grid Workshop @ KNU

  13. Gfarm Test • PC farm used: Athlon 1.2GHz 16nodes • Fast Ethernet • Data replication between two 8-nodes • 8 parallel copy of 1GB fragments : 8 GB data file % gfrep –H <nodes> gfarm:userdata.bin • Throughput 89.4 MB/s vs. 90.2 MB/s (sum of independent transfers) Overhead in access to the metadata ~ 1% Metadata DB Server Gfarm <nodes> Data file gfarm:userdata.bin HEP Data Grid Workshop @ KNU

  14. NII Gfarm Demonstration at SC2002 Nov. 18 “Bandwidth Challenge” Data transfer, data replication over trans-oceanic network HEP Data Grid Workshop @ KNU

  15. Related activity : Collaboratory • “Virtual Laboratory” for Material Science • Funded 2001-2005 • 5 laboratories: KEK, IMS, U. Tokyo, Tohoku U., Kyoto U. • Implementation • Remote control of experimental equipments X-ray diffractometers at Photon Factory of KEK • Data sharing • Simulation by supercomputers linked • IP video conferencing system with application sharing • Demonstrated the prototype system Oct. 23, 2002 • Remote control from IMS(Nagoya) of diffractometers at the beam line of PF/KEK. • H.323 video conferencing with 4 other institutesIMS: Institute for Molecular Science HEP Data Grid Workshop @ KNU

  16. Related activity: GANGlobal Accelerator Network • Aimed to remote operation of large accelerator facility, such as future Linear Collider • It facilitates acc control and monitoring for remote operation shift • International workshop held 2001, 2002 • KEK Planning to implement at J-PARC( Japan Proton Accelerator Research Complex) • 50 GeV, 3 GeV proton accelerators at Tokai 50 km away from Tsukuba(KEK) • Construction 2001 – 2006 JAERI Tokai Campus KEK HEP Data Grid Workshop @ KNU

  17. Funding status for Grid • Gigabit Network MEXT: Ministry of Edu, Sci, .. • MEXTInformatics Project • A05: Application Grid projects for science ~ US$ 6M / 5 y • Networking supercomputers: ITBL ~ US$ 105M / 5y • AIST GTRC(Grid Technology Research Center)~ US$ 140M • Other Funding from Institutional Resources • LHC/Atlas(ICEPP,KEK), Gfarm, Virtual Observatory, Ninf-G, BioGrid(Osaka U ), OBIGrid(Riken),……. • Projects proposed (2003 ~ 5 years) • National Research Grid Initiative(US$ 360M): focused in Bio & Nanotech • National ‘Business’ Grid Project(US$ 250M): ASP business HEP Data Grid Workshop @ KNU

  18. Gigabit network • Super SINET • 10Gbps MEXT DWDM nationwide network • About 40 universities and labs • US$ 60M/yr • 5 application areas: HEP, Nanotech, Bioinformatics, Astronomy, Grid Computing • Tsukuba-WAN • 330Gbps DWDM ring in the Tsukuba area • Tens of national labs • 6 supercomputers in TOP100 • Testbed for Grid infrastructure and applications HEP Data Grid Workshop @ KNU

  19. MEXT Grid Projects • MEXT “Informatics” Project A05 “Grid” Area (US$ 6M, 5 years) • Lead by Shinji Shimozo(Osaka U.) & Satoshi Matsuoka(Titech) • 11 groups of universities and labs • Research and development on application, middleware, and infrastructure • Gfarm project is partly supported. • Tokyo Institute of Technology “Titech Campus Grid” (US$ 2M) • Lead by Satoshi Matsuoka(Titech) • Total of 800 CPU PC clusters at 13 locations in a 30km range connected by Super TITANET(1-4Gbps) backbone • Pilot project for virtual parallel computer infrastructure for high end computational e-Science HEP Data Grid Workshop @ KNU

  20. ITBL • ITBL(IT-based Laboratory) • Government Labs: NAL, RIKEN, NIED, NIMS, JST, JAERI • Project period: 2001-2005 (3-stage project) with total of US$ 105M funding • Applications: mechanical simulation, computational biology, material science, environment, earthquake engineering • Step 1: Supercomputer centers of government lab arenetworked via SuperSINET • Step 2: “Virtual Research Environment”: Grid-enabling laboratory applications • Step 3: Sharing information among researchers from widely distributed disciplines and institutions HEP Data Grid Workshop @ KNU

  21. Environmental Circulation Simulation for Pollutant Materials VPP300 （Vector Parallel Computer） Atmospheric Environment Simulation Two-Dimensional Data at Sea Surface Two-DimensionalData at Ground Surface Stampi Stampi COMPAQ α （High-Performance PC) Marine Environment Simulation AP3000 （Scalar Parallel Computer） Terrestrial Environment Simulation Large-scale Hartree-Fock Calculation SPring8 Fluid-Particle Hybrid Simulation for Tokamak Plasmas Control Diagonalization Orthonormalizarion Pool of task distribution Integral handling Partial accumulation Fij<-Fij+Dkl*qijkl Vector Machine Scalar Machine Electronic fluid /Electro-Magnetic field Ion Particles Stampi Vector Machine Scalar Machine Grid Computing Applications implementing key technologies of ITBL HEP Data Grid Workshop @ KNU

  22. Grid Technology Research Center • Part of AIST under METI (Minitry of Economy, Trade and Industry) • GTRC(Grid Technology Research Center) • Established in Jan. 2002 • Director: Satoshi Sekiguchi • US$140M for building, equipment, R&D and operations • To be central institute for Grid research in Japan • Not just R&D, but serve as central CA, ApGrid’s principal site • Collaboration with institutions and other Grid projects of both domestic and international scale • Organizing GRID Consortium Japan HEP Data Grid Workshop @ KNU

  23. Conclusions • Grid activities are starting. But not coordinated support for middleware development • International testbed for LHC/Atlas in 2003 • Possible collaboration in Asian region for KEKB/Belle • Network is emerging for heavy application users. • domestic and international use • Testbed of the Grid middleware in collaboration with CS people HEP Data Grid Workshop @ KNU

More Related