1 / 46

Status and Prospective in Japan

KEK High Energy Accelerator Research Organization. Status and Prospective in Japan. Atsushi Manabe Computing Research Center (CRC) KEK. Outline. Grid projects (budget) in Japan academic region. HEP Data Grid activities in Japan. KEK Needs in KEK Strategy of KEK CRC Pacifi-Grid

Roberta
Download Presentation

Status and Prospective in Japan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. KEK High Energy Accelerator Research Organization Status and Prospective in Japan Atsushi Manabe Computing Research Center (CRC) KEK Int'l HEP Data Grid Work Shop in Daegu

  2. Outline • Grid projects (budget) in Japan academic region. • HEP Data Grid activities in Japan. • KEK • Needs in KEK • Strategy of KEK CRC • Pacifi-Grid • Gfarm project • Grid/HPSS test bed on SupserSINET. • ICEPP: ATLAS Japan Regional Center Int'l HEP Data Grid Work Shop in Daegu

  3. Grid Projects(budget) in Japan(academic area only) • MEXT(Ministry of Education, sports …. and technology) projects • SuperSINET; Gigabit backbone for academic researchers in Japan • MEXTInformatics Project • A05: Application Grid projects for science ~ US$ 6M /5 y Astronomy (Virtual Observatory), io-Infomatics/Genome (BioGrid)…… • Networking supercomputers: ITBL ~US$ 100M /5y • National Research Grid Initiative ~US$ 300M/5y focused in Nano-tech application • (LHC/ATLAS(ICEPP,KEK) • METI (Ministry of Economy, Trade and Industry) project • AIST GTRC(Grid Technology Research Center) ~US$ 140M Int'l HEP Data Grid Work Shop in Daegu

  4. Gigabit network • Super SINET • 10Gbps MEXT DWDM nationwide network • About 40 universities and labs • US$ 60M/yr • 5 application areas: HEP, Nanotech, Bioinformatics, Astronomy, Grid Computing • Tsukuba-WAN • 330Gbps DWDM ring in the Tsukuba area • Tens of national labs • 6 supercomputers in TOP100 • Testbed for Grid infrastructure and applications Int'l HEP Data Grid Work Shop in Daegu

  5. Tohoku U OXC NII Chiba IP 10 Gbps WDM path NIFS IMS IP router Nagoya U NIG Nagoya Hub Hokkaido Osaka Hub Tokyo Hub KEK Osaka U Tsukuba US，EU Kyoto U Korea NII KyushuU ISAS U Tokyo ICR Kyoto-U Internet TITEC NAO Waseda IMS U-Tokyo Network Topology of SuperSINET Int'l HEP Data Grid Work Shop in Daegu

  6. KEK SuperSINET Backbone Titech ICEPP Int'l HEP Data Grid Work Shop in Daegu

  7. Network Infrastructure TEIN NII- NY TRANSPAC Taiwan-US Japan (NII) -NY : 2.4G x 2 Jan. 2003 Japan – US: 622M x 2 (TRANSPAC) Korea – US: 45 M Korea – Japan: 2.4G Jan. 2003 Korea – Europe: 2 M (TEIN) China(IHEP) – Japan(KEK): 128 kbps (HEP) China – US: 10 M Taiwan – Japan: 155 M Taiwan – US: 622 M (Dec. 2002) Int'l HEP Data Grid Work Shop in Daegu

  8. MEXT Grid Projects • MEXT “Informatics” Project : A05 “Grid” Area • Lead by Shinji Shimozo(Osaka U.) & Satoshi Matsuoka(Titech) • 11 groups of universities and labs • Research and development on application, middleware, and infrastructure • Support the Virtual Observatory(astronomy) and the BIO-Grid(bio-Infomatics/Genome) projects • Tokyo Institute of Technology “Titech Campus Grid” (US$ 2M) • Lead by Satoshi Matsuoka(Titech) • Total of 800 CPU PC clusters at 13 locations in a 30km range connected by Super TITANET(1-4Gbps) backbone • Pilot project for virtual parallel computer infrastructure for high end computational e-Science Int'l HEP Data Grid Work Shop in Daegu

  9. ITBL • ITBL(IT-based Laboratory) • To Connect Super-Computers in Government Labs: NAL, RIKEN, NIED, NIMS, JST, JAERI • Project period: 2001-2005 (3-stage project) with total of US$ 105M funding • Software development project. • Applications: mechanical simulation, computational biology, material science, environment, earthquake engineering • Step 1: Supercomputer centers of government lab arenetworked via SuperSINET • Step 2: “Virtual Research Environment”: Grid-enabling laboratory applications • Step 3: Sharing information among researchers from widely distributed disciplines and institutions Int'l HEP Data Grid Work Shop in Daegu

  10. NAREGI (National Research Grid Initiative) • NII (The National Institute of Informatics) under MEXT project. • Making Center of Grid Research & Development • Development of Grid Lower and Middleware(Grid scheduler,VM, RPC, MPI, GUI tools…) • Networking, Security and User management(Routing, robust TCP/IP protocol, Traffic management) • Applications for Nano-science and technology.Joint with Institute for Molecular Science. • http://www.grid.nii.ac.jp/ Int'l HEP Data Grid Work Shop in Daegu

  11. Grid Technology Research Center • Part of AIST under METI (Ministry of Economy, Trade and Industry) • GTRC(Grid Technology Research Center) • Established in Jan. 2002 • Director: Satoshi Sekiguchi • US$140M for building, equipment, R&D and operations • AIST super server; PC cluster2004-2Q 11TFLOPS/ AMD Operon x 2116CPUs • To be central institute for Grid research in Japan • Not just R&D, but serve as central CA, ApGrid’s principal site • Collaboration with institutions and other Grid projects of both domestic and international scale • Organizing GRID Consortium Japan • http://unit.aist.go.jp/grid/ Int'l HEP Data Grid Work Shop in Daegu

  12. Grid related Activity at KEK Int'l HEP Data Grid Work Shop in Daegu

  13. Major projects in KEK • KEK sponsors projects which use accelerators and also projects in related fields • KEKB accelerator • BELLE experiment • Proton synchrotron • nuclear and elementary particle physics • medical • Synchrotron Radiation Facility • bio, material science and so on • Also many theorists in related fields • Support International HEP collaboration • ATLAS Japan Int'l HEP Data Grid Work Shop in Daegu

  14. KEK Future projects • J-PARC • will be built in Tokai div. of Japan Atomic Energy Research Institute (JAERI) where ~50km north from KEK start the operation in JFY2005 • 50 GeV high-intensity proton synchrotron • nuclear physics and elementary particle physics • Super B factory • x30 Luminosity(require x10 CPU power). Under discussion. • Linear Collider • discussion is still under way Int'l HEP Data Grid Work Shop in Daegu

  15. Computing Facilities at KEK • Computing Research Center offers group oriented computing services • Exclusive computing resource for main groups • Networking, storage and computing resource • User group owns their own computer systems • PC farms Int'l HEP Data Grid Work Shop in Daegu

  16. Main computing facilities • KEKCC computer system • for experiments at PS, linear collider R&D, theory, ATLAS and so on • IBM SP3, 120TB HPSS storage • KEKB computer system for Belle • Sun WSs and PC farms. 500TB storage. • KEKSC: Super-computer system for mainly lattice QCD G. • Hitachi SR-8000 100 nodes; almost 1TFLOPTS Int'l HEP Data Grid Work Shop in Daegu

  17. Our needs in Grid Era • ATLAS Regional Center (Grid is premise) • Being set up at ICEPP (Int’l Center for Particle Physics, U. Tokyo) • Still KEK have large portion of ATLAS Japan members and they will make data analysis using computing resource at KEK. We will support them. • And • BELLE; international collaboration and require more and more CPU and storage. • J-PARC; joint project of other Inst. regionally separated. Int'l HEP Data Grid Work Shop in Daegu

  18. BELLE • BELLE is the major project at KEK • They have many collaborators world wide • They always needs additional CPU power because acc. luminosity improving and improving. • They seek a solution of their problems related computing issue using GRID. • We interviewed the BELLE computing coordinator and got the wish list shown in the following slides Int'l HEP Data Grid Work Shop in Daegu

  19. Wish lists from BELLE • What BELLE wants: • A way to integrate CPU powers and way to distribute data among collaborators’ Univ.s and Inst.s for their data production. • Automatic way to submit analysis jobs in the distributed environment. • once analysis codes have placed somewhere, process for all of experimental data and Monte Carlo should be done automatically and simply users gain the result • Automatic Monte Carlo production • Automatic version control for libraries, constants and others on all of the computer system in BELLE • Automatic update on reconstructed data or Monte Carlo if libraries or constants are once updated Int'l HEP Data Grid Work Shop in Daegu

  20. Wish lists from BELLE 2 • It is not easy to achieve their requests technically • simplest solution will be hire more Post-Docs or find more grad-students, however • We have decomposed their requests • should be done in application level (BELLE) or • can be solved by GRID Int'l HEP Data Grid Work Shop in Daegu

  21. What they need related GRID • An efficient way to distribute jobs and way to manage/share/distribute data for the jobs. • effective data catalogue • physical to logical mapping • Seamless and unique file tree • query • file names, contents and others • works on the Internet but secure • efficient and easy to distribute data to remote sites • easy to use • web interface and GUI also Int'l HEP Data Grid Work Shop in Daegu

  22. continued • mass storage management system • not only tapes but also large amount of disks • robust and fast transfer • easy integration with other component • We already have them partially on LAN, but we need to expand them to the Internet. We hope GRID will solve them. Int'l HEP Data Grid Work Shop in Daegu

  23. KEK CRC strategy towards GRID Int'l HEP Data Grid Work Shop in Daegu

  24. Done or On going. • Install GT2(Globus toolkit ver.2) and applications (LSF) to two CRC Systems (KEKCC & KEKB) ) and will open to some limited users. (remote Job submission) • Build temporal CA with single Virtual Organization(VO). • Build a inter-institute wide Grid test bed on SuperSINET and make a performance test. (talk later) • Trial use of SRB with Belle data in a test system. • Start to touch LCG tools by ATLAS Japan people. • watch technical trends carefully. Int'l HEP Data Grid Work Shop in Daegu

  25. SRB • SRB is available today and looks useful even in HEP ( http://www.npaci.edu/DICE/SRB/ ) • developed at San Diego Super Computer Center • seamless solution to HPSS • good for the KEKCC (central computer) users • Data catalogue and replication • SLAC is also testing Int'l HEP Data Grid Work Shop in Daegu

  26. Until early 2004 • Deploy GT3, evaluate LCG tools and other something new technology. • Build own VO at each sites and integrate them. • File sharing by SRB (or ?) and seamless job submission between KEKCC and KEKB • Pacifi-Grid test environment; Pan-Pacific collaboration (see later) • KNU (Korea), Univ. of Melbourne (Australia), SLAC and KEK will test SRB • Start to develop higher level software (experiment side) on the top of Globus and SRB. • Message passing with GSI authentication ? Int'l HEP Data Grid Work Shop in Daegu

  27. ~2004 • KEKB and KEKCC (5yeas rental )system replacement (2005-4Q) . • GRID will become concrete until this stage or die • Final specification have to be decided • Decision of middle ware (Grid or else?) • CA • Security • human resource • trained engineer on site Int'l HEP Data Grid Work Shop in Daegu

  28. Other topics discussed internally • CA • nation wide, institution wide or else? KEK certify all Japanese HEP user? • Rules for certification of users. • PKI • secure enough? Safety private key management (IC card?) • management costs? • seamless integration with local authentication • Network • How we can overcome latency? • better connectivity to the Internet, routing and throughput • middleware • Globus, Avaki or something else • SRB, LCG or other data replication solution • Storage • SAN • StorageTank (IBM) • security vs usability Int'l HEP Data Grid Work Shop in Daegu

  29. Pacifi-Grid:Proposal on Open Pacific Grid collaboration Takashi Sasaki (KEK) Int'l HEP Data Grid Work Shop in Daegu

  30. charter • Give research institutions and IT vendors opportunity of testing Grid technologies under inter-state environment. • Emphasis on Data Grid • Open to everybody who wants to join • Grid is a newly emerging technology and more experiences are necessary before deploying at the large scale production systems • Test on high latency network • robustness tests • interoperability tests among sites and vendors • security Int'l HEP Data Grid Work Shop in Daegu

  31. Collaboration (under discussion) • Korea • U of Melbourne, Australia • SLAC, USA • JAPAN • ICEPP (Univ. of Tokyo) • KEK • Some venders • Under discussing with Fujitsu, IBM Japan and so on Int'l HEP Data Grid Work Shop in Daegu

  32. Gfarm R&D Development R&DGrid middleware of Cluster File System with job execution. Int'l HEP Data Grid Work Shop in Daegu

  33. R&D of Gfarm (Grid data farm) • Gfarm • Development by CS scientist of AIST, Titech and KEK since 2000 • AIST :Advanced Inst. for Science and Technology • Architecture • PC farm with large local disk/node as a GRID fabric • Large data file is divided into fragments and stored in the disks • Data I/O by parallel file system • Affinity scheduling of process and storage for data residence Int'l HEP Data Grid Work Shop in Daegu

  34. user job user job job manager job manager computing node job A job A computing node storage element job B high speed switch job B storage node Data Access Locality Exploit the data access locality as much as possible Int'l HEP Data Grid Work Shop in Daegu

  35. http://datafarm.apgrid.org/ Gfarm • Authentication:Globus GSI and/orShared Private Key • Grid Data Farm : Beta 3 release is now available. • Parallel I/O: exploit the data access locality, store and access files by “fragments” • Parallel Job: program runs on thenodes where the file fragmentsreside: “owner computes” • Programs and file fragments,job history, data checksum,file replication are managed withMetaDB • Fragments are replicated forbackup and load balancing • User sees the fragmented files asa single Gfarm URL • Provide system call hooks foropen(), close(), read(), write() etc Int'l HEP Data Grid Work Shop in Daegu

  36. FADS/Goofy in ATLAS • Used in sub-detector software developments and physics validations • Detector modules also run in ATLAS mainstream framework (Athena) • Testbed for ROOT I/O • Bandwidth Challenge inSC2002 • Generated 106 fully simulatedhigss  4 events in 2 dayswith ~ 400 CPU Int'l HEP Data Grid Work Shop in Daegu

  37. Gfarm achievement • “File affinity scheduling” provides parallel processing capability for both I/O-bound and CPU-intensive jobs. • FADS/Goofy was successfully implemented on Gfarm architecture. In the process, ROOT I/O module works with Gfarm parallel file system using the system call hook, without a changes to the ROOT package. • ROOT I/O files has been successfully replicated at 2.286 Gbps using the SC2002 Bandwidth Challenge testbed with 12 nodes ~ 190Mbps/node. • Please refer to CHEP03 reports by Y.Morita and O.Tatebe Int'l HEP Data Grid Work Shop in Daegu

  38. Status of ATLAS Japan RC (ICEPP U. of Tokyo) • R&D is ongoing • Test-buy of PC farm and storage • 1U P3, P4 servers ( >150nodes and more) , LTO (10TB) with CASTOR and >20TB disk servers. • Nordu GRID was tested first • Data transfer tests between U. Tokyo and KEK was done • ATLAS data challenges • LCG 0 and 1 Int'l HEP Data Grid Work Shop in Daegu

  39. Int'l HEP Data Grid Work Shop in Daegu

  40. Int'l HEP Data Grid Work Shop in Daegu

  41. LCG-0 Deployment Status These sites deployed the LCG-0 pilot system and will be the first sites to deploy LCG-1 Int'l HEP Data Grid Work Shop in Daegu

  42. LCG Resource Commitments – 1Q04 Int'l HEP Data Grid Work Shop in Daegu

  43. Int'l HEP Data Grid Work Shop in Daegu

  44. Status of ATLAS Japan RC 2 • Budget requests have been submitted to the funding agency • still no decision has been made • and no decision will be made within this FY • All of national universities and some national laboratories including KEK will be non-nationalized (not privatized, but will be a kind of independent governmental agencies) Int'l HEP Data Grid Work Shop in Daegu

  45. Summary ofKEK Grid activity • KEK had a step forward for the Grid computing • People at ATLAS Japan RC are working on LCG • Realistic environment is necessary to test middleware and applications • We have started the discussion among Pan-Pacific sites to collaborate each other • We want to exchange knowledge and experiences Int'l HEP Data Grid Work Shop in Daegu

  46. End of my presentation Thank you Int'l HEP Data Grid Work Shop in Daegu

More Related