90 likes | 197 Views
Grid and Cloud Infrastructures for Agriculture Antun Balaz Institute of Physics Belgrade, Serbia. Joint agINFRA and SCI-BUS workshop & agINFRA Open Stakeholder Day 30 May 2013, SZTAKI , Budapest, Hungary. Overview. agINFRA infrastructure Hardware resources Grid resources
E N D
Grid and Cloud Infrastructures for AgricultureAntun BalazInstitute of Physics Belgrade, Serbia Joint agINFRA and SCI-BUS workshop & agINFRA Open Stakeholder Day 30 May 2013,SZTAKI, Budapest, Hungary
Overview • agINFRA infrastructure • Hardware resources • Grid resources • Scientific Gateways • agINFRA RESTful gateway • Key results
agINFRA Infrastructure • Technically divided into three main layers: • Hardware resources (Grid and Cloud) • Scientific Gateways (Catania SG, WS-PGRADE SG, RESTful gateway) • Integrates services (Organic.Edunet, AGRIS, CIARD RING, AgLR-TF)
Hardware resources • Resource centers • PARADOX cluster – 88 x (2 x 4-core Xeon E5345@2.33 GHz) = 704 CPU (1 GB of RAM / core)50 TB of disk space, Gigabit Ethernet • Roma Tre cluster – 350 CPUs; 100TB • Catania cluster – 800 CPUs; 700 TB • SZTAKI cluster – 8 CPUs • PARADOX ugrade – 106 x (2 x 8-core Xeon E5-2670@2.6 GHz) = 1696 CPU (2 GB of RAM / core)100 TB of disk space, Infiniband • Virtualized resources • SZTAKI Desktop Grid90 000 volunteer resources; 3 Tflops. • EDGeS@home - 15 000 CPUs
Grid resources [1/2] • agINFRA Virtual Organization: vo.aginfra.euhttps://voms.ipb.ac.rs:8443/voms/vo.aginfra.euhttps://voms.ct.infn.it:8443/voms/vo.aginfra.eu • MyProxy service (PX) • myproxy.ipb.ac.rs • Computing Elements (CREAM) • cream.ipb.ac.rs:8443/cream-pbs-aginfra – 704 CPUs • ce-01.roma3.infn.it:8443/cream-pbs-grid – 328 CPUs • ce-02.roma3.infn.it:8443/cream-pbs-grid – 208 CPUs • grid012.ct.infn.it:8443/cream-lsf-aginfra – 20 CPUs
Grid resources[2/2] • Logical File Catalogue (LFC) • lfc.ipb.ac.rs • Storage Elements (SE) – 850 TB • dpm.ipb.ac.rs • prod-se-02.ct.infn.it • storm-01.roma3.infn.it • Metadata Catalogue (AMGA) • amga.ct.infn.it • Workload Management System (WMS) • wms.ipb.ac.rs • wms-demo.ipb.ac.rs • wms-aegis.ipb.ac.rs • Logging and Bookkeeping (LB) • wms.ipb.ac.rs • wms-demo.ipb.ac.rs • wms-aegis.ipb.ac.rs • Information System (BDII) • bdii.ipb.ac.rs
Scientific Gateways • Catania Scientific Gatewayhttps://aginfra-sg.ct.infn.it/ • gUSE/WS-PGRADE Portal (workflows)http://aginfra-portal.lpds.sztaki.hu/liferay-portal-6.0.5/
agINFRARESTful gateway • Cataloging • User’s configurations of Grid-ported applications • Datasets’ metadata information and locations • Arbitrary user-defined metadata information • Querying of these information • Off-line processing • Collect new datasets • Transform existing datasets • Produce new datasets’ metadata information • Data management • Automatic replication of datasets • Ensures existence of the same dataset on different storage systems • Exposure through the HTTP protocol
Key results • Design, specification and deployment of the agINFRA infrastructure • Establishment and operation of agINFRA Grid infrastructure (VO: vo.aginfra.eu, VOMS, BDII, LFC, WMS/LB, PX, AMGA, DPMs, CREAMs) • Establishment and deployment of Cloud infrastructure (CLEVER and Okeanos) • Deployment of agINFRA Scientific Gateways (Catania and gUSE/WS-PGRADE SGs) • Development of agINFRARESTful gateway • Development and deployment of data infrastructure • dataset exposure layer (RESTful gateway) • data management layer (Grid RESTful API, HTTP exposure) • data transformation layer (agDCtoLOM, agLOMtoAK, agLOMtoRDF) • data processing layer (agDataHarvest, agMetaExtract, agRecommender)