260 likes | 269 Views
ITM general meeting 2007. ISIP: status & plans B.Guillerminet, G.Manduchi. Infrastructure & Software Integration Project. ISIP belongs to the European Integrated Tokamak Modelling project 15 members, 6 laboratories:
E N D
ITM general meeting 2007 ISIP: status & plansB.Guillerminet, G.Manduchi ITM General Meeting Garching 17-21 September 2007
Infrastructure & Software Integration Project • ISIP belongs to the European Integrated Tokamak Modelling project • 15 members, 6 laboratories: • M. Airaj, J-F Artaud, Y. Buravand, A. Duarte, B. Guillerminet, P. Huynh, F. Iannone, F. Imbeaux, B. Knaepen, G. Manduchi, A. Neto, V. Pais, D. Robson, J. Santos, J. Storrs • G.Bracco(?) • In charge of the infrastructure (soft + hard) for the ITM: • Code platform: framework, editor, … • Data handling: data structure, data storage & access, … • Tools: version handling, portal, … • Hardware: gateway, GRID/HPC, … • Many thanks to all the contributors • A lot of works has been done since 2.5 years • ITER IM workshop: confrontation with others simulation approaches • Major issue: • 1st ITM framework & gateway will be released in 3 months • Question: Are we ready??? => ISIP status & plans ITM General Meeting Garching 17-21 September 2007
Outline • What will be provided? Current status & short term plan (=> Jan08) • Computing infrastructure • Applications & tools • Codes • Data structure, access & storage • How does all that work? • Summary: are we ready? • Plans • Mid-term (2008) • Long term (2009-2010) ITM General Meeting Garching 17-21 September 2007
Lab. Lab. user login user login Lab. user login Architecture resources applications Gateway Simulation editor Data servers: MDS+, Pub-Sub, HDF5 Workflow editor portal Clusters: JET, IPP, … Workflow engine GRID EGEE Codes: Transport, heating,… HPC UAL: C/C++, F95, Java, … Data mining tools Code repositories: Mercurial, … Post-processing: Scilab,Visit,Root,… Private data: XML files, … Catalogues • Currently, no portal, no gateway, no GRID, … • Use itm.partenaires.cea.fr, one login per user ITM General Meeting Garching 17-21 September 2007
What will be provided? • Short term issue (January 2008) • Computing infrastructure • Gateway: cluster 32-64 CPUs (16GB), 10-100TB, 10Gb/s • Currently, use “itm.partenaires.cea.fr” => then, will used for developments & tests • Logins, private & shared storage area • Tutorial, documentations • No catalogues • No Web Portal, no SSO • No GRID computing (EGEE or DEISA) • Applications • Data structure: v 4.05 • XML tools • Include/import files for F90/95, C/C++ • UAL library for F90/95, C/C++, Java, Matlab (?), Scilab (?) ITM General Meeting Garching 17-21 September 2007
What will be provided ? (2) • Applications: • Workflow engine (KEPLER): • running • Simulation editor/director: • Parameters editor (Nov 07) • Simulation run/control (Jan 08) • SCILAB: post-processing tool • UAL access (?) • Control & monitoring tools: • Cluster tools • On-line data visualization: • 2D/3D actors in KEPLER (VTK based) • Scilab ITM General Meeting Garching 17-21 September 2007
What will be provided? (3) • Codes: • IMP#1: • Helena • Cedres • Mishka • Efit • Soloviev • Database: • MDS+ storage • Machines: ITER, JET, TS • Shots • Use Soloviev to populate it • In-memory ITM General Meeting Garching 17-21 September 2007
How does all that work? • XML schemas • Generate automatically the include files for C, Fortran, Java, Matlab, … • Use it in your code: #include …, import … • Fill the structure through the provided API (UAL): ITM General Meeting Garching 17-21 September 2007
Code adaptation • Data access through the UAL: • Independent of the storage • The framework does not know our data • Principles: • data transfer through a data server (in-memory for the simulations) • Advantages: • Solve the problem of languages mixing • Allow // computations • Small changes: • I/O => UAL • Read euitmget() • Write euitmput() • Structure: • Use/import ITM structure • Call euitmget(“CPO name”, CPOin) • Call compute(CPOin,CPOout) • Data access is separated from the computational part • Call euitmput(“CPO name”, CPOout) codes EFIT, … codes HELENA, … codes MISHKA, … UAL In-memory ITM General Meeting Garching 17-21 September 2007
Code integration => actor • =>one-click integration of C/C++, Fortran local codes • GUI tool: define the arguments • JNI based • =>one-click integration of C/C++, Fortran remote codes • GUI tool: define the arguments • Automatic generation of Web Services based on gSOAP • =>one-click catalogue insertion • GUI tool • XML based ITM General Meeting Garching 17-21 September 2007
Building a workflow • Based on KEPLER • =>design: drag & drop actors • =>many actors: database, math, display, web service, grid service, R-expression, … • Advanced models of computation • =>directors: DE, CT, SDF, PN and more in Ptolemy II (FSM, Giotto, DDE) • =>hierarchical design Run control buttons Scheduler: time behavior Catalogues ITM General Meeting Garching 17-21 September 2007
Directors • SDF (synchronous data flow) • Dataflow oriented model • Modules are triggered by the data arrival • DE (discrete event) • Time-oriented models • One global time • Events can be posted at some specific time • PN (process networks) • Message oriented model (network of processes) • Communication by passing messages through FIFO • CT (Continuous Time) • Use to solve differential equations (integrator with feedback). • Implemented solvers: • Forward Euler • Backward Euler • 2(3)-order Runge-Kutta • Trapezoidal rule DE Transformation component: Waveform generator Transformation component: Event generator CT ITM General Meeting Garching 17-21 September 2007
Plot 2D Plot 3D in-memory Data server SOLOVIEV HELENA MISHKA ITM General Meeting Garching 17-21 September 2007
Data visualization • In the workflow (on-line): • =>2D, 3D data plots • VTK based • Scilab • Data & synchronisation through Kepler • In the GUI for the simulations (ISE see later) : • => Y=f(t) • Java graphs • polling • Interactive tools (UAL access is mandatory): • Scilab (using the Kepler data) • VISIT (planned) ITM General Meeting Garching 17-21 September 2007
Data visualization (cont’d) Examples (Scilab) ITM General Meeting Garching 17-21 September 2007
Post-processing tools • =>Scilab • Signal processing, non-linear control, … • Data analysis • Visualization • =>Implementation • Scilab expression • Numerical actors ITM General Meeting Garching 17-21 September 2007
Define a simulation: ISE Simulation Editor: GUI • Objectives: • define friendly all the parameters for a simulation • run & control a simulation ITM General Meeting Garching 17-21 September 2007
Edition mode Simulation mode Monitoring window 2D plot (Kepler actor) Overview window ITM General Meeting Garching 17-21 September 2007
ISE windows Menu toolbar: Overview: Tabs: • General: all the parameters (machines, fields, equilibrium, …) • Favorites: parameters changed often by the user • Overview: global view of the simulation on a timeline • Monitoring: plotted data during the simulation and waveform • Workflow: parameters for the codes relevant to the current workflow • Simulation: special parameters for the simulation ITM General Meeting Garching 17-21 September 2007
Are we ready? • Status • ok • ok • ok with the Kepler data, × with ITM data • ok • version 4.05 • ok for Fortran 90, C/C++ • × for Java, Matlab & Scilab • ok • Last modifications not yet reported • tests in progress • × in development • × not yet scheduled • × not yet scheduled • ok • expected October • Kepler • Local code integration JNI GUI • Remote code integration WS GUI • Post-processing Scilab • Visualization VTK • Data structure • XML schemas • XML transformers: Fortran 90, C/C++, Java, Matlab, Scilab • Data access: UAL • API: • C/C++ • Fortran 90 • Java • Matlab • Scilab • IDL • Storage: • MDS+ • In-memory ITM General Meeting Garching 17-21 September 2007
Are we ready? (2) • Status • ok 1st draft, improvements expected • ok 1st release mid-November • × release January 2008 • ? • ok software & procedures (64 bits SL4) • × must be investigated • × provisional (catalogue in Kepler) • × provisional version must be provided • × provisional (catalogue in Kepler) • ok doc + this meeting • ? This meeting • US-EU collaboration • ISE: Integrated Simulation editor: • Requirements • Parameters setting • Simulation control & monitoring • Gateway • Purchase • Installation • System • applications • Catalogues • Requirements • Codes • Specific parameters • Workflow • Tutorial, doc • What is missing for the 1st release? • Workflow consistency (design) ITM General Meeting Garching 17-21 September 2007
Lab. Lab. user login user login Lab. user login Mid-term plan: 2008 resources applications Gateway Simulation editor Data servers: MDS+, Pub-Sub, HDF5 Workflow editor portal Clusters: JET, IPP, … Workflow engine GRID EGEE Codes: Transport, heating,… HPC UAL: C/C++, F95, Java, … Data mining tools Code repositories: Mercurial, … Post-processing: Scilab,Visit,Root,… Private data: XML files, … Catalogues • 2008 work plan: one objective of this meeting ITM General Meeting Garching 17-21 September 2007
Mid-term plan: 2008 • Catalogues • Database for the specific parameter, codes and workflows • Tools (editor), Java API • Expected: 1st semester 2008 • Portal • SSO, Web portal, applications integration • Based on GridCore & Migrating Desktop • Realized by Euforia • Expected: 4Q08 • Grid-HPC: • Java API for job launching & notification on GRID EGEE: • Based on Glite, integration to Kepler • Expected: 3Q08 • Similar java API for HPC • Euforia • Expected: 3Q08 • Data access for GRID EGEE • How? Objectives: UAL integration • Expected: 4Q08 • Data access for GRID HPC • Euforia • Expected: 4Q08 ITM General Meeting Garching 17-21 September 2007
Mid-term plan: 2008 • Database & data structure • Requirements • Parallel & distributed storage • Data mining tools • Expected: 4Q08 • Visualization • VISIT & 4D/5D visualization • Euforia • Expected: 4Q08 • UAL: • HDF5 • Expected: 1-2Q08 • Handling large memory simulation demand • Expected: ? • Scilab access • Expected: 1Q08 • IDL access • Expected: ? ITM General Meeting Garching 17-21 September 2007
Mid-term plan: 2008 • ISE • Improvements • Parameter distribution, not single value => error bar • Consistency checking • Kepler • Integration of various IMP# codes • Checkpoints: need special consideration? collaboration: CPES? • … • Meshing: • Defining the requirements • Studies must start early • Collaboration: OpenCascade, Salome….? ITM General Meeting Garching 17-21 September 2007
Long-term plan: 2009-2010 • GRID-HPC • Extension of the workflow on EGEE & HPC: • Kepler, MetaGrid … • Euforia project • Expected: 2Q09 • Whole device simulation • Sophisticated meshing tools • Experiments plan (error bar, parameter space …) • Material database ITM General Meeting Garching 17-21 September 2007