180 likes | 266 Views
OAS Contribution The ACDS The new XCATDB based on Saada. Laurent MICHEL Christian MOTCH François Xavier PINEAU. Observatoire Astronomique de Strasbourg (Fr) Saada: http://astro.u-strasbg.fr/websaada XCATDB: http://amwdb.u-starsbg.fr/jacds. The ACDS. Delivered Products
E N D
OAS Contribution The ACDS The new XCATDB based on Saada Laurent MICHEL Christian MOTCH François Xavier PINEAU Observatoire Astronomique de Strasbourg (Fr) Saada: http://astro.u-strasbg.fr/websaada XCATDB: http://amwdb.u-starsbg.fr/jacds
The ACDS • Delivered Products • FITS catalogue extractions • X-ray source counterparts • Archival sources in the XMM field of view • Graphical products (PDF) • Finding charts • Rosat Images • Catalogue plots • HTML product • Browsing pages for ACDS data with many links to CDS resources
The ACDS Simbad (Strasbourg France) NED (Caltech USA) Vizier at CDS (Strasbourg France) • ACDS: • 2 Sparcs 400 Mhz • 1Gb RAM • 8 Agents in // Vizier at CADC (Victoria Canada) Pipeline (Leicester) Vizier at Cambridge (UK) Vizier at CFA (Harvard USA) • Correlation computing • A lot of remote server accesses • A few computation • File product generation • Some remote server accesses (Aladin) • A lot of computation Aladin (Strasbourg France)
The ACDS • Actual Throughput (8 process in //) • 3300 sec /dataset • (8*3600*24)/3300 = 209 datasets/day • Most part of the time is pure CPU (graphical prd) • IDL, Distiller…. • Easy to improve • new hardware
The ACDS • Operation issues • Support the pipeline throughput (but not more) • > 2200 datasets processed since October 13th • 157 pending this morning due to a 2 days Aladin shutdown • No need to improve the hardware • CDS database upgrades • Vizier is being moved on PSQL without serious consequences for us • Major Simbad to be release late 2006 • New error definition (no effect for standard users) • External failures • Remote server shutdown • Corrupted products returned (cache miss-synchronization after crash) • Internal failures • Empty Rosat images/wrong PPS index • Hardware failure • Spare available/never used • Miss-configuration • Holidays
X Source 1 Arch Source 1 Arch Source 2 X Source 2 Arch Source 3 X Source 3 Arch Source 4 Arch Source 5 The XCATDB • The XCATDB is not a data repository but a tool to make science • N to M persistent relationship • Difficult to implement in an efficient way for complex queries • Possibility of doing more than with dynamic cross-matches • Data-mining feature
Saada in a few Words • Saada: An Astronomical Database generator • Making automatic the process of building a database • Hosting heterogeneous datasets • Highlighting scientific content • Publishing personal data into the VO • Databases are installed on local machines • Any Linux/Windows/Mac box • Tomcat 5.xx • PostrgreSQL 8.xx
Source X Arch Source 1 Source X Arch Source 1 Arch Source 2 Source X Source X Arch Source 1 Source X Source X Arch Source 2 The XCATDB • Based on Saada • Queries expressed in SaadaQL • Data loader • How to build quickly millions of correlation links • How to restore archival source uniqueness • GUI • Layout inspired from the 1XMM XCatDB • Architecture based on native Saada GUI • Prototype based on RC2 data open for the community • http://amwdb.u-strasbg.fr/jacds
2nd Release of the XMM Catalogue Native Saada Data-loader 150,000 X-ray sources In 2800 observations Implemented with Saada relationships Specific Module 1,000,000 links Native Saada Data-loader 1,400,000 archival sources Demo
Hardness ratio range for X-ray binaries A simplified example: Looking for High Mass X-ray Binaries HR3 = (F2-4.5keV – F1-2keV) / (F2-4.5keV + F1-2keV) HR4 = (F4.5-12keV– F2-4.5keV) / (F4.5-12keV + F2-4.5keV) kT = 5 – 50keV < 1.5keV Demo Hardness ratio limit for stars
HR Condition With bright counterparts Without WR counterparts
[phot.flux;em.X-ray] > 1e-16 [W/m2] Using UCDs in queries • Selecting X-Rays sources by correlation patterns on 200 classes of X-Ray sources requires to handle a lot of meta-data select oidsaada from Arch_9037AEntry where ( _Flux > 1e-13 ) select oidsaada from Arch_7181AEntry where ( _Flux2 > 1e-16 ) select oidsaada from Arch_9032AEntry where ( _FX > 1e-16 ) select oidsaada from Arch_9015AEntry where ( _fX > 1e-16 ) select oidsaada from Arch_9031AEntry where ( _Fx > 1e-13 ) …………. • Solution: Expressing queries using UCDs and Units Demo
(att1>x1 or att2>x1) and (att1<x2 or att2<x2) (att1>x1 and att1<x2) or (att2>x1 and att2<x2) or (att1>x1 and att2<x2)or (att2>x1 and att1<x2) UCD Issue [UCD]>x1 and [UCD]<x2 Usually makes no sense Solution: Using unambiguous operators: [UCD]In(x1,x2)
UCD Issue (continued) • Query language issue: using a simple syntax • One operator for one UCD • Query execution • Editable execution plan • UCD attribution: a sensible issue • Using multi-words UCDs as often as possible • Limit the columns accessible per UCDs • Take care with units • Existence • Name: known by the converter • Result presentation: another sensible issue • The client must understand how resources have been selected and accessed
XCATDB: Status and Prospects • Data loader • Correlation setup works fine • Other products can be loaded with the native data-loader • GUI • HTML form improvements • Display of constrained values • Interoperability • External Xmatch in SaadaQL (F.X. Pineau) • Connection with ESAC resources (XSA, VOSpec) • Connections with CDS clients (VO portals) • VO protocols