80 likes | 92 Views
IRMIS PV Crawler. Auto-populating the IRMIS PV Schema. IRMIS PV Crawler. Requirements Perl 5 with DBI and DBD:mysql modules MySQL 4.x (4.1 preferred) EPICS msi tool (for now…) Reasonably consistent ioc boot process Some ability to write/edit perl code To create *BootScan module
E N D
IRMIS PV Crawler Auto-populating the IRMIS PV Schema
IRMIS PV Crawler • Requirements • Perl 5 with DBI and DBD:mysql modules • MySQL 4.x (4.1 preferred) • EPICS msi tool (for now…) • Reasonably consistent ioc boot process • Some ability to write/edit perl code • To create *BootScan module • To handle site-specific boot peculiarities
IRMIS PV Crawler D B I db.properties pv_crawler.pl path.properties ioc Parser *BootScan Ioc_boot, ioc_resource Rec,fld,… MySQL bootparams st.cmd dbd db IOC boot file system(s)
IRMIS PV Crawler • What pv_crawler does (every 5 minutes): • Identify ioc’s to be scanned and their last known boot date (ie. APSBootScan.pm or SNSBootScan.pm) • foreach ioc • Check for new boot date • If so, insert new ioc_boot record in db and proceed • If not, skip to next ioc • Parse st.cmd (collecting all info on dbd and db files) • Check for any dbd or db file time mods • If so, insert all pv and field data in db for ioc
IRMIS PV Crawler • Sample output from test run [bacchus]saunders> pv_crawler.pl --go --boot-scan=APSBootScan --test --debug-level=debug I|20050304105259|pv crawler beginning usage: msi -V -Ipath ... -Msub ... -Ssubfile template Specifying path will replace the default '.' stdin is used if template is not given D|20050304105259|PVCrawlerDBLayer: begin ioc_find D|20050304105259|iocs to be boot scanned: iocpar01 I|20050304105259|Processing ioc iocpar01, boot date: 20050118153007 I|20050304105259|Parsing /usr/local/iocapps/R3.13.10/ioc/par/3/iocBoot/iocpar01/st.cmd D|20050304105259|script mod time = 20050114070431 D|20050304105259|sl = < cdCommands D|20050304105259|script mod time = 20050223113545 D|20050304105259|sl = startup = "/net/helios/iocapps/R3.13.10/ioc/par/3/iocBoot/iocpar01" <------cut--------> D|20050304105259|sl = dbLoadDatabase("dbd/par01App.dbd") D|20050304105259|parse_function_call D|20050304105259|function dbLoadDatabase with args ("dbd/par01App.dbd") D|20050304105259|opening /net/helios/iocapps/R3.13.10/ioc/par/3/dbd/par01App.dbd D|20050304105259|sl = dbLoadRecords "par01App/commonDb/par01status.db" D|20050304105259|parse_function_call D|20050304105259|function dbLoadRecords with args "par01App/commonDb/par01status.db" D|20050304105259|calling parse_db with /net/helios/iocapps/R3.13.10/ioc/par/3/par01App/commonDb/par 01status.db <------cut--------> D|20050304105259|check_for_file_mods(): no record found in db, so return 1 I|20050304105259|IOC Resources changed: 1, Parse error: 0 I|20050304105259|Write data to DB I|20050304105259|pv crawler done
IRMIS PV Crawler • What info do we store over time for a given IOC? • Boot history • Each new ioc boot is recorded with date, regardless of whether any configuration has changed • If there is an identifiable problem with the boot, or inability to parse it’s configuration, an associated error is recorded • File system resources(only if changed from last crawl) • Path to each dbd and db file that make up configuration, along with file modified date and any text substitutions applied • Record type definitions (dbd) (only if changed from last crawl) • The set of fields defined for each record type at that time • Record definitions (db) (only if changed from last crawl) • Record name, fields and their default/overriden values
IRMIS PV Crawler rdbCore PV Schema
IRMIS PV Crawler • Experience to date • Running continuously for weeks now • No involvement of ioc developers time • All but 3 of 265 IOCs crawled successfully (including soft iocs) • Continual crawler parser improvement required, but has settled down to trivial cases now. • Crawler log raised useful issues for about 16 IOCs • Ioc’s registered as active in ioc table, but were not • Some file system resources unreachable due to perms • Identified ioc’s booting out of home directory • Identified references to non-existent st.cmd, dbd, db files • Fields defined in db files that don’t exist in record type • Identified ioc’s outside of general boot guidelines • Duplicate record definitions