EPICS Internals

1. EPICS Internals Nick Rees

2. Outline • Introduction • Libraries and modules • Data Structures • Process types • Examples of processing

3. Introduction

4. Background • When I accepted this challenge I thought there was an EPICS internals talk I could copy. • It turns out there wasn’t. • I decided the aim of the talk was to provide EPICS developers with a vague idea of what to expect to see when they look at a running EPICS system. • It turns out this wasn’t that easy • It also was very difficult to do this in 40 minutes. • By focussing on this I missed out more than I covered. • The result is something which even I find a bit difficult to understand (and I wrote the talk), so I don’t know about the rest of you. • For the non-programmers out there, good luck.

5. What is EPICS core?

6. What is EPICS core? • 109 directories • 1567 files, 239241 lines of all types • 281 files, 93451 lines of C • 203 files, 49807 lines of C++ • 329 files, 34985 lines of header files • 28 files, 14964 lines of html • Over 20 years of development by a competent world-wide team. • Not an enormous body of code. • A pretty reliable body of code that does what it is designed to do. • A reasonably useful, flexible and portable tool

7. What isn’t EPICS core? • Consistently designed • Designed • Buzzword compliant, or even object-oriented • Well understood by lots of people. • Stylistically consistent. • Pretty, elegant or pure

8. Another view • The Application Developers guide states: • “EPICS consists of a set of core software and a set of optional components. The core software, i.e. the components of EPICS without which EPICS would not function, are: • Channel Access - Client and Server software • IOC Database • Scanners • Monitors • Database Definition Tools • Source/Release” • This talk focuses on the first 4 points, with most emphasis on the IOC Database.

9. Libraries and modules

10. Helper libraries • If you write EPICS base software you must know, understand and use the basic helper libraries it provides. For example: • ellLib – doubly linked list library identical to VxWorks listLib • gpHash – efficient hashes • freeList – malloc replacement giving efficient handling of fixed size blocks. • epicsRingBytes – ring buffers. • epicsThread – thead creation and control • epicsMutex – OS independent mutexes • etc...

11. Essential EPICS code • dbStaticLib.c • Static database access routines • Used during EPICS development or before iocInit. • dbAccess.c • Run-time database access routines. • dbNameToAddr, dbPutField, dbPut, dbGetField, dbGet, dbProcess etc. • dbConvert.c • Type conversions • camessage.c • Channel Access server function table implementations. • dbEvent.c • Implements database event handling • etc.....

12. What EPICS libraries are there? • Common • Cap5 • ca • cas • Com • gdd • Host and IOC: • asHost • asIoc • dbStaticHost • dbStaticIoc • IOC only • dbIoc • dbtoolsIoc • miscIoc • recIoc • registryIoc • rsrvIoc • softDevIoc • testDevIoc

13. EPICS library dependencies

14. Conclusion • If you want to know more, consult the source.

15. Data Structures

16. dbBase.h • Basic database data structures are in dbBase.h • These define the structures that hold the database. • The are defined in the dbd and db files, and created by dbStaticLib. • At run-time, they are accessed by dbAccess routines.

17. dbBase.h record tree • dbBase • recordTypeList • dbRecordType • recordList • dbRecordNode • name • precord • dset • lset etc. • ... • deviceList • ... • dbFldDes[] • rset • ...

18. dbBase.h tree

19. dbBase definition typedefstructdbBase { ELLLIST menuList; ELLLIST recordTypeList; ELLLIST drvList; ELLLIST registrarList; ELLLIST functionList; ELLLIST variableList; ELLLIST bptList; void *pathPvt; structdbPvd *ppvd; structgphPvt *pgpHash; short ignoreMissingMenus; short loadCdefs; }dbBase;

20. dbRecordType definition typedefstructdbRecordType { ELLNODE node; ELLLIST attributeList; /*LIST of attributes*/ ELLLIST recList; /*LIST of sorted dbRecordNodes*/ ELLLIST devList; /*LIST of device support*/ ELLLIST cdefList; /*LIST of Cdef text items*/ char *name; short no_fields; /* number of fields defined */ short no_prompt; /* number of fields to configure*/ short no_links; /* number of links */ short no_aliases; /* number of aliases in recList */ short *link_ind; /* addr of array of ind in papFldDes*/ char **papsortFldName;/* ptr to array of ptr to fld names*/ short *sortFldInd; /* addr of array of ind in papFldDes*/ dbFldDes *pvalFldDes; /*pointer dbFldDes for VAL field*/ short indvalFlddes; /*ind in papFldDes*/ dbFldDes **papFldDes; /* ptr to array of ptr to fldDes*/ /*The following are only available on run time system*/ structrset *prset; intrec_size; /*record size in bytes */ }dbRecordType;

21. dbRecordNode definition typedef struct dbRecordNode { ELLNODE node; void *precord; char *recordname; ELLLIST infoList; /*LIST of info nodes*/ int flags; }dbRecordNode;

22. Record (dbCommon) definition typedefstructdbCommon { char name[61]; /* Record Name */ char desc[41]; /* Descriptor */ char asg[29]; /* Access Security Group */ epicsEnum16 scan; /* Scan Mechanism */ epicsEnum16 pini; /* Process at iocInit */ epicsInt16 phas;/* Scan Phase */ epicsInt16 evnt; /* Event Number */ epicsInt16 tse; /* Time Stamp Event */ DBLINK tsel; /* Time Stamp Link */ epicsEnum16 dtyp; /* Device Type */ epicsInt16 disv; /* Disable Value */ epicsInt16 disa; /* Disable */ DBLINK sdis; /* Scanning Disable */ epicsMutexIdmlok; /* Monitor lock */ ELLLIST mlis; /* Monitor List */ epicsUInt8 disp; /* Disable putField */ epicsUInt8 proc; /* Force Processing */ epicsEnum16 stat; /* Alarm Status */ epicsEnum16 sevr; /* Alarm Severity */ epicsEnum16 nsta; /* New Alarm Status */ epicsEnum16 nsev; /* New Alarm Severity */ epicsEnum16 acks; /* Alarm Ack Severity */ epicsEnum16 ackt; /* Alarm Ack Transient */ epicsEnum16 diss; /* Disable Alarm Sevrty */ epicsUInt8 lcnt; /* Lock Count */ epicsUInt8 pact; /* Record active */ epicsUInt8 putf; /* dbPutField process */ epicsUInt8 rpro; /* Reprocess */ structasgMember *asp; /* Access Security Pvt */ structputNotify *ppn; /* addr of PUTNOTIFY */ structputNotifyRecord *ppnr; structscan_element *spvt; /* Scan Private */ structrset *rset; /* Address of RSET */ structdset *dset; /* DSET address */ void *dpvt; /* Device Private */ structdbRecordType *rdes; /* dbRecordType */ structlockRecord *lset; /* Lock Set */ epicsEnum16 prio; /* Scheduling Priority */ epicsUInt8 tpro; /* Trace Processing */ char bkpt; /* Break Point */ epicsUInt8 udf; /* Undefined */ epicsTimeStamp time; /* Time */ DBLINK flnk; /* Forward Process Link */ } dbCommon;

23. dbAddr definition • Virtually all runtime access to the database is via a dbAddr handle. typedefstructdbAddr { structdbCommon *precord; /* address of record */ void *pfield; /* address of field */ structdbFldDes *pfldDes; /* address of fldDes */ long no_elements; /* number of elements */ short field_type; /* type of field */ short field_size; /* size of the field */ short special; /* special processing */ short dbr_field_type; /* request type */ /* DBR_STRING,...,DBR_ENUM,DBR_NOACCESS */ } dbAddr;

24. Process types

25. EPICS and threads • A typical C/C++ programmer’s first reaction is that EPICS has lots of threads • However, they all have there uses, and once you understand them they are not so confusing. • The need arises from the need to have non-blocking processing and strict priorities. • It is actually simpler and more efficient than a lot of select() calls.

26. EPICS Threads • IOC management: • timerQueue • taskwd • errlog • Database processing: • cb* • scan* • Channel Access: • CAS-* • CAC-event • dbCaLink • Plus various driver threads...

27. IOC management threads • timerQueue • Implements all EPICS base delays • taskwd • Monitors tasks for any suspensions • errlog • Handles asynchronous processing of log messages. • Forwards log messages to the console or to files at low priority.

28. Database processing threads • cb* • callback tasks – used for event and I/O Interrupt scanned records. • Calls any specified function asynchronously • 3 priorities (low, medium high) • scan* • periodic scan tasks • Calls record process() routines at regular intervals. • Higher scan rates have higher priorities. • + all driver threads • Every asyn port has its own thread

29. Channel Access threads • CAS-* • Channel access server tasks • CAS-UDP listens for channel lookups • CAS-TCP listens for TCP connections • CAS-beacon looks for CA beacons • One CAS-client and CAS-event task per client • dbCaLink • Channel Access client that processes CA requests on behalf of the database. • CAC-event • Handles the CA client callback events (I think)

30. Examples of processing

31. Example sequence diagram Task name C filename C function Client IOC tasks IOC function calls

32. CA server call processing • Every CA client message has a function code defined in caProto.h of the form CA_PROTO_XXXX • E.g.: CA_PROTO_WRITE • This will convert into a function in one of the CA jump tables (tcpJumpTable or udpJumpTable). Functions are off the form xxxx_action(mp,pPayload,client) • Parameters are pointers to • CA header block • Data • client structure. • Performs: • Net to host byte swapping • Access security checks • E.g.: write_action(mp,pPayload,client)

33. CA server call processing • xxxx_action often calls a “CA db_access” routine with underscores, which handles an ugly “old to new” type enumeration conversion • E.g.: db_put_field(pAddr, oldDataType,pPayload,count) • This calls a dbAccess routine which is part of the database code. • E.g.: dbPutField(pAddr,newDataType,pDate,count) • This locks the database and implements the appropriate action.

34. caget from a record

35. Channel life cycle

36. caput to an ai record

37. dbPutField long epicsShareAPI dbPutField(DBADDR *paddr, short dbrType, const void *pbuffer, long nRequest) { long status = 0; dbFldDes *pfldDes = paddr->pfldDes; dbCommon *precord = paddr->precord; short dbfType = paddr->field_type; /* Various sanity tests deleted here... */ dbScanLock(precord); status = dbPut(paddr, dbrType, pbuffer, nRequest); if (status == 0) { if (paddr->pfield == (void *)&precord->proc || (pfldDes->process_passive && precord->scan == 0 && dbrType < DBR_PUT_ACKT)) { if (precord->pact) { precord->rpro = TRUE; } else { /* indicate that dbPutField called dbProcess */ precord->putf = TRUE; status = dbProcess(precord); } } } dbScanUnlock(precord); return status; }

38. dbPut dbPut(DBADDR *paddr, short dbrType, const void *pbuffer, long nRequest) { long special = paddr->special; short field_type = paddr->field_type; long no_elements = paddr->no_elements; if (special) dbPutSpecial(paddr, 0); if ( IS AN ARRAY ) prset->get_array_info(paddr, &dummy, &offset); dbPutConvertRoutine[dbrType][field_type](paddr, pbuffer, nRequest, no_elements, offset); if ( IS AN ARRAY ) prset->put_array_info(paddr, nRequest); if (special) dbPutSpecial(paddr,1); if ( not ( VALUE FIELD and PP )) db_post_events(precord, paddr->pfield, DBE_VALUE | DBE_LOG); return; }

39. camonitor from an ai record

40. caput callback from an ai record

41. What’s the stack trace like? CAS-client thread Socket recv call (caput starts here) Process CA messages caput CA action

42. caput to an ai record CAS-client thread Socket recv call (caput starts here) Process CA messages caput CA action

43. caput callback to an ai record CAS-client task

44. camonitor from an ai record scan1 task

45. Camonitor event CAS-event task

46. caget from an ai record CAS-client task

47. The end • What did I miss? • IOC initialisation • Database definition • Static database access • dbStatic • Test facilities • Access security • IOC shell • Registry • OSI • libCom • Build rules • Record, device and driver support • Basically everything.....

48. Resources • EPICS Application Developers Guide • Channel Access Protocol Specification • http://epics.cosylab.com/cosyjava/JCA-Common/Documentation/CAproto.html • Training slides on EPICS web site • Eclipse is your friend... • “module load ddd” gives you the ddd graphical debugger.