Centera Integration Training

1. Centera Integration Training Advanced Centera Topics Centera Corporate Systems Engineering Sept 2007

2. Centera Integration Training: Advanced Centera Topics Agenda • Retention Management • Advanced Retention Management features • Retention Governors • Event Based Retention • Litigation Hold • Extending Retention (non-EBR) • Replication of Mutable Metadata • Content Addresses • Standard Formats • ClipID Canonical Form • SDK Logging • Wrap Up and Questions

3. Centera Integration Training: Advanced Centera Topics Advanced Retention Management • Introduced in CentraStar / SDK 3.1 • Separate licence • 3 main features • Retention Governors • Event Based Retention • Litigation Hold

4. Centera Integration Training: Advanced Centera Topics Enforced Retention • The presence of retention information can now be enforced • reject C-Clips without retention period or class. • Applies to pre-3.1 SDK only • 3.1 SDK adds Default Retention Period if Application does not • disabled by default • Valid retention value constraints can also be added • Min / Max retention Governors set on CentraStar • SDK enforces RetentionPeriod values to fall within the range

5. Centera Integration Training: Advanced Centera Topics Min / Max Retention Governors • Applied on Write, Replication and Restore • Only applies to Retention Period values • C-Clips with a Retention Class are not checked against Min / Max • Not examined during Delete • Available on a per pool basis • SDK Returns an error if value is outside the Min/Max constraints: • FP_RETENTION_OUT_OF_BOUNDS_ERR • FPPool_GetCapability(FP_RETENTION, attribute) can check the limits • FP_FIXED_RETENTION_MIN • FP_FIXED_RETENTION_MAX • FP_VARIABLE_RETENTION_MIN • FP_VARIABLE_RETENTION_MAX

6. Centera Integration Training: Advanced Centera Topics Advanced Retention Management – Retention Governors Application Application creates C-Clip Application deletes C-Clip SDK SDK throws error SDK checks C-Clip Retention Information Present?   Apply defaultretention for the pool (Infinite if CE+) Retention Fits in Fixed Min/Max?   SDK writes C-Clip CentraStar Server persists CDF and returns CA Server deletes C-Clip

7. Centera Integration Training: Advanced Centera Topics Advanced Retention: Default Values • Default values for all pools after enabling license: • Fixed retention minimum 0, maximum infinite • No enforced presence of fixed retention • Event based retention minimum 0, maximum infinite • Default Retention Periods must be within the Min/Max settings if present (except max in CE+, which can exceed it)

8. Centera Integration Training: Advanced Centera Topics Event Based Retention (EBR) • Fixed retention can still be specified as before (period or class) • Must now comply with rules for Min/Max Governor • EBR can also be set on new C-Clips or on copies of existing C-Clips • Disabled by default • Enabled using FPClip_EnableEBRWithPeriod / Class • Period must also comply with rules for Min/Max Governor • Stored in the C-Clip as before (immutable) • No default value, but period or Class must be specified up-front

9. Centera Integration Training: Advanced Centera Topics Event Based Retention (EBR) • EBR values applied when “triggered” onceby afuture event • FPClip_TriggerEBREvent uses the policy set when C-Clip created • New policy can be applied instead • FPClip_TriggerEBREventWithPeriod /Class • The longer of the old and new values will be used • New value only applied if longer than original setting • Must comply with rules for Min/Max Governor • Event Date is set by SDK based on the cluster time • Note: EBR information is not exported / imported by the Raw API commands

10. Centera Integration Training: Advanced Centera Topics Event Based Retention – Reporting • FPClip_IsEBREnabled: • Return TRUE if EBR is enabled, FALSE otherwise. • FPClip_GetEBREventTime: • Returns the time the EBR event was triggered • 0 if the event has not been triggered orthe C-Clipis not EBR enabled.

11. Centera Integration Training: Advanced Centera Topics Event Based Retention – Reporting • FPClip_GetEBRPeriod: • Returns the greater EBR period associated with the clip • Initial value when C-Clip created / value set on Event trigger • If EBR set with a class, the value of the class is used • 0 returned if the Clip is not EBR enabled. • FPClip_GetEBRClassName: • Returns name of the EBR retention class associated with the clip • Returns empty string if no class is set • … or if EBR period set on clip > period value of the class

12. Centera Integration Training: Advanced Centera Topics Event Based Retention - Expiry • Deletion is possible when: • Event has occurred • AND Current time > creation date + fixed retention • AND Current time > event date + variable retention • Note: Privileged Delete can still overrides these rules. event not specified fixed retention variable retention deleteallowed C1 fixed retention event not specified variable retention deleteallowed C2 fixed retention event not specified variable retention deleteallowed C3 Time C-Clip Created Event

13. Centera Integration Training: Advanced Centera Topics Litigation hold • Litigation Hold(s) can be set / removed at C-Clip level • Retention is held indefinitelyuntil removed. • Privileged Delete cannot override the Hold! • When a C-Clip is put under litigation hold via FPClip_SetRetentionHold the server tracks (outside the CDF): • Litigation identifier – text string uniquely identifying the Hold • Litigation hold flag – true/false • Date/Time when the C-Clip was put under litigation hold. • Profile that put the C-Clip under litigation hold.

14. Centera Integration Training: Advanced Centera Topics Litigation hold • C-Clip is released from a Hold by setting the hold flag to false • A C-Clip can be under multiple Litigation Holds at any time. • FPClip_GetRetentionHold returns TRUE if there are active Hold(s) on a C-Clip • Note: Cannot query for C-Clips that have active Hold(s) or obtain keys of C-Clips on hold

15. Centera Integration Training: Advanced Centera Topics Advanced Retention Management – Event Based Retention Application Application creates C-Clip Application sets event SDK SDK throws error SDK checks C-Clip SDK checks event data Retention Information Present?  Variable Retention Information Present?    Apply defaultretention for the pool (0 if CE+) Retention Fits in Variable Min/Max?  Retention Fits in Fixed Min/Max?    SDK writes C-Clip SDK sends event CentraStar Server persists CDF and returns CA Server processes event

16. Centera Integration Training: Advanced Centera Topics Extending Retention without EBR • Centera has a mechanism for placing a C-clip under fixed retention • Once a C-clip has been placed under fixed retention, the retention period cannot be modified • Customers may want to place a document under short term temporary retention, and then extend the retention period multiple times in the future • Note: this approach works well for extending the retention period • Decreasing retention may require Privileged Delete • To delete older CDF versions where retention has not yet expired • Not allowed on CE+

17. Centera Integration Training: Advanced Centera Topics Extending Retention – Step 1 – C-Clip is created • C-clip is created with a retention period / class • e.g. 30 days • Application stores the content address of the C-Clip • e.g. in local database DQG22UC39LHJOe66R2K2JIBD4EF Client Database CDF retention = 30 days blob

18. Centera Integration Training: Advanced Centera Topics Extending Retention – Step 2 – Retention is extended • At some point in time, retention requires to be extended • C-Clip is opened • New retention value is set in the CDF e.g. 60 days • C-Clip is written and the new ContentAddress is stored by the application • Replaces the original content address in the application database • Standard SDK metadata in the new C-Clip contains old content address DQG22UC39LHJOe66R2K2JIBD4EF EFG242769LH32e57R23E2IBC4FE Client Database CDF retention = 30 days blob CDF retention = 60 days prev.clip=DQG22UC39LHJOe66R2K2JIBD4EF

19. CDF retention = 30 days EFG242769LH32e57R23E2IBC4FE E3423ABE3D5EBe34E3D32156A3a Client Database CDF retention = 60 days blob CDF retention = 7 years prev.clip=EFG242769LH32e57R23E2IBC4FE Centera Integration Training: Advanced Centera Topics Extending Retention – Step 3 – Further extension of Retention • Retention could be extended multiple times • EBR can only extend retention once

20. Centera Integration Training: Advanced Centera Topics Extending Retention – Step 4 - Deletion • C-Clip has had retention extended several times • “List” of CDFs to delete • Traverse the list deleting the CDFs • use previous clip ID as the “pointer” to the “ancestor” CDF • FPClip_GetDescriptionAttribute (clipID, “prev.clip”, outputBuffer, size) • Original CDF has empty value for previous Clip ID • Blob is now unreferenced and will be Garbage Collected CDF retention = 30 days prev.clip= CDF retention = 60 days blob prev.clip=DQG22UC39LHJOe66R2K2JIBD4EF CDF retention = 7 years prev.clip=EF242769LH32e57R23E2IBC4FE

21. Replication will replicate any change in EBR and LH information Profile used for replication needs several capabilities clip-copy (c) for regular replication delete (d) for global delete privileged-delete (D) global privileged delete clip-copy (c) for EBR information litigation-hold (h) for LH information Centera Integration Training: Advanced Centera Topics Replication of Mutable Metadata

22. Content Address Formats Centera Integration Training: Advanced Centera Topics From 2.4 CentraStar, MG and GM forms are no longer used.

23. Centera Integration Training: Advanced Centera Topics ClipID Canonical Format • Binary encoding for Content Address • Enables independence from character encoding specific to, e.g., Mainframe and Open Systems • Content addresses generated by Mainframe use EBCDIC encoding! • Reduces byte count of CA from 65 bytes to 40 bytes • Handy for saving space, e.g., in a database record • Is not NULL-terminated FPClipID_GetCanonicalFormat (const FPClipID inClipID, FPCanonicalClipID outClipID) FPClipID_GetStringFormat (const FPCanonicalClipID inClipID, FPClipID outClipID)

24. Centera Integration Training: Advanced Centera Topics SDK Logging • Highly configurable by category • Debug messages / API calls / Warnings / Errors • … component • POOL, RETRY, XML, API, NET, TRANS, PACKET, EXCEPT • REFS, MOPI, STREAM, CSOD, CSO, MD5, SHA • APP (for application level logging) • LIB (for library loading) • and file properties • Format / Path / Size / Overflow (backups)

25. Centera Integration Training: Advanced Centera Topics Enabling SDK Logging with Configuration File • FPLogState.cfg file in application directory • Default method • Behaviour is dynamically controllable via changes in file state • Enable by adding file to directory • Disable by deleting file from directory • Alter by editing existing file in directory • Polling interval is configurable (default is 5 minutes) • FP_LOG_STATE_PATH environment variable • SDK tests for this setting only ifdefault configuration file not found • determines non-default location for configuration file • Behaviour is then the same as above

26. Centera Integration Training: Advanced Centera Topics Enabling SDK Logging via Environment Variables • FP_LOGPATH – location of the output file • FP_LOGKEEP – creation mode • CREATE / APPEND / OVERWRITE • FP_LOGLEVEL – category of messages to log • FP_LOGLEVEL_ERROR / WARN / LOG / DEBUG • FP_LOGFILTER – components to log messages from • FP_LOGFORMAT – XML or TAB formatting • FP_LOG_DISABLE_CALLBACK • Disable the use of any application supplied logging callback method • FP_LOG_MAX_OVERFLOWS – number of backup logs • FP_LOG_MAX_SIZE – maximum size of log file • FP_LOG_STATE_POLL_INTERVAL • Polling frequency in minutes, default=5 (-1: off, 0: on each log event)

27. Centera Integration Training: Advanced Centera Topics Configuring SDK Logging using API Calls • FPLogStateRef object holds current status • Get / Set methods change the property values • Serializable i.e. can be saved to or loaded from file • API calls to change logging status dynamically • FPLogging_Start(FPLogStateRef) • FPLogging_Stop() • Application can also log messages to the SDK logfile • FPLogging_Log(const FPLogLevel, const char *) • … or register its own logging method for the SDK to use • FPLogging_RegisterCallback(FPLogProc) • Can be disabled if required

28. Centera Integration Training: Advanced Centera Topics Questions? • Please Visit SDK Forums on Centera Developer’s Portal • http://lighthouse.emc.com • Corporate Systems Engineering Team provides: • Info, Training, Whitepapers • Design Advice / Design Reviews • Debugging Support • Code Reviews • Email Support Available from: • CenteraCSE@emc.com