Information Types and Registries

1. Information Types and Registries Corporation for National Research Initiatives Giridhar Manepalli Corporation for National Research Initiatives Strategies for Discovering Online Data BRDI Symposium – Feb 26, 2013

2. Research Data Interoperability Enabling Technologies ID ID ID ID ID 0100 0101.. 0100 0101.. 0100 0101.. 0100 0101.. 0100 0101.. Discovery ID ID ID ID ID ID ID ID Access ID ID ID ID ID Corporation for National Research Initiatives Scientists, Data Curators, End Users, Applications Interpretation Datasets Accessed via Repositories Reuse

3. Research Data Interoperability (cont.) • Interoperability of research data allows discovery, access, interpretation, and reuse of datasets by researchers • Examples • Discovery: A scientist from US “discovers” datasets from research in Germany, in related or even unrelated domain • Reuse: A scientist from US “re-uses” or processes datasets from the discovered research in Germany • For interpretation of accessible datasets, Types and Type Registries play a significant role Corporation for National Research Initiatives

4. Information Types – Our Definition • What they are not: • Programmatic data types (string, integer, double, etc.) • Mime types as normally used (text/xml, application/rdf) • Types are identifiers that, with the help of associated metadata, characterize data structures used for managing information • Data structures could be at multiple levels of granularity • Individual observations, to sets of observations within a time series, to multiple time-series sets that explain a phenomenon • Usually • Spread across multiple files (each with specific mime type) • Distributed on the network (managed by various repositories) • We call such data structures used for managing information digital objects • Types (aka type identifiers) are unique across their user base • Types are associated with machine-readable metadata to support interpretation of information • CNRI’s focus is to support infrastructure for enabling inter-discipline types Type IDMachine Readable Metadata Digital Object File File File File Network Corporation for National Research Initiatives Typed Digital Object

5. Value Proposition of Info. Types • Typing allows • Grouping of digital objects generated in different times and domains for reasoning and establishing correlations between different types of objects • Grouping is an aspect fundamental to humans for reasoning about things • Creation of services that can automate information processing based on information types • Advanced information processing can be performed for finding unforeseen correlations, trends, etc. • This type of advanced processing has different names: data-intensive science, fourth paradigm, big-data analytics, etc. Type B Type A Digital Object Type C Typed Digital Object Collection Corporation for National Research Initiatives

6. Value Proposition of Info. Types (cont.) Rights Visualization Interaction Terms:… Terms:… Terms:… I Agree I Agree I Agree SUITE OF SERVICES Data Set Dissemination Data Processing 2 5 4 3 1 10100 11010 101…. 10100 11010 101…. Corporation for National Research Initiatives 10100 11010 101…. User requests Type from a Digital Object of interest. Digital Objects Type ID is returned to the user. User requests the Type Registry for the Type info. Type Registry Type Info is returned to the user containing Services Info. User requests a Service for processing.

7. Info. Typing Challenges • Challenge: When are two digital objects assigned the same type? • When the bit-level encoding matches? • Or when the higher-level structures and intent matches? • If two observations are made by two similar instruments at the same time on the same entity, would the data generated by those two observations be constituted as being of the same type? • Even if the data generated by each observation, similar in concept, has a different format (e.g., JPEG vs. PNG)? • Our approach: • Intent wins over optics (formats, encodings, etc.) • The metadata associated with the type could list possible formats, encodings, etc. • Alternative approach: • Establish a base type and then sub-type for accommodating variations • Our experience was that it was too cumbersome to deal with multiple formats, encodings at the type definition level Corporation for National Research Initiatives

8. Info. Typing Challenges (cont.) • Challenge: Can the same digital object be assigned multiple types? • If so, how do we deal with duplicate types? • If not, how do we manage multiple types assigned by several domains? • Our approach: • An object is assigned an inter-discipline type only once. • Any domain-specific types are listed in its metadata Inter-discipline Type Type I Machine-readable Metadata Type α Type β Corporation for National Research Initiatives Type β Type α Computer Scientist Biologist Typed Digital Object Collection

9. Info. Typing Challenges (cont.) • Challenge: How can existing information be typed under this new scheme? • A lot of information exists already • One approach: • Start with domain-specific types, if any, and generate domain-neutral types and list the domain-specific types in their metadata records Corporation for National Research Initiatives

10. Info. Types – Machine-readable Metadata • Machine-readable metadata for Info. Types is still an area of research for us • Type interdependence • It is clear that sub-typing is needed for building on previously defined types • Our experience shows that sub-typing based on variations in formats and encoding is a cumbersome process • Instead, an exhaustive list of possible formats and encodings may be specified in the metadata • Domain-specific Types • Cross-domain Types could list or point at domain-specific types which could be multiple for a given object, and which might define detailed semantics for interpretation • Metadata for automated interpretation • For the few types of information we prototyped, defining metadata that helps services process datasets is loose ended and sometimes impractical • A parsing-language or a pseudo-code may instead be captured that transforms datasets into domain-specific ontologies or semantics Corporation for National Research Initiatives

11. Info. Type Registries • Info. Type Registries are metadata registries that • Support recording of information types and associated metadata records • Perform federation across other registries • De-duplicate (or match types) to control registration requests of existing types • Include manual moderation and/or crowd sourcing function for spotting redundant registrations (optional) • Cross-domain Type Registries may optionally link to domain-specific Type Registries • Type Registries may manage or reference services that process information of certain types • CNRI has vast experience building metadata registries Corporation for National Research Initiatives

12. Next Steps • Received Sloan Foundation funding to research Type Registries within scientific and financial communities • CNRI employees lead and participate in a Type Registry working group within the Research Data Alliance • Technical goal is to define the scope of ‘Information Type’ by working in aforementioned projects, and build and release an open-source Type Registry in the next 18 months. Corporation for National Research Initiatives