TUE / Information Retrieval Reference Structures part 1 - introduction Trezorix & RNA-project

TUE / Information RetrievalReference Structurespart 1 - introductionTrezorix&RNA-project 1

TUE / Information Retrieval / Reference Structures (1) contents chapter 1 - backgroundTrezorixRNA-project chapter 2 - dynamic knowledge systemswhat is a knowledge system?reference structureswhat is dynamic? chapter 3 - overview workflowsinfrastructurereference structuresdescription networkscontent metadatasearch systems chapter 4 - dutch register of species (RNA-casus 8)taxonomic thesaurusregister of species 2

TUE / Information Retrieval / Reference Structures (1) background / Trezorix officially founded on february 1st 2000 the early history of Trezorix is closely related to the building of the new Naturalis (1996-1998) Naturalis and partners and Trezorix and partners share a vision about the role modern cultural and science institutions should and/or can play as knowledge providers networking plays a key role in this vision, digital heritage is a good example 4

TUE / Information Retrieval / Reference Structures (1) background / RNA-project the RNA-project deals with findability in knowledge networks, with emphasis on practical application of existing technologies and economic feasability the organization of the project is based on the linking of a number of related cases from the area of cultural heritage and science concerning retrieval of web content with a number of promising technical solutions the project is funded by the Ministry of Economic Affairs and the Ministry of Education, Culture and Science in The Netherlands - the project started at Januari 1st 2005 and will be finished at December 31st 2007 5

TUE / Information Retrieval / Reference Structures (1) background / RNA-project Cultural heritage institutions of GroningenDutch Army MuseumDutch National Museum of Natural History - NaturalisDutch National Service for Archaeological Heritage - ROBDutch Public Library AssociationFree University Amsterdam - Computer ScienceUniversity of Tilburg – Language and Information ScienceAduna BVCheperu BVTextkernel BVTrezorix BV (project holder) 6

TUE / Information Retrieval / Reference Structures (1) dynamic knowledge systems / what is a knowledge system? user reference content ? content producer end user what do we mean with knowledge system? “a system which is used for storage and retrieval of knowledge” for instance: a private (yours) or public library for instance: a network of knowledge intensive websites reference systems play a key role in findability in knowledge systems 8

TUE / Information Retrieval / Reference Structures (1) dynamic knowledge systems / reference structures user reference content concepts content producer end user The term reference structure is used as an umbrella term for all types of structured concept lists, such as ontologies, taxonomies, thesauri, etc., which are primarily used to enhance and optimize findability. A reference network is a set of reference structures, connected in a transparent way for both content producer and end user. Content which references concepts from the different reference structures in the network, can be accessed as if belonging to one single collection. 9

metadata TUE / Information Retrieval / Reference Structures (1) dynamic knowledge systems / role of reference structures user reference content concepts content producer end user reference structures are powerful navigation guides: they can show the user the structure of the knowledge domain reference structures are important for improved search: concepts (can) contain naming variations, symantic relations content metadata can improve this further because they enrich unstructured content with structured information 10

TUE / Information Retrieval / Reference Structures (1) dynamic knowledge systems / what is dynamic? user reference content content producer end user it must be possible to edit, add or delete content and concepts continuously maintenance of connections between newly added or existing content and concepts should be realised in a highly automated way this makes a dynamic knowledge system, in which findability is updated and optimized continuously also ‘old fashioned’ paper based systems can be dynamic 11

TUE / Information Retrieval / Reference Structures (1) overview workflows a. framework b. reference structures c. reference network a. structure management b. concept card management c. scope management a. lemma networks b. object descriptions a. editing metadata b. managing candidate concepts c. linking content and concepts a. concept search b. search term utilities c. facet navigation infrastructure 1 reference structures 2 description networks 3 content metadata 4 search systems 5 13

TUE / Information Retrieval / Reference Structures (1) overview workflows / infrastructure user reference content concepts metadata technical manager descriptions the RNA-project uses RDF as a basic format for representation of: reference structures other structures, like object descriptions metadata linked to content 14

TUE / Information Retrieval / Reference Structures (1) overview workflows / infrastructure / a. rdf-framework user reference content concepts metadata technical manager descriptions with a RDF-framework all structures are stored and retrieved in an uniform way 15

TUE / Information Retrieval / Reference Structures (1) overview workflows / infrastructure / b. building reference structures user reference content technical manager most knowledge institutions have all sorts of reference structures, like thesauri, glossaria and taxonomies hardly any of these structures is available in RDF – most of them are stored in a ‘regular’ database or spreadsheet the SKOS-format (Simple Knowledge Organisation System) is very suitable to handle these ISO-2788 type of reference structures 16

other reference structure other reference structure register TUE / Information Retrieval / Reference Structures (1) overview workflows / infrastructure / c. reference networks user reference content technical manager reference networks are created by connecting reference structures a central register administrates which reference structures are part of the reference network - it is the base for all services of the infrastructure access control takes care of distribution in the network of rights pertaining to individual reference structures 17

TUE / Information Retrieval / Reference Structures (1) overview workflows / reference structures user reference content reference manager non-technical reference experts must be able to edit reference structures 19

TUE / Information Retrieval / Reference Structures (1) overview workflows / reference structures / a. structure management user reference content reference manager functionality to change the structure of a reference structure, by adding, moving or deleting (groups of) concepts functionality to make connections with (parts of) other reference structures 20

TUE / Information Retrieval / Reference Structures (1) overview workflows / reference structures / b. concept card management user reference content reference manager functionality to add or delete synonyms, name variants, etcetera within the concept data (concept card) functionality to establish links from within a concept card to other concepts (possibly from other structures) 21

TUE / Information Retrieval / Reference Structures (1) overview workflows / reference structures / c. scope management user reference content reference manager often reference structures are targeted at specific user groups so several versions of basically the same structure may exist within one domain such combinations of related structures for different contexts can become very complex and very difficult to maintain scope management should make this job easier 22

TUE / Information Retrieval / Reference Structures (1) overview workflows / description networks descriptions user reference content lemmas description expert in the RNA-project for practical reasons a category is introduced: descriptions descriptions are more or less in between reference and content they can form networks based on relations other then the semantic relations of reference structures examples in the RNA-project are lemma networks and object descriptions 24

TUE / Information Retrieval / Reference Structures (1) overview workflows / description networks / a. lemma networks user reference content lemmas description expert lemmas are encyclopedia-like descriptions of concepts on the one hand they are often closely coupled to concepts in a reference structure... ... on the other hand the descriptions itself can be considered as content independent links can be made based on the occurence of lemma keywords in descriptions of other lemma keywords 25

TUE / Information Retrieval / Reference Structures (1) overview workflows / description networks / b. object descriptions descriptions user reference content description expert object descriptions often exist of a structure of entities (fields), which places them close to reference structures but both as a whole and because of the often elaborate information in the description fields they can be treated as content as well here too, independent links can be made based on occurence of keywords in descriptions 26

TUE / Information Retrieval / Reference Structures (1) overview workflows / content metadata user reference content metadata writer / editor for flexibility linking concepts and content is done most of the time from within the content entities in content metadata are matched to concepts in reference structures 28

TUE / Information Retrieval / Reference Structures (1) overview workflows / content metadata / a. editing metadata user referenc e content metadata writer / editor in most cases an writer or editor applies metadata to the content he/she is producing an editor must know which reference structures are available, and which concepts are present in these structures he must be able to access specific concept data (the concept card), for instance to check the availability of certain synonyms or alternative spelling 29

TUE / Information Retrieval / Reference Structures (1) overview workflows / content metadata / b. managing candidate conceps user reference content metadata writer / editor candidate concept a ‘regular’ editor can submit candidate concepts first he does a number a number of checks: is the concept already present in some form (for instance as a synonym)? an editor with a higher authorisation level must finally accept or reject the candidate concept 30

TUE / Information Retrieval / Reference Structures (1) overview workflows / content metadata / c. linking content and concepts user referentie content concepts metadata writer / editor when a new concept is added to a reference structure, a check is done: is there existing content which is related to that concept? to link large amounts of existing, not-yet-classified content (no metadata) to reference structures language technology can be very useful 31

TUE / Information Retrieval / Reference Structures (1) overview workflows / search systems user reference content ? ! end user general search engines bring a user from anywhere in the big wide world to a specific site once the user has arrived in a knowledge network, the requirements for search result can go to a higher level search utilities within the network, coupled to exact knowledge of what is inside, can help the end user to get to a better search result 33

TUE / Information Retrieval / Reference Structures (1) overview workflows / search systems / a. concept search user reference content broader term term term synonym end user concept search extends the range of a single search term to: other terms within the same concept and other terms in related concepts this gives the end user a semantic overview 34

TUE / Information Retrieval / Reference Structures (1) overview workflows / search systems / b. input support gebruiker referentie content tea term test te... eindgebruiker several utilities can help the user (searcher) with submitting a search term search term anticipation offers a list of available keywords based on the characters the user has typed so far search term suggestion makes suggestions for search terms which might be good alternatives for a submitted term with no hits this type of support gives the end user a lexicografical overview 35

TUE / Information Retrieval / Reference Structures (1) overview workflows / search systems / c. facet navigation user reference content select 1 select 2 select 3 end user a combined search entry can be formulated very precisely by making selections in an overview of available search facets this makes it easy for the end user to see the structure of the source 36

TUE / Information Retrieval / Reference Structures (1) NSR / taxonomic thesaurus the term taxonomic thesaurus means literally: a thesaurus structure to organize specifically taxonomic information but most of the time the term is used for a taxonomic classification system used in natural history which is based on the Taxonomy of Linnaeus Carl Linnaeus (1707-1778) is often called the Father of Taxonomy - his system for naming, ranking, and classifying organisms is still in wide use today 38

TUE / Information Retrieval / Reference Structures (1) NSR / taxonomic thesaurus a taxon is a named group of organisms of a certain rank or: a taxon is a biological concept in Linnaeus's original system, taxa were grouped into genera, genera into orders, orders into classes, and classes into kingdoms the kingdom Animalia contained the class Vertebrata, which contained the order Primates, which contained the genus Homo with the species sapiens later biologists added additional ranks between these to express additional levels of similarity 39

TUE / Information Retrieval / Reference Structures (1) NSR / taxonomic thesaurus at present the followong ranking is used (italic ranks are optional): KingdomPhylumSubphylumSuperclassClassSubclassSuperorderOrderSuborderSuperfamilyFamilySubfamilyGenusSubgenusSpeciesSubspecies 40

TUE / Information Retrieval / Reference Structures (1) NSR / taxonomic thesaurus the example of the homo sapiens: Kingdom:Phylum:Class: Order: Superfamily: Family: Genus: Species: life on earthAnimalia (animals)Chordata (chordates) Mammalia (mammals)Primates (primates)Hominoidea (man-like primates)Hominidae (apes)Homo (man)Homo sapiens (modern man) 41

TUE / Information Retrieval / Reference Structures (1) NSR / taxonomic thesaurus the example of the homo sapiens according to the Tree of Life project: Eukariotes:Animals:Bilateria:Deuterostomia:Chordata:Craniata:Vertebrata:Gnathostomata:Scarcopterygii:Terrestrial vertebrates:Amniota:Synapsida:Therapsida:Mammalia:Eutheria:Primates:Catharrhinii:Hominidae: Homo: Homo sapiens: life on earthProtists, Plants, Fungi, Animals, etc.Metazoa most animals, including vertebrates, etc.vertebrates, echinoderms, tunicates, etc.vertebrates and relativeshagfishes and vertebrateslampreys and jawed vertebratesjawed vertebrates lobe-finned fishes and four-legged vertebratesmammals, reptiles, amphibians, birds, etc.mammals, reptiles, birds, dinosaurs, etc.mammals and their extinct relativesmammals and their extinct relativesmammalsplacental mammalsmonkeys, lemurs, gorillas, chimps, humans, etc.humans, great apes, gibbons, old world monkeyshumans, gorilla, chimpanzee, bonobo, urangutanhumansmodern humans 42

TUE / Information Retrieval / Reference Structures (1) NSR / taxonomic thesaurus the search for a ‘natural system’ of classification is still going on – so there are many versions of ‘the’ taxonomic thesaurus, and even within versions things change continuously all changes have to be published (the old situation, the actor, the date of change, the authority who approved the change, the date of the approvement, etc.) change-records have to stay linked to the original taxon 43

TUE / Information Retrieval / Reference Structures (1) NSR / taxonomic thesaurus an example of change in structure suppose a Class A with three Orders beneath: Order B1, Order C1 and Order C2 suppose Order C1 and Order C2 are very much alike, so it is decided to group them under a Subclass C then Order B1 and Subclass C become both children of Class A but this violates the taxonomic rule which states that siblings must be of equal rank therefore it is necessary to place a new Subclass B between Class A and Order B1 44

TUE / Information Retrieval / Reference Structures (1) NSR / register of species Nederlands Soortenregister (NSR): a register of Dutch species a tree of life of Dutch species a trustworthy register of Dutch biodiversity a trustworthy name reference of Dutch species authorized links to image library authorized links to occurrence and protection data continuously updated and improved by an extensive network of experts 45

TUE / Information Retrieval / Reference Structures (1) NSR / register of species Nederlands Soortenregister has direct links with other databases and sites: protection status database of the Dutch Ministery of Agriculture image library of Naturalis database and site of nature obeservations database and site of whale beachings and observations (soon) database and site of sea mammal observations (soon) many others (soon) 46

TUE / Information Retrieval / Reference Structures (1) NSR / register of species Nederlands Soortenregister is a work in progress: completion of register of speciesnow: 35,000 speciessoon: 45,000 species name referencesoon: authorized name reference Dutch speciessoon scientific names plus: Dutch names, English names, etc. aimed target groups (scopes)now: scientistsnow/soon: applied scientistssoon: general audience, schoolkids 47

TUE / Information Retrieval / Reference Structures (1) NSR / register of species a simple example of ranking and scope: scientific namepercidae. perca. . perca flavescens. . perca fluviatilis. sander. . sander lucioperca. stizostedion. . stizostedion canadense. gymnocephalus. . gymnocephalus cernuus English name (Dutch name)perches (baarzen). yellow perch (amerikaans gele baars). european perch (gewone baars). zander (snoekbaars). pike perch (canadese snoekbaars). ruffe (pos) 48

TUE / Information RetrievalReference StructuresTrezorix&RNA-projectend of part 1 www.rnaproject.orgwww.soortenregister.nl 49

TUE / Information Retrieval Reference Structures part 1 - introduction Trezorix & RNA-project