Technical Details

1. CPK NLP Suite for Spoken Language Understanding(http://www.cpk.auc.dk/~tb/nlpsuite)Tom Brøndsted, CPK. Aalborg Universitytb@cpk.auc.dk

2. Technical Details • Programs for syntactic/semantic parsing, generation of language models … • Reads/writes a number of grammar formats • Free for research/non-commercial use • Available in source (C++, lex/yacc) • Compiles and runs on “any” machine under “any” OS having a 32 bit C++ compiler

3. 25 Programs e.g. psgtrec(.exe) (what is done with/to the grammar) (grammar format)

4. Grammar Formats I • Augmented Phrase Structure Grammer format based on a subset of the “EUROTRA User Language” • Compound (but non-nested) features and structure sharing (“variables”) • Semantic frames are created either • through percolation of features • with mapping rules creating frames when specified conditions are met

5. Grammar Formats II • grammar format based on PC-PATR/PATR II (Shieber etc.) • Compound (including nested) features and structure sharing (“variables”) • Semantic frames are created • through percolation of features • no mapping

6. Grammar Formats III-IV-V • Simple symbol-based (non-unification based) formats including: • Simple BNFs intended for teaching • Finite state grammars intended for speech recognition • [further details in the paper]

7. Program Types I • Natural language parser • Input: 1-best/n-best. Output: semantic frames • Unification-based left-corner chart parser • Bottom-up, left-right with top-down filtering • Exhaustive parsing • API: allows interface to speech recogniser and dialogue management

8. Program Types II • Grammar converter • Input: NL-Grammer. Output: equivalent language model • Types of language models: RTNs, FSNs, Word Pair, “No grams” • can be written in the HTK standard lattice format etc. • can substitute the network loading routine of a speech recogniser

9. Program Types III-IV • Grammar constrained typed text recogniser, simulation of speech recognition/understanding • Input: typed sentence. Output: a “similar” sentence covered by the grammar (or corresponding frames) • standard linear Viterbi pattern-matching algorithm • uses FSN (“full-gram”) constraints • supports “garbage models”

10. Program Types V • Random sentence generator, exploring the coverage of grammars • Input: a NL-grammar. Output: a number ofsentences covered by the grammar • Uses a fully equivalent RTN derivation of the grammar • Corresponds to the HTK-utility HSgen

11. ANSI C API (+JAVA) • Functions for • Loading grammars • Activating/deactivating subgrammars • retrieving semantic frames from recognition results • ANSI C structure implemented for semantic frames

12. Adding a NL Grammar Format XXX lex/yacc code

13. Adding a Recognition Grammar Format RTN FSN WPG NOG HTK write procedure VOC write procedures XXX write procedure (ca. 80 lines of code)

14. Conclusion • Represents work in progress • Available for non-commercial use “as is” • Known inconveniences • Users are welcome to extend the suite

15. Basic idea: Controlling speech understanding via one grammar speech Speech recogniser grammar conversion “sentence” Natural language parser grammar semantic frames

16. Tutorial Example in Paper speech GrapHvite (Entropic) HTK standard lattice APSCONV “sentence” EUROTRA APS Gr. APSPARS semantic frames

17. Application Example • Voice driven email application from the GrapHvite tutorial (Entropic) • A. Using the Entropic grammar tools (HTK and GrapHvite tools) to make HTK standard lattice • B. Using the CPK NLP suite to implement the APS grammar and the HTK standard lattice.

18. Using the HTK Grammar Tools • An EBNF-like grammar definition language • HTK tools: Hparse, Hbuild, HSGen, HDMan $digit = ONE | TWO | THREE | FOUR | FIVE | SIX | SEVEN | EIGHT | NINE | OH | ZERO; $name = [ JOOP ] JANSEN | [ JULIAN ] ODELL | [ DAVE ] OLLASON | [ PHIL ] WOODLAND | [ STEVE ] YOUNG; ( SENT-START ( DIAL <$digit> | (PHONE|CALL) $name) SENT-END )

19. Using GrapHvite Grammar Tool • Visual Netbuilder:

20. Using the CPK NLP suite • Implementing the grammar in the APS format using a texteditor • Testing the grammar • Simulating speech recognition and understanding • Converting the APS into HTK standard lattice

21. BNF Rules consist of symbols, e.g. BritishCity, DanishCity Basic operation: string comparison Boolean (true or false) APS Rules consist of feature sets, e.g. {cat=city,country=UK}, {cat=city,country=DK} Basic operation: unification Null or a new feature set The APS formalism

22. Semantics in the APS • Simple: Percolation of values to the top node • Mapping, postprocessing (see paper) STRUCTURE BUILDING RULE {cat=person, emailaddr=$A} [{cat=lastname, emailaddr=$A}]. LEXICAL RULE {lex=brondsted,cat=lastname, emailaddr=tb@cpk.auc.dk}.

23. the msg_spec subnetwork Examples: delete message number three forward previous message go to first Semantics: a) an absolute number b) a relative number

24. the msg_spec subnetwork • LEXICAL RULES • {lex=previous,cat=spec, relpos=-1}. • {lex=current, cat=spec, relpos=0}. • {lex=this, cat=spec, relpos=0}. • {lex=next, cat=spec, relpos=1}. • {lex=first, cat=spec, abspos=0}. • {lex=last, cat=spec, abspos=10000}. • {lex=message,cat=msg}. • {lex=email,cat=msg}. • <number>.

25. Testing/running msg_spec testing on a test database apspars msg_spec.aps msg_spec.snt testing random generation apssgen msg_spec.aps simulating recognition apstrec msg_spec.aps 0 simulating speech understanding apstslu msg_spec.aps 0 fullgram HTK standard lattice apsconv -f -h msg_spec.aps