BioMedCEP shaping for NBIS Call V0.91

1. BioMedCEP shaping for NBIS Call V0.91 • Includes comments from (see http://forum.complexevents.com/viewtopic.php?f=13&t=316&p=1325#p1321 ff) • Dimitris Iakovidis 27 Dec 2011 • Andrew Hunter 3 Jan 2012 • Dimitris 4. Jan • Bernard de Bono 5. Jan • Dimitris 6. Jan • Bernard de Bono 6 Jan, 9 Jan • Wolfgang Maass 13 Jan • RvA 18 Jan • -RvA 5 Feb SotA and What is Beyond materials

2. Objective ICT-2011.9.11 NBIS • learn more about the relationship between structure, dynamics and function in neuronal circuits and assemblies, and how information is represented or “coded” in a brain. • develop deeper and more comprehensive theories of neural processing, possibly building on results obtained in the domains of dynamic and complex systems. • close the gap between neuroscience and engineering by motivating interdisciplinary work that ties data with theories, novel computing paradigms, models and implementations. • Target outcome • Developing and applying radically new neural recording, imaging or interfacing concepts and designs for a deeper understanding of neural information processing. • New multi-scale dynamical theories of neural representation for the development of neuro-bio-ICT systems that can perform high-level tasks (e.g. robust object recognition, or classification), going beyond purely sensory-driven information processing. Exocortex systems • Development and prototyping of modularbrain-like computing architectures that combine neural processing primitives to give a better understanding of brain function and facilitate the design of more complex processing systems for real-time and optimized performance. • World-class global research cooperation and alliances in this area, and links with similar actions outside Europe, in particular with participants from USA and Japan. • Expected impact • 3 Target outcome in the case of IP: • New computing paradigms leading to advanced bio-inspired sensing and processing systems, which are naturally able to learn and adapt • New concepts leading to new brain-computer interface technologies • Target outcome in the case of CSA: New EU and global collaborations between researchers in multiple disciplinesspanning engineering, physical and life science domains

3. Essence of the NBIS Call: The objectives of NBIS are more IT- than Health-related. If the IT objectives are met it would be a “plus” to have an impact on health, although it is not a requirement of the call. Main idea of NBIS is: “Study, analyze, model brain function(s), not necessarily of a human brain, and use this/these models to build artificial processing/sensing system(s) more efficient/effective than the state of the art ones, exploitable in the framework of novel brain-computer interface technologies.”

4. Enhancing human intelligence and cognitive or physical abilitiesconnect humans to more events of the universe (e.g. also Internet of Things and Services, “smart dust”) Integration with other proposals: e.g. - Ray Kurzweil: Singularity is Near / Henry Markram Blue Brain / - Bruce H. Lipton: Epigenetics – Intelligent cells / - Karlheinz Meier: Design, construction and Operation of a Neuromorphic Computation facility - Plamen Simeonov Integral Biomathics, - Francois Képès, Marc Schoenauer : Using Evolution to compute - Kevin Warwick: Brain Computer Interface - Cyborg NeuroColumn Process Modeler Event Modeler Monitor / Analyze / Act Protein machinery Process Engine tbd: ProcessExecutionLanguage Process Models Intracellular effectors = Event Processing Agents Intracellular effectors = Event Processing Agents Event ProcessingModel Intracellular effectors = Event Processing Agents Protein machinery… „unus mundus“- Internet services and their events CEP Enginetbd: Event ProcessingLanguage for U-CEP analysehistory… Middleware Event Store IF …AND …FOLLOWED BY…WITHIN…ACTION Normalizedevents,buildhigherlevelevents Event Type Adapters Extracellular receptors = event adapters • Whicheventsareimportant? • Howareeventscorrelated? • Whenandhowshouldwereact? e.g. JMS pub/sub Low Level Event Streams e.g. GPS-signal e.g. Traffic Message Controls e.g. Weather Forecast FET-F U-CEP presentation Jan, June 2010 e.g. RFID …

5. BioMedCEP and clinical applications Carel Meskers/LUMC

6. BioMedCEP and clinical applications – U-CEP based automatic treatment -Biomarkers as complex event patterns, and therapeutics as (pre-) modelled processes- „Understanding“ and „rebooting“ the brain - Mind reading for clinical applications and for HETs BrainPort and CN-NINMBCI as interfaceto the brain(U Wisconsin / TCNL) Biosensors and biomarkers, e.g. Mind reading – based on (complex) event patterns BCI as interfacefrom the brainU Würzburg, U Tübingen, MPI Leipzig, UCLA, MIT, IBM… http://www.silicon.de/technologie/mobile/0,39044013,41556997,00/gedanken_malen_am_computer_bilder.htm, VPH-FET 2011, Marco Viceconti et al.Paper 2012 Filippo Castiglione, Andrea Gaggioli et al „Physio-environmental sensing and live modeling based on U-CEP“

7. BioMedCEP abstract • tbd – • http://forum.complexevents.com/viewtopic.php?f=13&t=316&p=1332#p1330 • http://www.citt-online.com/downloads/BioMedCEP-preprop_en.doc

8. BioMedCEP EPSS para. 1.1ff “State of the Art / What is Beyond” • 1 Scientific and Technical Quality, Relevant to the Topics addressed by the Call • 1.1 Concept and Objectives • 1.2 Progress beyond the State-of-the-art • Tbd, this extended abstract/paper has to adapted or enhanced according to the Call/Objective – • http://www.citt-online.de/downloads/3-Danilov-Tyler-Ammon-Etzion.pdf • + e.g. enhancement of Erwin Vlugt about clinical applications and Ageing/Mobility of elder people, etc. • Tbd, overview comparison Brain projects and BioMedCEP positioning • http://forum.complexevents.com/viewtopic.php?f=13&t=257&p=1364#p1364

9. The basic workpackage structure of BioMedCEP WP1Management WP2Reference model + referencearchitecture (forintegrating U-CEP, Brain Computer Brain Interfaces, BAN, Quantum Computing orcogntivecomp. chips, etc.) WP3(Complex Event) Processing, Analysis and Modeling WP6Real-world clinical applications WP 4 Model validation WP 5 Beyondsenses BCI WP7Impact assessment WP5.1Sensitivity Range WP5.2Add senses WP8Training WP6.1Ageing WP6.2De-pression WP6.3Diabetes WP6.4Obesity WP9 Dissemination andExploitation

10. Comments wrt the Workpackages: WP2 (Reference Model and Reference Architecture for the BioMedCEP approach): RM and RA integrate U-CEP, Event adapters, Brain Computer Brain Interfaces, BAN, Quantum Computing orCognitive Comp. Chips, etcas a combinedtechnologyfor WP5.1 „Enhancingsensitivityrange“ and WP5.2 „Add newsenses“ via an Exocortex.The term “reference model” could also mean the selection of the biological model which we would like to get inspired from. If this is what it means then we should provide 2-3 justified alternatives. So far, BioMedCEP uses the 6-layered neuro-columns and the model that “old” or originally born event patterns are stored on the lower levels, the acquired or “historical” event patterns are stored in the middle layers and current, “inflying” events are processed on the top-level layer. Although the human brain can only process 120.000 events per second unconsciously, and around 7 event patterns logically or consciously, but the performance comes from the combination of the event patterns of several layers and the fact that event patterns are not processed every time again. (more in our papers and references) However, in the case of a “fault” in event processing (or trauma situations, etc.), this is the reason for diseases which could be treated or healed with the proposed U-CEP based approach, what should be covered by WP6. Andrew Hunter (Ulincoln): …the focus that now really makes sense for us is on sensor/signal interpretation, real-time, using specialized parallel hardware (FPGA) and neurally-inspired architectures.  In the BRAINS project, funded by the UK's Trade and Strategy Board, we have built a neurally-inspired architecture that has a number of modular computational capabilities, including collision detection, pattern recognition, anomaly detection and operates at high-speed, low current-consumption and therefore suitable for portable and/or wearable devices, and because FPGA-based has low reconfiguration/redesign costs. On the back of this we have built the world's first fully functional video-based security surveillance system programmed solely on FPGA, incorporation the full pipeline of video decoding, filtering, feature extraction, tracking, pattern recognition, anomaly detection of signal output. We would therefore like to contribute to WP2 in the development of FPGA-based cognitive computing chips, the key concepts being:  Design a neurally-inspired architecture with an architecture sufficiently well-aligned with natural neuronal structure to allow a "sympathetic" mapping, while respecting the architectural differences of the hardware platform (a key realisation for us is that you need to "respect the architecture" to make a really functional FPGA-based neural analogue); Ensure the resulting specialized hardware-based architecture can be integrated directly with sensors on a wearable system, is low-cost, low energy consumption and capable of massive computational demands on real-time scale, exploiting the FPGA inherently parallel architecture; Build neurally-inspired algorithms for signal translation/interpretation for this architecture.

11. Comments wrt the Workpackages: WP3(Complex Event) Processing, Analysis and Modeling): Signal and complex event processing/analysis from multisource data in order to get a deeper understand neural information processing and build a novel dynamic model that would serve the simulation of complex dynamic systems. This model should be able to solve more efficiently real-world problems. Modelling approaches of (class) diagrams for thoughts or bio-markers will be some of the most challenging tasks in the BioMedCEP project. "Mind reading„ and If we model a class diagram of "Thought" as software engineers, we would probably model it similarly to a "Notification Event Architecture of <domain>" like NEAR for the Retail domain - as a reference model of "Thought", not of "Thinking", because a class diagram does not model dynamics. The class diagram might become very complex, starting from a superclass of "Thought" with a lot of subclasses of special thoughts or components (Gen/Spec) and their attributes and operations or methods and a lot of associations, compositions, aggregations and so on... When we have this, we can define patterns and look for them in the brain and map them to machines or robots and manage or control them via "just thinking". Or vice versa and enhance the human by HET or by an exocortex as we have sketched it in our papers and references. Mind reading and what a human is thinking - based on a catalogue of thoughts or thinking patterns correlated with realtime fMRI brain activity patterns. The most challenging part is probably the modelling of the class diagram We shall cooperate with other accordant projects from John Dylan-Haynes, Max-Planck-Institute für Kognitions- und Neurowissenschaften in Leipzig , UCLA's Laboratory of Integrative Neuroimaging Technology, etc. See also the Comparison of neuro-science or brain projects http://forum.complexevents.com/viewtopic.php?f=13&t=257&p=1312#p1301 The problem of modeling the context: What "context" or “world knowledge" means and why it is important as a precondition to model and trigger the accordant (re-) actions based on complex event patterns… When do we have to start on which level with modeling the context? Is there a hierarchy of contexts or a relationship between contexts? How do we model context? By UML and its class-, state- or activity diagrams? Or directly as SQL-like or another EPL code without any abstraction level? We have to check what the "studios" of all the CEP- or ed(B)PM platforms can offer or must be enhanced. There was also already an initiative of the OMG (Jim Odell) to standardize a notation, but is not started yet. We should such standardizing make a task in BioMedCEP. http://forum.complexevents.com/viewtopic.php?f=13&t=252&start=20#p1317 This WP has to work not only on modelling notations but also on executable models http://www.citt-online.com/downloads/EDBPM-workshop09.ppt, slide 20, fig. 2 in http://www.citt-online.com/downloads/Integrating_Complex_Events_for_Collaborating_and_Dynamically_Changing_Business_Processes_MONA_Final.pdf Uncertainty modeling (as also mentioned in Ch. 5 of the VPH-FET roadmap). We experiment with modeling approaches from UAI (Uncertainty in Artificial Intelligence, http://www.auai.org/, http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf, http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf) as well as the suggestion of a reference model for non-deterministic U-CEP applications (see workshop paper Danilov/Tyler/Ammon/Etzion) and special aspects of uncertainty in the disciplin of CEP (Etzion/Niblett 2010). Bernard de Bono:knowledge representation (and associated visualization) of biological structure, and related disease mechanisms, across multiple scales, as well as the use of ontology-based reasoning in support of discovering novel relations between resources annotated by such ontologies. If this IP plans to manage clinically-related resources (i.e. models and data), and to ensure that the modeling frameworks emerging from its effort is semantically interoperable with standards being established by the VPH and Pharma community, then the application of the RICORDO metadata infrastructure may have key contributions to offer to BioMedCEP. The newly-emerging ApiNATOMY toolkit is being developed for a GUI to sit on top of the RICORDO infrastructure and provide a visual schematic of anatomy ontologies (including, therefore, neuroanatomy) as a way to overview brain nuclei, pathways and associated metadata. We are also in the early stage of applying RICORDO and ApiNATOMY in the knowledge-based resource management of the Virtual Physiological Rat co-ordinated by Wisconsin Wolfgang Maass: It would be fun to see, whether some of our theories for spike based processing and learning could also be fruitful in this expanded context. http://scholar.google.com/scholar_url?hl=de&q=http://193.54.228.31/fr_vers/documents/thorpe_s_01_715.pdf&sa=X&scisig=AAGBfm0NI--D7r024kKVfT9InkYrGNIGDA&oi=scholarr http://scholar.google.com/scholar_url?hl=de&q=http://www.t35.ph.tum.de/addons/publications/Kempter-2001b.pdf&sa=X&scisig=AAGBfm2CyQgvivUYtfTUrw4DMZt7A0Vjuw&oi=scholarr http://scholar.google.com/scholar_url?hl=de&q=http://neurophysics.huji.ac.il/~guetig/papers/guetig06.pdf&sa=X&scisig=AAGBfm2vzJudatx95o_e6ew1Eq4PFSuKPQ&oi=scholarr http://www.em.mpg.de/index.php?id=281 Rainer von Ammon: Perhaps this idea of spike based processing could be brought together/connected with indexing idea/problem (of Jeff Hawkins On Intelligence 2005) of how to find the “stored” and already processed “historical” or acquired or a”already born with” event patterns with the current inflying events depending on the context or the situational conditions. Question is: how to retrieve and correlate event patterns from the different 6 layers of the neocortex. What would spikes do? Rainer von Ammon: Another question in this connection might be where the memory or the memories (= stored event patterns?) is/are? Neuroscientists identify a master controller of memory: When you experience a new event, your brain encodes a memory of it by altering the connections between neurons. This requires turning on many genes in those neurons. Now, neuroscientists at MIT’s McGovern Institute for Brain Research have identified what may be a master gene that controls this complex process. How would that be related to the theory of the six layers of the neurocolumns and where the "old" or "historically" acquired event patterns are stored (on the lower levels) or the current events are processed (on the top levels)? (e.g. Kevin Hawkins, On Intelligence, 2005). What would we find respectively what would be marked with the help of Npas4? http://forum.complexevents.com/viewtopic.php?f=13&t=257&p=1316#p1316

12. Comments wrt the Workpackages: WP4 (Model Validation): This WP validates the theoretically based results of WP 3 and brings together the international key players of relevant technologies as named in WP3. …

13. Comments wrt the Workpackages: WP5 (Beyond Senses BCI) We focus on enhancing the sensitivity range of a sense and adding new senses – through brain-computer interfaces (BCI) exploiting the model developed in WP 3. This WP will develop and exploit an U-CEP based Exocortex and will experiment with an approach to map such enhanced sensitivity ranges and additional senses to the limited event processing capabilities of a human brain. Examples might be to smell pollution or radioactivity via new body area networks, bio-sensors, event adapters, etc., as sketched http://forum.complexevents.com/viewtopic.php?f=13&t=299 plus to (re-) act accordingly via (pre-) modeled processes. Another example might be to add a magnetic sense and to experiment with human behaviour and adapting brain functioning (according to first hints from belt technology, see U Osnabrück). More applications and examples for enhanced sensivity ranges and additional senses will be elaborated.

14. Comments wrt the Workpackages: WP6 (Real-World Clinical Applications): All clinical applications are organized in WP6 where we work on understanding the human brain and healing diseases by changing the event processing of a human brain or to “reboot” a brain (see BrainPort, …) or to delete “false event patterns” in the memory (see http://forum.complexevents.com/viewtopic.php?f=13&t=257&start=10#p1316). This is seen as a “fault” in event processing (as a consequence of trauma situations, accidents, etc.), what is the reason for diseases (balance, depression, …) which could be treated or healed with the proposed U-CEP based approach. To find accordant “treatments” will also help to learn more about the brain and to find more functions and applications of neuro-biologically inspired systems. Typical, main health problems and diseases of the today’s society (like ageing, depression, diabetes, obesity) are addressed. Each problem or disease represents a discriminant challenge of the BioMedCEP approach. …

15. Comments wrt the Workpackages: WP7 (Impact Assessment): …. Insert your suggestions

16. Comments wrt the Workpackages: WP8 (Training): …. Insert your suggestions

17. Comments wrt the Workpackages: WP9 (Dissemination and Exploitation) …. Insert your suggestions … isn’t exploitation already covered by WP5 and WP6??? So we should only focus on Dissemination with WP9??? Dimitris: “Exploitation” usually refers to a formal viability study/exploitation plan that refers to the way the project’s results will and can be exploited after the end of project e.g. commercial products, possible chain reaction of future research that could be triggered etc.

18. BioMedCEP - Use Cases related to the Objectives, Outcomes and Impacts - Will be changed or deleted UC1: Enhancingsensitivityrangeof a sense: Increasing / decreasingsensitivityrange, Howtomaptothebrain, Superimpositionof „normal“ eventsbynewevents, Will thebrainchange / adapt, Ego States / Consciousness / Reality… Applicationsof an Exocortex UC2: Addingnewsenses: Applicationsof an Exocortex, Howtomapnewsensestothebrain, Superimpositionof „normal“ eventsbynewevents, Will thebrainchange / adapt, Ego States / Consciousness / Reality… UC3: LearnmoreaboutAgeingandthebrainSubstitute typicalproblemsordiseaseswrtAgeing, Providemobilityofelderpeople, <insertbullets, …> UC4: Depressionas a fault ofeventprocessing Treatments on organlevelandcell- orproteinlevel, changemetabolismby U-CEP orBioMedCEPapproach, <insertbullets, …> UC5: Diabetesandnewtreatmentsbased on U-CEP Biosensingand Biomarkers Change orinfluencethinkingandbehaviour, addiction, Treatments on organlevelandcell- orproteinlevel, changemetabolismby U-CEP orBioMedCEPapproach, <insertbullets, …> UC6: Obesityandnewtreatmentsbased on U-CEP Biosensingand Biomarkers Change orinfluencethinkingandbehaviour, addiction, Treatments on organlevelandcell- orproteinlevel, changemetabolismby U-CEP orBioMedCEPapproach , <insertbullets, …>

19. Exploitation and dissemination: Use cases and the workpackage structure of BioMedCEP Will be changed or deleted WP1Management UC1: Enhancingsensitivityrangeof a sense: WP2Reference model + referencearchitecture (forintegrating U-CEP, Brain Computer Brain Interfaces, BAN, Quantum Computing orcogntivecomp. chips, etc.) UC2: Addingnewsenses: WP3(Complex Event) Processing, Analysis and Modeling WP6Real-world clinical applications WP 4 Model validation UC3: LearnmoreaboutAgeingandthebrain WP 5 Beyondsenses BCI WP7Impact assessment WP5.1Sensitivity Range WP5.2Add senses UC4: Depressionas a fault ofeventprocessing WP8Training WP6.1Ageing WP6.3Diabetes WP6.2De-pression WP6.4Obesity WP9 Dissemination andExploitation UC5: Diabetesandnewtreatmentsbased on U-CEP Biosensingand Biomarkers UC6: Obesityandnewtreatmentsbased on U-CEP Biosensingand Biomarkers

20. Tasks related to the use cases and the workpackage structure of BioMedCEP Will be changed or deleted UC1: Enhancingsensitivityrangeof a sense: WP1Management T2.1 T3.1 T4.1 T5.1 T5.1.1 T7.1 T8.1 T9.1 WP2Reference model + referencearchitecture (forintegrating U-CEP, Brain Computer Brain Interfaces, BAN, Quantum Computing orcogntivecomp. chips, etc.) UC2: Addingnewsenses: WP3(Complex Event) Processing, Analysis and Modeling T2.2 T3.2 T4.2 T5.2 T7.2 T8.2 T9.2 WP6Real-world clinical applications WP 4 Model validation UC3: LearnmoreaboutAgeingandthebrain T2.3 T3.3 T4.3 T5.3 T6.1 T7.3 T8.3 T9.3 WP 5 Beyondsenses BCI WP7Impact assessment WP5.1Sensitivity Range WP5.2Add senses UC4: Depressionas a fault ofeventprocessing T2.4 T3.4 T4.4 T5.4 T5.2.1 T6.2 T7.4 T8.4 T9.4 WP8Training WP6.1Ageing WP6.3Diabetes WP6.4Obesity WP6.2De-pression UC5: Diabetesandnewtreatmentsbased on U-CEP Biosensingand Biomarkers WP9 Dissemination andExploitation T2.5 T2.5 T4.5 T5.5 T6.3 T7.5 T8.5 T9.5 UC6: Obesityandnewtreatmentsbased on U-CEP Biosensingand Biomarkers T6.4 T2.6 T3.6 T4.6 T5.6 T7.6 T8.6 T9.6

21. Consortium Consortium should at least have experts on a) brains (mainly neuroscientists), b) intelligent systems – data/event processing and analysis, c) sensors, d) brain-computer interfaces, and if we would like to address specific health issues such as dementia, obesity etc then e) experts on these specific health issues.

22. Workpackages list

23. Deliverables list

24. List of milestones

25. Work Package Description