Environmental Data Integration (Fusion Synthesis)

1. Environmental Data Integration (Fusion Synthesis)

2. Data Fusion (1) M.A. Abidi, R.C. Gonzalez • Autonomy of behavior requires mechanisam to fuse, assimilate, metabolize information from various sources for knowledge accumulation, decision making and actions.(p1) • Intelligent (autonomuous) interaction with the enviroment requres the synergistic use of multiple sensors (p7) • Monitoring, anlaysis and data evaluation is required since, the state of the environment is not known a priory to the observer. • Totally unknown environment • Partial knowledge available but full world model is not feasible • The external world is changing in an unpredictable manner. • Sensors allow the monitoring the state of the world and updatating our internal mental model of the world. • The two main abilities (of humans, p8) that interprets info are • 1.flexible body of knowledge and • 2.ability to synergistically integrate information fro multiple inputs • The nature of human knowledge and the multisensory information fusion of the human sesnses can give an indication as to what is achievable by artificial intelligent systems. (p8). • However, both the human knowledge and information fusion can be augmented by mashine support. • 1.Mashine-aided deposition, storage and retriaval of knowledge • 2.Mashine-aided multi-sensory data integration. • The other difference is that most environmnetal sensors are not part of the human body, so they are not conneted to the built-in fusion system in he barin. • Sensor integration in humans through (p9) • ·duplicate sensing (pair of eyes) • ·sensor fusion (seeing and touching the same object) • ·distributed sensing (network of sensors in the skin)

3. Data Fusion (2) M.A. Abidi, R.C. Gonzalez • Multisensory Data Integration • ·deals with the synergistic combination of information from various data sources in order to better understand a given scene. [related to perception, intelligence] • ·combination of incomplete, inconsistent, impricise, redundant or conflictiong information (p xi, 1) • ·fused data reflects information from data sources as well as previous knowledge • ·data fusion, sensor fusion, sensor integration used interchangebly • Multisensory integration: synergistic use of multisensory information for a given task.(p13) It is systems and architectural level activity. • Multisensory data fusion: Any stage of data integration where the actual combination (fusion ) takes place. It is mathematical, statistical in nature. • MDI uses many paradigms ( inspiration ), frameworks ( specific form, structure) and tools (means of acomplishing MDI). These are borrowed from the fields of systems analysis, control theory, AI, information theory, cogitive sciece, information science, computer science...(p21). • Common themes include modularity, hierarchical structures, adoptability. • Data Fusion techniques: • Probabilitistic fusion • Bayesian reasonong (chapter 3) • Evidence theory (chapter 4) • Robust statistics (chapter 5) • Recursive fusion operators (chapter 6) • Least squares models • Kalman filtering - robotics (chapter 7) • Kalman filtering - military (chapter 8) • Optimisation (chapter 9) • Regularization (chapter 10) • Uncertainty ellipsoids (chapter 11) • Data fusion and neural networks (chapter 12) • Data fusion and fuzzy logic (chapter 13)

4. Data Fusion (3) M.A. Abidi, R.C. Gonzalez • Means of integrating information (p8) • 1.Separate inputs to the system controller. Each sensor provides info on different aspect of the environment. Sensors interact indirectly through the controller. • 2.Data fusion by one sensor directly influencing another. • 3.Combining and fusing info from different sensors. Fusion can take place at different levels of abstraction, signal, pixel, feature, symbol. The goal is to provide higher grade info (feature, symbol), not available from individual sensors. Benefits of multi sensory integration:(p14) Redundancy - Improved accuracy, reliability; fused at the signal, pixel level. Complementarily-gives unique, synthetic info; fused at feature and symbol level Data fusion also reduces data quantity, hence communication costs Problems of multisensory integration: Registration - info from multiple sensors refer to the same thing; geocoding, time coding, param coding Sensor error: - bad reading - systematic, random Operation error: -

5. Data Fusion (3) M.A. Abidi, R.C. Gonzalez • 2.3 Multisendor Data Integration, MDI • A group ( Fig 4) of sensors peovides the input to the integration process. • 2.3.1 Basic integration functions • The sensor model describes the behavior, uncertainty and data quality provided for integration • Sensor registration (e.g. geotransforms) that make the data from each sensor commensurate in space, time • Sensory processing , fusion at sensor, pixel, feature, and symbol level • The result is input into the world model. - the state of the environment. • 2.3.3 Paradigms and Framworks for MDI • World Model stores info about the environmnet, including a priori and recently sensed info. High level processing can use the world model to direct the sensory pocessing (perception) • 2.3.3.1 Logical Sensor. A useful paradigm for MDI is the logical sensor (p22, Fig 5). It is a member in a hierarchical structure. (a holon?) . It recieves logical sensor inputs from below, performs MDI (sensory modeling, registration and processing) and transmits an output vector up the hierarchy. Also, it recieves control commands from above, interprets the command (using the world model) ,issues commands to logical sensors below and influences the processing of logical sensor inputs. [Similar to the holon’s four components: InSignal processor, Reporter, Command Interpreter and Commander] • 2.3.3.4 Artificial Neural Networks. • 2.3.3.5 Object Oriented Programming (p28) is a framwork for immplementing MDI. An implementation framwork • 2.3.3.6 Logical Behaviors. and extention to logical sensors • 2.3.3.7 Generalized evidence Processing. • 2.3.4 MDI Control Structures • 2.3.4.1 Bayesian Networks and Rule-Based Systems Useful for dissimilar sensors. • 2.3.4.2 NBS Sensory and Control Hierarchy Ascendings sensory processing hierarchy, couppled with a descending task decomposition (conrol ) hierarchy, via the world model. Separatin of low and high level tasks. (very much like the holarchy) (Fig 8). World model same as PNPs model of the mind? • 2.4 Multisensory Data Fusion, MDI

6. Participatory Environmental Management as an Information System • Environmental management is the process of observing the environment, detecting possible hazards, identifying the causes, evaluating the mitigation options and acting to reduce the hazard. The approach to the environmental management has been has been changing recently in several important ways. • ·The command and control management style is giving way to broader participatory approach involving the key stakeholders throughout the management process. • ·The environmental decision makers have become interested in monitoring and evaluating the effectiveness of past environmental control actions. • ·The conectedness of air, water and land pollution is recognized by multimedia approach to the issue of permits. • These envimanagement changes • In the recent past, the mitigation of environmental problems has been accomplished primarily by laws and regulations that were prepared and enforced by governmental agencies. Other stakeholders, such as industry and public interest groups have participated in the management process indirectly by influencing the relevant governmental agencies. Conflict resolution among the stakeholders occurred primarily through the political mechanisms in the Congress where the laws created and through litigation with the regulatory agency, EPA. Once, the laws and regulations were in place, they constituted a command that had to be executed by those who performed the environmental control action. Thus the term command and control approach. • The management processes • Environmental management process can be broken down into two major activities, risk assessment and risk management. Risk assessment may be further broken down into • Risk Assessment • 1.Environmental monitoring • 2.Estimating exposure • 3.Detecting hazards and quantifying risk • Risk Management • 1.Identifying the causes of the risk • 2.Evaluating on the mitigation options • 3.Deciding which option(s) to take • 4.Acting to reduce (control) the hazard • 5.Monitoring the effectiveness of the controls.

7. Information, Entropy and Environmental Informatics • In the physical world, the main resources are mass and energy • Living systems maintain their adoptive behavior by the storage, transfer and use of an added resource – information • Information helps creating structure and order from simple, unordered matter -> information reduces entropy or chaos • Tentative definitions and characteristics of information are needed • Environmental informatics is the application of information science and engineering to the study of environmental problems • Theoretical and practical examples of informatics will be given

8. Engines of Creation (1) K Eric Drexler Anchor Books, New York, 1990 • p21 The Principles of Change. • Think of design process as involving first the generation of alternatives and then the testing of alternatives against a whole array of requirements and constrains. • Herbert A. Simon. The Search of the Artificial. MIT Press 1981, Cambridge, MA. • ENGINEERING & SCIENCE • Design of Policy: It is engineering, not science. • p10 Scientist work at predicting how the environmental processes work, not at designing processes that will minimize the environmental damage. The first is a scientific challenge, the second an engineering challenge. • p46 Science aims at knowing, but engineering aims at doing. • p46 Engineers an escape he inherent risk of precise, universal scientific theories. Engineers need only show that under particular conditions particular objects will perform well enough. • p148 Engineers can make do with approximations and special cases. And given tools, materials, and time they can demonstrate possibilities directly. Even when doing exploratory design, they can stay well within the realm of possible by staying well away from limits. • Though measurements cannot prove precise equality, they can prove inequality. • Because science aims to understand how everything works, scientific training can be of great air in understanding specific pieces of hardware. Still, it does not automatically bring engineering expertise; designing an airliner requires much more then a knowledge of the sciences of metallurgy and aerodynamics. • Scientist are encouraged by their training and colleagues to focus on ideas that can be tested with available apparatus. • Policy - social engineering • Provide a context of the problem. • Stating the specific problem, issue, • Define requirements • Defining constraints • Generating and testing the alternatives against requirements and constrains (variations and selection the stuff of evolution) • Evolution of Design. • P 32 Design practice evolves. Engineers accumulate design methods that work. (describes Design Rationale) . Design itself proceeds by variation and selection. Test simulated designs before building. Circle of design, calculation, criticism, redesign - avoids cutting and wasting metal. • p 31 In engineering, enlightened trial and error, not the planning of flawless intellects (artifacts) has brought most advances; this is why engineers build prototypes. Same holds for advances in corporate products and policies. Excellent companies create ‘an environment and a set of attitudes that encourage experimentation’ and why they evolve in a very Darwinian way.

9. Engines of Creation (2) K Eric Drexler Anchor Books, New York, 1990 • SCIENCE • Science and engineering intertwine. Engineers use knowledge produced by scientists; scientists use tools produced by engineers. They both work with mathematical descriptions of physical laws and test ideas with experiments. But science and technology is radically different in their methods and aims. Both fields consists of evolving meme systems, but evolve under different pressures.[driving forces] • Science is not a mechanical process of observations generating conclusions but an evolutionary process. (Popper, Kuhn) . A process in which ideas compete for acceptance. The meme system of science is special: it has a tradition of deliberate idea mutation and a unique immune system for controlling mutants. Scientists • - ignore ideas that lack testable consequences • - seek replacements for ideas that have failed • - seek ideas that make the widest possible range of applicability (gravity) • p 42 Scientific community provides an environment where such memes spread, forced by competition and testing to evolve toward power and accuracy. Agreement on the importance of testing theories holds the scientific community together, [science knows of only one commandment, scientific contribution] • p 49 When basic laws of technology are known, future possibilities can be foreseen. • P 32 Quality control is a sort of evolution, aiming at eliminating harmful variations. • Science is in the sensory end engineering is at the motor end of the sensory-motor system • Computer Supported Collaborative Work • Computer Mediated Communication • Commercial Implementations of groupware • Lotus Notes, Microsoft Exchange ? • http://www.well.com/user/spiff/ - Release 1.0 - Multi-User Virtual Environments. • Workers are drowning in e-mail messages, which lie, unfiltered, unthreaded, unread in swelling in-boxes. • Shared virtual worlds are not just for scientific and technical communities. It is shared context in the form of a persistent environment. Objects such as session transcripts, document drafts stay in place visit after visit. The place itself becomes familiar. • Workers are drowning in e-mail messages, which lie, unfiltered, unthreaded, unread in swelling in-boxes. • People can set rooms aside for different topics, and can furnish and decorate them appropriately. Some participant may create tools or objects ( e.g. work aids, idea prototype, discussion maps) that everyone uses to move discussion forward. Creating such space together is a significant barn-raising experience in the information age. The resulting space may hold special meaning for the participants. Of course it can be the site of major crises and it will. • Goal oriented MOOs such as work groups, have to enhance the task/communication features at the expense of entertainment.

10. Engines of Creation (3) K Eric Drexler Anchor Books, New York, 1990 • CHAOS & ORDER • p 22 Order can emerge from chaos without anyone giving orders. • P25 In detail molecules change haphazardly, but stepping back, the outcome could be said to have a purpose to achieve a goal of replication. • EVOLUTION • p26 Evolution proceed by variation and selection (elimination of unsuccessful) of replicators. • Richard Dawkins has a language of purpose? ->language of evolution. • MEMES • p 23 Things that give rise to copies of themselves are called replicators. RNA molecules qualify. A single molecule soon becomes two, then four, eight ..Later the replication rate levels off. Fixed stock of protein machines is can churn out RNA copies only so fast, no matter how many template molecules vie for their services. Later still, the raw materials for making RNA become scarce and replication starves to a halt. The exploding population of molecules reaches a limit to growth and stops reproducing. • P 35 Like genes, mental replicators - ideas split, combine and take multiple forms. (Genes can be transcribes from DNA to RNA and back. Ideas can be translated from language to language.) Ideas mutate, replicate and compete. - Ideas evolve. • P 35 Richard Dawkins call bits of replicating mental patterns (ideas) “memes” . Just as genes propagate themselves in the gene pool by leaping from body to body, memes propagate themselves in the meme pool by leaping from brain to brain via process which , in a broad sense can be called imitation.” • Memes replicate because people both learn and teach. They vary because people create the new and misunderstand the old. They are selected (in part) because people don’t believe everything they hear. As test tube RNA molecules, they compete for scarce copying machines, so memes compete for scarce resource - human attention and effort. Memes shape behavior, their success or failure is as a serious mater. • Memes are selfish. They care only about their own replication. They evolve solely to survive and spread. Like viruses, they can replicate, without aiding the hosts survival. Parasitic memes exist. Mental immune systems do exist. • Selection of ideas - Mental immune systems. “Believe the old, reject the new” - traditions. Much of the philosophy of science has been a search for a better mental immune system, for better ways to reject false, the worthless, and the damaging. • As genes, memes survive by many strategies. • Selfish motives can encourage cooperation. • P7 DNA - The genetic material of cells - Instructions to direct molecular machines (ribosomes) - they don’t do anything (a metafile). First DNA is transcribed to RNA ‘tapes’. Then ribosomes build proteins based on instructions on the tape. Protein molecules fold up to form small objects that can do things. • nSome are enzymes, machines that build up and tear down molecules • nOther proteins are hormones that signal other cells to change their

11. Engines of Creation (4) K Eric Drexler Anchor Books, New York, 1990 • FROM FEUDS TO DUE PROCESS (an orderly way of sorting out facts and ideas?) • p 206 In policy making (social engineering) the (conflict resolution, problem-solving ) methods determine what causes what. • Actors: Parties in conflict • Jury • Judge • Legal System: Courts • Actors: Parties in conflict • Jury • Judge • Principles of legal due process • Allegations must be specific • Both sides must have a chance to speak, and confront each other, to rebut and cross examine • The process must be public to prevent hidden rot • Debate must proceed in front of a jury that both sides accept as impartial • A judge must referee the process and enforce the rules • Scientific Evaluation: Peer Reviewed Publication • Actors: Scientists (parties in conflict) • Referees (jury) • Editor (judge) • Principles of scientific publication: • Scientific statements must be specific • All sides may state their views in a dispute. • The publications are public • Referees, like juries evaluate evidence and reasoning • Editors, like judges enforce the rules and procedures • - the methods of making rational decisions • Policy Making: Policy Forums • Actors: Major stakeholders (parties in conflict) • Policy evaluators (jury) • Policy ‘hunchos’ (judge) • Principles of Policy Forums: • Policy issues must be stated explicitly • Alternative policy options expressed • The issues, options and arguments are made public • Expert policy panels evaluates the reasoning and arguments • Policy ‘hunchos’ enforce the rules and procedures

12. Engines of Creation (5) K Eric Drexler Anchor Books, New York, 1990 • Technical Support for Policy: Technical Working Groups • Actors: Technical experts (parties in conflict) • Technical panels (jury) • Policy makers (judge) • Principles of Technical support groups: • Problems/questions must be stated explicitly • Alternative solutions/explanations/arguments expressed clearly • The problems, solutions, arguments (supporting data) are made public • Technical panel evaluates the reasoning, arguments • Group chair(s) enforce the rules and procedures • Informal conferences and networks, telephone, e-mail, copy machines also accelerate exchange and discussion. • P 209 Conferences, journals, informal networks share similar limitations: They focus on technical questions of scientific importance, rather then on questions of public policy importance. Scientific institutions evolved to advance science, not policymakers. • FACT FORUM • Badly needed: • Focused, streamlined, journal-like process to speed public debate on crucial facts (& policy options) • Distilling the results of the debate into a balanced picture of our state of knowledge (&ignorance) • Differences from a court: • The technical panel, the forum’s ‘jury’ must be technically competent, not have vested interest in the outcome • Focus on technical questions, not suggest actions - no power to govern

13. Engines of Creation (6) K Eric Drexler Anchor Books, New York, 1990 • NETWORK OF KNOWLEDGE • p 217 In order to cope with change, society has to learn to learn faster. [learning as in STELLA]. Fact forums can help but new technologies for spreading refining and combining information can help even more. • Today’s info systems are clumsy and hamper progress, information overload is a known problem: • Someone published a piece of information - how do you find it? • You found it - where do you file it? • You judge it to be good - how do you spread it? • You see an error - how do you correct it? • Your file system grow - how do you organize it? • P 218 Our trouble in finding, organizing, correcting, spreading, information leaves our shared knowledge relatively scarce, incorrect and disorganized. Because established knowledge is hard to find, we often find it easier to [re-invent it] or do without it, making s more ignorant then we need be. [Recycling of knowledge - knowledge need to be in small chunks so that it can be reassembled into different creations - the LEGO idea of reusable components. - object orientation] • p 218 Organization • Hierarchical - classical approach • Threads - hyperlinks - Networks of relationships • p 220 Readers, authors and editors of hypertext will ignore the computer, the same way as the photo composition and lithography was ignored in the past • p 222 Information gathering to be effective, it must be decentralized; information scattered among many minds can not easily be put into a system by a few specialists. • Hypertext systems comments will be easy to publish and easy to find. Imagine the questions that have bothered you. You could publish them - someone may be out there who can answer. [the selfish meme - it will try to spread] • 222 Hypertext networks will accumulate garbage. Remedies • Electronic Publishing - Publisher has a reputation for quality using high standards • Approval ratings by anyone, using links to the document, recommendations, • Virtual libraries with annotations, filters • Two-directional links: anyone that has referenced your articles - citation index • The evolution of knowledge requires the generation, spread and testing of memes. Hypertext will speed this process. • P 224 Brief critical comments are hard to publish and hard to find • p 224 Human knowledge forms an unbroken web and human problems sprawl across fuzzy boundaries between field [Hierarchies] • Hypertext will allow representing knowledge in a more natural way. Using hypertext, people will associate ideas through published links, enriching their meaning and making them more available. • When we change our minds, we change our internal network of connections. Reasoned change often requires comparing competing patterns of ides [ model - reality - STELLA] • In a book related info must be remembered or repeated by the author, so authors vacillate between too much (boring readers - [chewing their time] ) or leaving too much out (losing context) or both at one. • Authors will write pithy exciting summaries and link the to the lengthy and boring explanations. • Hypertext will help us with the great task of our time • p 230 The Gutenberg revolution reduced the labor cost of producing text by several hundred fold, the hypertext revolution will reduce labor cost of finding text . A revolution indeed! • Revolution in quality as well. Better indexes, critical discussion will weed out nonsense, better representation of wholes, will highlight holes in our knowledge. With abundant, available, high quality [high density] information, we will be more intelligent.