490 likes | 627 Views
Toward Networks with Cognitive Packets. Erol Gelenbe Ricardo Lent Esin Seref Zhiguang Xu School of Electrical Engineering & Computer Science University of Central Florida Orlando, FL 32816. Overview. Introduction to CPN Cognitive Packet Networks
E N D
Toward Networks with Cognitive Packets Erol Gelenbe Ricardo Lent Esin Seref Zhiguang Xu School of Electrical Engineering & Computer Science University of Central Florida Orlando, FL 32816
Overview • Introduction to CPN • Cognitive Packet Networks • Packet Routing Using Neural Networks • Random Neural Networks & Learning • Random Neural Networks with Reinforcement Learning • Simulation (LFRNN, RNN RL, & Bang-Bang) • Cognitive Adaptive Routing • Analytical Model Predictions using G-Networks • CPN Test-Beds • Future Work
Problem Definition Packet networks have capabilities for flow control, error control and routing Cognitive Packet Networks(CPN) concentrate intelligent capabilities for routing in the packets rather than in the nodes and protocols For the time being, we assume that error and flow control in CPN are handled using current techniques
Introduction to CPN • A packet switching network where packets route themselves • Packets are assigned goals before entering the network • Packets learn to achieve their goals • Learning is performed by sharing information between packets • Packets sharing same goals can be grouped into classes • Packets do not rely on nodes for routing
OSI Layers • TCP/IP is a layered protocol stack • Application handles particular applications, Presentation handles compression and encryption of data, Session controls establishment, management, termination of sessions • Transport provides flow of data • Network handles the transmission of packets in the network • Data-link is responsible for the interaction of the device driver in the operating system and the network card in the machine • Physical defines electrical and mechanical specifications CPN
Transport Layer (TCP) • Connection-oriented: client and server applications establish connection before data exchange • Reliable: sends checksum on data and header, takes care of the order of arriving packets, takes care of duplicate packets, sends acknowledgments, maintains timer • Flow-control: receiving side allows the sending side to send as much as it can buffer
Network Layer (IP) • Unreliable: best effort service, no guarantees to get to the destination • Connectionless: no state information about successive datagrams, each handled independently • Routing: done on a hop-by-hop basis according to the routing tables, required information is kept in the IP header (Routing table information comes from the routing protocols)
Current Routing Protocols • Distance-vector Protocols • Routing tables updated according to a vector of distances (hop counts) • Regular and triggered updates • Whole network topology is distributed • No information about the network topology Drawbacks: • Regular updates consume bandwidth • Takes too long to stabilize after failure • Using only hop count omits other variables
Current Routing Protocols • Link-state Protocols • Each router tests the status of its link to its neighbors and sends this information to its other neighbors • Converges faster than distance-vector protocols • Each router can be assigned a cost based on throughput,reliability, round-trip time, etc • Load balancing possible • Multicasting reduces load on systems not participating Drawbacks: • No policy based routing • Needs more memory and CPU power
CPN and CP CPN: Packet switching networks in which intelligent capabilities for routing are concentrated in packets rather than in the nodes and protocols CP: Packets in CPNs which have the route themselves
Structure of CPs • Identifier Field: The unique identifier of a packet • Data Field:Payload • Cognitive Map: S-D information, the packet’s view of the state of the network, etc • Executable Code: Code used to update the CM and the decision algorithm
Storage area for CPs: Input and Output Buffers Mailboxes are used to exchange data between CPs The Node executes the code for each CP in the input buffer Nodes in a CPN
Update of CM by Node • Packet arrives at the input buffer of a node • The code of each CP in the input buffer is executed • The CP retrieves relevant information from the mailbox • The packet’s CM is updated • Some information is moved from the CP to the MB • The CP is moved to an output buffer
Cognitive Packet Classes A Class is a set of packets having the same QoS requirements, sets of internal states, control rules, input-output signals Class (S,D,L)
Random Neural Networks • A spiked RNN with a mathematical structure similar to a queuing network (Gelenbe in Neural Computation 93, IEEE Trans. on NN 99 etc.) • Probability that the neuron is excited: where • is the rate at which neuron i sends excitation spikes to neuron j when i is excited • is the rate at which neuron i sends inhibition spikes to neuron j when i is excited and
Learning Feed-Forward RNN • Input to each neuron is represented as a pair of excitatory and inhibitory signals • Each connection is represented as a pair of weights • Output of the system is the q of the output layer neurons
LFF RNN Equations • Normalized training set • Network parameters are computed from the training set • Parameters adjusted to minimize the error function • Weight updates
LFF RNN Equations • Where if u=i,v=i if u=i,v!=i if u!=i,v=i otherwise if u=i,v!=i if u!=i,v=i otherwise
RNN with Reinforcement Learning • Fixed input values • Each connection is represented as a pair of negative and positive weights • Output of the system is the largest q of all the neurons
RNN RL Algorithm • Decision threshold • Compute If
RNN RL Algorithm • Re-normalize all weights by calculating and
Simulation • 100 nodes • Link speeds normalized 1 • Arrivals are Poisson • Buffer sizes are unlimited • Packet loss simulated probabilistically, intentionally high in some areas • Goal is to minimize a weighted combination of delay W and loss L
Cognitive Adaptive Routing (CAR) • There are three types of packets: • The packets sent into the network by the source with small rate which travel intelligently to their destination (SMART or Cognitive Packets) – • A SMART packet which reaches its Destination generates an ACK back to the source; ACKs travel back along the same (reverse) route as the packet • Reception of an ACK at the Source may trigger the transmission of a DUMB packet along the route which the older packet had taken • ACKs may not be necessary for each packet
Cognitive Adaptive Routing (CAR) • The ACK coming back from Destination D deposits Delay Information W(n,O,D) when it passes through Node n, entering it from Node O • W(n,O,D) is used to Update the CAR Learning Algorithm (eg Reinforcement Learning) either by the Smart Packet, or by the Node itself
Analyzing CAR with G-Networks • G-Networks are Queuing Networks with Three Types of Customers: • Normal Customers • Negative Customers: They Destroy Normal Customers • Triggers: They Move Normal Customers to Another Queue • Multiple Class Models Allowed • G-Networks Have Product Form … They Can be Solved Analytically and then Numerically • Work of Gelenbe (89, 91, 93, 98) and Many Others • Key Papers Mainly in Journal of Applied Probability • G-Networks Can Model “Worst-Case CAR” and “Best-Case CAR”
G-Network Equations • Non-Linear Equations for the Queue Utilizations - queue S is Source, queue D is Destination - is a route from S to D with queue i and queue j as components and j is the predecessor of i - ACKs will trigger packets from queue 0, which constantly feeds queue S with small rate 0<S,D>
Analytical Solution • Calculate the average No of packets in each queue: • Calculate the average transit delay for a packet
Cognitive Adaptive Routing Performance All Packets Are Smart but Not Always Successful Fewer Packets Are Smart: Dumb Packets Follow Successful Smart Packets
CAR Performance as a Function of Smart Packet Injection Rate
Test-beds for CPN Goals • Performance evaluation and capability enhancements • Demonstration of applications Requirements • Explicit separation and clear borderline between the different protocol layers • Standardized interfaces both between the application layer and the CPN layer, and between the CPN layer and the data transport network • Compatibility with a wide range of computing platforms • Interchangeability of the data transport networking layer
A CPN-VN (Virtual Network) • Standard Linux on Intel-PCs • New CPN Code Instead of IP
CPN Test-Bed Different Selected Physical Connectivities Different Selected Logical Connectivities
CPN-VP Test-Bed Current Test-Bed Connectivity
Smart & Dumb Packets • CPN • Header Routing Data + Code Payload Identifier SMART or DUMB Packet Indicator Cognitive Map and Executable Code
Packet Contents • CPN • Header
Mail Boxes in CPN-Routers • The Mailbox (MB) is a LRU stack where information is stored: delaysweights for different directions associated with certain <S,D>, ... • Packets (esp. ACKs) deposit information in the MB
State Diagram of CPN-Router PDU Arrived (either from higher layer or from peer entity) PDU Assembled Complete Route Available No Route Available Incomplete Route Available Update Route Consult MB Update CM Complete / Incomplete Route No Route Found PDU Ready Proceed at Random Send
Work Accomplished CPN: packet network architecture in which routing is carried under Packet Control -- Cognitive Adaptive Routing: Smart or Cognitive Packets Control the Routing of the Dumb Packets Cognitive Packet Routing is Designed Using Random Neural Network based Reinforcement Learning, Feed-Forward Learning, Bang-Bang Control Analytical Modeling and Simulation Results are Presented Test-Bed architectures are being implemented
Work to be Done: I Run and Measure Significant Applications (eg File Transfer, Video-Conferencing, Voice, Web Browsing …) Evaluate Network Adaptation under Time-Varying Workloads Deploy a “Large” Test-Bed Examine Implications with respect to TCP Consider CAR as a Joint Routing & Flow Control Scheme
Work to be Done: II Can CAR be used to enhance IP ? Can CPN run in conjunction with IP ? Can Generalized CAR be a way of selecting paths in BGP ?