AntNet: Distributed Stigmetric Control for Communications Networks

1. AntNet: Distributed Stigmetric Control for Communications Networks Gianni Di Caro & Marco Dorigo Journal of Artificial Intelligence Research 1998 Presentation by Tavaris Thomas

2. Presentation Contents • Introduction/Background • Model Description • AntNet: An Adaptive Agent-based Routing Algorithm • Other Routing Algorithms • Experimental Networks Used • Results • Conclusions and Future Work

3. Introduction/Background • Increase in the supply and demand of network communication services • Network Control – online and off-line monitoring and management of the network resources • Routing – process or method of determining and prescribing incoming packets to an outgoing path (forwarding messages)

4. Swarm Intelligence (SI) • New research field • Collective behavior of social insects and other organisms • ants, honey bees – states/actions • Stimergy – Complex and intelligent behavior performed through the interaction of thousands of autonomous swarm members

5. Ant Colony Optimization(ACO) • Foraging behavior of ants and is used successfully to solve combinatorial optimization problems. • traveling salesman • genome matching • routing in telecommunications networks • load balancing

6. Model Description • WAN Irregular topology connection-less network • Network communication is mapped on a directed weighted graph with N processing/forwarding nodes • Links characterized by bandwidth (bit/sec) and transmission delay (sec) • 2 types of packets (routing and data) routing have greater priority • C++ based discrete event driven simulator

7. AntNet • Adaptive, distributed, and mobile agent-based routing algorithm • Reinforcement learning problems with hidden state (Bertsekas & Tsitsiklis, 1996; Kaelbling, Littman, & Moore, 1996; McCallum, 1995).

8. AntNet Algorithm Overview • Mobile agents are asynchronously launched towards randomly selected destination nodes. • Each agent searches for a minimum cost path joining its source and destination nodes. • Each agent moves step-by-step towards its destination node. At each intermediate node a greedy stochastic policy is applied to choose the next node to move to. The policy makes use of (i) local agent-generated and maintained information, (ii) local problem-dependent heuristic information, and (iii) agent-private information. • While moving, the agents collect information about the time length, the congestion status and the node identifiers of the followed path.

9. AntNet Algorithm Overview • Once they have arrived at the destination, the agents go back to their source nodes by moving along the same path as before but in the opposite direction. • During this backward travel, local models of the network status and the local routing table of each visited node are modified by the agents as a function of the path they followed and of its goodness. • Once they have returned to their source node, the agents die.

10. Routing Table Contents Goodness (desirability) Routing table Mean, variance, and best Array of ds defining parametric statistical model for the traffic distribution over the network as seen by local node k

11. AntNet Algorithm • The heuristic correction ln is a [0,1] normalized value proportional to the length qn (in bits waiting to be sent) of the queue of the link connecting the node k with its neighbor n: • The value of alpha weights the importance of the heuristic correction with respect to the probability values stored in the routing table. Agent's decisions are taken on the basis of a combination of a long-term learning process and an instantaneous heuristic prediction. • Ideal alpha between 0.2 and 0.5

12. AntNet Algorithm • The backward ant updates the routing table and arrays stored at each node as it propagates through network. Positive reinforcement Negative reinforcement Reinforcement to be a function of the goodness where

13. Other Routing Algorithms Compared • OSPF (static, link state)Open Shortest Path First • SPF (adaptive, link-state) Shortest Path First • BF (adaptive, distance-vector) Bellman Ford • Q-R (adaptive, distance-vector): Q-Routing • PQ-R (adaptive, distance-vector): is the Predictive Q-Routing algorithm • Daemon (adaptive, optimal routing): is an approximation of an ideal algorithm

14. Networks Used • SimpleNet (1.9, 0.7, 8) 10Mbit/s and propagation delay of 1msec mean shortest path distance, in terms of hops, between all pairs of nodes, the variance Of this average, and the total number of nodes

15. Networks Used • NFSNET(2.2,0.8,14) 1.5Mbps propagation delays 4-20 msec

16. Networks Used • NTTnet(6.5,3.8,57) • 6Mbps propagation delay 1 to 5 msec

17. Metrics for Performance Evaluation • Throughput • Delay Distribution- the authors used whole empirical distribution or to use the 90th percentile statistic, which allows one to compare the algorithms on the basis of the upper value of delay they were able to keep the 90% of the correctly delivered packets • Network Capacity Usage (as expressed by the as the sum of the link capacities divided total available link capacity)

18. SimpleNet Throughput Results • SimpleNet: Comparison of algorithms for F-CBR traffic directed from node 1 to node 6) • The delay distribution showed similar results • *note AntNet outperformed

19. NFSNET Delay Results • Comparison of algorithms for increasing load for UP traffic. The load is increased reducing the MSIA (mean inter arrival time) value from 2.4 to 2 seconds • ** note that throughput results were similar amongst all algorithms but SPF and BF were the best

20. NTTnet Delay Results • NTTnet: Comparison of algorithms for increasing load for UP-HS traffic. The load is increased reducing the MSIA value from 4.1 to 3.7 seconds. • ** note that throughput results were similar amongst all algorithms but SPF and BF were the best

21. Routing Overhead Routing Overhead: ratio between the bandwidth occupied by the routing packets and the total available network bandwidth. All data are scaled by a factor of 10^-3

22. Conclusions and Future Work • AntNet showed superior performance and robustness to internal parameter settings for almost all the experiments. • AntNet's most innovative aspect is the use of stigmetric communication to coordinate the actions of a set of agents that cooperate to build adaptive routing tables.

23. Future Work • To add flow and error control to the algorithm • Change the priority of ants as the propagate through the system • Greater study of the negative reinforcement of connection • Greater survivability in the presence of faults (disaster situations)