Evolving Strategies for Contentious but Efficient Coexistence in Unlicensed Bands

1. N. Clemens C. Rose WINLAB Evolving Strategies for Contentious but Efficient Coexistence in Unlicensed Bands

2. Spectrum Management • Licensed • Mandated non-interference • Mediated • Spectrum Servers/Advisors • Laissez Faire • Raw greed sometimes works well • Memory can help when not

3. Our Problem: “survivor” • Scenario: A bunch of transceiver pairs are thrown into an unlicensed band • Transceiver Skills: • ‘cognitive’ • agile • Question: can transceivers learn to get along?

4. Toy System Model Everything is Gaussian IGNORE formal interference channel

5. Greed = Mutual Water Filling BAD

6. Two-Player Repeated Game • Symmetric model: • 2 signal space dimensions • Interference gains g • Actions: • Spectral power allocation x • Metric: • Average capacity Ci=( kCi(k))/K w/ K large • Strategy: • Play sequence of powers xi(k)

7. Actions and Payoffs Moderate Interference + Low Noise = Water filling is BAD Reputation Matters • Same player pool • Players remember

8. Pictorial Actions and Payoffs

9. Strategy Structure • Players with a memory of 2 games • Number of possible histories = 9x9 = 81 • Each history is associated with an action in response • Strategy = string of beads (actions)

10. Strategy Search HUGE Search Space Use genetic algorithms Basic Idea: Each possible solution is a ‘genome’. Generate populations and evaluate ‘fitness’. ‘Mate’ members to evolve new populations. Repeat until “good enough”

11. The Experiment Populations are collections of policy genomes Devise an evaluator set Completely random strategies Squatters, hoppers, avoiders No-learn fools (legacy?) Run tournaments, evaluate ‘fitness’ Result: Populations evolve effective strategies We distill essential features

12. Winning Strategy Performance • When played among themselves (left graph) they negotiate to segregate in signal space to reach a ‘fair’ settlement • When thrown into a population of random strategies, it outperforms the others

13. Winning Strategy Interactions • When two of our evolved strategies interact, there is an initial probing stage followed by a mutually agreed segregation • The intelligence of the opposition is necessary for attaining this ‘fair’ settlement. • What are the key features of these strategies?

14. Identifying Traits • Positive Traits: preference for an action • Negative Traits: avoidance of an action • Compose histogram and see what pops up

15. Examples of Useful Traits If the opponent doesn’t react to exploitation, continue to exploit Do not break a status-quo of segregation You push me, I push you back! Occasionally forgive to encourage cooperation Randomness to avoid repeated collision Avoid following the opponent in signal space

16. The Schema Skeleton • Fix useful traits and choose the rest of the genome randomly • Now test the fitness of this constructed strategy • The performance is good! Simple strategy descriptions might be possible!

17. The Seeds of Evolution Strategy structure depends on the other players Spread-only strategies spawned by Single channel squatters Spread channel squatters Random action Adaptive strategies emerge from interactions with other adaptive policies You’re as good as the company you keep

18. Questions Might we be able to use distributed competitive strategies in cognitive radios? Deploy fixed strategies? evolve via user purchase decisions? Radios evolve strategies in real time? Extensible to multi-player games? Formal strategy description grows exponentially with strategy memory size What is marginal improvement w/ size?

19. Cognitive Radio Kindergarten • The Golden Rule • Be Polite • Play Nice • The REAL Golden Rule • Be aware of your surroundings • A sucker and his toys are soon parted • Develop a good left hook • Bruises are bad • Forgiveness is divine (when you have a good left hook)