Trading Agent Competition (TAC)

1. Trading Agent Competition(TAC) Jon Lerner, Silas Xu, Wilfred Yeung CS286r, 3 March 2004

2. TAC Overview • International Competition • Intended to spur research into trading agent design • First held in July 2000 • TAC Classic and TAC SCM Scenarios

3. TAC Classic • Each team in charge of virtual travel agent • Agents try to find travel packages for virtual clients • All clients wish to travel over same five day period • Clients not all equal, each has different preferences for certain types of travel packages

4. Travel Packages • Each contains flight info, hotel type, and entertainment tickets • To gain positive utility from client, agents must construct feasible packages. Feasible means: • Arrival date strictly less than departure date • Same hotel reserved during all intermediate nights • At most one entertainment event per night • At most one of each type of entertainment ticket

5. Flights • Clients have preferences for ideal arrival/departure dates • Infinite supply of flights sold through continuously clearing auctions • Prices set by a random walk • Prices later set to drift upwards to discourage waiting • No resale or exchange of flights permitted

6. Hotels • Two hotels – high quality and low quality, 16 rooms per hotel per night • Sold through ascending, multi-unit, sixteenth-price auctions: one auction for all rooms for single hotel on single night • Periodically a random auction closes to encourage agents to bid • Clients have different values for high and low quality hotels

7. Entertainment • Three types of entertainment available • Clients have value for each type • Each agent has initial endowment of tickets • Buy and sell tickets through continuous double auction

8. Agent Themes • Agents have to address: • When to Bid • What to Bid On • How Much to Bid • Combinatorial preferences, but not combinatorial auctions

9. Strategies • What strategies come to mind? • What AI techniques might be useful? • Simple vs. Complicated Strategies • How quickly should you adapt as game progresses? • Use of historical data vs. Focus on current game only • Play the game vs. Play the players

10. living agents (Living Systems AG)Winner: TAC 2001 • Makes two assumptions • 1. Steadily increasing flight prices favor early decisions for flight tickets. • 2. Especially the good performing teams are following a strategy to maximize their own utility. They are not trying to take the risk to reduce other team’s utility. • Simple strategy • Makes substantial use of historical data. • Barely any monitoring/adapting to changing conditions • Benefits from other agents’ complicated algorithms to control price; Open-loop, Play the Players

11. living agents: Determining Hotel and Flight Bids • Assume hotel auction will clear at historical levels • Using these as hotel prices, initial flight prices, and client preferences, determine optimal client trips • Immediately place bids based on this optimum • Purchase corresponding flights immediately • Place offers for required hotels at prices high enough to ensure successful acquisition

12. Entertainment Auction • Immediately makes fixed decision as to which entertainment to attempt to buy/sell assuming the historical clearing price of about $80. • Opportunistically buy and sell around this point • Put in final reservation prices at seven minute mark.

13. How good is living agents? • Risky • If hotel bids are not high enough, fails to complete trips, resulting in huge loss of points. • If hotel clears at living agents’ bid, potentially pays much more than necessary • After placing initial bid, does not monitor hotel or flight auctions at all • Clearly not all agents could use this strategy (Hotel auctions) • Simple • Buys flights immediately, avoiding cost of waiting • Relies on historical data • Contains information from many games • But how sensitive is evolution of game to changes in client preferences, or changes in opponents’ strategy?

14. Applicability • Use of historical data for predictive information • Feasibility of simple strategies that ignore feedback • Play against the players (not prices), under the assumption that other agents keep things relatively efficient.

15. ATTac (AT&T Research)Winner: TAC 2002 • Uses sophisticated machine-learning techniques to predict future hotel prices based on the current situation • Buys flights based on cost-benefit analysis of committing versus waiting • Minute-by-minute reoptimization of bids based on holdings and predictions

16. The heart of ATTac • Assumption: Because of many unknowns, exactly predicting the price of a hotel room is hopeless. • Instead, regard the closing price as a random variable that needs to be estimated, conditional on our current state of knowledge • Number of minutes remaining in game • Ask price of each hotel • Flight prices • Historical Date • Construct a model of the probability distribution over clearing prices (based on a boosting algorithm), stochastically sample prices, and compute expected profit

17. The high-level algorithm • Denote the most profitable allocation of goods at any time by G* • When first flight quotes are posted: • Compute G* with current holdings and expected prices • Buy the flights in G* for which the expected cost of postponing commitment exceeds the expected benefit of postponing commitment • Starting 1 minute before each hotel close: • Compute G* with current holdings and expected prices • Buy the flights in G* for which expected cost of postponing commitment exceeds expected benefit of postponing commitment • Bid hotel room expected marginal values given holdings, new flights, and expected hotel purchases • Last minute: Buy remaining flights as needed by G* • In parallel (continuously): Buy/sell entertainment tickets base on their expected values

18. The boosting algorithm: solving conditional density estimation problems • Start with ordered pairs (x,y), with x being a vector that describes auction-specific features, y being the difference between closing price and current price • Aim of boosting is, given current x, to estimate the conditional distribution of y • Construct conditional distribution function that minimize the sum of negative log likelihood of y given x, for all training samples. • Use this condition distribution function to map x to y

19. living agents vs. ATTac • Two very different approaches • Statistically insignificant difference in scores in TAC2001

20. Open and Closed Loop Processes • Closed-loop: system feeds information back into itself. Examines the world in an effort to validate the world model. • appropriate for real-world environments in which feedback is necessary to validate agent actions. • Open-loop: no feedback from the environment to the agent. Output from processes are considered complete upon execution. • appropriate for simulated rather than real environments (tasks not performed perfectly by agent generally.) • generally more efficient for the same reason.

21. Walverine: (Closed-loop) • Model Based: Flight and Hotel • Predicts hotel prices by Walrasian equilibrium • Derives expected demand from 64 clients’ preferences and initial flight prices, which influence clients’ choice of travel days, and • Construct bids that max expected value of bid • Model Free: Entertainment • Q-Learning from thousands of auction instances (aside on model vs model-free learning) • No empirically tuned parameters

22. SouthamptonTAC: (Closed-loop) • Adaptive agent, varies strategy to mkt cond. • 3 classifications for environments: • Non-competitive (agent gets hotel at low prices) • Semi-competitive (medium prices) • Competitive (prices of hotels high) • Based on curr game and outcomes of recent games • Non-competitive: • Buys all flights at beginning of game • Never change itinerary of clients

23. SouthamptonTAC: (Closed-loop) • Competitive: • Rapidly rising prices – buy at beginning • Stagnant prices – buy near the end • Fuzzy reasoning to predict hotel clearing prices • 3 rule bases • Factors inc: price of hotel, counterpart, price change in prev minute, price change in counterpart hotel in prev minute • Continuously assesses game type

24. ROXY-BOT: (Open-loop) • Two phase bidding policy: • Solve completion problem • Optimization based on a tree structure using beam search that only partially expands the tree. [Greenwald] • Valuate goods in that set • Marginal utility calculator MU(x) = V(N) – V(N|x) • Computing Prices: (historical data) • Point estimates (’00) • Estimated price distributions (’01) • Averaging MU across many samples of estimated price dist • Monte-Carlo simulation to evaluate bidding policy (’02)

25. Whitebear (Winner in ’02, Open-loop) • Flights: • A: buy everything • B: buy only what is absolutely necessary • Combination: buy everything except dangerous tickets • Hotels: (predictions simply historical averages) • A: bid small increment greater than current prices • B: bid marginal utility • Combination: Use A, unless MU is high, use B • Domain specific, extensive experimentation • No necessarily optimal set of goods, no learning

26. Summary: Open vs Closed • All else equal open-strategy better: • Simple • Avoids waiting costs (higher prices) • Predictability of price is determining factor • Perfectly predictable – open-loop • Large price variance – closed-loop • Open-loop picks the good at the start and may pay a lot • Small price variance – optimal closed loop • But complexity for potentially small benefit