Class-constrained Packing Problems with Application to Storage Management in Multimedia Systems

1. Class-constrained Packing Problems with Application to Storage Management in Multimedia Systems Tami TamirDepartment of Computer ScienceThe Technion(Based on Joint papers with Hadas Shachnai)

2. A Resource Allocation Problem: Devices:Each device has a limited amount of some resource. - A disk with a limited broadcast capacity. - A processor with specific rate. - A machine with a limited amount of material. Clients:that present requests for resources. - Transmission of some video-program file. - Processing of a job that needs sprocessing units. - Production of a product that needs s units of the material.

3. devices clients The Multiple Knapsack Problem (MK) • A set of N knapsacks, K1,...,KN. Each knapsack, Kj, has limited volume, vj. • A set, I, of n items, u1,...,un. Each item, uI, has a size, s(u), and a value, p(u). • The objective: A legal placement of some U  I with maximal value. • Legal = For all j, the total size of items placed in Kj is at most vj.

4. 2 1 3 1 1 4 4 3 2 2 3 3 4 3 3 3 3 3 An Example (MK)For all u, p(u) = s(u). v=8 n=6 Optimal value = 16

5. A Class-constrained A Resource Allocation Problem: Devices:Each device has a limited amount of some resource, - A disk with a limited broadcast capacity, - A processor with a specific rate, Clients:that present requests for resources, - Transmission of a video-program file. - Processing of a job that needs uprocessing units. and a limited number of ‘settings’. and a limited storage capacity. and a limited number of configurations. and need specific ‘settings’. the video program file “Tom & Jerry” of type ‘red’ that needs sprocessing units.

6. The Multiple Knapsack Problem (MK) The Class-constrained Multiple Knapsack Problem (CCMK) • A set of N knapsacks, K1,...,KN. Each knapsack, Kj, has limited volume, vj, • A set, I, of n items, u1,...,un. Each item, u  I, has a size, s(u), and a value, p(u). • The objective: A legal placement of some U  I with maximal value. • Legal = For all j, the total size of items placed in Kj is at most vj, and a limited number,cj, of compartments. a value,p(u), and a type (color), c(u)  {1,…,M}. and the number of different colors included in Kj is at most cj.

7. 1 4 3 2 3 3 2 3 1 4 3 2 3 4 3 3 3 An Example (CCMK)For all u, p(u) = s(u). v=8, c=2 n=6, M=3 Optimal value = 15

8. v = 8 c = 2 A Special Case: Unit-sized Items • When each client’s request is for one unit of some resource, then in the resulting packing problem each item, u  I, has size s(u)=1. 6 + 7 + 2 + 6 + 3 = 24 • An important application: Data Placement for Multimedia-on-demand Systems.

9. Multimedia-on-Demand Systems • A centralized server. • A large database of video programs transmitted to clients. • Storage devices (disks). • The system has to “play” multiple streams of many different videos simultaneously, based on customer demand. The Challenge: Balance the load on the storage devices.

10. Multimedia on Demand - Some Special Properties • Good Properties: • A stream is a logically defined unit of load. • Basically predictable, both in forecast and real-time. • Most of the time: read only. • Flexibility: can make multiple copies of popular files, can choose (and change on-line) the source. • Bad Properties: • Rigid real-time scheduling requirement. • Sustained broadcasts with fixed transfer rates.

11. The I/O Subsystem • A database of M video-program files {f1, f2, …, fM}. • N storage devices (shared disks) {d1, d2, ..., dN}. • Each disk, dj,has: cj = Storage Capacity: the number of program files that can be stored on dj. vj= Load Capacity: the maximal number of concurrent multimedia streams that can be transmitted by dj.

12. The I/O Subsystem (Cont’) The number of broadcasts allocated to each file must be determined by its popularity pi = the popularity of the file fi

13. Maximizing the Throughput • Given the file popularities, how many copies of each file should be kept?Where should we store each copy? • How many broadcasts should be allocated to each of these copies? • Given the allocation of file copies to the disks, and a request to “play” a video, from which copy do we allocate a data stream? • How do we dynamically balance the load on the disks?

14. A Load Balancing Scheme Static Phase(run periodically): Input:Forecast of movie play demand. Output: Good quality assignment of storage and load capacity to the video files. Modes: From scratch / incremental. Dynamic Phase(run online): Input:Assignment of files to disks; current movie play demands. Output: Good load balancing decisions. Modes: incremental.

15. Related Work Data sharing techniques: • Batching – delaying requests with the hope that more requests for the same data will arrive during the batching interval. • Buffering - closing the temporal `gaps' between successive requests through the use of buffer space. • Adaptive piggybacking - adjusting broadcast rates of requests in progress for the same data until their corresponding data streams can be merged into one. These techniques can be applied independently of the load balancing scheme

16. The Static Phase • A Static Assignment Algorithm allocates the resources to the files. The output of a static assignment consists of two M  N matrices: • The Assignment matrix, A • a{0, 1} – matrix, • Aij =1 iff a copy of fi is stored on dj. • The Broadcast matrix, B • Bij {0, 1, ..., vj}. Bij is the number of broadcasts allocated to fi on dj.

17. The Static Assignment (Cont’) A legal static assignment must satisfy the following conditions: 1.Aij = 0  Bij = 0 [A disk can broadcast a video program only if it holds the video program file]. 2.j, i Bij < vj [the sum of allocated broadcasts does not exceed the broadcast capacity of the disk]. 3.j, i Aij < cj [the number of copies stored on a disk does not exceed its storage capacity].

18. Definitions • The broadcast potentialof the system is V = • The broadcast potential offi is Bi = • The broadcast request offi, denoted by Wi , is the desired number of broadcasts of fi . •  i ,pi V< Wi <pi V and

19. The Goals • 1. Fairness: An assignment is -fair if for every • file fi, Bi> Wi • allocated required •  [0, 1] • 2.Utilization: The utilization of an assignment is • U =i min(Wi , Bi) • required allocated • U is the total number of broadcasts that may be required and provided. • A Perfect Assignment is both 1-fair and with • utilization V (i , Wi =Bi).

20. B22=B23=20 B11= 60 d2: c2= 2 v2=40 d1: c1= 1 v1=60 Maximizing Fairness and Utilization f1: Pokemon p1 = 0.8 f2: Telletubies p2= 0.1 f3: The Flinstones p3= 0.1 Most Fair! V=100 W1 = 80, W2 = W3 = 10 B1= 60, B2= 20, B3= 20  3/4fair (60/80 for f1) ;  U = 60 + 10 + 10 = 80

21. B12=B22=20 B11= 60 d2: c2= 2 v2=40 d1: c1= 1 v1=60 Maximizing Fairness and Utilization f1: Pokemon p1 = 0.8 f2: Telletubies p2= 0.1 f3: The Flinstones p3= 0.1 Most Utilized! V=100 W1 = 80, W2 = W3 = 10 B1= 80, B2= 20, B3= 0  0fair,  U = 80 + 10 + 0 = 90

22. Our Results • The fair-allocation problem and the utilization problem are strongly NP-hard. • For some instances a perfect assignment always exists and can be found in polynomial time. • How a perfect assignment can be approximated (the approximation ratio depends on the uniformity of the disks). • The utilization problem can be solved optimally in polynomial time by adding a single storage unit to each disk (a dual approximation algorithm).

23. 3 , 4 , 6 , 6 , 8 , 8 16 20 13 The Moving Window Algorithm The knapsacks are sorted such that v1/c1  v2/c2  vN/cN. The sets of items are sorted such that |I1|  |I2|   |IM|. To fill the knapsack Kj, scan the list of items with a moving window that covers items of cj colors, until for the first time the window contains at least vj items. vj=17 cj=3

24. The Moving Window Algorithm We use the moving window algorithm to show: • When the knapsacks are homogenous (with identical v/c ratio) and cj  N+M-1, a perfect assignment always exists, and can be found efficiently. • If we are allowed to add a single compartment to each knapsack, then we can achieve the optimal utilization.

25. The Dynamic Phase • File popularities typically change periodically. • The MOD system may support the following operations: Reallocate storage capacities (by deletion of file copies, replications...). Reallocate load capacities (internal balance on each disk). • Based on the work of J.L. Wolf, P.S. Yu, and H. Shachnai, 1997: The current assignment is represented in two graphs: the file graph and the disk graph.

26. The Disk Graph, GD=(VD, ED) • VD: {d1, d2, …, dN} • ED: A directed edge (d1, d2) if there exists a file fi such that: • . Ai1 = Ai2 =1 • . There is a stream allocated to fion d1. • . Total load on d2 is at most v2-1. • A directed edge (d1, d2) signifies the potential of transferring a single stream from d1 to d2.

27. An Example of Dynamic Balancing A system with four movies: 1.  The lady in red (f1) 2.  Blue velvet (f2) 3.  Pink panther (f3) 4. The green mile (f4) d1 f1, f2 d2 f3 , f2 d4 f4 d3 f3, f4 More broadcasts of f1 are available

28. 1 4 3 2 3 3 2 3 3 4 3 Back to General Instances…. where items may have arbitrary sizes and arbitrary profits v=8, c=2 n=6, M=3

29. The Class-constrained Bin- Packing Problem (CCBP) • A set of Identical bins, each having volume v and c compartments. • A set, I, of items. Each item, u, has a size, s(u) < v, and a color, c(u)  1,...,M. • The goal: A legal placement of all the items in I in a minimal number of bins. In resource allocation terms, CCBP = complete all the requests using as few (identical) devices as possible.

30. In resource allocation (examples): • A request to print a file of 600 pages, both yellow and pink papers are fine. • A request to watch a video-program-file, • any episode from Seinfeld will do. Generalized Class-constrained Packing (GCCP) • Each item, u  I, has a size, s(u), a value, p(u), and a set, c(u)  {1,...,M}, of colors. • It is legal to place u in any knapsack that allocates a compartment to some color in c(u).

31. Our Results for General Instances • A dual PTAS for CCBP, (packs any instance, I, into m  OPT(I) bins of size (1+ )v ). • A PTAS for CCMK, (packs a set U  I such that P(U)  (1-)OPT(I)). • An FPTAS for the 0-1 class-constrained knapsack. - optimal when all the items have the same value. • APX-hardness proofs for GCCP (already for the case where all items have the same size and the same value). - For the PTASs we assume that M is a constant. - The FPTAS is suitable for M=(n). - In the APX-hardness proof M= (n).