Online Implementation of the Delta Design Game for analyzing Collaborative Team Practices

1. Online Implementation of the Delta Design Game for analyzing Collaborative Team Practices Sharad Oberoi, Susan Finger and Eric Rosé Carnegie Mellon University

2. Collaborative design • No single actor in complex design tasks has all the knowledge needed to realize the project goal. • Collaborative design requires individual design expertise as well as collaboration skills. • Team members have to ensure that the relevant knowledge is shared and integrated effectively. • To achieve this outcome, team members need to have skills for facilitating the integration of specialized knowledge across boundaries. DETC2013-13319

3. Delta Design Game • Created by MIT Professor Louis Bucciarelli in the 1980s • Aim: • To engage undergraduate engineering students in a team design project requiring communication and negotiation • Interdependence of the objectives of the roles requires the players to reveal their constraints in order to achieve a satisfactory outcome DETC2013-13319

4. Delta Design Game: Design Objectives • Planet DeltaP is a planned flat world with the design done in a diagonal 2-D space. • The team designs the residence for the citizens of DeltaP using red and blue deltas shaped like equilateral triangles. • Red deltas provide heat • Blue deltas have a cooling effect • Each team is asked to design a house that meets all the structural, thermal and aesthetic constraints. DETC2013-13319

5. Delta Design Collaboration • Actors • Architect • Project manager, • Thermal engineer • Structural engineer • DeltaP’s unique design environment and its skewed coordinate system levels the playing field for all players. DETC2013-13319

6. Delta Design Collaboration • Each player only knows the rules for his/her own role. • The rules are designed so that certain roles can easily form alliances, while others have conflicting objectives. • Unless the team realizes that they need to see the big picture and compromise, they cannot be successful in the game. DETC2013-13319

7. Delta Design Game: Board version • Computations required to evaluate the objective function for each role are lengthy. • Metrics need to be recomputed manually each time a board tile is moved. • Even successful teams barely manage to achieve most of their design objectives in the allotted time. • Lasts between 1-1.5 hours • Key attributes of collaboration and the negotiations within the team cannot be captured adequately. →Researchers cannot analyze collaborative team practices during the game. DETC2013-13319

8. Delta Design Game: Online version • Computations required to evaluate the objective function for each role are done automatically in the background. • Metrics recomputed automatically each time a board tile is moved. • Most teams manage to achieve their design objectives in the allotted time. • Lasts between 1-1.5 hours • Key attributes of collaboration and the negotiations within the team are captured through text chat and board moves. →Researchers can analyze collaborative team practices! DETC2013-13319

9. Delta Design Application DETC2013-13319

10. Delta Design Application • Rules of the board-based Delta Design game have been adapted as functional requirements for the online game. • Every action taken on the shared virtual board and all text chat between participants are recorded. • The players can roll back the canvas to a prior configuration if they realize that they have made a wrong move → allows the players to consider various strategies to maximize their scores. • Both the automatic computation of objective functions and the ability to roll back moves change the nature of the collaboration from the physical game play. DETC2013-13319

11. Delta Design Application Constraints • Only one person is allowed to work on the board at a time. • In order to move a tile the player needs to lock the board first (up to 3 minutes). • When a player has locked the board, the other team members cannot talk to him in person, but can send comments and requests in the chat window. • In order to win, everyone on a team must meet his/her goals. DETC2013-13319

12. System Architecture • The Delta Design game is a network application. • Four players located on separate computers interact with each other by way of a shared game surface, a shared drawing surface, and a chat application. • A central server maintains the current state of all running games. • Each player who logs into one of these games has a pre-assigned role. DETC2013-13319

13. System Architecture DETC2013-13319

14. Event Model of Delta Design • Every interaction with the simulation involves sending an event to the server and the receipt of a response which may contain an arbitrary amount of data. DETC2013-13319

15. Delta Design Game Implementation in Classroom • Authors conducted a series of user studies with this online implementation over a period of 4 years • Data collected from 38 four-player teams (mostly undergraduate capstone design classes) • Provides a rich resource to explore different aspects of collaborative team practices. DETC2013-13319

16. Evaluation Criteria • Metrics used to evaluate success of a collaborative strategy: • Did the teams strategize before the game? • Did the players share their constraints before the game? • How long did the teams take for getting all green lights (or their highest score during the game)? • Proposed hypothesis: • The teams that are more communicative during the Delta Design game can better negotiate among themselves and hence should be more successful. DETC2013-13319

17. Results • Out of 38 teams analyzed, only 12 achieved a score of 3 or more: • 3 green + 1 yellow light • All 4 green lights DETC2013-13319

18. Results • Among the 12 teams (with score >=3): • 50% teams devised a strategy for moving the tiles on the canvas and discussed beforehand what each person’s goals were • 16% teams neither strategized nor discussed their goals beforehand. • Whether or not the players strategized before the beginning of the game does not have any marked effect on the game’s outcome. • Whether or not the players were transparent about their role’s constraints and functional requirements does not have any marked effect on the game’s outcome. DETC2013-13319

19. Results • Teams which strategize about their game-plan are also better correlated to share their constraints with each other. • However, this does not seem to make them any more successful. DETC2013-13319

20. Conclusions • The complex rules of Delta Design make it a suitable candidate for analyzing collaboration strategies in team-based design projects. • More transparent negotiation tactics and board-moving strategies, do not improve the teams’ chances of success. • The proposed hypothesis has been proved false. • The teams that are more communicative during the Delta Design game are not found to be significantly more successful. DETC2013-13319

21. Questions? DETC2013-13319

22. Acknowledgments • This research was supported in part by National Science Foundation grant 0935127. • We would like to thank: • Prof. Louis Bucciarelli, MIT • Prof. Daniel P. Siewiorek, CMU • Prof. Asim Smailagic, CMU • Prof. Carolyn P. Rosé, CMU • Peter A. Simon, CMU • Gahgene Gweon, CMU DETC2013-13319

23. Delta Design Application: Lights • Every player has met his goal when all the stop lights on the tabs are green. • All players can see the lights for the other players, but not their specific calculations. • A player within 10% of the goal for any one of his conditions who has met all the other conditions is assigned a yellow light. • This allows other members of the team to see how their moves on the canvas affect the performance of the team as a whole. DETC2013-13319

24. Coordinate System • Virtual board is composed of diamond shaped units laid out on a coordinate system on a 150° angle. • This can be seen as a grid composed of triangles called “quads,” each of which bisect one of the diamonds. • Each delta consists of four smaller quads. • The “origin” or center of a delta is considered to be the middle quad of the four • Each delta can be represented using three components: x-coordinate, y-coordinate, and orientation. • The placement of deltas on this grid follows certain rules which vary with the orientation. DETC2013-13319

25. Coordinate System DETC2013-13319

26. MySQL Database • The database stores the following: • The details of the game (start time, duration, agents, playing surface state) • A log of all of the events which take place to modify the playing board • A log of all of the chat communication between the players • A snapshot of the state of the game after every action which modifies the board • A record of any events affecting the Delta Design server DETC2013-13319