1 / 30

Agent-Based Computing in Software Systems: Exploring Models and Big Data Analysis

Discover the world of software agents, focusing on fine-grain entities with simple rules and coarse-grain entities for complex tasks like network administration. Topics include agent-based models, big data analysis, molecular dynamics, and multi-agent spatial simulations. Learn about agent-based data analysis and future goals in parallel-programming features. Explore agent-based simulation and the collision-free agent migration concept. This talk covers various disciplines' applications, from biology to economics. Finally, delve into the differences in counting triangles with agent-based big data analysis compared to the conventional approach.

hillj
Download Presentation

Agent-Based Computing in Software Systems: Exploring Models and Big Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Agent-Based ComputingCSS599 Winter 2018 Munehiro Fukuda, Ph.D. Computing & Software Systems University of Washington Bothell CSS599 Talk

  2. Outline • Softwareagents • Research Topic 1 • Agent-Based Models • Research Topic 2 • Agent-Based Big Data Analysis • Student Projects • Past students achievements • Currentstudentprojects • Expected activities CSS599 Talk

  3. Software Agents Cognitive Agents ReactiveAgents fine-grain entities, each reacting to its environment with simple rules. • coarse-grain execution entities that achieve network-administrative and computation-intensive task, based on their behavioral intelligence Ourfocus CSS599 Talk

  4. Topic 1Agent-BasedSimulation CSS599 Talk

  5. Macro-versusMicro-SimulationTop-downversusBottom-up Approach Macroscopic/Top-down Microscopic/Bottom-up Focus on an emergent, collective group behavior of many objects Molecular level Pedestrian/vehicle level Difficulty verification Calibration required Fewer real data available to the public (e.g. congestion at a traffic light) Scalable computation required Easy parallelization No partition required (yet object-CPU mapping needed) • Mathematical models • Schroedinger eq. (for wave) • Euler eq. (for fluid dynamics) • Easy verification • Mathematical verification • Easy access to public data (e.g. traffic flow) • Inaccurate enough • No support of object-interaction • Difficulty in parallelization • Space partition/mapping etc. CSS599 Talk

  6. MolecularDynamicsConventional Top-down Approach t=0 t=100 t=200 for ( int i = 0; i < N; i++ ) for ( int j = 0; j < I; j++ ) if ( sqrt( pow( x[i] – x[j], 2.0 ) + pow( y[i] – y[j], 2.0 ) ) < forceCutoff ) { ax[i] += fOver2 * ( x[i] – x[j] ); ax[j] -= fOver2 * ( x[i] – x[j] ); ay[i] += fOver2 * ( y[i] – y[j] ); ay[i] += fOver2 * ( y[i] – y[j] ); } i = molecular_range_begin[rank]; i < molecular_range_end[rank]; i++ ) MPI_Allreduce( ax, tmp_ax, N, MPI_DOUBLE, MPI_SUM, MPI_COMM_WORLD); MPI_Allreduce( ay, tmp_ay, N, MPI_DOUBLE, MPI_SUM, MPI_COMM_WORLD); computer 0 computer 1 Computer 2 CSS599 Talk

  7. MolecularDynamicsAgent-based bottom-upApproach Each agent: compute( int t ) { for ( Agent neighbor : place.agents ){ my_ax += fOver2 * ( my.x – neighbor.x ); my_ay += fOver2 * ( my.y – neighbor.y ); } } Agent-Based Computing Middleware computer 0 computer 1 Computer 2 CSS599 Talk

  8. Multi-Agent Spatial Simulation CSS599 Talk

  9. func1( ) func1( ) func1( ) func1( ) func2( ) func1( ) … … func2( ) func2( ) func3( ) func2( ) func3( ) MASS Specification Public static void main( String[ ] args ) { MASS.init( args ); Places space = new Places( handle, “MySpace”, params, xSize, ySize); Agents agents = new Agents( handle, “MyAgents”, params, space, population ); space.callAll( MySpace.func1, params ); space.exchangeAll( MySpace.func2, neighbors ); agents.exchangeAll( MyAgents.func3 ); agents.manageAll( ); MASS.finish( ); } CSS599 Talk

  10. Implementation Status • The latest status is given at: • http://depts.washington.edu/dslab/MASS/index.html • MASS Java • Intended for agent-based big data. • MASS C++ • Intended for agent-based models. • MASS GPU • In progress • Debugger • Available for MASS Java and C++ versions • Availability • Pilot used in CSS534 summer 2012, winter 2015, and autumn 2016 • Java and C++ versions arealdy released to publicly • https://bitbucket.org/mass_library_developers/ • https://bitbucket.org/mass_application_developers/ CSS599 Talk

  11. goal start goal 45 minutes start Agent-BasedModels in Other Desciplines • Biology/Ecology • Immune system simulation • Epidemic simulation • Neural network simulation • City Planning • Transport simulation • Electrical grid • Social, Behavioral, and Economic Sciences • Ambient intelligence • Social network • Economic crisis simulation CSS599 Talk

  12. Agent-BasedTransportSimulationParallelized byZachMaandpresentedatWSC2015 Road network converted to an adjacency list Programmability Execution performance Vehicle drives converted to agent migration CSS599 Talk

  13. Collision-Free Agent MigrationPresented at FedCSIS/JAWS 2017 • Sequential migration • Upper-left agents have a higher priority than lower-right agents. • Parallel migration • All agents can migrate at once. • We need some prioritization. MATSim Traffic simulation Greedy parallel migration Location-ordered migration Direction-ordered migration Sequential migration collision another direction Distributed termination detection required 5 ordered migrations 4 ordered migration direction CSS599 Talk

  14. Our FutureGoals Investigating what parallel-programming features attract scientists who are not necessarily computing specialists: • Parallelizing ABM applications of our collaborators. • Developing benchmark testsets that cover various agent behaviors. CSS599 Talk

  15. Topic 2Agent-BasedData Analysis CSS599 Talk

  16. Topic 2:Agent-BasedBigDataHow different from conventional approach when counting triangles in a graph for example MapReduce, Spark, and Storm Agent-based Big Data Analytics actively move over data. Only necessary portionofdatacanbeexamined. Data can maintain its structure. Code is written from the driver’s viewpoint. • Analytics stay passive to receive data. • All data must be examined. • Data must be flattend to feed. • (1,2), (2,1), (2,3), (3,2), (2,4), (4,2), (2,5), (5,2), (3,4), (4,3), (4,5), (5,4) 1 • (1, [2]), (2, [1,3,4,5]), (3, [2,4]), (4, [2,3,5]), (5, [2,4]) • [(1,2), -], [(1,3), 2], [(2,3), -], [(2,3), 4], [(2,4), -], [(2,4),3], • [(2,4), 5], [(2,5), 4], … 2 5 • (2,3) [-,4]  (2,3,4), (2,4) [-,3,5] -> (2,4,3), (2,4,5)… • (2,3,4), (2,3,4), (2,3,4), (2,4,5), (2,4,5).. • (2,3,4), (2,4,5) 3 4 CSS599 Talk

  17. n2(0.133) n1(0.1) n5(0.383) n3(0.183) n4(0.2) Our Target Applications Injecting a group of agents into big data • Scientific Computing • Climate analysis • Prof. Kim’s biological network motif search • Data Sciences / Machine Learning • K-means clustering • K-nearest neighbors • GIS/Spatial Data Analysis (represeted by graph problems) • Shortest path • Chinese postman problem • Triangle counting • page ranking CSS599 Talk

  18. Climate AnalysisparallelizedbyJasonWoodringandpresentedat ICDCS 2017 Detect time of emergence Thebiggestoverheadisreadingfilesintoaclustersystem CSS599 Talk

  19. Biological Network Motif Searchwith Prof. Wooyoung Kim and Presented at HCPP 2015 and BDCloud 2016 • Agent explosion • 400,000-node network with motif size 5 • 5.5 million agents • Needs to address • Memory allocation overheads • Agent management overheads • Controlling agent population • Don’t spawn too many agents. • In Java, • Size of each agent = 1M • Size of serialized agent = 300B A A R A R R R R A agent spawn request R CSS599 Talk

  20. Agent-basedClustering Single Clustering Algorithm Batch Clustering Algorithm K-means + Biologically-Inspired Algorithms • K-means X X A Space of Potential Cluster Centroid Sets X X C1 X agent X [c1, c2, c3] X X [c1, c2, c3] X X X X X X [c1, c2, c3] X X swarm X X X X C2 [c1, c2, c3] [c1, c2, c3] X [c1, c2, c3] X X X X X X X X C3 X X X X CSS599 Talk

  21. Agent-based K-Nearest Neighbors milliseconds X X X X X X T=5 T=4 X X T=3 X X X X X X K = 7 X T=2 X ? T=1 X X X X X X MASS Places size: 100 * 100 400 * 400 700 * 700 So that each place may have 1 or 2 data items. X X X X X X X X X X X Problem: An agent must be spawned by another agent but not any place. This in turn means that we need to populate an agent at each place at the beginning, which is a big overhead. CSS599 Talk

  22. Past StudentResearchOutcome(gradudate students in red and undergraduate students in blue, listed below) • Christopher Bowzer, Benjamin Phan, Kasey Cohen, Munehrio Fukuda, Collision-Free Agent Migration in Spatial Simulation, In Proc. of the Joint Agent-oriented Workshops in Synergy JAWS, Prague, Czech Republic, September 3-6, 2017 • Jason Woodring, Matthew Sell, Munehiro Fukuda, Hazeline Asuncion, Eric Salathe, A Multi-Agent Parallel Approach to Analyzing Large Climate Data Sets, The 37th IEEE International Conference on Distributed Computing Systems, pages 1639-1648, Atlanta, GA, June 5-8, 2017 • Andrew Andersen, Wooyoung Kim, and Munehiro Fukuda, MASS-based NemoProfile Construction for an Efficient Network Motif Search, 2016 Big Data and Cloud Computing – BDCloud’16, October 8-10, 2016 • Zhiyuan Ma, Munehiro Fukuda, A Multi-Agent Spatial Simulation Library for Parallelizing Transport Simulation, the 2015 Winter Simulation Conference - WSC 2015, Newport Beach, December 6-9, 2015 • Bhargav Mistry, Munehiro Fukuda, Dynamic Load Balancing in Multi-Agent Spatial Simulation, submitted to 2015 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing - PACRIM’15, Victoria, BC, Canada, August 24-26, 2015 –Best Paper Award • Matthew Kipps, Wooyoung Kim, Munehiro Fukuda, Agent and Spatial Based Parallelization of Biological Network Motif Search, submitted to 17th IEEE Int’l Conference on High Performance and Communication, New York, August 24-26, 2015 • Brett Yasutake, Niko Simonson, Jason Woodring, Nathan Duncan, William Pfeffer, Hazeline Asuncion, Munehiro Fukuda, Eric Salathe, Supporting Provenance in Climate Science Research, In Proc. of 7th International Conference on Information, Process, and Knowledge Management, eKnow 2015, Lisbon, Portugal, February 22-27, 2015 –Best Paper Award • Timothy Chuang, Munehiro Fukuda, A Parallel Multi-Agent Spatial Simulation Environment for Cluster Systems, In Proc. of the 16th IEEE International Conference on Computational Science and Engineering - CSE 2013, pages 143-150, Sydney, Australia, December 3-5, 2013 • Somu Jabayalan, Munehiro Fukuda, Field-Based Job Dispatch and Migration, In Proc. of 2013 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing - PACRIM'13, pages 52-57, Victoria, BC, Canada, August 27-29, 2013 CSS599 Talk

  23. Thesis/CapstoneProjects Future projects in blue CSS599 Talk

  24. Agent-Based Clustering: Collin Gordon Use MASS to perform clustering on data points Variant 1 Single Clustering: Uses Agents and Places to compute one set of clusters for a given data set. Example: Call the Ball Algorithm (in progress) Variant 2 Batch Clustering: Uses Agents and Places to compute multiple sets of clusters for a given data set. Example: Best Tool Algorithm (completed) Currently targeting PAAMS, MAS&S, and IEEE for paper submissions. Questions? Email me at colntrev@gmail.com! January 24, 2017 Collin Gordon

  25. Code and Execution January 24, 2017 Collin Gordon

  26. MASS CUDALisa Kosiachenko GPU programming model • Single Instruction Multiple Thread (SIMT) parallel programming model • GPU has many more arithmetic logical units than CPU (on NVIDIA GeForce GTX TITAN Black – 2880 cores in total) Lisa Kosiachenko

  27. MASS Current Development • Optimization of speed • Implementing new features (dynamic agents) Lisa Kosiachenko

  28. MASS CUDA Optimization:Utilizing GPU Memory Architecture Fast (visible to a block of threads) Very fast (visible to 1 thread) Slow Slow Fast (read-only) Fast (read-only) Lisa Kosiachenko

  29. Expected Activities • Research Group Meetings • Weekly status report to your lab mates • Quarterly research presentation • OptionalIndividual Meetings • Weekly progress discussions, code reviews, etc. with theprofessor, (i.e.,me). • Programming/Installation • We are a product-oriented group. You need to code somethingorinstall something useful to manage our projects • Publications • I recommend you to coauthor conference/journal papers. CSS599 Talk

  30. Questions? • Please check: http://depts.washington.edu/dslab/ CSS599 Talk

More Related