1 / 4

Project

Project . Design, build, and operate a large-scale distributed application. Issues to worry about: Scalability, reliability, efficient use of resources, easy to operate, reuse, Large-scale deployment platform. (PlanetLab) Limited handholding Groups of up to 3 students. TO DO: Start

jamil
Download Presentation

Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Project Design, build, and operate a large-scale distributed application. • Issues to worry about: Scalability, reliability, efficient use of resources, easy to operate, reuse, • Large-scale deployment platform. (PlanetLab) • Limited handholding • Groups of up to 3 students. • TO DO: Start thinking about your group.

  2. Gnutella Network Topo crawl Topo information (e.g., neighboring nodes) Main idea: recursively crawl the entire network Support provided: libraries, bootstrap node

  3. Project steps • P1. warmup: familiarize yourself with PlanetLab, setup the environment, develop a monitoring service • P2. start crawling in controlled environment: centralized / single node crawler. • P3. large –scalecrawler • Master-worker design • Deployed on planetlab (and using at least 100 nodes) • Options: • Single node vs. distributed. • Blocking vs. non-blocking IO; • Volume of data gathered

  4. Alternatives: Your own project Goal: Design, build, and operate a large-scale distributed application Some ideas • Crawl and analyze data form other p2p or social networks: • e.g., Twitter, Skype, YouTube • Hard: closed protocols (Skype) • Cool: no (or few) independent measurements • Explore Amazon service performance: e.g., S3 • Performance: latency, throughput, consistency • multiple vantage points (migration?) • Hard: limited budget (!), black box • Cool: real, well engineered service, huge scale • Others: • location services, • Network health monitoring

More Related