1 / 14

Progress on the Circuit-LDM Project

Progress on the Circuit-LDM Project. Jie Li and Malathi Veeraraghavan 2010.10.28. Outline. LDM Software Installation & Set Up CONDUIT Feed Type Data Analysis Justifications of Using Circuit Services Future Plans. LDM Software Installation & Set Up.

enid
Download Presentation

Progress on the Circuit-LDM Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Progress on the Circuit-LDM Project Jie Li and MalathiVeeraraghavan 2010.10.28

  2. Outline • LDM Software Installation & Set Up • CONDUIT Feed Type Data Analysis • Justifications of Using Circuit Services • Future Plans

  3. LDM Software Installation & Set Up • Downloaded LDM 6.8.1 Version and installed it on one of our lab machine (zelda2.ece.virginia.edu) • Configured the ldmd.conffile to add a REQUEST entry to receive CONDUIT data products from idd.unidata.ucar.edu:388 • Ran LDM in verbose mode to log information about every received CONDUIT product • Rotated log files in every 24 hours

  4. CONDUIT Data Analysis • Write Perl programs to parse the log files and extract information about each product: Product Size, Creation Time, etc. • Write R programs to do statistical analysis and make plots on the product information Product Size Creation Time Aug 25 19:47:56 zelda2 uni1.unidata.ucar.edu[18517] INFO: 688420100825193312.333 CONDUIT 412 data/nccf/com/sref/prod/sref.20100825/15/pgrb_biasc/sref_eta.t15z.pgrb212.n1.grib2 !grib2/ncep/NAM_84/#000/201008251500F084/HGHT/700 hPa PRES! 000412

  5. CONDUIT Data Analysis (Cont.) • Accumulated CONDUIT product size per minute (Date: 2010-10-18)

  6. CONDUIT Data Analysis (Cont.) • # of received CONDUIT products per minute (Date: 2010-10-18)

  7. CONDUIT Data Analysis (Cont.) • Histogram of CONDUIT product sizes

  8. Do We Need Circuit Service? • Peak transfer size for CONDUIT: ~250MB/minute, or an equivalent ~33Mbps bandwidth consumption • Minimum bandwidth allocated for a circuit by backbone circuit service providers (such as Internet2 and NLR): ~50-100Mbps • Do we really need virtual circuit services for CONDUIT? • Our answer: YES! And we propose to use static circuit service for CONDUIT.

  9. Why Circuit Service: Reason 1 • An alternative transfer pattern on circuits • On a circuit, the sender host can hold new data products for a certain period (e.g. 1 minute) and then send them in a burst utilizing the full circuit bandwidth • Potential question: How timely does the downstream LDM processes (the users) need to receive the data products?

  10. Why Circuit Service: Reason 2 • The Bandwidth-Hop-Duration Product of Network Flows Heavy-hitter Flows Bandwidth X Normal Flows Fig. 1 Normal Flows vs. Heavy-hitter Flows It is our hypothesis that under heavy IP traffic conditions, the heavy-hitter flows may have negative impacts on the overall delays in packet routing performances

  11. OWAMP Delay Measurements • One-way Active Measurement Protocol(OWAMP) • Packet interval: 0.1 sec • 10 packets per sec • 600 packets per minute • Use perl programs provided by perfSONAR • Sample columns of the OWAMP data file: • endTime, loss, maxError, max_delay min_delay, sent startTime • one report per minute

  12. Boxplots of min-delay and max-delay outliers min-delay max-delay ELPA-BOIS OWAMP measurements for 24 hours

  13. PerfSONAR OWAMP data analysis • One day’s data from 23:00 Sep 14 to 23:00 Sep 15, 2010 for ELPA to BOIS • minDelay ≈ 15ms • Inter-Quartile Range (IQR) of maxDelay:

  14. Future Plans • Simulation • Background IP traffic plus CONDUIT type flows • Simulate and study the impact of the CONDUIT flows on other flows • LDM Source Code Modification • Modify the RPC/TCP communication module to support virtual circuit service • Implement dual transfer mode with dual network services • Evaluation • Cost/Benefit evaluation & Delay Effects evaluation

More Related