1 / 31

High Throughput Computing in Your Backyard: Urban Hydrology Applications

Explore how urbanization affects water flows and learn how homeowner decisions impact urban hydrology. Discover interactions between practices, climate, and neighbor actions, and study models processing on HTC. Delve into hydrologic flux analysis through HTC to post-process on HPC, optimizing resource use and modeling scenarios for urban hydrology applications.

xylia
Download Presentation

High Throughput Computing in Your Backyard: Urban Hydrology Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. High Throughput Computing in Your Backyard: Urban Hydrology Applications OSG School, July 19, 2019 Carolyn Voter, University of Wisconsin-Madison cvoter@wisc.edu carolynbvoter.weebly.com @VoteWater

  2. Urbanization messes with how water flows NATURAL URBAN Rain Rain Evapotranspiration Evapotranspiration Runoff Runoff Infiltration Infiltration

  3. Urbanization messes with how water flows Last Millennium's Approach Today’s Approach

  4. Homeowner decisions affect urban hydrology Downspout Soil Condition vs. Driveway Microtopography Sidewalk

  5. Homeowner decisions affect urban hydrology How do these practices interact with one another?

  6. Homeowner decisions affect urban hydrology How do these practices interact with climate?

  7. Homeowner decisions affect urban hydrology How do these practices interact with climate?

  8. Homeowner decisions affect urban hydrology How do your actions combine with your neighbors’ actions to impact larger scale hydrology?

  9. Basic Lot Layout = 0.5m x 0.5m grid cell (to scale) House Garage Front Walk Driveway Front Walk Driveway Sidewalk

  10. Why these models take forever* *To me time Image: Staudinger et al., 2019 https://doi.org/10.1016/j.jhydrol.2019.01.058

  11. Why these models take forever* *To me Image: parflow.org

  12. Why I started on HTC Model too big for desktop  HPC Postprocessing too small for HPC  Postprocess on HTC Do everything on HTC!

  13. HTCondor Workflow submit server gluster file server SPLICE DAGman files fluxA fluxB out01 out03 out04 input fluxD out02 fluxC fluxA fluxA fluxC out03 out04 input input out02 fluxB input out01 fluxB fluxA fluxC fluxB DAGman convertInputs.sub *.sh, *.tcl convertInputs.sub *.sh, *.tcl compute resources runParflow.sub *.sh, *.tcl runParflow.sub *.sh, *.tcl rearrangeOutputs.sub *.sh rearrangeOutputs.sub *.sh • 6-10 models running at any given time • Nearly all take >3 days • Use “+is_resumable” • Send outputs back regularly 1 2 3 1 1 2 2 3 3 4 4 • Inputs need to be converted to parflow binary format and repackaged • Outputs saved by (model) time, I need to process them by hydrologic flux • Convert to Matlab files for analysis • Derive additional fluxes of interest, check water (mass) balance convertOutputs.sub *.sh, *.m convertOutputs.sub *.sh, *.mat 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 calculateOutputs *.sh, *.m calculateOutputs *.sh, *.mat 13 13 14 14 15 15 16 16 18 18 19 19 20 20 17 17

  14. Key wins along the way Figuring out what “compiling” means Using DAG and ultimately DAG splice Sending back output regularly, but not too frequently Custom script to extract current status of all models

  15. Current Pinch Points Limited to Gluster machines So much data, no (cheap) place for it to go As my models get bigger, back to hybrid HPC/HTC workflow

  16. Urban Hydrology Applications 1 7 scenarios x 20 processors each x 5-ish days each More resources + time than I ever imagined spending on hydrologic modeling

  17. Urban Hydrology Applications 1 Baseline Baseline Compaction = ? Lowest-Impact

  18. Urban Hydrology Applications 1 Baseline Baseline Baseline Highly Compacted Moderately Compacted Ksat ↓ 10x Ksat ↓ 2x Weather? Lowest-Impact Lowest-Impact Lowest-Impact

  19. Urban Hydrology Applications 1 Baseline Baseline Highly Compacted Moderately Compacted Ksat ↓ 10x Ksat ↓ 2x Average Lowest-Impact Lowest-Impact

  20. Urban Hydrology Applications 1 Baseline Baseline Highly Compacted Moderately Compacted Ksat ↓ 10x Ksat ↓ 2x Dry Lowest-Impact Lowest-Impact

  21. Urban Hydrology Applications 1 Highly Compacted Moderately Compacted Ksat ↓ 10x Ksat ↓ 2x 96 simulations

  22. Urban Hydrology Applications 2 WET DRY

  23. Urban Hydrology Applications 2 X 2 Lot Types 50 Largest U.S. Cities (plus Madison) Hourly weather data WY2014 Baseline 102 simulations ET0:P Low-Impact arid humid

  24. Urban Hydrology Applications 2 Reduction in runoff Estimates for WY1981 – WY2010

  25. Urban Hydrology Applications 3 Compaction ?

  26. Urban Hydrology Applications 3 99 simulations

  27. Urban Hydrology Applications 3

  28. Urban Hydrology Applications 4

  29. Urban Hydrology Applications 4 102 simulations

  30. Urban Hydrology Applications 3

  31. Thank You! Funding UW Water Resources Institute, WR12R002 UW Sea Grant Institute, RCE-05 LTER-NTL Others Hydroecology Lab UW Center for High Throughput Computing Carolyn Voter, Ph.D. Candidate, Civil & Environmental Engineering, UW-Madison cvoter@wisc.edu carolynbvoter.weebly.com @VoteWater Questions?

More Related