1 / 11

Project Organization

Project Organization. Organize the directory structure for the project at the beginning if possible Use consistent file extensions: .scr C shell script .c source code for C program etc. Name executables with the same name as the source code so you can find the source code later.

hubert
Download Presentation

Project Organization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Project Organization • Organize the directory structure for the project at the beginning if possible • Use consistent file extensions: .scr C shell script .c source code for C program etc. • Name executables with the same name as the source code so you can find the source code later. • Back up programs to the local hard disk in mirror directory or create them on this disk originally. You should generally be able to reproduce everything on the raid drive from what you have archived on the local disk.

  2. CPU Raid Drive Local Disk Redundant but NO BACKUP!! Tape Backup

  3. Backup Script /raid/hamleaf/backup/backup.scr # #This script backs up whole directories or specific subdirectories or file types #on the raid disk to mirror directories on the local hard disk that is backed up on tape. #back up whole directories including all subdirectories and files therein cd /raid/hamleaf/ foreach dirname (plot data_processing) echo backing up directory called $dirname cp -R $dirname /usr1/hamleaf/backup/$dirname end #back up all files in specific subdirectories (note target directory must exist) foreach subname (vic_forecast_0.25_b vic_forecast_0.25_b/SRC vic_forecast_0.25_b/INPT vic_forecast_0.25_b/rout ) echo backing up subdirectory called $subname cp /raid/hamleaf/"$subname"/* /usr1/hamleaf/backup/"$subname"/ end

  4. Raw data Process 1 Process 2 Process 3 product

  5. Why Integrate a Project in a Master Script • Helps organize the project by creating a step by step process and creating intermediary products that can be checked to isolate the source of problems. • Redo the entire project or pieces thereof in a single command • Keep a detailed record of programs used, locations of data, and the exact sequence of data processing steps so that you (or someone else) can come back later and figure out what you did. • Update specific components of a project without altering other components

  6. Monte Carlo Experiment for CE 574 /raid/hamleaf/cee574/numgen/mc_res_store.scr # # Set basic parameters set outf = quantiles_cedar_dem set cl = 40 set ss = 1000 set ncl = 140000 set phi = 0.97 set theta = 0.85 set sigma_e = 0.896706 set numseason = 3

  7. foreach demscenario (1 2 3) #demand scenarios if($demscenario == 1) then set dem1 = 33024 set dem2 = 22016 set dem3 = 55040 endif if($demscenario == 2) then set dem1 = 55040 set dem2 = 36693 set dem3 = 91734 endif if($demscenario == 3) then set dem1 = 77057 set dem2 = 51371 set dem3 = 128428 endif

  8. #generate random numbers rannumgen 1 0 $sigma_e 0 $ncl >! temp1 rannumgen 1 0 1 0 $ncl >! temp2 #generate ARMA11 numbers, seasonally disaggregate, and perform inverse ln3 transform mc_arma11 $cl $ss $phi $theta $sigma_e temp1 temp2 >! temp3

  9. #write out individual timeseries awk '{if(NR%3==1) print $1;}' temp3 >! arma11_time_series awk '{if(NR%3==1) print $2;}' temp3 >! cedar_annual_time_series awk '{if(NR%3==1) print $3;}' temp3 >! cedar_s1_time_series awk '{if(NR%3==2) print $3;}' temp3 >! cedar_s2_time_series awk '{if(NR%3==0) print $3;}' temp3 >! cedar_s3_time_series #perform storage analysis on the ensemble awk '{print $3}' temp3 >! temp3b mc_res_store $cl $ss $numseason temp3b $dem1 $dem2 $dem3 >! temp4

  10. sort -n -r temp4 >! temp5 echo $ss >! temp6 cat temp6 temp5 >! temp7 #quantile estimator from handbook of hydrology pg 18.25 Cunnane estimator awk '{if(NR==1){ n=$1;} if(NR>1) {q=(NR-1-0.4)/(n+0.2); print q, $1;} }' temp7 > temp8 mv temp8 $outf"$demscenario" echo Demand Scenario = $demscenario echo "" awk '{if($1==0.0205959) print "98th percentile storage estimate = " $2;}' $outf"$demscenario" end

  11. Output: graph:/usr1/hamleaf/backup/cee574/numgen> mc_res_store.scr Demand Scenario = 1 98th percentile storage estimate = 59611 Demand Scenario = 2 98th percentile storage estimate = 173111 Demand Scenario = 3 98th percentile storage estimate = 406187

More Related