1 / 22

SERPM-6 and Cube Cluster

SERPM-6 and Cube Cluster. Corradino’s Initial Experience. Why Cluster?. Reduce running time for long model runs. HIGHLY DEPENDENT ON YOUR HARDWARE! SERPM-6 2000 Base year (3.2 GH Pentium) = 22 hours. With 3 cores = 14 hours (overnight).

emmett
Download Presentation

SERPM-6 and Cube Cluster

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SERPM-6 and Cube Cluster Corradino’s Initial Experience The Corradino Group SE Florida Model Users Group

  2. Why Cluster? • Reduce running time for long model runs. • HIGHLY DEPENDENT ON YOUR HARDWARE! • SERPM-6 2000 Base year (3.2 GH Pentium) = 22 hours. • With 3 cores = 14 hours (overnight). • Much greater savings with more cores and Core 2 Duo hardware (we don’t have these yet). • For example, ODOT uses 32 cores. The Corradino Group SE Florida Model Users Group

  3. Issues • Networking and Security – the hard part (what Citilabs doesn’t tell you much about). • Modifying your setup to use Cluster – the easy part. • After modification the process will run on a single processor, so the ability to distribute the model setup is not affected. The Corradino Group SE Florida Model Users Group

  4. Computer Network Issues • All processors must see the model folder exactly the same way, even the host processor. • All processors must have the appropriate license. • All processors must have the appropriate rights. The Corradino Group SE Florida Model Users Group

  5. Licenses • Host must have a Cube Voyager Cluster license. • Nodes must have either: • Cube Voyager license • Node license (see your Citilabs representative) The Corradino Group SE Florida Model Users Group

  6. Where’s the model? • Map to a common drive – we like z: • Everybody, including the host computer, must map to z: • Everybody must have rights. • Share the folder. • On EVERY computer map to z: • Give every USER rights to READ and WRITE. The Corradino Group SE Florida Model Users Group

  7. How to share:(You must be an administrator) The Corradino Group SE Florida Model Users Group

  8. Add Users and Permissions • Usually the login name (who the network thinks you are) • We also had to do it for the “Host” – the user that is actually running Cube. • Required READ and WRITE permissions. Also the Security tab. Get help from your IT expert. • Test by trying to copy a file from the node location to the mapped Z: If this doesn’t work, Cluster won’t work. The Corradino Group SE Florida Model Users Group

  9. How to Map – everybody! The Corradino Group SE Florida Model Users Group

  10. Then start the nodes (Cluster) • Must use a consistent folder and process ID, Like • Z:\cube\SERPM6ID – this a folder, like cube, on the mapped drive z:, with some identifier SERPM6ID. • The identifier must be the same one you use in the Voyager script. The Corradino Group SE Florida Model Users Group

  11. Nodes (here are 2 remote nodes) The Corradino Group SE Florida Model Users Group

  12. How Ken Does it. • Automate with a Visual Basic Script • Map drives and folders • Establish the ProcessID • Starts the nodes (Cluster) The Corradino Group SE Florida Model Users Group

  13. Process ID • In the following slides note the process ID = SERPM6ID • This MUST be the same as in the ProcessID used when you start Cluster. The Corradino Group SE Florida Model Users Group

  14. VB Script This script maps a drive and then starts a CUBE Cluster 'KDK 2-6-2007 'The Corradino Group '****** INPUT VARIABLES *********************************** 'Specify what folder to map -- this must the shared folder visible to the network where the model catalog resides model_folder="\\Tcg-lvl-0218\S6-HOT" 'Specify drive to map to. Use Z: unless at sometime Z: is not available (very unusual) drive="Z:" 'Specify process ID. This must be a subfolder of the mapped folder and then an ID name, like \CUBE\SERPM6ID processid="\cube\SERPM6ID" 'Specify process numbers (list or range) ; for example 2-3 starts processes 2 and 3, 4-5,7 starts 4 and 5 and 7 nums="2-3" '*******end input variables************************************** ' DO NOT TOUCH THE FOLLOWING LINES IF YOU DO NOT KNOW WHAT YOU ARE DOING 'Create the shell for the process Set objShell = CreateObject("Wscript.Shell") 'Ignore the errors if drive not mapped or already mapped On Error Resume Next 'Map the drive Set objNetwork = CreateObject("Wscript.Network") objNetwork.RemoveNetworkDrive drive objNetwork.MapNetworkDrive drive, model_folder 'WScript.Echo "Drive Mapped!" On Error GoTo 0 'Start the CLUSTER program myargs=drive & processid & " " & nums & " Start Exit" CLUSTER="""C:\program files\citilabs\CUBE\CLUSTER""" & " " & myargs 'WScript.Echo CLUSTER objShell.Run CLUSTER The Corradino Group SE Florida Model Users Group

  15. Script Inputs (only 4)(always run from the remote nodes) • The model folder to be mapped to z: model_folder="\\Tcg-lvl-0218\S6-HOT" • The name of the mapped drive drive="Z:" • The ProcessID (also in the Voyager script) processid="\cube\SERPM6ID" • The remote nodes (on the remote machine) nums="2-3" The Corradino Group SE Florida Model Users Group

  16. Interstep - Intrastep • Interstep – You must program the logic. Can be used with any process where the logic fits. • Intrastep – Voyager takes care of the logic, but available only for HIGHWAY and MATRIX. The Corradino Group SE Florida Model Users Group

  17. Insert Control Boxes • Cluster is an option on the Control menu. • Insert a Start. • Insert an End. • Positions determined by execution order like all items in a flowchart. The Corradino Group SE Florida Model Users Group

  18. Interstep (multiple assignments) The Corradino Group SE Florida Model Users Group

  19. Pilot Script to “Wait” ; Do not change filenames or add or remove FILEI/FILEO statements using an editor. Use Cube/Application Manager. Wait4Files Files= SERPM6ID2.script.end, SERPM6ID3.script.end, CheckReturnCode=T, PrintFiles=MERGE, DelDistribFiles=F The Corradino Group SE Florida Model Users Group

  20. Intrastep ; Do not change filenames or add or remove FILEI/FILEO statements using an editor. Use Cube/Application Manager. RUN PGM=MATRIX PRNFILE="E:\S6-HOT\CUBE\ODMAT00B.PRN" MSG='Assemble 1+ Veh OPTrips & Create Temp File for Final Mode Choice' DISTRIBUTEINTRASTEP ProcessID='SERPM6ID',ProcessList=1-4 FILEI MATI[2] = "{OUTDIR}\PSCHOOL_{ALT}{YEAR}.MAT" The Corradino Group SE Florida Model Users Group

  21. Intrastep Programs • Highway • Matrix • NOT – Distribution (would be valuable) The Corradino Group SE Florida Model Users Group

  22. CONCLUSION • Great way to speed up the model runs • We like it, but • Should be easier. • We know a product that uses whatever processors it can find in the host machine and requires not setup. Can’t use remote machines. The Corradino Group SE Florida Model Users Group

More Related