Download
serpm 6 and cube cluster n.
Skip this Video
Loading SlideShow in 5 Seconds..
SERPM-6 and Cube Cluster PowerPoint Presentation
Download Presentation
SERPM-6 and Cube Cluster

SERPM-6 and Cube Cluster

124 Views Download Presentation
Download Presentation

SERPM-6 and Cube Cluster

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. SERPM-6 and Cube Cluster Corradino’s Initial Experience The Corradino Group SE Florida Model Users Group

  2. Why Cluster? • Reduce running time for long model runs. • HIGHLY DEPENDENT ON YOUR HARDWARE! • SERPM-6 2000 Base year (3.2 GH Pentium) = 22 hours. • With 3 cores = 14 hours (overnight). • Much greater savings with more cores and Core 2 Duo hardware (we don’t have these yet). • For example, ODOT uses 32 cores. The Corradino Group SE Florida Model Users Group

  3. Issues • Networking and Security – the hard part (what Citilabs doesn’t tell you much about). • Modifying your setup to use Cluster – the easy part. • After modification the process will run on a single processor, so the ability to distribute the model setup is not affected. The Corradino Group SE Florida Model Users Group

  4. Computer Network Issues • All processors must see the model folder exactly the same way, even the host processor. • All processors must have the appropriate license. • All processors must have the appropriate rights. The Corradino Group SE Florida Model Users Group

  5. Licenses • Host must have a Cube Voyager Cluster license. • Nodes must have either: • Cube Voyager license • Node license (see your Citilabs representative) The Corradino Group SE Florida Model Users Group

  6. Where’s the model? • Map to a common drive – we like z: • Everybody, including the host computer, must map to z: • Everybody must have rights. • Share the folder. • On EVERY computer map to z: • Give every USER rights to READ and WRITE. The Corradino Group SE Florida Model Users Group

  7. How to share:(You must be an administrator) The Corradino Group SE Florida Model Users Group

  8. Add Users and Permissions • Usually the login name (who the network thinks you are) • We also had to do it for the “Host” – the user that is actually running Cube. • Required READ and WRITE permissions. Also the Security tab. Get help from your IT expert. • Test by trying to copy a file from the node location to the mapped Z: If this doesn’t work, Cluster won’t work. The Corradino Group SE Florida Model Users Group

  9. How to Map – everybody! The Corradino Group SE Florida Model Users Group

  10. Then start the nodes (Cluster) • Must use a consistent folder and process ID, Like • Z:\cube\SERPM6ID – this a folder, like cube, on the mapped drive z:, with some identifier SERPM6ID. • The identifier must be the same one you use in the Voyager script. The Corradino Group SE Florida Model Users Group

  11. Nodes (here are 2 remote nodes) The Corradino Group SE Florida Model Users Group

  12. How Ken Does it. • Automate with a Visual Basic Script • Map drives and folders • Establish the ProcessID • Starts the nodes (Cluster) The Corradino Group SE Florida Model Users Group

  13. Process ID • In the following slides note the process ID = SERPM6ID • This MUST be the same as in the ProcessID used when you start Cluster. The Corradino Group SE Florida Model Users Group

  14. VB Script This script maps a drive and then starts a CUBE Cluster 'KDK 2-6-2007 'The Corradino Group '****** INPUT VARIABLES *********************************** 'Specify what folder to map -- this must the shared folder visible to the network where the model catalog resides model_folder="\\Tcg-lvl-0218\S6-HOT" 'Specify drive to map to. Use Z: unless at sometime Z: is not available (very unusual) drive="Z:" 'Specify process ID. This must be a subfolder of the mapped folder and then an ID name, like \CUBE\SERPM6ID processid="\cube\SERPM6ID" 'Specify process numbers (list or range) ; for example 2-3 starts processes 2 and 3, 4-5,7 starts 4 and 5 and 7 nums="2-3" '*******end input variables************************************** ' DO NOT TOUCH THE FOLLOWING LINES IF YOU DO NOT KNOW WHAT YOU ARE DOING 'Create the shell for the process Set objShell = CreateObject("Wscript.Shell") 'Ignore the errors if drive not mapped or already mapped On Error Resume Next 'Map the drive Set objNetwork = CreateObject("Wscript.Network") objNetwork.RemoveNetworkDrive drive objNetwork.MapNetworkDrive drive, model_folder 'WScript.Echo "Drive Mapped!" On Error GoTo 0 'Start the CLUSTER program myargs=drive & processid & " " & nums & " Start Exit" CLUSTER="""C:\program files\citilabs\CUBE\CLUSTER""" & " " & myargs 'WScript.Echo CLUSTER objShell.Run CLUSTER The Corradino Group SE Florida Model Users Group

  15. Script Inputs (only 4)(always run from the remote nodes) • The model folder to be mapped to z: model_folder="\\Tcg-lvl-0218\S6-HOT" • The name of the mapped drive drive="Z:" • The ProcessID (also in the Voyager script) processid="\cube\SERPM6ID" • The remote nodes (on the remote machine) nums="2-3" The Corradino Group SE Florida Model Users Group

  16. Interstep - Intrastep • Interstep – You must program the logic. Can be used with any process where the logic fits. • Intrastep – Voyager takes care of the logic, but available only for HIGHWAY and MATRIX. The Corradino Group SE Florida Model Users Group

  17. Insert Control Boxes • Cluster is an option on the Control menu. • Insert a Start. • Insert an End. • Positions determined by execution order like all items in a flowchart. The Corradino Group SE Florida Model Users Group

  18. Interstep (multiple assignments) The Corradino Group SE Florida Model Users Group

  19. Pilot Script to “Wait” ; Do not change filenames or add or remove FILEI/FILEO statements using an editor. Use Cube/Application Manager. Wait4Files Files= SERPM6ID2.script.end, SERPM6ID3.script.end, CheckReturnCode=T, PrintFiles=MERGE, DelDistribFiles=F The Corradino Group SE Florida Model Users Group

  20. Intrastep ; Do not change filenames or add or remove FILEI/FILEO statements using an editor. Use Cube/Application Manager. RUN PGM=MATRIX PRNFILE="E:\S6-HOT\CUBE\ODMAT00B.PRN" MSG='Assemble 1+ Veh OPTrips & Create Temp File for Final Mode Choice' DISTRIBUTEINTRASTEP ProcessID='SERPM6ID',ProcessList=1-4 FILEI MATI[2] = "{OUTDIR}\PSCHOOL_{ALT}{YEAR}.MAT" The Corradino Group SE Florida Model Users Group

  21. Intrastep Programs • Highway • Matrix • NOT – Distribution (would be valuable) The Corradino Group SE Florida Model Users Group

  22. CONCLUSION • Great way to speed up the model runs • We like it, but • Should be easier. • We know a product that uses whatever processors it can find in the host machine and requires not setup. Can’t use remote machines. The Corradino Group SE Florida Model Users Group