Serpm 6 and cube cluster
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

SERPM-6 and Cube Cluster PowerPoint PPT Presentation


  • 75 Views
  • Uploaded on
  • Presentation posted in: General

SERPM-6 and Cube Cluster. Corradino’s Initial Experience. Why Cluster?. Reduce running time for long model runs. HIGHLY DEPENDENT ON YOUR HARDWARE! SERPM-6 2000 Base year (3.2 GH Pentium) = 22 hours. With 3 cores = 14 hours (overnight).

Download Presentation

SERPM-6 and Cube Cluster

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Serpm 6 and cube cluster

SERPM-6 and Cube Cluster

Corradino’s Initial Experience

The Corradino Group

SE Florida Model Users Group


Why cluster

Why Cluster?

  • Reduce running time for long model runs.

  • HIGHLY DEPENDENT ON YOUR HARDWARE!

  • SERPM-6 2000 Base year (3.2 GH Pentium) = 22 hours.

  • With 3 cores = 14 hours (overnight).

  • Much greater savings with more cores and Core 2 Duo hardware (we don’t have these yet).

  • For example, ODOT uses 32 cores.

The Corradino Group

SE Florida Model Users Group


Issues

Issues

  • Networking and Security – the hard part (what Citilabs doesn’t tell you much about).

  • Modifying your setup to use Cluster – the easy part.

  • After modification the process will run on a single processor, so the ability to distribute the model setup is not affected.

The Corradino Group

SE Florida Model Users Group


Computer network issues

Computer Network Issues

  • All processors must see the model folder exactly the same way, even the host processor.

  • All processors must have the appropriate license.

  • All processors must have the appropriate rights.

The Corradino Group

SE Florida Model Users Group


Licenses

Licenses

  • Host must have a Cube Voyager Cluster license.

  • Nodes must have either:

    • Cube Voyager license

    • Node license

      (see your Citilabs representative)

The Corradino Group

SE Florida Model Users Group


Where s the model

Where’s the model?

  • Map to a common drive – we like z:

  • Everybody, including the host computer, must map to z:

  • Everybody must have rights.

  • Share the folder.

  • On EVERY computer map to z:

  • Give every USER rights to READ and WRITE.

The Corradino Group

SE Florida Model Users Group


How to share you must be an administrator

How to share:(You must be an administrator)

The Corradino Group

SE Florida Model Users Group


Add users and permissions

Add Users and Permissions

  • Usually the login name (who the network thinks you are)

  • We also had to do it for the “Host” – the user that is actually running Cube.

  • Required READ and WRITE permissions. Also the Security tab. Get help from your IT expert.

  • Test by trying to copy a file from the node location to the mapped Z: If this doesn’t work, Cluster won’t work.

The Corradino Group

SE Florida Model Users Group


How to map everybody

How to Map – everybody!

The Corradino Group

SE Florida Model Users Group


Then start the nodes cluster

Then start the nodes (Cluster)

  • Must use a consistent folder and process ID, Like

  • Z:\cube\SERPM6ID – this a folder, like cube, on the mapped drive z:, with some identifier SERPM6ID.

  • The identifier must be the same one you use in the Voyager script.

The Corradino Group

SE Florida Model Users Group


Nodes here are 2 remote nodes

Nodes (here are 2 remote nodes)

The Corradino Group

SE Florida Model Users Group


How ken does it

How Ken Does it.

  • Automate with a Visual Basic Script

  • Map drives and folders

  • Establish the ProcessID

  • Starts the nodes (Cluster)

The Corradino Group

SE Florida Model Users Group


Process id

Process ID

  • In the following slides note the process ID = SERPM6ID

  • This MUST be the same as in the ProcessID used when you start Cluster.

The Corradino Group

SE Florida Model Users Group


Vb script

VB Script

This script maps a drive and then starts a CUBE Cluster

'KDK 2-6-2007

'The Corradino Group

'****** INPUT VARIABLES ***********************************

'Specify what folder to map -- this must the shared folder visible to the network where the model catalog resides

model_folder="\\Tcg-lvl-0218\S6-HOT"

'Specify drive to map to. Use Z: unless at sometime Z: is not available (very unusual)

drive="Z:"

'Specify process ID. This must be a subfolder of the mapped folder and then an ID name, like \CUBE\SERPM6ID

processid="\cube\SERPM6ID"

'Specify process numbers (list or range) ; for example 2-3 starts processes 2 and 3, 4-5,7 starts 4 and 5 and 7

nums="2-3"

'*******end input variables**************************************

' DO NOT TOUCH THE FOLLOWING LINES IF YOU DO NOT KNOW WHAT YOU ARE DOING

'Create the shell for the process

Set objShell = CreateObject("Wscript.Shell")

'Ignore the errors if drive not mapped or already mapped

On Error Resume Next

'Map the drive

Set objNetwork = CreateObject("Wscript.Network")

objNetwork.RemoveNetworkDrive drive

objNetwork.MapNetworkDrive drive, model_folder

'WScript.Echo "Drive Mapped!"

On Error GoTo 0

'Start the CLUSTER program

myargs=drive & processid & " " & nums & " Start Exit"

CLUSTER="""C:\program files\citilabs\CUBE\CLUSTER""" & " " & myargs

'WScript.Echo CLUSTER

objShell.Run CLUSTER

The Corradino Group

SE Florida Model Users Group


Script inputs only 4 always run from the remote nodes

Script Inputs (only 4)(always run from the remote nodes)

  • The model folder to be mapped to z:

    model_folder="\\Tcg-lvl-0218\S6-HOT"

  • The name of the mapped drive

    drive="Z:"

  • The ProcessID (also in the Voyager script)

    processid="\cube\SERPM6ID"

  • The remote nodes (on the remote machine)

    nums="2-3"

The Corradino Group

SE Florida Model Users Group


Interstep intrastep

Interstep - Intrastep

  • Interstep – You must program the logic. Can be used with any process where the logic fits.

  • Intrastep – Voyager takes care of the logic, but available only for HIGHWAY and MATRIX.

The Corradino Group

SE Florida Model Users Group


Insert control boxes

Insert Control Boxes

  • Cluster is an option on the Control menu.

  • Insert a Start.

  • Insert an End.

  • Positions determined by execution order like all items in a flowchart.

The Corradino Group

SE Florida Model Users Group


Interstep multiple assignments

Interstep (multiple assignments)

The Corradino Group

SE Florida Model Users Group


Pilot script to wait

Pilot Script to “Wait”

; Do not change filenames or add or remove FILEI/FILEO statements using an editor. Use Cube/Application Manager.

Wait4Files Files= SERPM6ID2.script.end, SERPM6ID3.script.end, CheckReturnCode=T, PrintFiles=MERGE, DelDistribFiles=F

The Corradino Group

SE Florida Model Users Group


Intrastep

Intrastep

; Do not change filenames or add or remove FILEI/FILEO statements using an editor. Use Cube/Application Manager.

RUN PGM=MATRIX PRNFILE="E:\S6-HOT\CUBE\ODMAT00B.PRN" MSG='Assemble 1+ Veh OPTrips & Create Temp File for Final Mode Choice'

DISTRIBUTEINTRASTEP ProcessID='SERPM6ID',ProcessList=1-4

FILEI MATI[2] = "{OUTDIR}\PSCHOOL_{ALT}{YEAR}.MAT"

The Corradino Group

SE Florida Model Users Group


Intrastep programs

Intrastep Programs

  • Highway

  • Matrix

  • NOT – Distribution (would be valuable)

The Corradino Group

SE Florida Model Users Group


Conclusion

CONCLUSION

  • Great way to speed up the model runs

  • We like it, but

  • Should be easier.

  • We know a product that uses whatever processors it can find in the host machine and requires not setup. Can’t use remote machines.

The Corradino Group

SE Florida Model Users Group


  • Login