1 / 81

Co-processing with ParaView (HPC Visualization with ParaView )

Co-processing with ParaView (HPC Visualization with ParaView ). Shuaib Arshad. April 2, 2014. Agenda. I n t rodu ct ion t o Para V iew C a t al y s t C a t al y s t f or U s ers C a t al y s t f or D e v elopers. Online H e lp. E m ail li s t : para v ie w @para v ie w . org

elkan
Download Presentation

Co-processing with ParaView (HPC Visualization with ParaView )

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Co-processing with ParaView(HPC Visualization with ParaView) Shuaib Arshad April 2, 2014

  2. Agenda • IntroductiontoParaViewCatalyst • Catalyst forUsers • Catalyst forDevelopers

  3. OnlineHelp • Emaillist:paraview@paraview.org • Doxygen: • – http://www.vtk.org/doc/nightly/html/classes.html • – http://www.paraview.org/ParaView3/Doc/Nightly/html/classes.html • Sphinx: • http://www.paraview.org/ParaView3/Doc/Nightly/www/py-doc/index.html • Websites: • http://www.paraview.org • http://catalyst.paraview.org • Examples: • https://github.com/acbauer/CatalystExampleCode

  4. WhatisCatalyst

  5. WhyInSitu? DOEExascaleInitiativeRoadmap,Architectureand TechnologyWorkshop,SanDiego,December,2009.

  6. Catalyst Approach

  7. Two ways to run Simulation Simulation Separate MPI Catalyst Catalyst Disk Storage Disk Storage Visualization Visualization

  8. CatalystArchitecture

  9. HighLevelView • SimulationDevelopers • Passnecessarysimulation • datato ParaView • Need sufficientknowledge • ofbothcodes • VTKforgridsandfielddata • ParaViewCatalystlibraries • Transparentto simulation • users • Extensible • SimulationUsers • Knowledge of ParaViewas apost-processing/analysis tool • BasicinteractionwithGUI co- processingscriptgenerator plugin • Incrementalknowledge increasetousetheco- processingtoolsfrombasic ParaViewuse • Programmingknowledge canbeusefultoextend the tools

  10. UserPerspective Simulation # Createthereaderandsetthe filename. reader= servermanager.sources.Reader(FileNames=path) view= servermanager.CreateRenderView() repr= servermanager.CreateRepresentation(reader, view) reader.UpdatePipeline() dataInfo =reader.GetDataInformation() pDinfo=dataInfo.GetPointDataInformation() arrayInfo = pDInfo.GetArrayInformation("displacement9") ifarrayInfo: #getthe rangeforthemagnitudeof displacement9 range=arrayInfo.GetComponentRange(-1) lut=servermanager.rendering.PVLookupTable() lut.RGBPoints =[range[0],0.0,0.0,1.0, range[1], 1.0,0.0,0.0] lut.VectorMode="Magnitude" repr.LookupTable=lut repr.ColorArrayName="displacement9" repr.ColorAttributeType="POINT_DATA" Augmented scriptin inputdeck. ScriptExport Catalyst Output Processed Data PolygonalOutput withFieldData Statistics SeriesData RenderedImages

  11. DeveloperPerspective Catalyst Solver function calls function Co-processing calls Adaptor API INITIALIZE() ADDPIPELINE(inpipeline) REQUESTDATADESCRIPTION(intime,out fields) COPROCESS(invtkDataSet) FINALIZE()

  12. DeveloperPerspective 0.1 0.09 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0.5 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 DecimateTime(sec) SliceTime(sec) 0 5000 10000 15000 20000 25000 30000 35000 0 5000 10000 15000 20000 25000 30000 35000 Cores Cores Slice Decimate

  13. ParaViewCatalystfor SimulationUsers

  14. CreatingCatalystOutput • Twomainways: • Createand/ormodifyPythonscripts • ParaViewGUI plugintocreatePythonscripts • Modification withknowledgeofParaView PythonAPI • Developergenerated“canned”scripts • User provides parametersforalready created • Catalyst pipelines • User maynotevenneedtoknowParaView • SeeParaViewCatalystUser’sGuide

  15. CreatePythonScriptsfrom ParaView • InteractwithParaViewnormally • Exportascriptthatmimics thatinteraction • Queries duringeachco-processingstep • –(oneframeata time)

  16. ParaViewGUIPlugin • SimilartousingParaViewinteractively • Setupdesiredpipelines • Ideally,startwitharepresentativedatasetfromthe simulation • Extrapipelineinformationtotellwhattooutput duringsimulationrun • Addindataextractwriters • Createscreenshotstooutput • Bothrequirefilenameandwritefrequency

  17. InSituDemo • CreateaParaViewCatalyst Pythonpipeline • script • Specifydesiredoutputsfromrun • Export thescript • Runthescriptwithafictitiousinput • Timedependentgridandfielddatacomefromafile • insteadoffromanactualsimulation • Examineresults

  18. InSituDemo–BuildStep

  19. InSituDemo–LoadPluginStep

  20. InSituDemo–NewPluginMenus

  21. In Situ Demo–CreatingaCatalyst PythonScript • Loadcan.ex2 • – Notethatthereare44timesteps • Createdesiredpipeline

  22. InSituDemo–AddinginWriters • Parameters: • FileName–%tgets • replacedwithtimestep • WriteFrequency

  23. InSituDemo–Exportingthe Script

  24. InSituDemo–SelectInputs • Usually only asingleinputbutcanhavemultiple • inputs

  25. InSituDemo–SelectInputs • Eachpipelinesourceisapotentialinput

  26. InSituDemo–MatchUpInputs • Sourcename(e.g.“can.ex2”)needstobe • matchedwithstringkeyinadaptor(e.g.“input”)

  27. In Situ Demo– GeneratinganImage • Parameters/Options: • Livevisualization • RescaletoData • Range(allimages) • Individualimages • ImageType • FileName • WriteFrequency • Magnification • FittoScreen • %tgetsreplaced with timestep

  28. In Situ Demo– GeneratingTwoImages • Parameters/Options: • Livevisualization • RescaletoData • Range(allimages) • Individualimages • ImageType • FileName • WriteFrequency • Magnification • FittoScreen

  29. In Situ Demo–Write OuttheScript • Generatedscriptwilllook • somethinglikethis defDoCoProcessing(datadescription) input=CreateProducer(datadescription "input" ParallelMultiBlockDataSetWriter1=CreateWriter( XMLMultiBlockDataWriter "filename_%t.vtm"1

  30. InSituDemo–RuntheScript • Putcandriver.py andthegeneratedPythonscript • inthesamedirectory • LinuxandMacfromaterminal • <path>/pvpythoncandriver.py<generatedscript> • <path>/can.ex2 • Windowsfromacommandprompt • start“simpleexample” <path>/pvpython.exe • candriver.py <generatedscript> <path>/can.ex2

  31. LiveInSitu AnalysisandVisualization • Everythingbeforethis was“batch” • Presetinformationwithpossible logic • “Beta”functionalityforinteractingwithsimulation • dataduring simulationrun • WhenexportingaPythonscript,select“Live Visualization” • Duringsimulationrunchoosethe“Tools->Connectto • Catalyst” GUI menuitem

  32. LiveInSitu Example • LinuxandMacfromaterminal • <path>/pvpythonlivecoprocessing.py<path>/can.ex2 • Windowsfromacommandprompt • start“simpleexample” <path>/pvpython.exe • livecoprocessing.py<path>/can.ex2 • StartParaViewandselectTools→Connectto • Catalyst • Selectport(22222is default)

  33. LiveInSitu Example • Onlytransferrequested datafromserver (simulationrun)to client • –ExtractSurface1is • alreadygettingextracted • Use onclient tostoptransferringto client • Clickon totransfer toclientfromCatalyst

  34. Gratuitous Catalyst Images

  35. ParaViewCatalystfor Developers

  36. ParaViewPipelineConcept • Fundamentalconceptin • ParaView • Directedacyclic graph specifyinghow toprocess information • Filtersarenodes inthegraph • Performacertainactionona dataset(gridandfields) • Contours,streamlines,fileIO,etc. • Do notmodifyinputdataset • Catalyst executes user • pipelinesatspecifiedtimes

  37. CatalystPipelines • UsergeneratedPythonscriptsfromthe • ParaViewplugin • ExecutedwithvtkCPPythonScriptPipeline • Hard-coded pipelines(“canned”output) • Executedwithaclassthatderives fromvtkCPPipeline

  38. vtkCPPythonScriptPipeline • TypicallyfromParaViewscriptgeneratorplugin • InitializefromapathtoaParaViewCatalyst Pythonscript • –vtkCPPythonScriptPipeline::Initialize(constchar* • fileName) http://www.paraview.org/ParaView3/Doc/Nightly/html/classvtkCPPythonScriptPipeline.html

  39. vtkCPPythonScriptPipeline(2) • Advantages • EasilycreatedthroughParaViewplugin • Output shouldbemoderatelyreadable • Encourageadvanceduserstomodify • CanuseParaView’sPythontraceutilityto seeoptions • View/screenshotsettings canbedifficulttoset • Cameraangle,zoom,lighting,datarepresentations,etc. • Takescareofparallel imagecompositing • Canmodifywithoutrecompiling

  40. vtkCPPythonScriptPipeline(3) • Disadvantages • Slightoverheadcomparedto C++hard-coded pipeline • Roughly 10-5 secondsper timestep • Simulationcode mustbelinkedwithPython • Staticbuildissues • More complextominimizeexecutablesize

  41. Hard-codedPipelines • DerivesfromvtkCPPipeline • GenerallyC++butcouldbePythoncode • MostaredonedirectlycreatingVTKfiltersand connectingthemtogether • Creatingscreenshotsfromrenderingpipelinewith compositingcanbedaunting • PossibletodousingParaViewC++proxies • Lowlevelaccessis morecomplex • Simplersetupforrenderingpipeline • Lessdependencies forcompilingandlinking http://www.paraview.org/ParaView3/Doc/Nightly/html/classvtkCPPipeline.html

  42. Catalyst • Catalyst’sjobis tocreateandexecutepipelines

  43. DataStructures • Simulationhasseparatedatastructuresfrom • VTKdatastructures • Useranadaptortobridgethegap • Trytoreuseexistingmemory • Alsoresponsibleforotherinteractionsbetween • simulationcodeandCatalyst Solver Adaptor

  44. InformationFlow Solver Adaptor • Initialization • –Informationforcreatingpipelines

  45. InformationFlow Pipeline Pipeline Solver Adaptor Pipeline • Aftersimulationcompletestimestepupdate • Time,timestep,forceoutputflag • Informationforcreatinggridandfieldinformation

  46. InformationFlow Pipeline Pipeline Solver Adaptor Pipeline • Aftersimulationcompletestimestepupdate • –Time,timestep,forceoutputflagpassedtoeach • pipeline

  47. InformationFlow Pipeline Pipeline Solver Adaptor Pipeline • Aftersimulationcompletestimestepupdate • –Flagindicatingwhichpipelines needtobe • executed/updated

  48. InformationFlow Pipeline Pipeline Solver Adaptor Pipeline • Aftersimulationcompletestimestepupdate • Ifanypipelineneedstobeexecuted • Adaptorcreates VTKobjectsto representgridsandfields

  49. InformationFlow Pipeline Pipeline Solver Adaptor Pipeline • Aftersimulationcompletestimestepupdate • –PassVTKdataobject representinggridsandfields • topipelinesthatneedtoexecute/update

  50. InformationFlow Pipeline Pipeline Solver Adaptor Pipeline • Aftersimulationcompletestimestepupdate • –Pipelinesexecute andoutputdesiredinformation

More Related