1 / 7

Creating an HPC application framework from a mere HPC application automatically

This presentation discusses the challenge of creating a high-performance computing (HPC) application framework from an existing application, particularly how application developers often overlook the importance of framework development. The concept of the "Cactus" phenomenon illustrates how domain-specific frameworks struggle to expand beyond their original intent. The process covered includes utilizing tools like Onramp and Appgen to identify components and methods, decompose the application, and generate a componentized, pluggable framework that maintains the original application's functionality while enhancing collaboration and expandability.

dalit
Download Presentation

Creating an HPC application framework from a mere HPC application automatically

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rob Armstrong SNL Creating an HPC application framework from a mere HPC application automatically

  2. The problem is that application developers see themselves at the center of the universe... ... and not framework developers as it rightly should be... • Explains the “Cactus” phenomenon: • domain-specific, special purpose framework that has a home-grown audience. • framework is adapted from a successful application in that domain. • has a “big event loop” architecture that is less flexible. • aside: why is this? • has difficulty expanding outside its original domain. • CCA has grown the ability to do this. • Onramp->Bocca->Ccaffeine->CCASpec->Babel

  3. Autogen'ing an HPC framework from an existing monolithic application /* CCA->COMP(Test)->BEGIN CCA->PORT(Hello)->BEGIN */ int Hello1(int x, char *arg)‏ { int i; i=2+3; return i; } //CCA->ARG(2,out)‏ int Hello2(int x, char *arg)‏ { int i; i=2+3; return i; } • Identify components and methods with Onramp. • The hard work is still there: • how is this to be decomposed into components? • what is the execution graph for these components? • Does it make sense? • Iterate

  4. Re-create the application using existing tools... • Use Appgen to create a main() that marshals the components. • Componentized framework does exactly what the original did... • but now is a plug-able framework • presumably more amenable to collaboration • is full-up CCA compliant, has a generally expandable architecture. • Resulting framework is, and can remain, an automated derivative of the original • A framework is generated as part of the build process for the original application.

  5. Workflow for “automatic” framework generation... CmdLineBuilderViewForHuman::~CmdLineBuilderViewForHuman() { out = NULL; bm = NULL; } /** Implements ComponentChangedListener. Signal a change in the Component's status. */ void CmdLineBuilderViewForHuman::componentChanged(ComponentChangedEvent* evt) { if(out != 0) { fprintf(out, "revalidate \"%s\"\n", evt->getComponentInstance()); } } void CmdLineBuilderViewForHuman::setOutputStream(FILE *out_) { out = out_; } void CmdLineBuilderViewForHuman::setBuilderModel(BuilderModel *bm_) { bm = bm_; bm->addComponentChangedListener(this); } void CmdLineBuilderViewForHuman::displayPallet() { if(bm == 0) { pn("!no pallet available"); return; } std::vector< std::string > pal = bm->getPallet(); pn("Components available:"); for(int i = 0;i < (int)pal.size();i++) { pn(pal[i]); } } void CmdLineBuilderViewForHuman::displayInstantiatedComponents() { if (bm == 0) { pn("!No instantiated builder."); return; } ::std::map< ::std::string, ComponentInfo_shared > arena = bm->getArena(); boolean shown = false; ::std::map< ::std::string, ComponentInfo_shared >::iterator pos; for ( pos = arena.begin(); pos != arena.end(); pos++) { ComponentInfo_shared ci = pos->second; if (ci == 0) { pn("!corrupted arena!!"); assert(ci!=0); exit(1); } p(ci->getInstanceName()); p(" of type "); pn(ci->getClassName()); shown = true; } if (!shown) { pn("!no instantiated components available"); } } void CmdLineBuilderViewForHuman::displayComponentInfo(const char *instanceName)‏ { ::std::map< ::std::string, ComponentInfo_shared > arena = bm->getArena(); ::std::map< ::std::string, ComponentInfo_shared >::iterator pos; pos = arena.find(instanceName); if (pos == arena.end()) { pn("!displayComponentInfo component instance not found"); return; } ComponentInfo_shared ci = arena[instanceName]; if (ci == 0) { pn("!corrupted arena!!"); assert(ci!=0); exit(1); } pn("------------------------------------"); p("Instance name: "); pn(ci->getInstanceName()); p("Class name: "); pn(ci->getClassName()); pn("------------------------------------"); p("UsesPorts registered for "); pn(ci->getInstanceName()); pn(""); ::std::vector< UserPortData > pi = ci->getUsesPortRegister(); size_t i; if (pi.size() != 0) { char tmp[40]; for(i = 0; i < pi.size(); i++) { sprintf(tmp,"%d. Instance Name: ",int(i)); p(tmp); p(pi[i].getPortName()); p(" Class Name: "); pn(pi[i].getPortType()); } } else { p("!No UsesPorts in "); pn(ci->getInstanceName()); } pn("------------------------------------"); p("ProvidesPorts registered for "); pn(ci->getInstanceName()); pn(""); const ::std::vector< ProviderPortData > ppi = ci->getProvidesPorts(); if (ppi.size() != 0) { for(i = 0; i < ppi.size(); i++) { p("Instance Name: "); p(ppi[i].getPortName()); p(" Class Name: "); pn(ppi[i].getPortType()); } } else { p("!No ProvidesPorts in "); pn(ci->getInstanceName()); } pn("------------------------------------"); } void CmdLineBuilderViewForHuman::pullDownComponent(const char *className, const char *instanceName) { p(instanceName); p(" of type "); p(className); pn(" \nsuccessfully instantiated"); } void CmdLineBuilderViewForHuman::connect(const char *fromInstance, const char *providesInstance, const char *toInstance, const char *usesInstance) { p(toInstance); p("))))"); p(usesInstance); p("---->"); p(providesInstance); p("(((("); pn(fromInstance); pn("connection made successfully"); } void CmdLineBuilderViewForHuman::disconnect( const char *fromInstance, const char *providesInstance, const char *toInstance, const char *usesInstance) { p(toInstance); p("))))"); p(usesInstance); p("-\\ \\-"); p(providesInstance); p("(((("); pn(fromInstance); pn("connection broken successfully"); } provides port Component1 uses port Component2

  6. Needful things ... SWIG-like typemaps a = dict()‏ a[“asdf”]=”;lkjj” a_component.a_method(a)‏ • Onramp only supports argument types that are convertible to Babel, many needed types do not exist: • Types that are not supported in all languages: e.g. hashtables. • User-defined types that need to be preserved across languages. • Nothing new here, for this purpose Babel is no different than SWIG: most of SWIG is typemaps (measured in lines of code). Arg marshaling in Py IOR in Arg marshaling in C IOR out Arg marshaling in Java /* in java AComponent */ static void a_method(Hashtable a) { String value = (String)a.get(“asdf”)‏

  7. Toolkit Update • Masha has made progress with Scalapack • Standards for extra-bocca components: • https://www.cca-forum.org/wiki/tiki-index.php?page=Component+Builders+Guide+Sans+Bocca • (Just mouse down from the “Toolkit” menu)‏ • BoF Friday • Another Telecon in a week. • Possible “intensive” efforts for a small number of toolkit components at a time • you visit us • we visit you

More Related