cycle stealing kind of
Download
Skip this Video
Download Presentation
Cycle Stealing (kind of)

Loading in 2 Seconds...

play fullscreen
1 / 13

Cycle Stealing (kind of) - PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on

Cycle Stealing (kind of). Christopher Salembier CSC-8530. Review. Cycle Stealing Network of Workstations (NOW) as a parallel computer Making use of unused resources Workstation may be active or inactive Used in many volunteer computing projects

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Cycle Stealing (kind of)' - ingrid


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
cycle stealing kind of

Cycle Stealing(kind of)

Christopher Salembier

CSC-8530

review
Review
  • Cycle Stealing
    • Network of Workstations (NOW) as a parallel computer
    • Making use of unused resources
    • Workstation may be active or inactive
  • Used in many volunteer computing projects
    • Berkeley Open Infrastructure for Network Computing (BOINC)
    • Distributed.net
  • GridGain middleware used
    • Java based cloud enabling platform
    • Configured through Spring framework IoC container
intended project focus
(Intended)Project Focus
  • The cycles / resources themselves
  • Research findings
    • Fine-grained kernel-level stealing
    • Complex algorithms for scheduling and allocation
    • Existing platforms
  • Did none of it
    • Needed C or lower level language
    • Wouldn’t have any distributed element
  • “Application” level stealing instead
what got missed
What Got Missed
  • Client-Side Issues
    • GridGain only active within JVM
    • No recovery on shutdown / offline
      • Cannot save task
      • Grid registers as a failure
    • “Pull” jobs
  • Bandwidth monitoring
  • WAN usage
the distributed task
The Distributed Task
  • Parsing the contents of Wikipedia
    • 24 GB+ of text
    • Requires 3 passes
  • Unused processing power on work servers
  • Ultimate goal is general text parsing system
    • General purposes grid for processing large files
    • Use method injection to configure algorithm
implementation worker
Implementation - Worker

public final void unregister() {

StealFactory.unregisterThread( this );

}

  • public abstract Serializable process() throws GridException;

public final Serializable execute() throws GridException {

Serializable result = null;

register();

result = process();

unregister();

return result;

}

public final void register() {

pauseLock = new ReentrantLock()

waiting = pauseLock.newCondition();

StealFactory.registerNewThread( this );

if ( StealFactory.isBlockThreadActive()

pauseProcess();

}

public final void pauseProcess() {

if ( !isBlocked ) {

isBlocked = true;

while ( isBlocked )

waiting.await();

}

}

public final void wakeProcess() {

pauseLock.lock();

isBlocked = false;

waiting.signal();

pauseLock.unlock();

}

implementation worker1
Implementation - Worker
  • public void setMonitoredValue( Long value ) {
  • currentSystemState = value;
  • if ( isWaiting ) {
  • lock.lock();
  • isWaiting= false;
  • waiting.signal();
  • lock.unlock();
  • }
  • public void run() {
  • while ( running ) {
  • checkSystemState(); lock.lock();
  • isWaiting= true;
  • while ( isWaiting )
  • waiting.await();
  • lock.unlock();
  • }
  • private void checkSystemState() {
  • if ( currentSystemState >= systemStateThreshold && isBlocking ) {
  • isBlocking= false;
  • StealFactory.notifyRegisteredThreads();
  • else if ( currentSystemState < systemStateThreshold && !isBlocking ) {
  • isBlocking= true;
  • StealFactory.blockRegisteredThreads();
  • }

public void addTask( CycleSteal job ) {

synchronized ( mutex )

registeredJobs.add( job );

}

public void removeTask( CycleSteal job ) {

synchronized ( mutex ) {

registeredJobs.remove( job );

}

public void blockTasks() {

synchronized ( mutex ) {

for ( final CycleSteal job : registeredJobs ) {

new Thread( new Runnable() {

public void run()

job.pauseProcess();

} ).start();

}

public void notifyTasks() {

synchronized( mutex ) {

for ( CycleSteal job : registeredJobs ) {

job.wakeProcess();

}

}

implementation master
Implementation - Master

public void onFinished( GridTaskFuture<?> taskFuture ) {

GridNode node = null;

HashMap<String, String> map = null;

UUID sesID = taskFuture.getTaskSession().getId();

node = assignThread.removeFuture( sesID );

map = (HashMap<String, String>)taskFuture.get();

processJobResult( map );

if ( !assignThread.isRunning() ) {

if ( !assignThread.hasWorkOutstanding() )

shutdownMasterNode();

else

assignJobsToNode( node );

}

configuration
Configuration
  • All components configured as Spring beans
    • Application Context XML file
    • Inversion of Control pattern
  • Allows run-time configuration
    • Grid Itself
    • Node type
      • Master / Worker
      • User / CPU monitor
    • Stealing properties
configuration1
Configuration

<bean id="blockManager" class="edu.villanova.salembier.csc8530.steal.block.StealBlockManager" scope="singleton" />

<bean id="blockThread" class="edu.villanova.salembier.csc8530.steal.block.BlockThreadLong" scope="singleton">

<constructor-arg type="long" value="955"/>

</bean>

<bean id="activityMonitor" class="edu.villanova.salembier.csc8530.steal.monitor.SystemActivityMonitor" scope="singleton" >

<constructor-arg type="long" value="500"/>

</bean>

<bean id="stealFactory" class="edu.villanova.salembier.csc8530.steal.StealFactoryLong" scope="singleton" >

<constructor-arg type="edu.villanova.salembier.csc8530.steal.block.BlockManager" ref="blockManager"/>

<constructor-arg type="edu.villanova.salembier.csc8530.steal.block.BlockThread" ref="blockThread"/>

<constructor-arg type="edu.villanova.salembier.csc8530.steal.monitor.ActivityMonitor" ref="activityMonitor"/>

</bean>

<bean id="grid.cfg" class="org.gridgain.grid.GridConfigurationAdapter" scope="singleton">

<property name="gridName" value="CSC 8530 GRID"/>

<property name="userAttributes">

<map> <entry key="nodeType" value="cpu" /> </map>

</property>

<property name="gridLogger">

<bean class="org.gridgain.grid.logger.log4j.GridLog4jLogger">

<constructor-arg type="java.lang.String" value="config/default-log4j.xml"/>

</bean>

</ property>

<property name="executorService">

<bean class="edu.villanova.salembier.csc8530.execute.StealExecutorService" >

<constructor-arg type="int" value="10"/>

<constructor-arg type="int" value="25"/>

</bean>

</property>

</bean>

testing
Testing
  • No UI to verify
  • Reliant on log files
    • Massive amount of data generated
    • Log entries not always sequential
  • 1st Round of Testing failed
    • Though code was working
    • Block Thread & Manager needed significant rework
    • Though difficult, log ultimately helped solve problem
slide12
Demo
  • I hope this works!!!
  • Run nodes of each types
  • View logs
  • View output
conclusion
Conclusion
  • Original Goal not really met
  • Project a Success
  • Learned a lot about…
    • Cycle stealing
    • Complex multi-thread Java systems
    • General distributed programming issues
ad