accelerating file check outs with the starteammpx cache agent n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Accelerating File Check-outs with the StarTeamMPX Cache Agent PowerPoint Presentation
Download Presentation
Accelerating File Check-outs with the StarTeamMPX Cache Agent

Loading in 2 Seconds...

play fullscreen
1 / 26

Accelerating File Check-outs with the StarTeamMPX Cache Agent - PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on

Accelerating File Check-outs with the StarTeamMPX Cache Agent. Randy Guck, Chief Scientist, Borland. Overview. StarTeam architecture overview Configuring MPX Cache Agents Using bundled Cache Agent-aware clients Checking-out files via the StarTeam SDK The basic file check-out application

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Accelerating File Check-outs with the StarTeamMPX Cache Agent' - bendek


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
accelerating file check outs with the starteammpx cache agent

Accelerating File Check-outs with the StarTeamMPX Cache Agent

Randy Guck, Chief Scientist, Borland

overview
Overview
  • StarTeam architecture overview
  • Configuring MPX Cache Agents
  • Using bundled Cache Agent-aware clients
  • Checking-out files via the StarTeam SDK
    • The basic file check-out application
    • The new CheckOutManager class
  • Cache Agent performance comparison
  • New Cache Agent options
starteam basic architecture

StarTeam

Client

StarTeam

Client

StarTeam

Client

SDK

SDK

SDK

StarTeam Basic Architecture

StarTeam

Configuration

workspace

Data-

base

StarTeam

Server

workspace

Command API

Vault

workspace

starteam architecture with basic mpx

StarTeam

Client

StarTeam

Client

StarTeam

Client

SDK

SDK

SDK

StarTeam Architecture with Basic MPX

Data-

base

StarTeam

Server

Vault

publish/subscribe

update events

Message

Broker

starteam with mpx cache agents

StarTeam

Client

StarTeam

Client

StarTeam

Client

SDK

SDK

SDK

StarTeam with MPX Cache Agents

Data-

base

StarTeam

Server

Vault

Remote

Message

Broker

Root

Message

Broker

forwarding

and catch-up

file check-out

Root

Cache Agent

Remote

Cache Agent

Local

Cache

mpx cache agent configuration order

StarTeam

Client

SDK

MPX/Cache Agent Configuration Order

7) Enable MPX andCache Agent

Data-

base

StarTeam

Server

1) MPXEventTransmitter.xml

Event Transmitter

Vault

File Transmitter

5) STMessageBroker64.ini

2) MPXFileTransmitter.xml

Remote

Message

Broker

Root

Message

Broker

3) STMessageBroker64.iniand optionallySTMulticastService64.ini

4) RootCAConfig.xml(example)

6) RemoteCAConfig.xml(example)

Remote

Cache Agent

Root

Cache Agent

bundled cache agent aware clients
Bundled Cache Agent-aware Clients
  • Using Cache Agent check-out wth the StarTeam Cross-Platform Client

After check out

Tools -> Personal Options

File -> Check Out…

bundled cache agent aware clients cont
Bundled Cache Agent-aware Clients (cont.)
  • StarTeam command-line (stcmd) utility
    • Add “-useMPXCacheAgent <address>” to “co” command
      • Use “autolocate” to use the network-nearest Cache Agent
      • Use “host:port” for <address> to use a specific Cache Agent
    • Add “-mpxCacheAgentThreads <#>” to set fetch threads
      • 3 is the default
    • Note: Cache Agent mutually exclusive with keyword expansion
    • Example:

stcmd co -p "Administrator:Administrator@localhost:49201/StarDraw/Beta Release"-is -filter OM -useMPXCacheAgent autolocate

bundled cache agent aware clients cont1
Bundled Cache Agent-aware Clients (cont.)
  • StarTeam bulk check-out (BCO) utility
    • Similar to “stcmd co”, but optimized for large check-outs
    • Options:
      • -useca <address>: “autolocate” or host:port
      • -threads <#>: to set fetch threads (3 is default)
      • -t: to get post-check out statistics
      • -verbose: to get a warm fuzzy feeling about what it’s doing
    • Limitation: can’t lock/unlock files during check-out
    • Note: Cache Agent mutually exclusive with keyword expansion
    • Example:

bco -autologon -p "localhost:49201/StarDraw/Beta Release " -is -filter OM –t-useca "192.168.0.4:5201"

quick refresher on sdk flavors
Quick Refresher on SDK Flavors
  • StarTeam SDK supports multiple language “flavors”
    • Pure Java applications: starteam80.jar
    • Pure .Net applications: Borland.StarTeam.dll
    • Cross-compilable Java/J# applications: starteam80.jar for Java; Borland.StarTeam.Core.dll for .Net
    • Hybrid core/edge applications: Java/J# mixed with pure .Net: starteam80.jar for Java; Borland.StarTeam.dll and Borland.StarTeam.Core.dll for .Net
    • COM application languages: StarTeamSDK80.dll
      • VB, C++, scripting languages, etc.
checking out files via the sdk
Checking-out files via the SDK
  • The basic check-out application
    • Most SDK applications work in the context of a view
      • Open a Server and log-on
      • Find the Project you want
      • Select the View you want
        • Change ViewConfiguration if needed: tips, label, date, or promotion state
      • Shortcut: StarTeamFinder
    • Find the base Folder of interest
    • The old way to check-out
      • Examine each File in a Folder
      • If you want a file, call File.checkOut() or File.checkOutXxx()
      • Recurse to subfolders
old way to check out files in java
Old Way to Check-out Files in Java

// Find view as a URL.String viewURL = "Administrator:Administrator@localhost:49201/StarDraw/Beta Release";View view = StarTeamFinder.openView(viewURL);

// Fetch all the file properties we're going to need in one request.

Server server = view.getServer();

String[] filePropNames = new String[] { /* list of property names here */ };

Folder folder = view.getRootFolder();folder.populateNow(server.getTypeNames().FILE, filePropNames, -1);

// Check-out all files in this folder whose status is "missing" or "out of date".Item[] itemArray = folder.getItems(server.getTypeNames().FILE);for (int i = 0; i < itemArray.length; i++) { File fileItem = (File)itemArray[i]; // StarTeam File, not Java.ioint fileStatus = -1;try {fileStatus = fileItem.getSmartStatus();} catch (IOException ex) {}if (fileStatus == Status.MISSING || fileStatus == Status.OUTOFDATE) {try { fileItem.checkout(Item.LockType.UNCHANGED, false, true, true); } catch (IOException ex) {// Couldn't check-out file

}

}}// Recurse to subfolders...

old way to check out files in c
Old Way to Check-out Files in C#

// Find view as a URL.string viewURL = "Administrator:Administrator@localhost:49201/StarDraw/Web1";View view = StarTeamFinder.OpenView(viewURL);

// Fetch all the file properties we're going to need in one request.

Server server = view.getServer();

string[] filePropNames = newstring[] { /* list of property names here */ };

Folder folder = view.RootFolder;folder.PopulateNow(server.TypeNames.FILE, filePropNames, -1);

// Check-out all files in this folder whose status is "missing" or "out of date".ItemCollection folderItems = folder.GetItems(server.TypeNames.FILE);foreach (Item item in folderItems)

{ // Get this item as a file and check its status

File fileItem = (File)item; // StarTeam File, not Java.io

int fileStatus = fileItem.SmartStatus;

if (fileStatus == Status.MISSING || fileStatus == Status.OUTOFDATE)

{ // Check out this file

try

{ // Checkout could throw an IOException.

fileItem.Checkout(Item.LockType.UNCHANGED, false, true, true);

}

catch (IOException ex) { /* Error checking-out this file. */ }

}

}

// Recurse to subfolders...

the new way to check out files
The New Way to Check-out Files
  • New CheckoutManager class
    • “The mother of all check-out classes”
    • Checks-out a single file or many files in one call
    • Can check-out from an MPX Cache Agent
    • New CheckoutOptions class for bundling options together
    • New CheckoutListener (Java) or CheckoutEventSource (.Net) class for monitoring progress and setting per-file options
    • New CheckoutProgress class provides statistics on snapshot or final basis
    • New ContentFilter for in-stream manipulation
  • “SDK New Features” session covers each in more depth
steps for cache agent check out
Steps for Cache Agent Check-out
  • Create a CheckoutOptions object
    • Create from View or StarTeamClientOptions to initialize with workstation settings
    • Set desired options; some non-default examples:
      • Set ViewConfiguration by date, by label, by promotion state
      • Insert a ContentFilter into the check-out stream
      • Override project setting for keyword expansion and file encoding
      • Lock files during check-out (exclusive or non-exclusive)
      • Mark non-locked files as “read only”
      • Set workfile timestamp to check-out time instead of revision modification timestamp
      • Enable “delta check-out” for out-of-date text files
steps for cache agent check out cont
Steps for Cache Agent Check-out (cont.)
  • Create a CheckoutManager object
    • Initialize with CheckoutOptions or just from View
    • Set Cache Agent options
      • setMPXCacheAgentEnabled(boolean bEnabled)
      • setMPXCacheAgentAddress(String address)
        • null or not called means “auto locate”
      • setMPXCacheAgentPort(int port)
        • default is 5201
      • setMPXCacheAgentThreadCount(int nThreads)
        • default is 3
    • Call addCheckoutListener() to add a CheckoutListener (Java)
    • Call NewCheckoutEventSource() to create a CheckoutEventSource for .Net
steps for cache agent check out cont1
Steps for Cache Agent Check-out (cont.)
  • Call one CheckoutManager.checkOut or checkOutTo
    • void checkout(File file): single file
    • void checkout(Folder folder): all files in a folder
    • void checkout(Folder folder, int depth): all files in a folder and child folders to the specified depth (-1 means “all children”)
    • void checkout(Item[] files): all files in Item array
    • void checkout(ItemList files): all files in an ItemList
    • void checkout(Items files): all files specified by Items interface
    • void checkoutTo(File file, java.io.File workingFile): single file to an alternate location
    • void checkoutTo(File file, java.io.OutputStream stream): single file to a stream (System.IO.Stream in .Net)
    • Note: all check-outs are “force” check-outs
steps for cache agent check out cont2
Steps for Cache Agent Check-out (cont.)
  • While the check-out is in progress…
    • If a CheckoutListener object (or CheckoutEventSource delegates) are registered
      • onStartFile() and onNotifyProgress() get called once for each file
        • onStartFile() can modify each file’s CheckoutOptions or its workfile
        • onNotifyProgress() can see each file’s result
    • CheckoutManager.setCanceled() can be called to cancel the check-out
    • CheckoutManager.getProgress() can be called to get a snapshot of in-progress or final statistics
    • Be careful of Cache Agent check-out with threads > 1
      • ContentFilter and listeners must be thread safe!
cache agent check out simple example
Cache Agent Check-out: Simple Example

// C#

void SimpleCacheAgentCheckout(View view)

{

CheckoutManager checkoutManager = new CheckoutManager(view);

checkoutManager.MPXCacheAgentEnabled = true;

checkoutManager.Checkout(view.RootFolder, -1); // all files in view

}

// Java

void SimpleCacheAgentCheckout(View view)

{

CheckoutManager checkoutManager = new CheckoutManager(view);

checkoutManager.setMPXCacheAgentEnabled(true);

checkoutManager.checkout(view.getRootFolder(), -1); // all files in view

}

cache agent check out more sophisticated
Cache Agent Check-out: More Sophisticated

// Initialize CheckoutOptions from StarTeamClientOptions

StarTeamClientOptions defaultClientOptions = null;CheckoutOptions checkoutOptions = null;try { // StarTeamClientOptions.getDefault() could throw an exception. defaultClientOptions = StarTeamClientOptions.getDefault(); checkoutOptions = new CheckoutOptions(defaultClientOptions);} catch (IOException ex) {

checkoutOptions = new CheckoutOptions(view);}// Create a CheckoutManager and enable the use of an auto-located Cache Agent.CheckoutManager checkoutManager = new CheckoutManager(view, checkoutOptions);checkoutManager.setMPXCacheAgentEnabled(true);// Add ourself as a CheckoutListenter and check-out all files from root.

checkoutManager.addCheckoutListener(this);checkoutManager.checkout(view.getRootFolder(), -1);// Get some final statisticsCheckoutProgress finalStats = checkoutManager.getProgress();System.out.println("Total files checked-out: " + finalStats.getTotalFilesCheckedOut());System.out.println("Total bytes checked-out: " + finalStats.getTotalBytesCheckedOut());System.out.println("Total check-out time: " + ((float)finalStats.getTotalCommandTime() / 1000) + " seconds");

performance comparison
Performance Comparison
  • Test #1: Project Documents
  • 238 files: Word docs, PPTs, PDFs, etc.
  • 115,132,745 total bytes; 483,751 average file size
  • Check-out over cable mode, VPN
  • Remote Cache Agent network-near over 100Base-T
performance comparison cont
Performance Comparison (cont.)
  • Test #2: Source Files
  • 172 files: source files, IDE workspaces
  • 857,044 total bytes; 4.983 average file size
  • Check-out over cable mode, VPN
  • Remote Cache Agent network-near over 100Base-T
new cache agent options for 2005 r2
New Cache Agent Options for 2005 R2
  • Remote Cache Agent tracking-by-project
    • Allows a remote Cache Agent to cache a subset of each StarTeam server’s files
    • Enabled by adding <project> options to <ContentSource> group:

<ContentSource> <ServerGUID>bd1352a8-68a3-4ac1-9f28-0604b3d9267a</ServerGUID><Projects> <Project>FelixTools</Project> <Project>WebFoundation1</Project> <Project>Bank*</Project> <Project>Insurance*West*</Project> </Projects> <UpstreamCache> <AutoLocate/> </UpstreamCache></ContentSource>

new cache agent options for 2005 r2 cont
New Cache Agent Options for 2005 R2 (cont.)
  • Multiple Cache Agents on the same machine
    • CacheAgentService.exe makes it easier to register multiple Cache Agent services on the same machine
      • -name <name>: sets service display name
      • -dependson <service list>: declares service start dependency
      • -log <log file>: sets name of log file (default based on <name>)
      • -verbose: generates secondary debug log file
    • Unregister supports -name <name> as well
    • Example:

CacheAgentService -register Auto"C:\Program Files\Borland\StarTeamMPX Cache Agent 2005 R2\RemoteCAConfig.xml"-dependson StarTeamMessageBroker6.4 -verbose

new cache agent options for 2005 r2 cont1
New Cache Agent Options for 2005 R2 (cont.)
  • New Cache Agent Configuration Options
    • New common Cache Agent options
      • CacheCheckInterval: Frequency (seconds) for checking cache limit
      • InitialRequestThreads: Start-up request threads
      • MaxCatchupSize: Maximum content transfer provided (root Cache Agent) or requested (remote Cache Agent) for catch-up requests
      • RequestReadTimeout: Connection timeout (seconds)
    • New remote Cache Agent options
      • CatchupCheckInterval: Catch-up check frequency after a network outage
      • PrechargeSize: Initial “pre-charge” size for new remote Cache Agents
questions
Questions?

Contact me at:

randy.guck@borland.com