1 / 26

Accelerating File Check-outs with the StarTeamMPX Cache Agent

Accelerating File Check-outs with the StarTeamMPX Cache Agent. Randy Guck, Chief Scientist, Borland. Overview. StarTeam architecture overview Configuring MPX Cache Agents Using bundled Cache Agent-aware clients Checking-out files via the StarTeam SDK The basic file check-out application

bendek
Download Presentation

Accelerating File Check-outs with the StarTeamMPX Cache Agent

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Accelerating File Check-outs with the StarTeamMPX Cache Agent Randy Guck, Chief Scientist, Borland

  2. Overview • StarTeam architecture overview • Configuring MPX Cache Agents • Using bundled Cache Agent-aware clients • Checking-out files via the StarTeam SDK • The basic file check-out application • The new CheckOutManager class • Cache Agent performance comparison • New Cache Agent options

  3. StarTeam Client StarTeam Client StarTeam Client SDK SDK SDK StarTeam Basic Architecture StarTeam Configuration workspace Data- base StarTeam Server workspace Command API Vault workspace

  4. StarTeam Client StarTeam Client StarTeam Client SDK SDK SDK StarTeam Architecture with Basic MPX Data- base StarTeam Server Vault publish/subscribe update events Message Broker

  5. StarTeam Client StarTeam Client StarTeam Client SDK SDK SDK StarTeam with MPX Cache Agents Data- base StarTeam Server Vault Remote Message Broker Root Message Broker forwarding and catch-up file check-out Root Cache Agent Remote Cache Agent Local Cache

  6. StarTeam Client SDK MPX/Cache Agent Configuration Order 7) Enable MPX andCache Agent Data- base StarTeam Server 1) MPXEventTransmitter.xml Event Transmitter Vault File Transmitter 5) STMessageBroker64.ini 2) MPXFileTransmitter.xml Remote Message Broker Root Message Broker 3) STMessageBroker64.iniand optionallySTMulticastService64.ini 4) RootCAConfig.xml(example) 6) RemoteCAConfig.xml(example) Remote Cache Agent Root Cache Agent

  7. Bundled Cache Agent-aware Clients • Using Cache Agent check-out wth the StarTeam Cross-Platform Client After check out Tools -> Personal Options File -> Check Out…

  8. Bundled Cache Agent-aware Clients (cont.) • StarTeam command-line (stcmd) utility • Add “-useMPXCacheAgent <address>” to “co” command • Use “autolocate” to use the network-nearest Cache Agent • Use “host:port” for <address> to use a specific Cache Agent • Add “-mpxCacheAgentThreads <#>” to set fetch threads • 3 is the default • Note: Cache Agent mutually exclusive with keyword expansion • Example: stcmd co -p "Administrator:Administrator@localhost:49201/StarDraw/Beta Release"-is -filter OM -useMPXCacheAgent autolocate

  9. Bundled Cache Agent-aware Clients (cont.) • StarTeam bulk check-out (BCO) utility • Similar to “stcmd co”, but optimized for large check-outs • Options: • -useca <address>: “autolocate” or host:port • -threads <#>: to set fetch threads (3 is default) • -t: to get post-check out statistics • -verbose: to get a warm fuzzy feeling about what it’s doing • Limitation: can’t lock/unlock files during check-out • Note: Cache Agent mutually exclusive with keyword expansion • Example: bco -autologon -p "localhost:49201/StarDraw/Beta Release " -is -filter OM –t-useca "192.168.0.4:5201"

  10. Quick Refresher on SDK Flavors • StarTeam SDK supports multiple language “flavors” • Pure Java applications: starteam80.jar • Pure .Net applications: Borland.StarTeam.dll • Cross-compilable Java/J# applications: starteam80.jar for Java; Borland.StarTeam.Core.dll for .Net • Hybrid core/edge applications: Java/J# mixed with pure .Net: starteam80.jar for Java; Borland.StarTeam.dll and Borland.StarTeam.Core.dll for .Net • COM application languages: StarTeamSDK80.dll • VB, C++, scripting languages, etc.

  11. Checking-out files via the SDK • The basic check-out application • Most SDK applications work in the context of a view • Open a Server and log-on • Find the Project you want • Select the View you want • Change ViewConfiguration if needed: tips, label, date, or promotion state • Shortcut: StarTeamFinder • Find the base Folder of interest • The old way to check-out • Examine each File in a Folder • If you want a file, call File.checkOut() or File.checkOutXxx() • Recurse to subfolders

  12. Old Way to Check-out Files in Java // Find view as a URL.String viewURL = "Administrator:Administrator@localhost:49201/StarDraw/Beta Release";View view = StarTeamFinder.openView(viewURL); // Fetch all the file properties we're going to need in one request. Server server = view.getServer(); String[] filePropNames = new String[] { /* list of property names here */ }; Folder folder = view.getRootFolder();folder.populateNow(server.getTypeNames().FILE, filePropNames, -1); // Check-out all files in this folder whose status is "missing" or "out of date".Item[] itemArray = folder.getItems(server.getTypeNames().FILE);for (int i = 0; i < itemArray.length; i++) { File fileItem = (File)itemArray[i]; // StarTeam File, not Java.ioint fileStatus = -1;try {fileStatus = fileItem.getSmartStatus();} catch (IOException ex) {}if (fileStatus == Status.MISSING || fileStatus == Status.OUTOFDATE) {try { fileItem.checkout(Item.LockType.UNCHANGED, false, true, true); } catch (IOException ex) {// Couldn't check-out file } }}// Recurse to subfolders...

  13. Old Way to Check-out Files in C# // Find view as a URL.string viewURL = "Administrator:Administrator@localhost:49201/StarDraw/Web1";View view = StarTeamFinder.OpenView(viewURL); // Fetch all the file properties we're going to need in one request. Server server = view.getServer(); string[] filePropNames = newstring[] { /* list of property names here */ }; Folder folder = view.RootFolder;folder.PopulateNow(server.TypeNames.FILE, filePropNames, -1); // Check-out all files in this folder whose status is "missing" or "out of date".ItemCollection folderItems = folder.GetItems(server.TypeNames.FILE);foreach (Item item in folderItems) { // Get this item as a file and check its status File fileItem = (File)item; // StarTeam File, not Java.io int fileStatus = fileItem.SmartStatus; if (fileStatus == Status.MISSING || fileStatus == Status.OUTOFDATE) { // Check out this file try { // Checkout could throw an IOException. fileItem.Checkout(Item.LockType.UNCHANGED, false, true, true); } catch (IOException ex) { /* Error checking-out this file. */ } } } // Recurse to subfolders...

  14. The New Way to Check-out Files • New CheckoutManager class • “The mother of all check-out classes” • Checks-out a single file or many files in one call • Can check-out from an MPX Cache Agent • New CheckoutOptions class for bundling options together • New CheckoutListener (Java) or CheckoutEventSource (.Net) class for monitoring progress and setting per-file options • New CheckoutProgress class provides statistics on snapshot or final basis • New ContentFilter for in-stream manipulation • “SDK New Features” session covers each in more depth

  15. Steps for Cache Agent Check-out • Create a CheckoutOptions object • Create from View or StarTeamClientOptions to initialize with workstation settings • Set desired options; some non-default examples: • Set ViewConfiguration by date, by label, by promotion state • Insert a ContentFilter into the check-out stream • Override project setting for keyword expansion and file encoding • Lock files during check-out (exclusive or non-exclusive) • Mark non-locked files as “read only” • Set workfile timestamp to check-out time instead of revision modification timestamp • Enable “delta check-out” for out-of-date text files

  16. Steps for Cache Agent Check-out (cont.) • Create a CheckoutManager object • Initialize with CheckoutOptions or just from View • Set Cache Agent options • setMPXCacheAgentEnabled(boolean bEnabled) • setMPXCacheAgentAddress(String address) • null or not called means “auto locate” • setMPXCacheAgentPort(int port) • default is 5201 • setMPXCacheAgentThreadCount(int nThreads) • default is 3 • Call addCheckoutListener() to add a CheckoutListener (Java) • Call NewCheckoutEventSource() to create a CheckoutEventSource for .Net

  17. Steps for Cache Agent Check-out (cont.) • Call one CheckoutManager.checkOut or checkOutTo • void checkout(File file): single file • void checkout(Folder folder): all files in a folder • void checkout(Folder folder, int depth): all files in a folder and child folders to the specified depth (-1 means “all children”) • void checkout(Item[] files): all files in Item array • void checkout(ItemList files): all files in an ItemList • void checkout(Items files): all files specified by Items interface • void checkoutTo(File file, java.io.File workingFile): single file to an alternate location • void checkoutTo(File file, java.io.OutputStream stream): single file to a stream (System.IO.Stream in .Net) • Note: all check-outs are “force” check-outs

  18. Steps for Cache Agent Check-out (cont.) • While the check-out is in progress… • If a CheckoutListener object (or CheckoutEventSource delegates) are registered • onStartFile() and onNotifyProgress() get called once for each file • onStartFile() can modify each file’s CheckoutOptions or its workfile • onNotifyProgress() can see each file’s result • CheckoutManager.setCanceled() can be called to cancel the check-out • CheckoutManager.getProgress() can be called to get a snapshot of in-progress or final statistics • Be careful of Cache Agent check-out with threads > 1 • ContentFilter and listeners must be thread safe!

  19. Cache Agent Check-out: Simple Example // C# void SimpleCacheAgentCheckout(View view) { CheckoutManager checkoutManager = new CheckoutManager(view); checkoutManager.MPXCacheAgentEnabled = true; checkoutManager.Checkout(view.RootFolder, -1); // all files in view } // Java void SimpleCacheAgentCheckout(View view) { CheckoutManager checkoutManager = new CheckoutManager(view); checkoutManager.setMPXCacheAgentEnabled(true); checkoutManager.checkout(view.getRootFolder(), -1); // all files in view }

  20. Cache Agent Check-out: More Sophisticated // Initialize CheckoutOptions from StarTeamClientOptions StarTeamClientOptions defaultClientOptions = null;CheckoutOptions checkoutOptions = null;try { // StarTeamClientOptions.getDefault() could throw an exception. defaultClientOptions = StarTeamClientOptions.getDefault(); checkoutOptions = new CheckoutOptions(defaultClientOptions);} catch (IOException ex) { checkoutOptions = new CheckoutOptions(view);}// Create a CheckoutManager and enable the use of an auto-located Cache Agent.CheckoutManager checkoutManager = new CheckoutManager(view, checkoutOptions);checkoutManager.setMPXCacheAgentEnabled(true);// Add ourself as a CheckoutListenter and check-out all files from root. checkoutManager.addCheckoutListener(this);checkoutManager.checkout(view.getRootFolder(), -1);// Get some final statisticsCheckoutProgress finalStats = checkoutManager.getProgress();System.out.println("Total files checked-out: " + finalStats.getTotalFilesCheckedOut());System.out.println("Total bytes checked-out: " + finalStats.getTotalBytesCheckedOut());System.out.println("Total check-out time: " + ((float)finalStats.getTotalCommandTime() / 1000) + " seconds");

  21. Performance Comparison • Test #1: Project Documents • 238 files: Word docs, PPTs, PDFs, etc. • 115,132,745 total bytes; 483,751 average file size • Check-out over cable mode, VPN • Remote Cache Agent network-near over 100Base-T

  22. Performance Comparison (cont.) • Test #2: Source Files • 172 files: source files, IDE workspaces • 857,044 total bytes; 4.983 average file size • Check-out over cable mode, VPN • Remote Cache Agent network-near over 100Base-T

  23. New Cache Agent Options for 2005 R2 • Remote Cache Agent tracking-by-project • Allows a remote Cache Agent to cache a subset of each StarTeam server’s files • Enabled by adding <project> options to <ContentSource> group: <ContentSource> <ServerGUID>bd1352a8-68a3-4ac1-9f28-0604b3d9267a</ServerGUID><Projects> <Project>FelixTools</Project> <Project>WebFoundation1</Project> <Project>Bank*</Project> <Project>Insurance*West*</Project> </Projects> <UpstreamCache> <AutoLocate/> </UpstreamCache></ContentSource>

  24. New Cache Agent Options for 2005 R2 (cont.) • Multiple Cache Agents on the same machine • CacheAgentService.exe makes it easier to register multiple Cache Agent services on the same machine • -name <name>: sets service display name • -dependson <service list>: declares service start dependency • -log <log file>: sets name of log file (default based on <name>) • -verbose: generates secondary debug log file • Unregister supports -name <name> as well • Example: CacheAgentService -register Auto"C:\Program Files\Borland\StarTeamMPX Cache Agent 2005 R2\RemoteCAConfig.xml"-dependson StarTeamMessageBroker6.4 -verbose

  25. New Cache Agent Options for 2005 R2 (cont.) • New Cache Agent Configuration Options • New common Cache Agent options • CacheCheckInterval: Frequency (seconds) for checking cache limit • InitialRequestThreads: Start-up request threads • MaxCatchupSize: Maximum content transfer provided (root Cache Agent) or requested (remote Cache Agent) for catch-up requests • RequestReadTimeout: Connection timeout (seconds) • New remote Cache Agent options • CatchupCheckInterval: Catch-up check frequency after a network outage • PrechargeSize: Initial “pre-charge” size for new remote Cache Agents

  26. Questions? Contact me at: randy.guck@borland.com

More Related