slide1 n.
Download
Skip this Video
Download Presentation
Building high-quality Windows PCs using the Assessment and Deployment Kit

Loading in 2 Seconds...

play fullscreen
1 / 39

Building high-quality Windows PCs using the Assessment and Deployment Kit - PowerPoint PPT Presentation


  • 182 Views
  • Uploaded on

TOL-147T. Building high-quality Windows PCs using the Assessment and Deployment Kit. Sylvain Goyette Scenario Adoption PM Microsoft Corporation. Agenda. ADK introduction Assessment tools scenarios Assessment investments Windows Assessment Console Windows Assessment Services

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Building high-quality Windows PCs using the Assessment and Deployment Kit' - alameda


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
building high quality windows pcs using the assessment and deployment kit
TOL-147T

Building high-quality Windows PCs using the Assessment and Deployment Kit

Sylvain Goyette

Scenario Adoption PM

Microsoft Corporation

agenda
Agenda
  • ADK introduction
  • Assessment tools scenarios
    • Assessment investments
    • Windows Assessment Console
    • Windows Assessment Services
    • Assessment execution engine
  • Windows Performance Toolkit (WPA/WPR)
assessment and deployment kit
Assessment and Deployment Kit
  • Consolidate OPK toolkit and WAIK
    • Core deployment tools and assessment toolkit
  • Create, assess, and maintain quality Windows images
  • How we measure the PC today lacks the needed customer view
    • Processor speed, megahertz, gigabytes, etc.
    • How exactly does this relate to an individual’s experience?
    • The ADK system assessment tools help engineers deliver high-quality, differentiated devices that measure system quality through a customer experience
  • One-stop shop for all partners
system assessment basics
System assessment basics
  • System assessment is a process that uses the ADK tools to measure and analyze a PC
  • Assessments are core functionality tests
  • Combinations of these tests provide additional measures of the entire PC experience
  • Quality expectations are changing
  • Software + hardware + Windows = PC experience
  • The way we measure PC quality must also change
benchmarking doesn t go far enough
Benchmarking doesn’t go far enough
  • Advantages
    • Accepted and easy to discuss by techies
    • Used as part of certification or comparison
    • Widely adopted, especially in the press
  • Disadvantages
    • Provided with little context on how benchmarks are created
    • A “black box” to users
    • Less useful in system engineering
system assessments
System assessments
  • CheckLogo and driver assessments
  • File handling
  • Photo handling
  • Internet Explorer launch/tab create
  • Hybrid boot
  • On/off assessments (boot/shutdown/S3/S4)
  • Browser assessment
  • Media transcode performance
  • Metro performance
  • Memory footprint
  • First boot experience
  • Media streaming
  • WinSAT comprehensive
  • Battery life (and idle efficiency analysis)
  • MiniFilter driver performance impact (option for other assessments)
  • Internet browsing workload for battery life assessment
  • Windows Media Player performance and quality
assessment tenets
Assessment tenets
  • System quality assessments should:
    • Focus on assessing the end user’s experience, not just the underlying technologies
    • Assess system quality in a quantifiable way
    • Allow for system comparisons
    • Identify issues and provide remediation steps wherever possible
    • Provide the ability to dig deeper using tools like Windows Performance Analyzer
    • Make collaboration easier
    • Be widely available
evolution of tools
Evolution of tools

ADK assessment tools

Velocity Test Suite

(VTS / VOS)

Velocity 2.x (VOS)

Velocity 4.x

Velocity 3.x (VOS)

Fundamental Quality Tools Suite

(FQTS)

SPPTS

(WLK 1.4)

Velocity (VTS)

Velocity 3.x (VTS)

SPPTS

(WLK 1.5)

System Power and Performance Tools Suite

(SPPTS)

FQTS

2.x

FQTS

v3

overview of problems with existing tools
Overview of problems with existing tools
  • Existing tools reuse solutions designed to assess quality of single computer
  • Introduce significant manual work, which grows as number of computers increases:
    • Computer preparation (image deployment and drivers installation)
    • Assessment execution and monitoring progress
    • Collection of results from computers
  • Assessing quality of multiple computers
two ways of using assessments
Two ways of using assessments

Single Machine and Basic Architecture

Lab scenarios and Scalability

what you get in the windows adk
What You Get in the Windows ADK
  • Remediation tool used for analysis of performance traces

WPA

Windows Performance Analyzer

Used to access, configure, create, save, package jobs to run on different machines. Good for running assessments on one to a few machines. Used to view results from assessment run

Scenario-based tests of system quality, delivering metrics, identifying issues, and providing remediation steps wherever possible

ADK

Assessment and Deployment Kit

WAC

Windows Assessment Console

Assessments

WAS

Windows Assessment Services

Used for larger environments with the potential of running assessments on hundreds of computer

three key concepts
Three key concepts
  • Jobs run, not assessments
    • Jobs contain one or more assessments
    • Jobs contain the parameters that are used to configure individual assessments (as well as a few job parameters)
  • All jobs run in administrator mode
    • It’s a requirement of the Assessment Execution Engine (AXE)
  • It’s the assessment that has the “power”
    • The GUI and platform provide a test harness to expose things the assessment wants to expose—it’s the assessment that does (and should do) most of the heavy lifting, analysis, and subject matter expertise
assessment infrastructure
Assessment infrastructure

Windows Assessment Service (WAS and WASC) – lab environment

Discover

Compose

Execute

Interpret

Monitor

Diagnose

Manage

  • Alignment with WAC on the four primary goals
  • Extends concept to distributed environment with multiple assets (machines, images, assessments)
  • Adds capability to manage assets, monitor progress on multiple jobs and assets, and facilitate centralized diagnosis
assessment execution engine
Assessment Execution Engine
  • AXE core components:
  • AXE.exe
  • AL.exe
  • Etc.
  • Small set of binaries to support “platform”
  • XML-driven definition and configuration
    • Assessment manifest defines the assessment, including parameters, metrics, thresholds, etc.
    • Job manifest defines the job (which assessments to run and parameter values)
    • Results file defines the output
    • Each “subsequent” XML contains the previous files in their entirety
  • Ability to launch any code/script/process
  • Assessment resources:
  • asmt.exe
  • Assessmentmanifest.xml
  • Axecore.dll
  • Other resources (including samples)
integration goals
Integration goals
  • Modular design provides multiple ways to extend and varying levels of integration
  • Provide both interactive and automated tools
  • APIs and schema will be made available publicly
    • Both native and managed execution APIs for running assessments
    • Assessment, job, and results XML schema definitions
    • Advanced managed administrative APIs for creating XML documents

We used these same APIs and schemas to produce the tools and assessments delivered in the kit.

ways to integrate and extend
Ways to integrate and extend
  • Convert or build your own assessments
  • Automate execution in your existing infrastructure
  • Import the results
  • Create jobs and packages
windows assessment services
Windows Assessment Services
  • Infrastructure to support lab environments
  • Automation to support large scale deployment and assessment of many machines
  • Results database allows for system comparison, analysis, and reporting
automation for the lab environment
Automation for the lab environment
  • Computer preparation
    • Using Windows Deployment Services (WDS) and customized WinPE image
  • Assessment execution
    • Using Windows Remote Management (WinRM) to copy assessment execution engine + assessments to computers and run them
  • Collection of results from computers
    • Using WinRM to copy results to lab execution server and preprocess them
  • Supports multiple computer scenarios
what was is not
What WAS is not?
  • Assessment execution in the cloud / over the Internet. It is designed to work within a single subnet.
  • Windows Hardware Certification framework
  • Testing computers on manufacturing floor
  • Large scale testing (there is an limit of 300 computers)
  • Testing computers that are already shipped and are in use by end users
  • Debugging / fixing issues that are found by assessments
  • Issue management
was results database
WAS results database
  • Provide an optional platform for aggregated results collection based on the AXE assessment results schema
  • Focus on results collection for trending, comparison, and business reporting needs across large amounts of results data
    • By contrast, WAC is focused on engineering views of assessment results for immediate analysis and triaging purposes
benefits of wpt
Benefits of WPT
  • Enables follow up on issues raised via ADK performance assessments
  • Enables holistic performance analysis
    • All processes/threads
    • All key system resources
    • User and kernel mode
  • Provides a way to analyze what customer actually experiences
    • Catch the problem as it happens
    • Capture-Anywhere-Decode-Anywhere
  • Very low overhead versus alternatives
  • Region of interest profiling
    • You get both what and when
windows performance recorder and analyzer
Windows Performance Recorder and Analyzer
  • Tools for:
  • Trace capture (WPR)
  • Trace analysis(WPA)
  • Based on ETW (event tracing for Windows)
  • Used within Microsoft to build Windows
  • Core code reused in-box
  • Documentation on MSDN
  • Part of ADK and SDK in Windows 8
high level changes
High-level changes
  • Windows Performance Analyzerreplaces xperview
  • Windows Performance Recorder replaces xbootmgr
    • Also replaces xperf trace capture functionality
  • xperf command line actions remain in WPT
benefits of wpa in windows 8
Benefits of WPA in Windows 8
  • Broaden the target audience to mainstream developers
  • Leverage familiar UI experiences
  • Enable quick ramp up
  • Reduce “noise” to focus on the “right” issues
wpa in windows 8
WPA in Windows 8

Detailed Graphs

Graph Explorer

Graph

Legend

Grouping Columns

Aggregating Columns

Graphing Columns

Table

Blue Bar

Gold Bar

call to action
Call to action
  • Evaluate the ADK assessment tools (file bugs, send feedback) until Beta
  • Work with your local contact to start planning your implementation
  • Integrate ADK assessments into your test processes to optimize user experience
  • Training and evangelization will be communicated
  • POC for adoption: adkcontact@microsoft.com
  • Requests and Timeline
further reading and documentation
Further reading and documentation

Event Site:

  • http://channel9.msdn.com/Events

Dev Center:

    • Windows Performance Analysis

MSDN documentation:

    • WPA and WPR
    • Event Tracing for Windows
    • Assessment and Deployment Kit
    • Assessment Execution Engine

Videos

  • Channel9 – BUILD ADK Videos

Books:

  • Windows Internals by Mark Russinovich, David Solomon, and Alex Ionescu

Reference:

  • ADK_TechnicalReference.chm in the ADK
related sessions from build conference
Related sessions from //build conference

Go to - www.buildwindows.com/sessions and search for these numbered sessions.

  • [HW-141T]Reducing the Memory Footprint of Drivers and Apps
  • [HW-147T] Building High Quality Windows PCs using the Assessment and Deployment Kit
  • [HW-148P] Scaling and Extending Windows Assessments to Improve System Quality
  • [HW-149P] Improving Battery Life and Energy Efficiency
  • [HW-459T] Delivering Great Performance and Battery Life
  • [HW-59T] Improving Performance with the Windows Performance Toolkit
  • [HW-915P] Introduction to Assessments
  • [HW-916P] Introduction to the Boot Performance Assessment
  • [HW-917P] Introduction to the IE Launch Assessment
  • [HW-920P] Introduction to the Streaming Assessment
related sessions from build conference cont
Related sessions from //build conference (cont.)

Go to - www.buildwindows.com/sessions and search for these numbered sessions.

  • [HW-921P]Introduction to ADK Custom Jobs
  • [HW-922P]Capturing and Analyzing Performance Traces
  • [HW-925P]Customizing WPA Trace Views
  • [HW-926P]Introduction to the New WPA User Interface
  • [HW-927P] Understanding Pool Usage using WPA
  • [HW-928P] Understanding Heap Data using WPA
  • [HW-975P] Customizing ADK Assessment Reports
  • [HW-976P] Packaging Assessments for Use on a Second Computer
  • [HW-977P] Understanding VirtualAlloc Usage using WPA
slide38
Thank You!

For questions, please visit me in the

Speakers Connection area following this session.

slide39

© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.