1 / 34

The New COM API for Accessibility and Automation in Windows 7

makya
Download Presentation

The New COM API for Accessibility and Automation in Windows 7

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. The New COM API for Accessibility and Automation in Windows 7 ? Michael Bernstein Principal Software Design Engineer Microsoft Corporation

    2. Programmatic UI Access: A Key Element of Accessibility Enables developers to write code that: Navigates between UI elements Gathers information about the UI Interacts with UI elements Receives notifications when the UI changes 2

    3. UI Automation (UIA): Programmatic Access in Windows Successor to Microsoft Active Accessibility (MSAA) Creates a universal object model for accessing UI Designed based on experience with MSAA usage Supported on Windows Vista, XP and Server 2003 UIA COM Client API introduced in Windows 7 3

    4. What’s new in Windows 7? UIA Clients can now use COM instead of managed code New Accessibility properties are available Native clients can register custom ‘proxies’ to make existing UI more accessible Developers can define custom Accessibility properties and events beyond the official UIA list

    5. Inspecting UIA Properties demo

    6. Context: Clients and Providers

    7. Taking advantage of the new COM Client API

    8. SAMPLE: Querying a window’s password status CComPtr<IUIAutomation> spAutomation; CComPtr<IUIAutomationElement> spElement; HRESULT hr; hr = spAutomation.CoCreateInstance(CLSID_CUIAutomation); if (SUCCEEDED(hr)) { hr = spAutomation->GetFocusedElement(&spElement); if (SUCCEEDED(hr)) { BOOL isPassword; spElement->get_CurrentIsPassword(&isPassword); } }

    9. Automation Elements Just as in the managed API, an automation element represents a piece of UI, regardless of underlying implementation First, create an instance of CUIAutomation, the factory for the object model Next, get an element: From an HWND: ElementFromHandle From a Point: ElementFromPoint From current focus: GetFocusedElement Each method returns an IUIAutomationElement

    10. Basic Properties IUIAutomationElement exposes properties that make sense for elements in general Properties are exposed in COM manner and are consistent across all underlying UI

    11. Three Key Properties NameProperty String that a user would use to explain which control was being referred to (Customer: Accessibility Tools) ControlTypeProperty Primary identifier for what type of control the automation element represents. (Customers: Both) AutomationIdProperty String that uniquely identifies an element among its siblings in the tree (Customer: Test Automation)

    12. Control Patterns Control Patterns allow access to a control’s particular abilities InvokePattern: ‘Push Me’ TransformPattern: ‘Move Me’ ScrollPattern: ‘Scroll Me’ SelectionItemPattern: ‘Select Me’ Patterns can contain properties, methods and events

    13. Find Requests Find requests allow you to locate elements based on conditions Conditions are based on property matching They can include Boolean operators as well Requests can be restricted by scope Find can return the first match or all matches Faster alternative to manual searching

    14. Visiting the Relatives: TreeWalker All UIA elements are organized into a tree TreeWalkers allow navigation around the tree GetParentElement, GetFirstChildElement, GetNextSiblingElement, etc. Can filter by condition

    15. Consuming Events Enable clients to be notified when changes occur in the UI Client implements a listener interface Register your listener with CUIAutomation Clients can scope what parts of automation tree to listen for events on Applicable events are always associated with a reference to automation element where change occurred http://msdn.microsoft.com/en-us/library/aa359583(VS.85).aspx 15

    16. DumpUIATree sample

    17. Accelerate your access: CacheRequests

    18. Revisiting DumpUIATree sample

    19. New properties and patterns

    20. Patterns for Control Virtualization Allows controls to have more children than are exposed in Automation tree at one time Parent container permits searching for children by property Children can be de-virtualized upon request Useful for: Containers with massive numbers of children

    21. Other new properties and patterns

    22. Building great UIA Providers

    23. When do you need a provider? You don’t always need to write a provider. Customizing beyond your framework Changing perceived properties Annotation API may be sufficient Adding new abilities Full custom controls Creating new frameworks Silverlight had to implement Accessibility

    24. Implementing the simple interface Create a COM object that implements IRawElementProviderSimple Retrieve VARIANT properties Implement the most important ones for your control, especially Name and ControlType Retrieve a pattern implementation Connect your COM object to the outside world through WM_GETOBJECT

    25. Implementing patterns Control patterns are much like interfaces Can reside on same object or different object from Simple interface Implement patterns that reflect your control’s abilities

    26. A basic custom-control UIA provider sample

    27. Beyond Simple Elements Representing children Free support for mirroring HWND tree When you have non-window children … Implement Fragment for all nodes in your tree Implement FragmentRoot for the root of your tree Firing Events If a user would notice a change in your control, you need to fire an event You may check for listeners first

    28. Custom patterns and properties Allows client and provider to exchange data beyond original Accessibility design Custom work must be done on both sides Properties and events: business as usual Methods: extra marshalling work required

    29. Custom ‘proxy’ support Enables a client to register a substitute provider for a UI that lacks a real provider Proxy implements UIA interfaces on behalf of UI Useful for legacy code that cannot be modified but must be made accessible

    30. Wrapping it up

    31. Is this all of Accessibility? Is your software usable with … Programmatic access? Keyboard-only access? High contrast / high DPI? Slow reaction times? Accessibility is not a yes/no question: it is usability for a specific audience http://msdn.microsoft.com/accessibility

    32. What about managed clients? The new COM API was designed with .NET interoperability in mind Pro: Fastest performance for programmatic access Con: Imported COM interfaces are not identical with older interfaces Con: Some Win32 controls have slight behavior differences

    33. Managed client option #2: Use the existing managed library We continue to support the existing System.Windows.Automation namespace Pro: No change to existing code Con: May be slower when accessing MSAA applications Con: Cannot access new properties and patterns with current version

    34. Call to Action Use the new UI Automation Client API to interact with existing user interfaces Create UI Automation providers for new custom controls and UI frameworks Consider the Accessibility needs of your customers when designing applications

More Related