XAML - Designing for Automation - Exposing ControlTemplate (s)

Application design and development is an art. It requires to be adept in tools of the trade. For Xaml technologies, the better you are in writing XAML and underlying framework, the better designer you are. For developing WPF applications, very commonly, we use custom controls, control templates and data templates but they make it really difficult to automate our applications. I thought it would be a good idea to write a two-part series to discuss the roadblocks of automating UIs and their possible solutions. This is the first part of the series. Here we are discussing how to expose elements from a ControlTemplate.

For this post, we will be assuming that you have Windows SDK installed on your machine. We would be using UI Automation Verify tool from the SDK. You can download the SDK from Microsoft's download page.



Let's consider a very simple application. It just consists of a button shown in the middle. The requirements suggested that it has to be elliptical and painted as green. It has to have some text at its center with some border around. In the bottom, it should have a copyright text. As ugly as it can look, it is shown as below :)



This is the XAML used for creating the above window. Here we are using a custom button. We are not adding anything else to the button but we are setting some properties. The underlying button should use them.


Since this greatest button is not available off-the-shelf, we need to update the existing button's ControlTemplate. Here we are adding Ellipse, Border and TextBlocks to the template. In order to respect the properties defined in parent control, here we are using TemplateBinding.


Here UIAutomationButton is derived from Button. Let's keep its definition empty for now. Let us run the application now. When we try to view the hierarchy in UI Automation, it is surprising to see that we see no elements under button. We know that they are in the ControlTemplate. Now your automation guys would complain that they cannot see any elements under the button, so they cannot automate this application.



Now previously we have discussed that Microsoft has provided UIAutomation API for nearly all of its UI technologies including WPF. For exposing elements for automation, we introduce AutomationPeer for elements. Generally, there is a 1:1 correspondence between a control type and its automation peer. It is the responsibility of control authors to provide automation peer for the controls for automation. We have discussed about AutomationPeer here [Discussion about AutomationPeer].

In order to expose the elements defined under the control template, we need to introduce a new AutomationPeer for the new button type. The type inherits from ButtonAutomationPeer defined under System.Windows.Automation.Peers namespace in PresentationFramework assembly. In order to provide the automation peers for its children, an AutomationPeer needs to override GetChildrenCore method. The method returns a list of child automation peers for its children. Now don't go recursive, it just needs to do it for its immediate children.

Just look at how it is finding the expected children from Owner's Template using FrameworkTemplate.FindName() method [MSDN].


There are no AutomationPeer(s) provided for the types which are not expected to be automated. They include shapes , decorators and panels [MSDN Reference]. In this example, I have included all three of them i.e. Border(Decorator), DockPanel(Panel) and Ellipse(Shape). It is possible that we want to automate and verify some of them. In the above example, we have assumed that we don't have to expose DockPanel but we do need to do it for Border and Ellipse. Since there is no custom automation peer for them, we have just used FrameworkElementAutomationPeer for Ellipse.

Since we have to expose a child TextBlock from Border, we need to define a custom AutomationPeer for border. Here we have defined UIAutomationBorderAutomationPeer for this purpose. Since the TextBlock is directly a child of Border, we can access that using FrameworkElement.FindName() method [MSDN].


Now the control needs to tell automation framework about AutomationPeer it prefers for automating itself. It does so by returning an instance of the intended automation peer in an overridden OnCreateAutomationPeer() method [MSDN].


Now let's run it again and try to see how it has affected the hierarchy for automation. Here we notice that although the Ellipse and Border is available but there are no TextBlock(s). Now we know we have two TextBlocks in our design. But why are they not visible here?



Actually TextBlocks are not always visible for automation. I have no idea why Microsoft decided to hide TextBlocks from automation when they are part of a ControlTemplate. They are visible only when they are part of a ContentPresenter in a DataTemplate. Since Microsoft has opened up its Source code for easy reference, you can verify this from here:



Since TextBlocks are defined under a ControlTemplate, they are not available for Automation. In order to fix this, we need to add a new AutomationPeer for our TextBlocks. You don't need to provide a derived type for TextBlock. Only a child TextBlockAutomationPeer with overridden IsControlElementCore() should be enough.


We also need to update the other Automation peers to use this instead of TextBlockAutomationPeer wherever we need to expose TextBlocks.


Now the UI Automation Verify should be able to show the hierarchy as follows:



Now we just need to assign AutomationIds to these controls in order for them to be automated. Zindabad!!!

Get It from GitHub



What about Template Parts [Part_SOMETHINGS]
Now let me give you an exercise to practice what you have learned here. As an example we have this Custom ListBox with Header as a TemplatePart. We are naming it as Part_Header.


And we have the ControlTemplate for the ListBox as follows:


We are using the HeaderedListBox as follows to create our UI.


But when we look at the UI Automation Verify. The whole hierarchy under the list box is empty.



Let’s see if you can create an automation peer for this. Keep me posted.

Simple App Orientation for Windows Store Applications With Prism

In this post we are trying to focus how we can control application state for an App. We have chosen Windows Store Apps as the platform of choice. It provides SimpleOrientationSensor as the device orientation changes.

Let's start with PCL
Let's create a simple Portable class library project. We are naming it as SimpleOrientationEx.Portable. The project is supposed to hold the orientation state type. The application state would be based on an enumeration type, also defined in the same PCL project. For this simpleton project, let's use the maximum numbers of target frameworks.



These are the simple possible orientations of the app. It includes portrait, landscape and their contra-states. It also has face-up and face-down states. These are the states when the device is held in a perfect horizontal position either face-up or down. As a developer, you might have realized that it is really difficult for an app to determine how the device is being held, and should the display be using portrait or landscape layout.

Now let's introduce an interface type to hold the current application orientation. It holds the state in a CurrentOrientation property. We are naming the interface-type as IAppOrientation.

Let's implement this now. In addition to implementing the CurrentOrientation member from the interface type, it also introduces a protected member to update the current orientation of the App.


Windows Store App Orientation
As an example application, let us consider Windows Store Apps for Windows 8.1. We are creating a specialized type of AppOrientationBase class to hold the orientation state of a Windows Store App. The specialized type is named as AppOrientationWindowsStore.

Windows Store Apps can use SimpleOrientationSensor to determine the device orientation. The sensor type is available in Windows.Devices.Sensors namespace. It provides the application orientation using enumeration type.



The enumeration type has similar values as our AppOrientationState. We can define a mapping between the enumeration values of AppOrientationState and SimpleOrientation. Here MapOrientation() method is defining this mapping for us.

Let's introduce Reactive Extensions
Here we are adding reactive extensions core nuget package [Reactive Extensions Homepage]. Not only this makes throttling these events a child's play but also provides the events as Obervable.OnNext() sequence. This can be achieved using Obervable.FromEventPattern().

Since updating the layout for orientation change can be very involved process rendering heavy load elements, let us throttle the orientation changes a little. Here we are throttling the OrientationChanged event from SimpleOrientationSensor for one second. The throttled events are not raised in the publisher's thread but a different thread pool thread. Since we are updating the UI in the subscription code, we can observe the sequence using SynchronizationContext.Current. This would execute the handler code in the required UI thread.


Prism for Windows Store App
Now let us add Prism for Windows Store App nuget package. We introduce the main view model for the App. It just has a string based property to hold some text data. The view model inherits from ViewModel type from Prism.

The view model keeps the App orientation. It instantiate the member using default SimpleOrientationSensor. Like other Windows Store App sensors, SimpleOrienationSensor singleton instance can be obtained using the static GetDefault() method.

Prism for Windows Store App provides the support of restoring the application state using RestorableState attribute. We are decorating the only property with this attribute. This should restore the application state when application is restored to Running from Suspended state. [MSDN: App Lifecycle] [Handling Suspend, Resume & Activation in a Windows Store App]



Prism for Windows Store App also provides MvvmAppBase type. We can inherit from this to further control the App. Here we are overriding OnLaunchApplication and OnNavigationFailed methods. Here OnLaunchApplication is being used to navigate to MainPage.xaml as the app is launched. Based on the convention, we must have [X]Page.xaml. Here [X] is the first argument to the NavigationService.Navigate() method. We can override this mapping by overriding GetPageType() method. It is passed with the same string as pageToken and it should return the mapped Page type.

Now we add an IValueConverter to convert the enumeration value to string.


App Orientation and View Updates
Now we turn our attention to the view. We have a simple view with a TextBlock and TextBox. They are bound to the same property in the view model. Here we are using Prism's view model auto-wiring capabilities. The view model should be named following the Prism's convention. For our case, it is named as MainPageViewModel [ [X]PageViewModel ].

Here we have also introduce a RotateTransform for the Grid to the RenderTransform property. We are defining the center of the grid as the origin of the transformation by assigning RenderTransformOrigin as "0.5, 0.5".

Now we add Visual states to the page. All of these states correspond to different states in AppOrientationState enumeration. In order to keep the example simple, we are just animating the RotateTransform defined above for the Grid. It just rotates it keeping the view always upright. We are also defining transitions to these states to ease the rotation effect.


Introduce Behavior SDK (XAML) extension
Now we need to define the transitions between these states. These transitions should be based on the states in IAppOrientationState in MainPageViewModel. This is very tricky in Windows Store Apps as there is no DataTrigger support for the platform. In Visual Studio 2013, Microsoft added support for some behaviors including DataTriggerBehavior. Let us add the extension SDK using Reference Manager dialog.



In addition to DataTriggerBehavior, we are also using GotToStateAction from the same extension. This can be used to transition to a particular visual state.


Get Code from GitHub