Working around the brittle UI Virtualization in Windows 8

posted by Craig Gidney on November 20, 2012

When doing UI programming, having to show a lot of results fast is not uncommon. I recently ran into this show-many-results-fast problem as part of implementing the search functionality for the Element music player. I found that although search results could be computed quickly, getting those results displayed quickly (within a windows store app) was a huge challenge. This post explains the obstacles that make showing a lot of results difficult in windows store apps, and also outlines the solution I ended up using. I’ve also uploaded an example solution to github.

The Solution that is the Problem: UI Virtualization

There is a “correct” (huge emphasis on those quotes) way to show large collections of controls in windows store apps, recommended by msdn: UI virtualization. UI virtualization is a feature inherited from WPF, and you can try it out by creating a new C# WPF project called ‘LayoutTest’ and pasting the following code into the appropriate files:

<!--MainWindow.xaml-->
<Window x:Class="LayoutTest.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="350" Width="525">
    <ListView x:Name="items">
        <ItemsControl.ItemTemplate>
            <DataTemplate>
                <Slider Value="{Binding Mode=OneWay}" Minimum="0" Maximum="1" Width="300"/>
            </DataTemplate>
        </ItemsControl.ItemTemplate>
        <ItemsControl.ItemsPanel>
            <ItemsPanelTemplate>
                <VirtualizingStackPanel />
            </ItemsPanelTemplate>
        </ItemsControl.ItemsPanel>
    </ListView>
</Window>
//MainWindow.xaml.cs
using System.Collections.ObjectModel;
using System.Linq;
using System.Threading.Tasks;
using System.Windows;

namespace LayoutTest {
    public partial class MainWindow : Window {
        public MainWindow() {
            InitializeComponent();
            this.Loaded += async (sender, arg) => {
                var data = new ObservableCollection();
                const int n = 10000;
                items.ItemsSource = data;
                foreach (var i in Enumerable.Range(0, n)) {
                    data.Insert(0, i/(double)n);
                    await Task.Delay(1);
                }
            };
        }
    }
}

When you run the project containing the above code, you should see a list of sliders that updates smoothly as values are streamed into the observable collection being used as a data source. Very nice. Very easy.

But things are not so nice in a windows store project. Doing the exact same thing results in something best described as “completely broken and useless”. Basically, you get a screen full of non-responsive teleporting sliders:

Virtualizing gone wrong

It really can’t be stressed enough how unusable this is. Inserting new items at the start or in the middle of a virtualizing control (like what you do when streaming sorted results) in quick succession is currently a very bad idea. Even appending results to the end is far too slow, although at least it doesn’t have such erratic behavior.

Approaching the Problem

I lost a lot of time because of UI virtualization being broken. Enough that we actually had to publish a version of Element with a search that slowly streamed results and truncated them after 250 items because anything more was simply non-responsive (the corrected version is still in never-ending approval limbo at the moment).

Because the recommended approach did not work, I ended up trying a lot of different approaches. The obstacle that consistently thwarted these attempts is the fact that many UI operations are extremely expensive. For example, constructing user controls is expensive (especially sliders), adding or removing controls from a container control like a grid is expensive, and even re-assigning the content of a blank user control is expensive. Profiling showed that it took up to a second to do ten thousand simple content changes.

Profiling ten thousand UserControl content changes

If you do any of these expensive operations more than a few hundred times without yielding the UI context, you’ve just created a very noticeable pause (>50ms). Of course all of these operations are things you tend to do a lot when trying to show lots of sorted items. As a result, searches took too long to get all the results on screen and there were hiccups (small pauses) as users typed to refine them. Eventually I realized it was time to stop trying to play nice and start blatantly cheating.

Implementing a Solution

Two UI operations that happen to be cheap are repositioning controls and changing the visibility of controls. If a control is already constructed and invisibly added to a container, we can quickly move it into a desired position and make it visible. This is useful because controls in a list often all have the same type. Since only a subset are visible at any one time, we can re-use controls as they exit the viewable area. In other words, we can implement our own specialized but limited UI virtualization. We just need to keep a cache of invisible already-in-the-container controls ready to go, and be able to efficiently determine where they need to be positioned. (I call this “cheating” because it’s likely to break functionality that would normally be implicitly supported, like accessibility or tabbing through results.)

Efficiently determining where controls need to be positioned is not trivial. It requires fast insertion of new items, deletion of stale items, aggregation of layout positions, and querying of ranges of items by those layout position. Just storing the items in a list won’t work here, because we’ll end up visiting half the list every time an item is changed just to reposition items or update the layout positions. We need something better, with guaranteed fast insertion, deletion, aggregation, and querying. If only there was some construct with all of these properties, covered in every data structures course ever… like balanced binary search trees.

A search tree can do all of the operations I’ve mentioned in logarithmic (plus output size for queries) time. Squaring the number of items in a search tree only doubles the amount of processing time. In more concrete terms that means, after the data is in the tree, laying out a million items will only be four times as expensive as laying out thirty three items (ignoring factors like locality and caching).

Unfortunately, as far as I know, .Net does not include a search tree that supports aggregation or querying against those aggregates. Implementing a custom balanced search tree might be a well-defined task, but they’re notoriously tricky to get right. Fortunately, I happened to have previously almost finished implementing and testing a red black tree with persistence in C# in my spare time. I had to modify it to add support for aggregation and querying, but that was trivial compared to getting the insertion/deletion cases right. The source code for the persistent aggregating red black tree is in the example project (note that it’s not optimized for raw speed, because that ended up not being necessary).

The search tree is really the hardest part of doing your own IO virtualization. The remainder is boring to describe because it’s so straightforward: define some interfaces to represent virtual controls, implement a simple cache that avoids adding or removing controls from a container, have a method to position the controls based on the tree’s state, and package it all up into a user control.

With our custom virtualizing control, we can finally do some fast searching. We just have to wire it all up, as is done in the main page of the example project. If you run the example project, you should see something that behaves like this:

Example searching with custom virtualizing

This is actually usable. Even though I’ve omitted a few tweaks I later added (like showing stale search results for 100ms when the user types in order to avoid a flicker of ‘no results’) and the search tree is in no way optimized, both scrolling and refining searches are fast and responsive. No hiccups. No pauses. Usable.

Summary

The UI virtualizing in windows store apps is broken, or at least brittle. You have to implement your own instead, while avoiding many operations that seem promising but are too expensive. I did it by aggregating and querying layout heights with a search tree, and re-using controls that had already been invisibly added to a container. The result is quite nice, in my opinion (see the example project on github).

Hopefully, I haven’t made a fool of myself by completely missing the actual right way to show lots of controls. Clearly there are apps out there that already show lots of controls. But, even if the comments are embarrassing, they’ll at least be enlightening.

Discuss on Reddit

  • kns98

    I just happened across your blog. It is two years later but MSFT has not fixed the virtualization completely. I found this link

    http://msdn.microsoft.com/nb-no/library/dn465797.aspx

    but it still fails to render after ~100,000 records. I am giving your code a try but I have not wrapped my head around it completely. I am wondering what license you offer the code under.
    Regards :)
    Kevin

    • CraigGidney

      Sorry to hear that. I added an MIT license to the repository.


Twisted Oak Studios offers consulting and development on high-tech interactive projects. Check out our portfolio, or Give us a shout if you have anything you think some really rad engineers should help you with.

Archive

More interesting posts (10 of 33 articles)

Or check out our Portfolio.