Multithreaded UI

Lets start this discussion with this assumptions:

It is desirable to have your user interface as threaded as possible.

Thus we guarantee less about the latency of the updates and more about the potential to display extremely intricate user interfaces on low-parallel machines.

First off, lets talk about levels of parallelism. First off, we are counting threads of execution (TOE), not cores:
low : On the order of 10's of cores (1 - 99 TOE)
mid : On the order of 1000s of cores (100 - 9999 TOE)
high : 10,000 TOE and up.

Currently, our machines exhibit low levels of hardware parallelism. Some big servers exhibit mid levels of hardware parallelism, and graphics cards as well as supercomputers exhibit high levels of hardware parallelism.

I want to think about extending UI implementations to low levels of parallelism from a single, or largely single execution model present today.

As a simple model, lets take a 3d view with an object along with a palette where the object's position is displayed. We have two different types of updates, where the user drags the mouse and when the user enters a discrete value.

Generically, we have some data model. We have two different views of the data model with most likely extremely different datastructures used to display the data.

We furthermore assume that rendering is only safe from one thread. This thread renders both views according to some schedule, either on dirty or as needed due to a framerate requirement. Breaking this assumption usually has extreme performance implications *or* it is not allowed (i.e. hard crash), at least with current rendering API's (opengl, .net, etc).

Starting from concrete and moving to generic, here is a design that has been slowly manifesting itself in my mind.

Per view, there is some translation from the model to some very view specific datastructures where the view's particular renderer iterates over them and takes over.

I propose that there is a thread queue where each view places its individual controller. So the entry points to these translation steps can all be started from parallel. The product of these translation steps is thrown into a threaded queue where the render thread takes over and iterates over them telling each view to update on its own accord.

In any case, something changes the base model. Now all the controllers need to have a chance to look at the model and decide if they need to produce a new render datastructure. This translation step can be done in parallel for each view. Furthermore, this translation step should translate the model quite far into view specific structures so that the render thread finishes rendering as fast as possible.

So, functionally, we have:

mutator->model, model->view structure(s), view structure(s)->renderer.

Ideally this design would allow for views that have simple translations to finish and render quicker. Thus a view with a particularly involved translation step to render wouldn't have the ability to throttle the application.

So, we can at least speed up rendering of complex applications with multiple views in a very basic sense without allowing an involved view to slow down the application. Each translation step should be further threaded if it makes sense for the amount of data and the problem domain but we haven't said anything about that yet so we can't assume anything about the specific translations.

UI's are not written with this design but if you want truly sophisticated graphics effects it will require allowing the machine more room to process complex views and components. Thus rendering the entire application cannot be held up by the rendering of a complex view and user interaction should not be noticeably slowed down by the rendering of a complex view.

Ideally you can also design a system where the more expensive views receive proportionally more compute time in a multicore system and this type of application design requires thinking very hard about your translation pipeline from model to view.

Chris