Benchmarks

A lot of people has been asking me about some performance comparison for the vector graphics framework we have. Rendering polygons, especially when we're dealing with stroke, tends to be the most expensive rendering operation performed in vector graphics. I constructed a little test, which tests raw polygon rendering power of Qt and Cairo.

For the test I used the latest Qt main branch, and the master branch from Cairo's Git repository.

The test is composed of rendering three complex polygons. The first one is a text path, the second is a small polygon with a large amount of vertices that fall on the same scanline and the third one is a huge polygon with about 100000 vertices.

The results measure frames per seconds at which each framework was capable of rendering the given testcase. Therefore the larger the better.

I tried being as objective as possible. All tests go through the whole pipeline, meaning I tried to make sure that the framework doesn't cache too much and actually renders what's being asked. I used latest version control code for all frameworks. The data used in the tests is available here: http://ktown.kde.org/~zrusin/complex.data (newline separated polygons whose coordinates are comma separated). All tests were written to utilize antialiasing.

Oh and all tests have been done on a machine with Pentium(R) 4, 3.20GHz processor, 1 GB RAM and NVIDIA 6600 with 1.0-9625 drivers.

Having said that the results were (charts follow):
First just pure Cairo vs Qt native performance:

Qt was respectively 7, 5 and 6 times faster. Than Cairo in those plain tests. This is a direct result of Qt's new wicked tessellator in 4.3.

But all the frameworks have many backends, the most interesting one being the OpenGL backend. For 4.3 I devised a new method of rendering polygons for the OpenGL, based on stencil clipping. So lets see how Qt's OpenGL engine compares to Qt's native XRender engine:

The difference is huge. Qt OpenGL rendered the first polygon at 487 frames per second vs 76 in Qt XRender engine. Note though, that in XRender we tessellate which is of NlogN complexity, in OpenGL I'm able to render polygons in a completely linear fashion. The method does deteriorate for polygons with 80000+ vertices due to large amount of triangles that has to be processed - it's a GPU bottleneck though which means that with a more powerful graphics card those results would skyrocket.

Finally, lets combine the results of all the frameworks and see how they match-up:

The reason for Cairo with Glitz backend yielding the same results as Cairo with XRender backend is that polygon rendering in both of those goes through the same client-side steps all the way until the final blit and it's not the blit but the tessellation and rasterization that are the bottlenecks. I added Amanith to the results because some people mentioned it in my blog before. Both Amanith and Cairo (Cairo only with native XRender backend) crash on the last polygon. In Amanith the tessellator seems to fall apart. In the Cairo case application crashes in the XRender code, so most likely rasterization code is not keen on one the trapezoids that Cairo sends it. Cairo with Glitz backend render the last polygon at about 0.2 frames per second (but doesn't crash which again shows that it's likely XRender's trapezoid rasterization code, especially that Carl couldn't reproduce the given crash on his laptop). Interesting fact right now is that Qt with XRender is way faster at rendering polygons than Amanith and Cairo with Glitz, both of which are OpenGL accelerated.

Notes: I know Carl is working on a new tessellator for Cairo which should exhibit the same logarithmic behaviour as the current Qt one. Carl was kind enough to even send me a tarball of the branch in which he's working on it. Unfortunately, although the results for the first polygon were at about 13 FPS (2 frames per second better than the current Cairo tessellator) they were degenerating for other two polygons. This is most likely due to large precision of the new tessellator (in both #2 and #3 testcases you get vertices close enough to consider them coinciding without any visual artifacts). Once Carl will get the precision down and integrate the new tessellator I'm going to run the tests again.

Conclusion from all those tests is that right now Qt is leaps ahead of any other vector graphics framework in terms of raw performance. Nothing comes even close. Qt's OpenGL engine is so fast it's basically unfair to compare anything else to it.

Objectivity aside, Qt rocks. It really does. And if you're using Qt and not using Qt's rendering architecture, everyone should point at you and make fun of you for not having complete and utter trust in me, as the only true graphics ninja and the team of Trolltech's Samurai Graphics Assassins.