Tuesday, October 24, 2006

Disappointing

The responses to my latest blog have been hugely disappointing. I thought about ignoring them but since they might be representative to more than just one individual I decided to address them in a whole new blog entry. In no particular order:

Q: Results show that Cairo is bad and slow, is that true?
A: Of course not. That was never what I wanted to show or ever implied. It's so disappointing to see people being so negative and then putting me in the middle of it all. Let me explain using a simple analogy: when two great soccer teams are playing, lets say Brazil and Italy and Brazil wins 3:0, it doesn't mean that Italy is horrible. It means that Brazil was really good at that particular day, a lot better than Italy. Surely before the next game Italy will be better prepared. Graphics is difficult. Trust me I do it every day. One graphics framework being better at something doesn't imply anything negative about the other. All it says is that the framework that was better at something, is really good at that something. Qt is really good at rendering polygons. _Really_ good.

Q: Those tests don't mean anything for most use cases which is rendering images and rectangles!
A: Right... Of course frameworks that focus on those are not vector graphics frameworks. Writing a vector graphics framework that is not good at rendering polygons is a little silly. One might as well stick to pure raster graphics with images/rectangles and it's going to be a lot easier. I'd love to see KDE move towards dynamic, resolution independent interfaces. For the things we're planning, polygon rendering is representative of exactly what we're doing.

Q: Cairo will catchup and it will be better! It's actively developed.
A: That's great. Of course, it's not like the whole graphics team at Trolltech is being fired because Qt is so good there's nothing to be done anymore. We work on graphics every single day so you have to expect Qt to be getting rapidly better.

Q: You will not repeat the benchmarks when Cairo or something else is faster!
A: Right, because that would imply that I have to update and compile every other framework our there every single day and repeat the tests every single day. There's no way I could physically do that (or even be willing to waste my time like that every day). It's up to the framework developers. If you work on a framework, test it after you make some changes and let me know when something changes. If anything will come even close to Qt I'd love to see it. I'd love to stop having to compare Qt to Qt when I do tests. It'd be great if someone would make my life a little bit more challenging.

Q: Tests have to be unfair!
A: What are you basing this on? You either trust me as an Open Source developer or you don't. And if you don't then don't waste mine and yours time by reading my blog. The code for Cairo benchmark is at http://ktown.kde.org/~zrusin/examples/cairorender.tar.bz2 if you think it's unfair, feel free to send me fixes (and don't be silly by trying to render polygons to a cached surface and then repeatedly composing the surface, like I said the tests measure polygon rendering, not image blitting).

Q: Ha! Qt is not Open Source, Cairo is the standard!
A: Not even mentioning that both of those are wrong, how does that relate at all? I am first and foremost an Open Source developer. Yes, I do prefer to write desktop code in C++ than C which was the basis for me joining KDE in the first place, but that's all. If I'll decide to quit Trolltech at some point and will get an offer from company supporting GNOME, I'll start contributing to GNOME the next day. It's all Free Software, that's what's really important to me. Qt is Free Software, that's why I work on it. I'm an Open Source developer, I just happen to work on KDE. I moved half way across the globe, to a country where I knew I'm going to be hungry all the time, just to be able to work on Free Software full time so please skip my blog if you want to bring something as ignorant as that here.

Q: Give me the code?
A: Why? I gave you the dataset and I told you what to test. If you question my results, go write an application rendering polygons and see for yourself. It's an absolutely trivial operation. If you can't do that then what's the point of you fetching my code? The bottom line is that if you don't believe me already, you won't either way and the whole discussion is moot. The people who cared knew where the code was and if you'll look at the date you'll notice the code was there from before I posted the blog (the link to the code is two paragraphs above).

Q: The test is not representative of anything!
A: Of course it is. What would be the point of me writing a test not representative to anything? Like I said in the second paragraph the test is completely representative to the things we want to do in Plasma. If your application renders SVG's, Flash, fonts as paths, complex shapes or actual vector shapes, the test is fully representative of the results you'll see. But again, it doesn't imply anything negative - so it's not that Cairo will be very slow in those cases, it's that Qt will simply rock them.

Q: But what if something else would be faster than Qt? Would you post results then?
A: Well, I don't exhibit self-destructive behaviour so no, of course not. I'd fix my code to make sure it's faster and then I'd post them. This is how this works. If there's anything, and I do mean anything that Qt is slower at than any other vector graphics framework, be it Microsoft's WPF, Java2D or Cairo we'll fix Qt and make sure it's either faster or at least at the same level. Like I said, I'd love to be able to test Qt against something else, rather than Qt itself. If you can provide me a benchmark that shows Qt slower at anything in comparison to any other vector graphics framework, I'll make sure that it's not the case for long.

38 comments:

Anonymous said...

Rather puzzled by the need to respond to any of these questions.

The questions are surprisingly superficial. Sorta shows the disconnect between quality techies like you, and the hordes desiring food and entertainment.

Anonymous said...

Well, I found it interessting to read about your benchmark. And I wonder what people seem to complain about. On the one hand people say the advantage of open source is the competition on the other hand they can not accept if there is something that doesn't fit to their "idiologies". Obviously the people Trolltech did a better job regarding this benchmark. So either the others learn from it, find really weak points of the benchmark which matter or they just accept it. Moreover I guess that the Cairo find such benchmark interessting. No point to be angry about I would say.
So Zack, thumbs up for you. Good work.

Anonymous said...

Well, I found it interessting to read about your benchmark. And I wonder what people seem to complain about. On the one hand people say the advantage of open source is the competition on the other hand they can not accept if there is something that doesn't fit to their "idiologies". Obviously the people Trolltech did a better job regarding this benchmark. So either the others learn from it, find really weak points of the benchmark which matter or they just accept it. Moreover I guess that the Cairo find such benchmark interessting. No point to be angry about I would say.
So Zack, thumbs up for you. Good work.

Anonymous said...

Well, I found it interessting to read about your benchmark. And I wonder what people seem to complain about. On the one hand people say the advantage of open source is the competition on the other hand they can not accept if there is something that doesn't fit to their "idiologies". Obviously the people Trolltech did a better job regarding this benchmark. So either the others learn from it, find really weak points of the benchmark which matter or they just accept it. Moreover I guess that the Cairo find such benchmark interessting. No point to be angry about I would say.
So Zack, thumbs up for you. Good work.

Anonymous said...

Keep up the exceptionally good, hard work! Comments either positive or negative, constructive or not, are just words and thus ... vaporware! The action is the only criteria upon which a person gets separated from the heap and might make history! Your entire distinguished contribution to the open source world has been proven invaluable numerous times!
A plain 'Thank you' is simply not enough to express my deepest gratitude to you concerning your work!

Anonymous said...

I'm sure the majority of people look at your work, think it's really cool and wonderful. But those people wouldn't have anything to write in a comment.

So just naturally you end up with only the critical critics posting.

So anyway, you rock :-)

Anonymous said...

I've read through all the comments to your other blog post - and to be honest: don't care about the negative ones!

These are only a few people who are not willing to see that you are actually in contact with Cario people (Carl) and who are not willing to realize that you tested a specific task.
And: most of these "critics" were the usual "KDE is bleh because KDE didn't do xyz and is against standards" - which is bullshit.

Most of your readers, and most of the people writing comments, love to read your posts about new and exciting developments in Qt, X or the graphics world at all. And these people thank you.

So: don't blame yourself, you did everything right. And don't think that the comments you've mentioned represent the majority of your readers - they just don't.

liquidat
(Who hopes that these possibilities will find their way into KDE 4!)

Anonymous said...

You ARE the true graphics ninja! I should have said it in your previous post. I just landed a carpet, pointing towards the northern europe and I'm gonna get down on my knees and pray to ensure an healty, fast and optimized life for you !! ;-)
As a developer I haven't dreamed more than 1% of the possibilities you're creating with this new vectorial graphics paradigm yet! Thanks man.

Anonymous said...

Excellent writing! Congratulations and thanks.

Anonymous said...

not sure which of the "Q"s was supposed to address my point -- thats it counter productive for free software to compete when it makes more sense to cooperate. (perhaps if you dont like people misinterpreting your words, you might go about trying to avoid running theirs through the blender.)

regardless, i think its fairly safe to say the real issue i have is not with kde or zack, rather the 12 year old fanboys that like to take sides and fling mud at other peoples work from their safe and secure sandboxes.

so, zack to you, best of luck with all your work. heres hoping you can make the pie bigger instead of dividing what tiny bits there are to have.

Anonymous said...

I believe in you!! :)

Anonymous said...

As said earlier, I just love reading this blog and see the amazing work. Although I'm a Gnome-guy, I've thought many times about learning QT instead of GTK after seeing the really nice work done.

However; I don't understand what you mean about being hungry here in Norway. I've just moved to Oslo to study, and I can't say I'm hungry although I don't have the money of a employed person. :-) Anyway, great to see someone is not thinking about bloating things up with features and throwing optimization and speed on the sea.

Anonymous said...

Don't let these, 'evil' people get too you. Your blog posts are fabulous, especially with the pretty pictures.

Keep up the work Zack.

Luke Parry

Anonymous said...

Digg it! http://digg.com/linux_unix/Exciting_V3ctor_Gr4phics_with_Qt_4_3

Anonymous said...

Thanks for the benchmarks.

QT is really great. It makes the other frameworks catchup.. hopefully.

Keep up the good work!

Anonymous said...

Zack, the passion you put in your amazing work is just inspiring. Needless to say, keep rocking!

Anonymous said...

Zack, I really enjoy working with the results of your efforts, using Qt in my commercial Windows-only program.

I can confirm that Qt 'even' outperforms the Windows GDI ;)

Great work!

(And by the way, I'm just making fun here, I like open source and am even working on it. Doesn't put bread onto the table tho ;)

Anonymous said...

Awesome post, Zack. Very nicely worded. I echo your thoughts and sentiments exactly.

cies said...

This struck me: "I moved half way across the globe, to a country where I knew I'm going to be hungry all the time"

I guess that has to be do with you being a veg. Okay let me give a positive swing to this: if you ever happen to be in Holland your invited to a massive vegetarian dinner. Since that is the way I'd like to welcome a "true graphics ninja from the team of Trolltech's Samurai Graphics Assassins" to my country.

Anonymous said...

Zack, the only thing I took from it is that Qt is many things without being many things weakly. Generalyl when you touch as many areas as Qt does, its called a platform. Qt is a phenominal platform for business apps, and network apps. But everyone who gets those right generally falls short of graphics capability. What these tests show is that TT nails yet another area. Even if Qt was slower, but only marginally so, I'd still consider Qt a huge success. The fact that even if you cherry picked (or maybe you didn't -- it doesn't matter much) means that at least in some areas Qt is a contender, and that proper emphasis is being put on optimization (maybe not everywhere yet, but it shows it is a goal , and done significantly).

In short, I have nothing but wonderful things to say about you, your blog, Qt, and TrollTech. Don't let the nay-sayers detract or distract you from doing more great work.

Zack said...

Thank you all very much. Those comments made my day.

Anonymous said...

I love you Zack! Seriously, I love your work and Qt. Qt is proven to be a very powerful framework for developer. Those trolls (may I say so?) maybe just came to open source world so they got some culture shock! ;)

Anonymous said...

Yo Zack, you do rule! And this if from a gnome/gtk person :-)

What really matters to me is to beat the proprietary guys, so, do you happen to know or have interest in knowing how does qt/arthur stand against java2D and both microsoft's and apple's latest offerings in the area?

Thanks for all your work!

Rui

Anonymous said...

Seems Aaron wasn't completely right: constructing criticisms

I was doing exactly as he suggested: I ignored the negative remarks (difficult, very).

But what Aaron (and I) forgot is that sometimes there is someone who is the target of those remarks. At those moments you have to stand up and say something to balance out the negativism.

So I join myself to the chorus: nice job Zack! Love reading your blog. It gives me a warm fuzzy feeling knowing that KDE will have all this goodness in the future. (Still looking for that itch to scratch so I can join the KDE developer league, but for now I'm happy to be "just" a user)

Anonymous said...

Anonymous said:
"thats it counter productive for free software to compete when it makes more sense to cooperate"

Um, this code is Open Source under the GPL, so the Cairo guys are free to have a look see how Zack and the Assassins pulled it off and use their methods (if not the code, Cairo is LGPL I believe?).

I fail to see how the two wildly disparate toolkits (C vs C++, Qt vs Gtk, Glib vs Qt, etc) could do more co-operation then they have now with the free exchange of ideas that goes on at a personal level (you can see the same thing with the GEGL and Krita guys). These guys know how hard this stuff really is, and I see a lot of mutual respect and communication going on that the fanbois out there completely miss.

And we're humans after all, competition spurs us on to do better, which is what Zack talks about here.

Zack, dude, you rock, as does your work. I've seen a couple of projects now dump all their own hand-rolled canvas/vector code for your high-performance stuff and the developers are loving it as they can focus on their real task of building great apps without having to figure out all that real hard vector performance stuff. Keep it up.

Heh, "Zack and the Assassins", cool band name...

P.S. Hey Zack, do we need to organise an emergency care package of organic vege's to be parachuted into Norway for you? :-)

Anonymous said...

If anyone takes those benchmarks as anything other than an opportunity to make Cairo and others faster, they're just not going to get it.

Anonymous said...

Zack:
I love QT and KDE, personally, and your work on them is amazing. However, parts this particular post comes off as kind of... self-contractadictory.

"Let me explain using a simple analogy: when two great soccer teams are playing, lets say Brazil and Italy and Brazil wins 3:0, it doesn't mean that Italy is horrible."

This is really a terrible analogy. It begs the question... IS Cairo really a graphics framework capable of competing with QT? With soccer, the teams are mostly equal... they operate by the same rules, have the same goals, etc. But my understanding (which is poor, admittedly) is that Cairo and QT are two very different graphics frameworks. It's more like there is a soccer and a kickball team playing. They still do basically the same thing (kick spheres around into goals), but the rules they abide by and the entirety of the organization is entirely different.

"One graphics framework being better at something doesn't imply anything negative about the other."

But it does mean that it's slower. Which, in any graphics framework, is a bad thing. Which means that it DOES imply something negative about the framework. This may not matter much (GDI -- ugh -- is fast enough for a lot of applications, for instance), but it most certainly does imply a negative.

"Q: You will not repeat the benchmarks when Cairo or something else is faster!"

Then

"Q: But what if something else would be faster than Qt? Would you post results then?
A: Well, I don't exhibit self-destructive behaviour so no, of course not."

So the first question is entirely correct. You admit that you wouldn't post stats that would show another graphics library being faster than QT. It's for a good reason (it's better to spend time improving things than producing useless reams of data), but people do have a point here. Of course, if they don't like it they can (like you suggest) produce their own tests.

"Q: Give me the code?
A: Why? I gave you the dataset and I told you what to test. If you question my results, go write an application rendering polygons and see for yourself. It's an absolutely trivial operation. If you can't do that then what's the point of you fetching my code?"

This seems a snotty response, honestly. One of the foundations of science is repeatable results with the same setup and data. Sure they COULD write their own code, but that would mean they couldn't be sure that the same code paths are excerisized. It would be like not providing a methodology document in a scientific paper... there's no way to check the experiment on its own terms. Sure, one could take the hypothesis and write a new experiment to test it, but with different methodology different errors will creep in, making improvement and refinement in the test difficult if not impossible.

And then: "The code for Cairo benchmark is at http://ktown.kde.org/~zrusin/examples/cairorender.tar.bz2"

So code for testing Cairo is offered up anyway. Huh? Either you object to people testing QT by the same code you use, or you don't. This seems really self-contradictory. What makes the QT benchmark code so much more precious than the Cairo one that it must be held and hoarded?

Besides which, one of the best ways to learn to use code is by example. Your code would probably be the best, most up-to-date examples for QT.

Other than these spots, though, the responses to rather flameworthy questions were sensible. Once again, you're doing great work so please don't take this as anything other than friendly critique of your non-code writing. :)

Anonymous said...

I like the way you deliver information...

Very interesting thing that...

Hope Trolltech keep up the good work and doesn`t fall apart one day...

Anonymous said...

Here there is just truth, that is "YOU ROCK"

Anonymous said...

"What would be the point of me writing a test not representative to anything?"

Indeed; that's exactly what's so suspicious. The obvious (and cynical) answer is that you're excited about "Qt's new wicked tessellator" (case in point!) and want to show off what it's best at.

"Like I said in the second paragraph the test is completely representative to the things we want to do in Plasma." [sic]

Rendering a 100,000-vertex polygon is "representative"?

I've been doing graphics for over 10 years and I can count on one *finger* the number of times I've needed to render 100,000-vertex polygons (line map data).

I'm all for fast graphics and innovative UIs, but no existing or proposed UI (incl. radical ones like "lowfat") seems to involve rendering *any* 100,000-vertex polygons.

It's great that you made this corner-case fast -- congrats! -- but it's still a corner-case.

Anonymous said...

Strange that so many people get so agitated just because of this benchmark. So Cairo is not as fast as QT? So what!? This just proves that the programmers at Trolltech are doing a very good job and that there is a lot of room to improve for Cairo.
Since I mostly use GTK+ applications this is great news for me; my desktop may even get much better!

I don't think that any of the Cairo developers feels offended by the benchmark. So why do the users feel the urge to behave so unrational?

Zack said...

"Rendering a 100,000-vertex polygon is "representative"?"

For someone who claims to do graphics, that's a pretty ignorant statemant to make. That polygon is a wonderful testcase to see the actual complexity of your algorithms and check whether your algorithms degenerate. That's the purpose of the testcase, not comparing the actual perforamnce. All of that should be pretty clear from the charts, especially given that I emphasized what testcase is doing what. I mean, if you didn't feel like even reading my blog carefully then why would you even try to comment? (of course, I appreciate the comments, just try to actually read the blog in reference before doing so)

Zack said...

"But it does mean that it's slower. Which, in any graphics framework, is a bad thing."

No. "Being too slow to do X" is bad, "being slower than Y" is not. If you can render an application at 1% CPU usage at 60FPS in one framework and 1% CPU usage at 500FPS in the other, it doesn't mean a thing because noone would ever be able to see the difference.

"So code for testing Cairo is offered up anyway. Huh? Either you object to people testing QT by the same code you use, or you don't. This seems really self-contradictory. What makes the QT benchmark code so much more precious than the Cairo one that it must be held and hoarded?"

Of course if you actually went to that address you'd see the code for Qt tests, that's been there for a day longer than Cairo code (as date shows)...

Anonymous said...

Is image performance on X11 going to be improved in 4.3 as well?
I am currently playing with Qt 4.2 GraphicsView and on Unix/X11 when moving for
example a text element over an big image (say 1500x1500 pixels or more) is not very fast. On windows it seems to perfom much better.

Sree Kotay said...

Hmmm - so looking at the two benchmarks, I have to admit they don't appear apples to apples.

It appears in the Cairo case you're allocating and freeing the backbuffer EACH iteration, in addition to drawing, whereas in the QT case, you're just drawing the polygon?

Not sure - I'm not that familiar with QT rendering. I just signed up for an evaluation license to see :)

That said, very nice of you to release the code and data.

Sree Kotay said...

Also, the data set appears to have **4 ** polygons: the text, a small path, a big path, and a stroked square?

Anonymous said...

zack!
Its a blog!And as such it will OF course addict ALWAYS more negativ response then POSITIVE. Maybe if u would post something like: im giving u money all, freee as much as u want that would change.. may be even there some people would complain. Dont hurt your self too much :)
IMO ur doing a great job workinon such cool OS technology.
And ur blog is a nice place to get informed about it . So keep up the good work :)

Anonymous said...

I'm porting my app, Freecycle (freecycle.redsteamrecords.com) from Qt3 to Qt4 and I must admit that Qt4 rendering code is fantastic. I'm mainly using QCanvas architecture and what you guys were able to achieve with QGraphicsScene is unbelievable.

I have spend MONTHS coding the drawBackground() method in order to display the audio wave in the QCanvas.

With Qt4, I really just add millions of GraphicsItems and Qt4 handles all the scaling, zooming etc. Floating point geometry makes thigs so easy comparing to before, where I had to compute roundoff errors in order to display audio amplitudes at correct places!!

Thank you very much.

Predrag Viceic
http://freecycle.redsteamrecods.com