During the last three weeks I've spent most of my spare time writing a GUI for Jose's amazing ApiTrace project. ApiTrace is a project to trace, analyze and debug graphics api's. Both OpenGL and Direct3D. To some extend inspired by gDEBugger and Windows PIX. We wanted a tool that would let us slice through huge games and CAD apps to the exact call which causes problems and be able to inspect the entire graphics state, including the shaders, textures and all the buffers. We ended up doing that, plus a lot more and we're just getting started. In other words it's the best thing since "Human/Robot Emancipation Act of 3015".
You begin by tracing your target application. You can do that either from the console or from the GUI. A trace file is created and we can do some amazing things with it. You can open it in a GUI and
- Inspect the state frame by frame, draw call by draw call:
- Replay the trace file:
- Check every texture:
- Every bound framebuffer:
- Every shader:
- Every vertex buffer:
- You can see if OpenGL threw an error at any point during the replay and if so what was it:
- And to go completely nuts, as graphics developers like to do, you get to edit any shader, any uniform and large chunks of the state to immediately see the effects it would have on the rendering:
As a driver developer you no longer have to install all the games just to debug a problem, the report can simply include a short trace which you can use to immediately figure out what's wrong. As an application developer you can inspect every graphics call your app makes, you can analyze your api usage and you could automatically produce standalone testcases which you can send to driver developers.
ApiTrace is hosted on github and it's BSD licensed. It works on Linux and Windows (we're planning to add OSX support as well). Gui is written using Qt and requires the QJson library.
Jose just announced the first release so let us know if there's anything that would make your life a lot easier. Next is support for multiple GL contexts, ability to export just a single frame from a trace (either as a trace file or a standalone C application), ability to start and stop tracing on a hot key and lots of other features soon. So whether you're a driver developer, working on games, CAD apps or 2D scene-graphs this is a tool that should make your life significantly easier and better.
40 comments:
Judging from the screenshot of Unigine Heaven, I suppose ApiTrace supports OpenGL >= 3.0? That'd be really cool!
Seems to be a nice tool! Are there any prebuilt binaries to download and try? I found none at GitHub.
There was already an OpenGL debugger, but ApiTrace seems much better:
http://www.opengl.org/sdk/tools/BuGLe/
Seems awesome!
Have to hope game developers dont go ape-shit over sharing small debug traces that feature their 'proprietary' game code.
Sounds very cool. Has anyone tried thisn on KDE KWin already?
Truly amazing tool.
I just started testing it fifteen mintes ago, so I'm not sure whether or not this would be possible given it's archtecture, but the ability to inspect the contents of non-bound objects (textures, shaders, etc.) would be extremely useful.
@A.: Yes, everything up to latest 4.1 should be working.
@R-ride: Unfortunately not yet.
@slasdotaccount: Yes, there was a few, the README in apitrace lists the most important ones.
@johndrinkwater: the engine code is safe, the tool decomposes everything and traces only graphics api calls.
@Tsiolkovsky: compositors won't work right now because some of the GLX extensions haven't been implemented, in particular GLX_EXT_texture_from_pixmap. It would be great to see someone who cares about those to step up and implement them.
oh now that looks very handy, good work!
I was able to build glretrace.exe and tracedump.exe - I have a Qt 4.7.1 dev environment but I don't see how to build the gui portion. Same set of CMake commands in the gui dir? That fails immediately for me with a warning about qt4_automoc.
Something wrong on my end or different build instructions?
I only have Visual Studio 2010 - seems like I can't build it using that?
@Anonymous: sounds like something on your end. You shouldn't be running cmake in the gui directory. Just like the instructions say you just run it in the top-level directory. The gui build is simply skipped if either Qt or QJson aren't detected, which is probably the case for you. The output of cmake will tell you which one it is.
@SomeGuy: you can. In fact you could probably even generate VS project files from cmake. There's lots of tutorials online about how to use cmake with visual studio, they should get you started. Of course you still need Qt and Qjson.
Very very nice tool for debugging and optimize gl pipeline. Good work !!
Tested with F15, was very easy to build and use, and fascinating to explore. For anyone using fedora, the prerequisites include (i.e. maybe there are more I already had installed):
yum install qt-webkit-devel qjson-devel
I've got it built for Windows and the game I'm developing runs the logo screen fine (which is basically just one quad), but when it transitions into the actual game (which initializes shaders, FBOs, etc) I get a crash. The generated trace seems fine up till the crash.
The crash happens on line 976 of apitrace\zlib\deflate.c
Anything I could be doing wrong to cause this? Also, is there a better place to bring this issue like a forum?
@sinoth: Unfortunately we haven't even thought about forums yet. For now upload exactly what you're doing and a stack trace as a paste (ideally to https://gist.github.com/ ) and open a new issue on the "issues" page at:
https://github.com/apitrace/apitrace
with links to those.
Any chance we get built binaries for windows?
Understood. CMake finds Qt but not qjson.
Where should the qjson files be so CMake can find them? They are currently in "C:\program files (x86)\qjson" - I'm using Win7/64 - I've tried putting them in just "C:\Program Files\qjson".
Btw, when was the last time you changed your desktop wallpaper? :D
I just noticed that you're using the same wallpaper that you mentioned as your favorite in a blog post from 2007:
http://zrusin.blogspot.com/2007/02/core-dump.html
This looks like an incredible tool. I look forward to getting it out next time I do any OpenGL programming.
Outstanding work.
This is probably the best thing ever to happen. I just debugged and fixed a bug in Qt in *minutes* with the help of the GUI :) Thanks!
This looks like a really useful tool.
Would be great if you could provide prebuilt exe's (Qt and I aren't on friendly terms =) )
Sorry for hijacking this Blog entry.
I know you are a very busy programmer. Nonetheless I would like to ask you for a status update on the clover branch of mesa. The last git commit was from 2010-11-26. Is the Clover project still alive? If yes: is there any sort of roadmap? I have been thinking about using OpenCL, but I would be glad to have an implementation that is not vendor specific and that is free and open source software.
Hi Zack. ApiTrace looks pretty awesome, and I'm especially impressed by the GUI.
As it happens, Nokia recently open sourced a very similar set of tools for capturing and playing back graphics API call traces. The project is called Tracy, and the code is at
https://gitorious.org/tracy
(Apologies for the bare-bones content at the moment). The Tracy core is API independent and there are front-ends for OpenGL ES, OpenVG and EGL.
@victor: Maybe for the next release.
@Max: hah, yea, sad. What can I say, I dig that one, it's dark! Or more specifically I haven't had the time to look for a new wallpaper in a few years now.
@Anonymous: Clover (it's really called Coal, just that the project page hasn't been updated in a while) needs lots of tedious work to happen in Gallium first. I'll try to write a status update about that stuff soon.
@Eskil: Awesome!
@Sami: Ah, so what you're saying is that now you can abandon it and use ApiTrace :) Also, OpenVG? Was your abacus tracer not good enough? ;)
@Zack: Mm, I think we should let all the flowers bloom :) Besides, I think Tracy has a couple of tricks up its sleeve that ApiTrace lacks. For instance, you can convert trace files to C code to generate benchmarks that most of the time run as fast as (or faster than) the original app.
As for OpenVG, well, let's say the times were different back in 2006 :)
@Sami: apitrace can do that, you just run the retrace program in the benchmark mode. Generating c program is pretty trivial, in fact you could almost tracedump > app.c right now but I don't think it has any benefits whatsoever over the current retracing method unless you can extract a single frame from a trace. In which case it could be useful to make standalone testcases but I find the advantage of that over retracing still highly dubious.
very nice tool!
while trying it out, I found out that glretrace doesn't seem to show texture data if it is attached to a framebuffer object by using "glFramebufferTexture"
@Zack: At least on the embedded side interpreted trace playback turned out to be too much of an overhead, which is why we went for C code generation. You are probably right that on desktop hardware the advantage is not that great.
Tracy can extract a single frame from a longer trace while maintaining just the needed state setup calls prior to that frame. I'll be the first to admit, though, that this feature has not received enough real world testing.
@Sami: tbh I find that a little surprising. If a cost of a hash lookup of a function call via its name is a bottleneck then your graphics cods is pretty good. Either way I find it a little disingenuous to be comparing Tracy and ApiTrace. At least until ApiTrace supports EGL or Tracy supports full OpenGL and GLX/WGL and can trace across various forms of dlopen's that apps use (that part could be trivially lifted from ApiTrace of course). Everything else is like saying that Windows PIX has some nice features, which it does but ultimately it's irrelevant.
I'm trying to add "realtime preview" of front buffer to QApiTrace, but I have a problem - how to embed GL window created by glretrace? I've tried using the QX11EmbedContainer but without a luck. Any ideas?
it'll be nice to get prebuilt binaries.
Are you guys thinking about the ability to play maybe a single frame or a range from the trace files
any plans on working on a 64 bit version too
Zack, first of all THANKS A LOT. Its one of the best tools I have ever used. I have installed Qt & QJSON. But the CMAKE is not compiling GUI. It always says "-- Could NOT find QJSON (missing: QJSON_LIBRARIES QJSON_INCLUDE_DIR)"
I have added the QJSON_INCLUDE_DIR & QJSON_LIBRARIES in the CMakeCache.txt as follows
//Path to a file.
QJSON_INCLUDE_DIR:PATH=C:/Program Files (x86)/qjson/include/qjson
//Path to a library.
QJSON_LIBRARIES:FILEPATH=C:/Program Files (x86)/qjson/lib
I usually add this and try to run the cmake build command again. It just doesn't work !! Any obvious mistake that I'm doing ?
Jaw droppping cool.
Thanks guys.
PS: Any chance for X traces ?
> PS: Any chance for X traces?
Is there some problem with xtrace (in recent years it has gotten support for newer X extensions)?
Thanks a lot!
Not only is it the only OpenGL debugger that works on my machine (Arch x64) without constantly segfaulting, but it also is a great pleasure to work with!
Nice tool! I just discovered it. We had a similar internal tool at a previous company I worked at. We couldn't live without it. Writing/debugging OpenGL drivers is much easier with such a tool!
You guys are a-mazing!
apitrace helped me find a bug in my picking code where it worked in some screen locations and not in others, which was driving me crazy. I couldn't see any difference in what the code was doing until I used the trace diff tool. It really made it obvious! (Culling mode set one place and not the other.)
Super helpful! Thanks!
Post a Comment