Cairo Summit 2008 Notes

Tuesday, August 26, 2008

10:30 Welcome/Introductions

Carl: Been working on cairo since the beginning, (6 years ago). Much of that work was funded by Red Hat, but recently changed jobs to work for Intel. Hoping to talk about things today, but get some real work done Wednesday and Thursday

Vlad: Firefox 3 is using cairo and an update was offered (yesterday!) to Firefox 2 users to upgrade to Firefox 3. This should put cairo out onto 150+ million machines. Very interested in cairo on mobile platforms.

Joe: Working for Mozilla from Toronto office. Really want to hardware-accelerate cairo. Will probably write an OpenGL backend for cairo.

Jeff Muizelaar: Joined Mozilla in July. Worked on cairo in the past due to work with poppler. Interested in cairo performance.

Fred Plourde: Freelance consultant---been working with Mozilla. Interested in improving upscaling/downscaling quality and performance.

Søren Sandmann: Work for Red Hat, on the X team. Maintaining pixman library, (not full time). Want to hear ideas about pixman improvements, and talk about damage tracking.

Chris Wilson: Generally working on fixing cairo. Interested in talking about some rounding issues, (double-to-fixed and back, etc.).

Adrian Johnson: Mostly working on printing side of things, (PDF and PostScript). Want to talk about what needs to be done there.

Antoine Azar: Background in 3D graphics. Very interested in OpenGL/Direct3D backends and various performance optimizations in Cairo/Pixman.

Travis Griggs: Got into cairo through work with touchscreen interfaces for food-processing equipment, (smalltalk-based). More recent work continues to use cairo from smalltalk. Lots of Mac OS X and Windows deployment, and would like to learn how to improve that.

Behdad: Work for Red Hat. Co-maintaining cairo with Carl. Want to get back on track with maintenance. The past 4 or 5 months, (post 1.6), we haven't been doing a good job of keeping things running. Also want to talk about some larger plans, (post 1.8 stuff). We need to know where we want to go after we fix all the bugs.

10:45 Brainstorming/Scheduling

Topic ideas:

Hacking ideas:

11:00 What people are doing with cairo

Vlad: With Firefox, every pixel on the screen is going through cairo. For the current web, cairo is working quite well: Text, images (some issues with scaling), simple lines, etc. But future web stuff, (canvas, SVG, video, etc.), will stress cairo much more. So there's lots of performance work to be discussed/done there. Image clamping issue. Most pain points are just performance.

Carl: Working on Linux performance improvements [inside X server drivers].

Chris: Looking at PostScript---looking at fixing up ghostscript to have a fully-functional cairo backend---also writing a custom PostScript interpreter for use with the test suite. Is that cheating?

Travis: I write tools with cairo. I'm an IDE developer. And I get it even easier than people using Cocoa or .NET, etc. We run our stuff on 16 platforms, (multiple flavors of Linux and Windows, etc.). Mesh gradients would sure be nice.

Søren: In addition to maintaining pixman, I've been doing other stuff on the side. I've got my own little GTK+-replacing prototype, so it needs damage tracking and hit detection, etc.

Adrian: Biggest issue in PDF was fallback images---this is all fairly under control now. One of the next big things left is to not re-emit an image when a pattern is reused.

Antoine: On my website I've posted a performance evaluation comparing Quartz and pixman backends. This identified some big outliers on the pixman side---hopefully identifying some low-hanging fruit for pixman performance.

Fred: Cairo's downscaling quality has been very poor for a long time. Over the past couple of months, I've done a mipmapping implementation for pixman. Uses about 30% more memory and the performance is reasonable.

13:00 1.8 Planning

Behdad: We want to release 1.8 in two weeks (!). There are still a lot of failing tests though, (some of which might be poppler issues, etc.). But meanwhile, we've put over 500 commits in place and we don't really have good testing to know if all of those commits are ready. We've only done one development snapshot so far.

This release is targeting the GNOME release date which is September 24. So tar files are due September 22, with pre-releases on September 8 and September 1: http://live.gnome.org/TwoPointTwentythree

1.7.4 is out already and there have not yet been any bad reports yet. Who is using it? It's in Fedora at least, but not Ubuntu. Is Mozilla using it yet?

Vlad: Current cairo in Mozilla is about a month behind, (missing the LCD filtering stuff and a few fixes from Chris and Jeff's group device-scaling stuff). Vlad will update Mozilla this week to latest cairo and pixman which will get at least some daily testers.

Behdad: Pixman release?

Søren: Yes, we need a pixman release, (currently depending on an unstable version). Latest pixman bits are in Gentoo and Fedora and things are in good shape. Should be able to do a stable release in a couple of days.

Carl: So let's take the results of our test-suite hacking session from this summit this week, and make a 1.7.6 snapshot by September 1 and a 1.8 release-candidate by September 8.

Behdad: There's nothing I really want to change at this point other than fixing any bugs identified by the test suite.

Carl: What I'm more worried about is the new stuff we've added since 1.6. Is there anything we don't actually want to release?

Behdad: Review of stuff since 1.6: http://cairographics.org/news/cairo-1.7.2

[Carl got too involved in this discussion to keep good notes. Most of the new API is uncontroversial, but Carl dislikes the 3 lcd_filter API entries---it seems we should be able to address the use cases that have come up without any new API here.]

14:00 Automated testing (Carl's notes)

Vlad: Already have several Mac minis---just need to get them all set up and running. One issue is building cairo and pixman in an integrated way. Mostly just need help setting it all up, (adding hooks to pixman and cairo git hooks to make it go). Let's do a hack session this week to get something setup.

Antoine: How about allowing a developer to ask the system if a patch is good or not?

Vlad: Should be possible, definitely.

Behdad: Let's choose Fedora rawhide as our initial reference platform to make the whole test suite work and then extend from there, (Fedora, Ubuntu, Win32, OS X, etc.).

14:00 Automated testing (Travis' notes)

Interesting discussion about testing issues, things like how to compute tolerable differences between images, what to do about reference images that contain text and change from platform to platform. One interesting idea from 3D, do a minimal blur on reference and output, and then compare. Other options are standard image registration algorithms such as SSD, SAD, Mutual information, etc.

14:30 Project management issues (Carl's notes)

Carl: To some extent, I was "checked out" from cairo development the past few months, (for various reasons), and the project clearly stalled. When Behdad went to make a new release, there were 5-600 commits piled up with no intervening snapshots. What metrics do we need to know that the project is staying healthy?

Behdad: We definitely need snapshots every couple of weeks or so. And the only way to make that happen is to keep the tree in a releasable state at all times, (see buildbot discussion above).

Vlad: Would be great to have a fixed schedule for snapshots: Say, there must be a new snapshot every month. And we do let other external projects set the major release schedule, (GNOME, Mozilla, etc.), but we should also guarantee some maximum time that will pass between major releases, (say one year or so).

Carl: We need a person to own stable releases. I'm failing to keep up with them.

Chris: I'm willing to do that.

Behdad: It doesn't need to be the same person every time, either. We can nominate a new person for each stable branch.

14:30 Project Management issues (Travis's notes)

Background: Carl changed jobs this summer, and for a while the project stalled. Last release largely pushed by Gnome->Pango->Cairo, Behdad did the work.

Vlad: More and more Cairo consumers may mean timed release better model to keep up.

Carl: Tree has to stay releasable. If one falls too far behind, the effort to get to a snapshot-able point is too big. Not in favor of automating it completely.

Behdad: Six month cycle makes stable releases easier to maintain.

Chris: Willing to maintain the stable release.

Carl: If "new" features done on a separate branch, it makes merging it back into both unstable and stable easier.

Automated Testing WITH feedback important part of keeping project management rolling.

15:00 Printing/PDF

Adrian: The next feature we need is to avoid re-implementing the same image multiple times. Also, there are miles and miles of PDF specification without cairo API. We probably don't want to implement all of that. One idea is a generic API for inserting an arbitrary PDF object, allowing us to offload the work to an external library---that's especially useful for the non-graphical portions of PDF. As for graphical stuff, probably the only piece we're really missing is mesh gradients.

Vlad: To what extent is cairo the be-all-and-end-all solution for PDF generation? Where do we draw the line?

Adrian: It's a hard question to answer. No matter what we generate, it's always easy for users to want one more feature. Another idea rather than having API for the application to shove objects into cairo's PDF file would be for cairo to instead shove objects into the application's PDF file, (or that of some external PDF-generating library).

Adrian: Another topic is metadata for PDF, (author, creator, date, etc.). There are basically three ideas:

  1. PDF-specific API

  2. Generic API for any surface type

  3. Offload to one of the approaches above for external PDF library.

[Good consensus about option 2.]

The API should be very generic, and perhaps even more generic than add_metadata. It could be even "add_property", and could even control things like compression for PDF etc. A good "hack session" topic would be to draft up this "property" API.

Adrian: JPEG embedding---I've got some code already, but what we have to agree on is the API:

  1. A setter on a surface/pattern to provide the JPEG image data

  2. cairo_surface_create_from_jpeg (like cairo_surface_create_from_png)

Consensus that option #1 is better. And it should be on the pattern rather than the surface, and that it should allow for other data types rather than just JPEG, (can accept a mime type and the blob of data). So maybe something like this:

cairo_pattern_set_image_data (cairo_pattern_t *pattern,
                  const char *mimetype,
                  unsigned char *data,
                  unsigned int length);

[Discussion of mesh gradients.]

16:00 Performance

[Lots of interesting discussion with little in the way of notes. A high-performance, high-quality OpenGL backend would be interesting since it has the potential to actually be a cross-platform, high-performance backend. It won't be easy though, (Vlad thinks 12 months or so). Targeting OpenGL ES 2.0 seems like the right thing.]

Joe plans to make a first attempt at an OpenGL backend and will talk about it on the cairo list. Antoine and Fred expressed interest in contributing to that work.

Wednesday and Thursday, August 27-28, 2008

On days two and three of the summit, we switched gears from group discussions of topics to instead have hacking sessions. For the hacking sessions, we chose tasks which we thought we could make some significant progress on in a couple of days, and which would benefit from 2 or more people who don't normally get the chance to work together.

We didn't make any written notes at the time, but here are some writeups made after the fact. (Some of the details and participants of these hacking sessions will be missing---we had up to 3 or 4 going on simultaneously, so I couldn't attend them all. Please add anything missing.)

Making the PDF backend emit shared source images only once

Adrian, Carl, and Chris spent quite a bit of time on this issue and made some excellent progress. After initial discussion, we were able to divide the work up into 3 separate pieces with agreed-upon interfaces.

Chris implemented a new global counter for every surface created, and another per-surface counter for each operation that modifies a surface. Carl reviewed this, and Chris completed round 2 with new naming.

Adrian completed most of the work necessary to augment the current array of source patterns stored by the PDF backend with a new hash table keyed off the two counters implemented by Chris.

Carl planned to provide the final piece which is a copy-on-write implementation for _cairo_surface_snapshot, (so that when the meta-surface makes duplicate snapshots of an identical surface the PDF backend will see identical keys). Without the copy-on-write implementation, the current snapshot code is currently making explicit copies, so no sharing is possible. Carl did not get much time to work on this at the summit, so this task will need to be finished later.

Improving the build on Windows and OS X

Travis, Antoine, and Fred spent a lot of time battling configure scripts, and libtool, etc. trying to get builds more friendly to Windows and OS X. The consensus was to remove the version number from configure.in and the file lists from Makefile.in putting them into simple files. These files could then be included by configure.in/Makefile.in as well as by alternate platform-specific Makefiles. Behdad did some work on this extraction, which we hope to see soon.

JIT for pixman

Søren has been investigation some JIT ideas for pixman for a while. He had the chance to meet with Mozilla developers, (didn't catch the names), that have been working on Nanojit---a JIT engine that Mozilla is using (will use?) for Javascript. This looks potentially interesting for pixman, but there are some licensing issues to work out, (Nanojit comes from Adobe originally, with the MPL license while pixman will need an MIT license).

Setting up cairo buildbot instances at mozilla

Vladimir took on this task, and made some good progress getting things working. He also succeeded at locating suitable hardware already at Mozilla. We had a useful conversation about many desirable features: Developer access to machines of various platforms, the ability to submit patches to the buildbot for testing across the platforms, reports on test suite and performance suite, etc. Clearly the right answer is to get the minimal thing working and to add additional features incrementally.

Fixing test-suite failures

Chris and Carl had only a very little time to look at this issue. Carl is getting 8 failures on the image backend, (which should really always have 0 failures). Five of these have apparently never passed and are related to the LCD_FILTER API, which Carl had already wanted to just remove. Behdad agreed grudgingly. The remaining 3 failures look simply like rasterization differences which Carl didn't track down yet. On that topic, Antoine proposed to work on a smarter image comparison algorithm to avoid these errors. Carl expressed interest in seeing that, and removing the reference images for each individual platform.

Improving image downscaling quality

Fred has had some patches for some time in this area, and appeared to have some very fruitful discussions with Søren about them. Various ideas could be heard being thrown out about the best way to address the problem, and we hope for good things soon.

Getting JPEG data into PDF output

Adrian has had reasonable code for this for a while, and just needed some help nailing down the API. Everyone seemed to have their own idea about the ideal API, (should it be on the pattern or the surface? on any pattern or just a surface pattern? on any surface or just an image surface?). But we finally did reach consensus, (which Adrian will have to remind us of on the list).

Using doubles in the tessellator

Antoine has a patch for this, but we only had time for a very brief conversation about it. Antoine knew that Carl had some concerns about precision, but we only really had time for Jeff and Antoine to convince Carl that 52 is in fact greater than 32 and the conversation ended there. (Only on the way home did Carl realize that the current fixed-point implementation is actually using 64 and 128 bits, so this issue remains open.)

Reviewing Joonas' rasterizer work (Summer of Code)

Unfortunately, we did not succeed at getting Joonas to join us at the summit, but he did dial in to give us a nice overview of his progress. He did some great work surveying/benchmarking various existing polygon-rasterization implementations, implementing his own ideas, and integrating the best into cairo. The current results show up to 2x improvement on complex polygons. (The most simple, pixel-aligned rectangular paths still have 10x slowdowns, so some special-case code for those will still be needed---the code for this should all be within cairo already and just needs to be hooked up).

Static analysis for cairo

David Mandelin made a special guest appearance at the Cairo Summit to give an overview of two static-analysis tools being developed and used at mozilla:

DeHydra (http://developer.mozilla.org/En/Dehydra)

TreeHydra (http://developer.mozilla.org/en/Treehydra)

These hook into the gcc frontend and export the ability to write rules with Javascript that operate on the gcc data structures. We have various things we've been wanting to do with static analysis in cairo, (verifying that integers and fixed-point values don't get confounded, that user-space and device-space don't get confounded, that cairo_status_internal_t values don't leak out, etc.).

So it will be worth looking into these tools, (as well as sparse), to see how to implement some of those.

Image clamping

This may have been the only hacking session idea that we didn't get to spend any time on. Too bad---it would have been fun.

Mesh gradients

Adrian and Travis spent a lot of time poring over the mesh gradient specification in the PDF reference. The big challenge is coming up with "cairo like" API to capture the tremendous amount of data necessary to specify a mesh gradient. With Carl's help, a sketch of such an API did form, (with a series of cairo_mesh_pattern_curve_to calls to incrementally construct the mesh similar to the way a path is constructed).

There are lots of details missing to turn the sketch into a complete API, but Adrian feels like he has enough of an idea now to no longer be blocked.

The "chain of bugs" issues

We didn't come up with any better name for these bugs, but they are all detailed in Carl's blog post here, (and google knows that this is what "chain of bugs" means):

http://www.cworth.org/cairo/a_chain_of_bugs/

Of the three bugs mentioned there, bug #1 does not appear in current Mozilla builds, but similar issues are sure to appear again, (basically, the rest of the bugs are ready to be exposed by anything triggering fallbacks).

For bug #2, Adrian found one problem in cairo's PDF output, (an overly-large BoundingBox property that was causing the fallback-region to expand), and Jeff agreed that it should be possible to convince poppler to just use SOURCE for simple knockout groups, (which is what cairo really wants). We opened this bug report to remember that issue:

Simple knockout groups should just be rendered with CAIRO_OPERATOR_SOURCE https://bugs.freedesktop.org/show_bug.cgi?id=17346

We didn't get a chance to look closer at bug #3, so that will have to wait for another day.