Table of Contents
Effects with Overlays
A common scenario presents itself when developing plug-ins with FxFactory: you may be designing a filter or transition that can trivially handle media at any pixel aspect ratio (e.g. a color correction) but you need to composite an overlay over the effect.
Often times, this overlay is generated assuming square pixels geometry, and will look incorrect when displayed over interlaced and/or anamorphic footage. Let us look as the simplest scenario:
The output of the color correction step is composited with some text (the “A”) generated as a progressive image in square pixels aspect ratio1).
| Something rather obvious is happening: the output looks correct because the overlay and the color-corrected image have the same pixel aspect ratio (square) and they are both progressive. |
|---|
In practice, the simple scenario described above rarely happens. All broadcast standards demand interlaced footage, and media (even progressive one) is often anamorphic2).
Let us look at the worst-case scenario, next:
In the above diagram, the color correction is applied to a single anamorphic field, an image with dimensions 1440×540 that will be combined with the other field and ultimately be displayed at 1920×1080.
There is a pretty big difference between what you were trying to do, and what will actually happen:
One Solution
Ideally, you should design the filter so that it renders the overlay correctly, no matter what the media characteristics are.
One way to solve this problem is to continue generating the overlay as a progressive image with square pixels aspect ratio, and to resize it so that it has the same dimensions as the original frame:
Though it may not be obvious from the diagram, by the time the output image is displayed to the user, it will actually look correct.
No matter what host your plug-in is running in, filters always expect the input and output to have the same characteristics. In other words, the input image and the output image always have the same pixel aspect ratio. Unless your filter happens to be expanding the frame3), the input and output image will also have the same dimensions.
Looking at the Issue in Quartz Composer
In Quartz Composer, this translates roughly to the following:
The above composition does not use the Rendering Destination Dimensions patch at all. It calculates the ideal dimensions of the output4) by multiplying the Pixels Wide and Pixels High outputs with the Scaling X (native to square) and Scaling Y (native to square) outputs.
To conform the overlay to the pixel aspect ratio of the filter’s input/output image, the image is scaled by the values specified in the Scaling X (square to native) and Scaling Y (square to native) outputs, and then composited over the filtered image using a simple Source Over operation5).
Conclusions
There is no general approach that will work for all your effects. Sometimes, to create the overlay you need to know the ideal dimensions of the input/output image. Other times, the overlay can be generated independently of the input/output image, and composited at the correct location via a different technique6).
The fundamental requirement for compositing images with different pixel aspect ratio is that your output image should always be the result of conforming multiple inputs to the pixel aspect ratio requested by the host.