FX Motion Vector Flipbooks
Motion Vectors:
The idea of scrolling UVs in games has been around since the beginning- changing the per-vertex sampling of a texture map:
The idea of per-pixel UV scrolling, otherwise known as Flow-mapping, has also been around for a long time. The idea is you distort the sampling of a texture by using another texture to offset the UV coordinates:
Over the years, these techniques have had an advantage over flipbooks, since they use much less memory, and can interpolate between frames for smoother playback. This flowmap cloud system was created for Blacksite:Area 51:
Motion Vectors:
It's only in recent years that a technique came out to use flow maps, or motion vectors, to morph a flipbook frame to match the next frame in the sequence, allowing interpolation between frames, once again as per-pixel UV scrolling: Klemen Lozar's Demo.
The movie below is a series of Houdini simulations initially created for Guild Wars 2. They started as 200 frame .exr files, with motion vectors created using Slate. Only every 4th frame is kept, so the sequence is reduced to only 49 frames, or a 7x7 .tga file.
They generally hold up well stretched out to just under 10 seconds, with artifacts more visible on hard-edged liquids. These are depected in realtime in the Guild Wars material editor:
Flipbook Map:
The above animations are created with RGBA .tga files like the one below (depicted as RG,B,A channels here). Most of our motion vector flipbooks shipped as 1024x1024 Targas, but occasionally we used a 2048:
The R and G channels show the normal map of the simulation, and the B channel is a heatmap, or a depth map for backlit subsurface scattering. Alpha is the usual transparency. For Guild Wars, we left the diffuse out as most of our effects are tintable.
If an up vector, or the blue channel, is needed for the normal map, it can be derived. A normal is a vector in 3d space. If you think back to high school math, a vector can be represented as a right triangle on a graph, with the vector itself being the long side, and the axis being the shorter sides. Since we know the normal length is 1.0, we can use the Pythagorean theorem to solve for the missing side, with 1 being the the length of the long side of the 3D right triangle: B = sqrt(1-(sqrt(R) + sqrt(G))) [ Theodox- aka Steve Theodore]. This returns a value between .5 and .707, which isn't much detail, and often the Blue channel can be given an arbitrary value depending on the needs of the effect.
Motion Vector Map:
This is the Slate-generated Motion Vector Map, originally calculated for 200 frames, but once again reduced to 49 frames. A higher number of initial frames allows more accuracy, so long as you increase the amount of per-pixel uv distortion between frames. Red controls the U interpolation, and G controls the V. This image can be small- we used 256x256 .tgas on Guild wars:
Fake Global Illumination Textures:
On Guild Wars, we tried to avoid per-pixel lighting in favor of faking Global Illumination with another 256x256 texture (depicted as RGB, and separate R,G,B channels here:
By using the difference between the vert's position in camera space, and the emitter's position in camera space, we were able to create UVs which stretch across the entire particle system. This enabled us to map the GI texture across the entire particle system.
These are images of the Red, Green and Blue channels mapped across a particle system, respectively:
In the shader, we created a fake lightsource position at or near the center of the particle system. By doing a dot product between the sprite to camera vector and the sprite to fake-light vector, we could tell when a sprite was facing the interior fake-light or not, and blend between GI textures accordingly. The blue channel was slightly additive, to fake sub-surface scattering when a sprite was backlit.
Then we used the Red and Green channels of the flipbook to offset the UVs on the sample of the GI texture, to make it look like the particle was actually being lit:
It holds up fairly well, especially in an engine that is more than a decade old.
Reflection Maps:
A reflection map can be added and UVs offset with the flipbook normals, to make it look like liquid. The normal map can also be used to distort the frame buffer:
