Mikkel Gjoel

Removing Banding in Linelight

2022-01-22

TL;DR If your game has banding, it is often very easy to get rid of simply by adding noise to the shader-outputs:

Bandy Linelight	Beautiful Linelight

Intro - Linelight

Linelight is a magnificient puzzlegame with a beautiful artstyle. You should go play it! The contrast between the sharp, focused gameplay-layer and the soft background makes for an appealing and functional aesthetic.

Having severe banding-PTSD from working on INSIDE, it always felt like the elegant artstyle of Linelight was disrupted by the sharp edges caused by banding. So when I had the good fortune to get to work with the creator, and after an indecent amount of pestering on my part, he gratiously allowed me a few hours in the company of the sourcecode of the game.

As the visuals of the game are fairly simple, I thought it would be a nice example to explain how little it takes to remove the banding-artefact otherwise common in many games, and what impact it can have visually.

A quick note that we are looking at Unity 2020.1.0f1 using the built-in render-pipeline

Removing Banding in Unity’s Built-In Sprite Renderer

All of Linelight is rendered using the SpriteRenderer in Unity. This means that, while the game-project does not explicitly contain shaders, all shapes are rendered with the same built-in shader:

//note: Sprites-Default.shader
fixed4 SpriteFrag(v2f IN) : SV_Target
{
  fixed4 c = SampleSpriteTexture (IN.texcoord) * IN.color;
  c.rgb *= c.a;
  return c;
}

Which we replace with the following (all of which we will dive into in the article below, but here it is because everyone loves spoilers and copy-pasting code):

//note: uniform pdf rand [0;1[
float4 hash43n(float3 p)
{
    p  = frac(p * float3(5.3987, 5.4421, 6.9371));
    p += dot(p.yzx, p.xyz  + float3(21.5351, 14.3137, 15.3247));
    return frac(float4(p.x * p.y * 95.4307, p.x * p.y * 97.5901, p.x * p.z * 93.8369, p.y * p.z * 91.6931 ));
}

fixed4 SpriteFrag(v2f IN) : SV_Target
{
    float4 c = SampleSpriteTexture (IN.texcoord) * IN.color;
    c.rgb *= c.a; //note: premultiplied alpha

    //note: color dithering
    float4 r0f = hash43n( float3(IN.texcoord, fmod(_Time.y, 1024.0)) );
    float4 rnd = r0f - 0.5; //symmetric rpdf
    float4 t = step( 0.5/255.0, c) * step( c, 1.0-0.5/255.0 );
    rnd += t * (r0f.yzwx - 0.5); //symmetric tpdf

    const float4 target_dither_amplitude = float4(1.0, 1.0, 1.0, 10.0);
    float4 max_dither_amplitude = max( 1.0/255.0, min( c, 1.0-c ) ) * 255.0;
    float4 dither_amplitude = min( float4(target_dither_amplitude), max_dither_amplitude );
    rnd *= dither_amplitude;

    c += rnd / 255.0;

    return fixed4( c );
}

(see the end of this article for a version that is a bit easier to reuse)

Four Easy Steps to Banding-Free Bliss

Removing banding in Linelight pretty much came down to

Downloading the built-in-shaders from Unity
Making a copy of the used shader (Sprites-Default.shader)
Inserting code in the shader-copy, to add noise to the color before output.
Creating and assigning the material to all sprite-renderers

Before (using default material `Sprites-Default`)	After (custom material `mdz_Sprites_Default`)

That’s it. That is the article.

Back up, tell me Everything: How does adding noise to an otherwise beautiful image help?

The human eye is really good at distinguishing details in dark areas - evolution-wise it rather helps with not getting eaten by creatures lurking in the dark. We can perceive quite a bit more than the 8bits/channel often used in rendering (somewhere around 14bits) - and as the human vision-system is also great at finding edges, the abrupt color-changes caused by banding in dark areas is quite noticeable to us. Conversely, our perceptual system is very robust against noise in dark areas, which we barely notice - probably because it occurs naturally in low-light conditions, and rarely eats us.

Color-banding is caused by numbers being quantized to lower precision integers - i.e. rounded to nearest integer (in rendering typically 8bit integers for each color-channel, so 32bit-RGBA = R8+G8+B8+A8, or 256 values per color-channel). There is no way around turning color-values into integers. However, by adding noise per pixel we can make sure that, no matter the precision, we see the right value on average, while also breaking up the noticeable banding-edges and replace them with less noticable noise.

Signal quantized (signal in orange, quantized signal in blue)	Noise added to Signal before quantization

a smooth input turns into a staircase due to conversion to integers	adding noise, the integers now vary between Too Much and Too Little (on average, we now hit the original signal)

Interactive version below (drag line with mouse, white point marks average across 32 random values)

Read here and here for more details on banding.

In order to generate the noise used for dithering, we will be using the following hash-function because it is fast and GoodEnough(TM) (go here for a rigorous analysis of hash-functions on GPUs). It takes a 3-component input (say screenposition.xy and time), and calculates four pseudo-random values which we will use to dither RGBA independently:

//note: uniform pdf rand [0;1[
float4 hash43n(float3 p)
{
    p  = frac(p * float3(5.3987, 5.4421, 6.9371));
    p += dot(p.yzx, p.xyz  + float3(21.5351, 14.3137, 15.3247));
    return frac(float4(p.x * p.y * 95.4307, p.x * p.y * 97.5901, p.x * p.z * 93.8369, p.y * p.z * 91.6931 ));
}

Removing banding is then achieved by simply adding noise to the output of our shader, in the case of Linelight, our copy of the Sprites-Default-shader:

fixed4 SpriteFrag(v2f IN) : SV_Target
{
  fixed4 c = SampleSpriteTexture (IN.texcoord) * IN.color;
  c.rgb *= c.a;

  //note: add RGB-noise to dither color
  float4 rnd = hash43n( float3(IN.texcoord.xy, _Time.y) ); //note: uniform noise [0;1[
  c += (rnd.xyzw+rnd.yzwx-1.0) / 255.0; //note: symmetric tpdf noise 8bit, [-1;1[

  return c;
}

(for reasons bordering magic, you should use a noise with a triangular distribution and an amplitude of 2LSB (Least Significant Bit) - which luckily is easily obtained by simply adding two random-numbers together, like here where we add different components of the calculated hash-values. In the above illustrations we used 1LSB noise with a uniform distribution, which is why there are vertical bands with no noise visible in the dithered gradients)

no dithering on large white-sprite (noticeable banding) (image contrast increased)	rgb dithering (some subtle banding) (image contrast increased)

Blacks and whites should not change

Since the output is clamped to [0;255] upon write to the rendertarget, we need to limit the amplitude of the noise used for dithering, to not add noise to pure blacks and whites (more in depth treatment here). Another way to think about this, is that the average of the dithered signal should always be equal to the signal itself.

fixed4 SpriteFrag(v2f IN) : SV_Target
{
    fixed4 c = SampleSpriteTexture (IN.texcoord) * IN.color;
    c.rgb *= c.a;

    //note: color dithering
    float4 r0f = hash43n( float3(IN.texcoord, fmod(_Time.x, 1024.0)) );
    float4 rnd = r0f - 0.5; //symmetric rpdf
    float4 t = step(0.5/255.0,c) * step(c,1.0-0.5/255.0); //note: comparison done per channel
    rnd += t * (r0f.yzwx - 0.5); //symmetric tpdf
    c += rnd / 255.0;

    return c;
}

triangular PDF (notice the average signal in yellow diverges at the ends, and blacks/whites are noisy)	triangular PDF, noise switch to 1LSB rpdf noise at boundaries (issue at both ends fixed)

zoom or open image in new tab to see details

Blending

When rendering sprites on top of each other, they are blended - which means the GPU will render one sprite to the rendertarget - then render the next sprite, read back the color in the rendertarget, do a blending operation like (1-alpha)*dst_col + alpha*src_col - and then write the result to the rendertarget.

Ideally, what we would want to do is to first blend then add noise like so: (1-t)*dst + t*src + noise - all at high precision. We could potentially achieve this by doing the t*src in the shader, then add noise just before the output. Unfortunately directx specifies that colors can be reduced to rendertarget precision before blending. This means our noise also gets quantized removing its dithering effect. Don’t worry if this all sounds a bit hairy - our workaround is to simply add noise to the alpha-channel:

rgb only dithering (some subtle banding) (image contrast increased)	full rgba-dithering (image contrast increased)

We need to add more noise to alpha than to RGB. In the above example around 10x was “enough” (in INSIDE it was adjusted for each transparent object). It is not generally possible to get it perfect for every pixel, so you tend to have to error on the side of Too Much Noise. It is worth noting that even the simple solution of “adding more noise” is not entirely trivial, as we risk adding so much noise that we noticeably ruin pure blacks/whites:

notice large unsightly square (image contrast increased)

So to solve this we again change the amplitude of the noise to increase the amount of noise, while still limiting it at the boundaries, so blacks and whites remain noise-free.

const float4 target_dither_amplitude = float4(1.0, 1.0, 1.0, 10.0);
float4 max_dither_amplitude = max( 1.0/255.0, min( c, 1.0-c ) ) * 255.0;
float4 dither_amplitude = min( float4(target_dither_amplitude), max_dither_amplitude );
rnd *= dither_amplitude;

triangular PDF with large amplitude noise	triangular PDF, noise amplitude limited to remove clamping

zoom or open image in new tab to see details

notice large unsightly square around player (image contrast increased)	square is gone now that we have restricted the noise (image contrast increased)

This leads us to the final shader-code, again, for our custom Sprites/Default, both forcing pure blacks/whites by limiting to an rpdf-noise where clamping occours AND increasing noise (but not “too much”):

fixed4 SpriteFrag(v2f IN) : SV_Target
{
    float4 c = SampleSpriteTexture (IN.texcoord) * IN.color;
    c.rgb *= c.a; //note: premultiplied alpha

    //note: color dithering
    float4 r0f = hash43n( float3(IN.texcoord, fmod(_Time.y, 1024.0)) );
    float4 rnd = r0f - 0.5; //symmetric rpdf
    float4 t = step( 0.5/255.0, c) * step( c, 1.0-0.5/255.0 );
    rnd += t * (r0f.yzwx - 0.5); //symmetric tpdf

    const float4 target_dither_amplitude = float4(1.0, 1.0, 1.0, 10.0);
    float4 max_dither_amplitude = max( 1.0/255.0, min( c, 1.0-c ) ) * 255.0;
    float4 dither_amplitude = min( float4(target_dither_amplitude), max_dither_amplitude );
    rnd *= dither_amplitude;

    c += rnd / 255.0;

    return fixed4( c.rgb, c.a );
}

A Quick Note on Banding in Source Textures

Most of the source-content in Linelight is made with 24bit rgb-images rendered as sprites, which can contain banding already in the source material. There is no easy way to remedy this after the fact, so the easiest approach is to fix the content. The way to fix it, depends on the cause:

If caused by texture-compression in Unity
- The default in Unity is to use compression. There is no magic wand here, just change compression-settings on the image to do less of it (it is a setting on the texture in Unity)
If caused during authoring
- use higher precision source images (e.g. 16bit). This is absolutely dobable, but of course comes with a performance-cost. Incidentally, this plugin for Photoshop turned out to be excellent https://www.exr-io.com/
- add noise to source images
  - noise in images can of course not be animated, so is more noticable :(
  - the images are scaled so noise becomes scaled and visible :(
- for Linelight, most sprite-images are simply radial gradients, so we could approximate with procedural shapes (i.e. write a shader that renders a radial gradient…)

A Quick Note on the Type of Noise Used

While this article is dithering using uniform random noise (“white noise”) because it gives a natural look, you are entirely free to use whatever type of pattern you like:

I also wouldn’t be able to look the rest of the Blue Noise Brigade in the eye, if I didn’t at least made a reference to the splendords achieveable by simply exchanging the noise-pattern used for dithering, with blue noise - making the noise a lot less perceivable simply by exchanging the dither-pattern:

White- vs Blue-noise for dithering

notice how the noise at the bottom looks much smoother due to less "clumping" shadertoy

Quick note that you should then pipe the noise through a TPDF-remapping function to get the best results. Here is a slightly more reuse-friendly version of the final code, also supporting bluenoise (though still using the old hash-sampling function).

//note: uniform pdf rand [0;1[
float4 hash43n(float3 p)
{
    p  = frac(p * float3(5.3987, 5.4421, 6.9371));
    p += dot(p.yzx, p.xyz  + float3(21.5351, 14.3137, 15.3247));
    return frac(float4(p.x * p.y * 95.4307, p.x * p.y * 97.5901, p.x * p.z * 93.8369, p.y * p.z * 91.6931 ));
}

//note: input [0;1[, output [-1;1[
//      source https://github.com/Unity-Technologies/Graphics/blob/master/com.unity.postprocessing/PostProcessing/Shaders/Builtins/Dithering.hlsl#L12
float4 remap_pdf_tri_unity_v4( float4 v )
{
    v = v*2.0-1.0;
    return sign(v) * (1.0 - sqrt(1.0 - abs(v))); //note: [-1;1[
}

//note: bluenoise-compatible dithering function, example usage (dithering rgb with 1/255 and alpha with 8/255):
//      col.rgba += calc_dither( col.rgba, tex2D(bluenoise,uv), float4(1,1,1,8)/255.0 );
float4 calc_dither( float4 col, float4 rnd01, float4 target_dither_amplitude_rcp255 )
{
    float4 rnd_rpdf = rnd01 - 0.5; //symmetric rpdf [-0.5;0.5[
    float4 rnd_tpdf = remap_pdf_tri_unity_v4( rnd01 );
    float4 rnd = (abs( col*(1.0/254.0) - (0.5/254.0) ) < 0.5/255.0 ) ? rnd_rpdf : rnd_tpdf;

    float4 max_dither_amplitude = max( 1.0/255.0, min( col + (0.5/255.0), 255.5/255.0 - col ) );
    rnd *= min( target_dither_amplitude_rcp255, max_dither_amplitude );

    return rnd;
}

fixed4 SpriteFrag(v2f IN) : SV_Target
{
    float4 c = SampleSpriteTexture (IN.texcoord) * IN.color;
    c.rgb *= c.a; //note: premultiplied alpha

    float4 rnd01 = hash43n( float3(IN.texcoord, fmod(_Time.y, 1024.0)) ); // <-- could be a bluenoise-texture-sample instead
    c += calc_dither( c.rgba, rnd01, float4(1,1,1,10)/255.0 );

    return fixed4( c );
}

References

Congratulations for making it this far - if you want to dive further into the topic, please explore the links below, and don’t hesistate to reach out if you have any questions.

Links directly relating to the article

Linelight the Game https://linelightgame.com/
Shaders used for illustrations
All the random functions your GPU can eat: Hash Functions for GPU Rendering
A bunch of presentations, too many of which are on dithering https://loopit.dk/publications/

More info on banding and dithering