Why does post-processing matter?

To really understand how post processing works, we have to take a detour.

This is a camera.

A traditional camera works by using a lens to focus an image into a sheet of chemically treated film. The film, being exposed to the light, undergoes a chemical reaction. In response to the light, the image is etched onto the film.

The properties of the lens - how perfect it is, how it's curved - lead to certain artifacts in that captured image. For example, a halo around bright lights is typically caused by imperfections in the lens scattering the light around.

The human eye works differently, but the basics are the same - light, lens, receptor, chemical reaction.

In the history of photography and image capture, a particular pattern has emerged. It seems obvious to say that a photograph is not as bright as the place it was captured from. But it still looks like the real thing. How is this possible?

It's because the chemical reaction between the light and film is carefully tuned to respond to different levels of light in different ways. If you've heard of 'film stock', this is the difference between them - different stock has a different chemical profile, and responds to light differently.

From this page, which goes into great depth.

In this image, the section labelled gamma marked by dotted lines is all that can be seen without post processing.

Going back to the main point; why is this needed to understand post processing? Because neither your monitor nor a VR headset are as bright as the places they are displaying. To work around this, the solution is the same as with film - having a light response curve that fits things to their natural appearance. This is "tone mapping". With this, you have the building blocks of a proper render.

That goes for cel shading and non-photo realistic rendering too. After all, paintings and artwork aren't as bright as the real world. To get things looking right in VR, we need to use the right shading to evoke that colour contrast for us, and that's what tone mapping is for.

These are the true basics of post processing.

And Unity, by default, doesn't do any of this until you add the post processing stack! In scenes without post processing, all you're seeing is the raw colour/lighting data, carelessly thrown on your display. Colours get shifted, bright areas and objects look weird when they get brighter - none of this matches what we would expect to see with our own eyes.

This applies to the Unity editor, too. When you set up lights in your world, you should have post processing set up and enabled - probably the second or third thing you should add. If you don't, you will end up with light intensities that make no sense, and lead to areas of your world being too dark/bright. Even though post processing is the last thing rendered in game, it should be set up very early in creating a world.