Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vanilla friendly HDR (effects) implementation #28

Draft
wants to merge 12 commits into
base: 1.20
Choose a base branch
from

Conversation

shBLOCK
Copy link

@shBLOCK shBLOCK commented Jul 23, 2024

This PR proposed a possible implementation of HDR-related effects (bloom and tonemapping) that avoids impact on the look of non-related stuff (vanilla & mods not using veil).

General Description

Since tonemapping influences the look of things that aren't designed for HDR, they need to be corrected. This is done by "inverse tonemapping" (similar to the concept of inverse functions in math). (old idea, turned out to be impractical)
Since tonemapping influences the look of things that aren't designed for HDR, we simply try to ignore them when doing tonemapping & bloom.

To distinguish between non-HDR things and things that are actually designed for HDR (from veil users), two techniques are used for the deferred rendering pipeline and gui stuff.
For the deferred pipeline, a bitmask flag is used in the red channel of the MaterialSampler. Therefore, dedicated shaders are needed for HDR things (although veil's shader definition system makes this very easy).
One problem that comes with this implementation is post-processing shader compatibility. If a post-processing shader warps the color buffer then the material buffer won't match up with the color buffer anymore. To work around this, any warping post-processing needs to update the material buffer accordingly to keep the relation.
For Gui, HDR things are rendered into dedicated framebuffer, the HDR framebuffer is then composed back to the main buffer based on depth testing in the end. The non-HDR gui parts also need to be rendered into a separate framebuffer to apply reverse tonemapping to it.

Note: In the following sections I might not include changes to the transparent pipeline since it's basically the same as the opaque one.

Pipeline Changes (Slightly out-of-date)

  1. Run the deferred rendering normally, using a bitflag in the material buffer to mark out HDR fragments, note that for the transparent pipeline this bitflag needs to not be overridden once set (for example when a tinted glass is in front of a glowy thing).
  2. When composing, also compose the material buffer (or at least the HDR bitflag) based on depth and if it's from the transparent buffer. Keep this buffer somewhere (maybe just as a color attachment of the veil:post framebuffer, if compatibility won't be a problem).
  3. Do the remaining of the veil pipeline and vanilla post-processing, but don't ever resolve back to vanilla SDR framebuffer. (This might cause compatibility issues with other mods, TBD)
  4. Render overlay and gui to a dedicated framebuffer (which we'll call the gui framebuffer), also have a dedicated framebuffer for HDR stuff in gui.
  5. Compose the two gui framebuffers and the world buffer, writing to our material buffer to mark out the HDR pixels.
  6. Apply bloom.
  7. Apply tonemapping to pixels marked as HDR in the material buffer.
  8. Resolve back to minecraft:main.

TODO List

  • HDR content identification in framebuffer
  • HDR render types
  • HDR examples
    • Block
    • Item
    • Entity
    • Custom Shader
  • HDR texture loader & HDR texture sheet (16bit-per-channel PNG)
  • Tonemapping
  • Bloom
    • Fix bloom "biased towards center of screen"
  • Gui & overlay Support
    • Fix first person stuff
  • Documentation

IMPORTANT NOTE!!!

Below here are descriptions of the original plan that uses inverse tonemapping, but during development many more problems were found that makes me think it doesn't really make sense (or it's not worth the trade-off) to do a proper full-image tonemap for MC if we were to implement HDR effects, while keeping the vanilla look.

Pipeline Changes

  1. Run the deferred pipeline normally up until light rendering, using the bitmask to mark out HDR and non-HDR fragments.
  2. Apply the inverse tonemapping to the non-HDR fragments in the CompatibilitySampler of veil:opaque.
  3. Make VanillaLightRenderer render before any other lights.
  4. Just after VanillaLightRenderer, apply the inverse tonemapping to veil:opaque_light.
  5. (to be determined) Apply inverse tonemapping to AlbedoSampler of veil:opaque?
  6. Compose HDR framebuffer to veil:opaque and veil:opaque_light (based on depth test).
  7. Finish the deferred pipeline and finish all veil post-processing (except tonemapping and bloom).
  8. Don't resolve back to vanilla main render target yet. (This might cause compatibility issues with other mods, TBD)
  9. Run vanilla post-processing on our framebuffer.
  10. Render overlay and gui to a dedicated framebuffer (which we'll call the gui framebuffer), also have a dedicated framebuffer for HDR stuff in gui.
  11. Apply inverse tonemapping to the non-HDR gui framebuffer.
  12. Compose the two gui framebuffers.
  13. Render gui framebuffer onto the world framebuffer.
  14. Apply bloom.
  15. Apply tonemapping.
  16. Resolve back to vanilla main render target.

Remaining Problems & Ideas

  • Perhaps the "inverse tonemapping" operation doesn't really make sense (eg. its output approaches infinity as the input color value approaches 1.0).

@shBLOCK shBLOCK changed the title Vanilla friendly HDR effects Vanilla friendly HDR effects & HDR workflow Jul 24, 2024
@shBLOCK shBLOCK changed the title Vanilla friendly HDR effects & HDR workflow Vanilla friendly HDR (effects) implementation Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant