Purpose
I aim to obtain linear RGB values based on ACEScg, the working color space of Unreal Engine.
Quick Conclusion
- Apply the OpenColorIO plugin to the UE's viewport mode to display images in the linear color space, utilizing ACEScg to Raw conversion.
- Use the Pixel Inspector in the viewport, with OpenColorIO applied, to derive linear RGB values of ACEScg from the "Final Color".
Note
The content has been translated from an original Japanese content using ChatGPT as a base. The original Japanese content was also written by me (https://note.com/tonali/n/n2a91fbef4584).
Introduction: Seeking Raw Data on Color in the Unreal Engine
Unreal Engine (UE) employs a scene-linear workflow when computing color, using linear RGB. This is grounded in the linear nature of light in the real world, ensuring a realistic simulation. However, the Computer Graphics (CG) images seen on the UE edit screen are constructed using non-linear RGB.
In typical game development, a combination of linear computation and non-linear display works effectively. However, for my current research in information engineering, I require both the visualization and values of linear RGB images. As such, I have explored methods to extract linear RGB from UE's default state. It's worth noting that this is my first foray into using UE, so there might be some areas where I've misunderstood.
Project Setup
Setting the Working Color Space to ACEScg
For this endeavor, I am running UE under the following environment. For the project's foundation, I've chosen the "Blank" project, found under the "Game" category.
Execution Environment:
Engine: Unreal Engine 5.2.1
OS: Windows 11
CPU: Intel Core i7-13700
GPU: RTX 2060 Super
RAM: 32 GB
Before diving into the operations, we must first determine the Working Color Space for this project. The Working Color Space decides which linear color space to use, as previously mentioned. For this project, I will set it to ACES AP1 / ACEScg.
ACEScg is a color space proposed by the Academy of Motion Picture Arts and Sciences. It was developed not for display purposes, but explicitly as a Working Color Space, and by default, it conforms to the linear RGB specification. The workflow that adopts ACEScg as the Working Color Space is known as the ACES workflow. It's not only being utilized in UE but is also starting to gain traction in movie production and image creation domains. While there are other color spaces under the ACES umbrella, such as ACES AP0, delving deep into the subject of ACES would make for an extensive discussion, so I'll omit the details here. Nonetheless, the philosophy and history behind ACES are truly intriguing.
For the purposes of my research, I have changed the Global Illumination and Reflections from the default Lumen to Screen Space. While Lumen is a highlighted lighting feature introduced with the release of UE5, I found that its behavior with HDRI (High Dynamic Range Image), which I use in my research, was not optimal. Hence, I opted to switch to Screen Space.
Setting Up HDRI Ambient Lighting and Objects
For my testing environment, I've integrated an appropriate HDRI lighting setup and included the SM_ColorCalibrator, which is already available within UE. Regarding the HDRI, I utilized the HDRI Backdrop plugin since it conveniently allows for simultaneous configuration of both lighting and background imagery. The background used in this test is the default one that comes pre-installed with UE.
https://docs.unrealengine.com/5.0/en-US/hdri-backdrop-visualization-tool-in-unreal-engine/
To ensure a consistent exposure, I added a PostProcessVolume object and configured its settings as follows:
- Infinite Extend (Unbound): Enabled
- Lens -> Exposure -> Metering Mode: Manual
- Lens -> Exposure -> Exposure Compensation: 0.0
- Apply Physical Camera Exposure: Disabled
Linear RGB Images Tend to Appear a Bit Dark
Before showcasing how linear RGB appears in UE, let's introduce what a linear RGB image looks like.
The image on the left was recorded using RGB that underwent non-linear gamma correction, while the one on the right was recorded in linear RGB. Although I captured both images, the left one appears closer to what our eyes perceive, giving the linear image on the right a somewhat darker impression. Why does this phenomenon occur?
To simplify the explanation, refer to the figure above. Starting from the top row, the far-left represents the display's characteristics. It would be ideal if the display could control the light linearly in relation to RGB. However, in reality, displays emit non-linear light in response to the inputted RGB, a behavior known as gamma characteristics. As a result, image formats apply gamma correction to the raw RGB values, deliberately creating non-linear RGB values. When these RGB values are inputted to the display, it can control the light in a linear manner relative to the raw RGB values.
On the other hand, the bottom row illustrates that linear RGB directly represents the gamma characteristics of the display. Consequently, in comparison to raw RGB values, a darker image than expected gets displayed. This phenomenon is lucidly explained on EIZO's display webpage.
https://www.eizo.com/library/basics/lcd_display_gamma/
Can UE Output Linear RGB Images by Default?
Is it possible for UE to output images in the Working Color Space by default? Upon investigating, I came across a mode in the Viewport called "Scene Color." Checking the documentation, it says:
Scene Color shows the result of the scene before any post processing is done. This means before any exposure, bloom, color correction, or antialiasing. In the above image, the scene appears very dark because exposure has not brightened it up for us. (from UE document: https://docs.unrealengine.com/5.2/en-US/viewport-modes-in-unreal-engine/)
"Post-processing" refers to various treatments applied to the calculated linear RGB before it's displayed on the screen, such as gamma correction, tone mapping, and color correction. With that in mind, let's take a look at Scene Color.
You can switch the Viewport mode as shown in the figure above. For reference, the image currently displayed has UE's preset post-processing applied.
Having switched to Scene Color, upon observation, I thought:
It doesn't seem that dark...
Yes, that's right. The Scene Color, which should be displaying an image in linear RGB, doesn't quite behave as expected for linear RGB. This makes me question if the RGB values are indeed accurate. Although this is becoming a bit long-winded, this very question was the catalyst for my investigation. Now that Scene Color doesn't seem very reliable, how can we effectively obtain images in linear RGB?
Solution: Using the OpenColorIO Plugin
For those familiar with color science, you might be thinking, "Just use OpenColorIO already." Well, your patience has been rewarded. Let's explore how the UE's viewport imagery changes with OpenColorIO.
In simple terms, OpenColorIO is a unified color space transformation plugin. In production environments, there's a history of using various color space materials and different color space displays for content creation. This made stable color management challenging. OpenColorIO was developed to address this by creating a standardized system, as depicted in the above image, ensuring consistent color management. Since Unreal Engine 5 supports OpenColorIO, we'll utilize it to display in a linear color space.
To activate OpenColorIO in UE, head to the plugins page. Search for OpenColorIO, and by ticking the checkbox, the OpenColorIO plugin will be enabled. Once activated, you can then prepare to use OpenColorIO.
1. Download the OCIO File
The Academy Color Encoding System (ACES) provides an OpenColorIO Config file which you can download. This Config file describes various color spaces and their corresponding transformations. Typically, using the Config file from ACES should suffice for most needs. The ACES Config file includes:
cg-config
studio-config
reference-config
https://github.com/AcademySoftwareFoundation/OpenColorIO-Config-ACES/releases/tag/v1.0.0
Each of these offers a slightly different list of color spaces, tailored to specific production scenarios. For use in Unreal Engine, you download the cg-config file.
2. Creating OpenColorIO Configuration in UE
Next, set up the OpenColorIO Configuration within Unreal Engine. Note that detailed settings for OpenColorIO can be found in the UE documentation.
https://docs.unrealengine.com/4.27/en-US/WorkingWithMedia/ManagingColor/OpenColorIO/
After creating the OpenColorIO Configuration in the Content Folder, configure it as shown above. Here are the details for each setting:
Configuration File: This is the OCIO Config file, packed with color space information. Typically, you'll be using the file provided by ACES.
Desired Color Spaces: Register the color spaces you intend to use within UE. While you have the freedom to choose which color spaces to include, if you're applying OCIO to the viewport, ensure to register the ones adopted in the working color space. In this context, we're talking about ACEScg.
Desired Display-Views: Register the intended display. When converting ACEScg colors, the method of conversion varies depending on the display's color space and gamma, hence registration is necessary. In our scenario, since we're using an sRGB display, we'll add three types: SDR Video (tone-mapped from HDR to SDR), Un-tone-mapped (without tone mapping), and Raw (no conversion to the display's color space).
Now that everything is set up, let's apply OpenColorIO to the UE viewport.
Applying OpenColorIO to the Viewport
Upon integrating the OpenColorIO plugin, a new mode is added to the Viewport. Here, you'll need to incorporate the OpenColorIO Configuration we created earlier. For the source (on the top), select ACEScg, and for the target (at the bottom), select sRGB - Display - Raw. Ensure the "Enable Display" box is checked.
Success! By applying OpenColorIO, we've managed to display the color reminiscent of ACEScg's linear RGB. While Scene Color seemed to retain some brightness, the application of OpenColorIO allowed us to showcase the darker imagery characteristic of linear RGB. Seeing this comparison, it becomes evident that, even though Scene Color represents RGB before post-processing, it might not precisely represent the raw information of ACEScg.
Retrieving ACEScg Linear RGB Values
Having successfully displayed an ACEScg image, let's dive deeper and extract the actual RGB values from the image. Unreal Engine provides a handy tool called the Pixel Inspector, which allows users to examine the RGB values of any pixel under the pointer.
https://docs.unrealengine.com/4.26/en-US/TestingAndOptimization/DevTools/PixelInspector/
You can access this tool by navigating to Tools -> Debug -> Pixel Inspector. Using this, we will extract the color values from the color checker object.
The image above depicts a color checker where we applied OpenColorIO to display the raw ACEScg information and used the Pixel Inspector on it. Within the Pixel Inspector, the Scene Color represents the pre-post-processing color, essentially the color based on the image before applying OpenColorIO.
In contrast, the Final Color showcases the color after all post-processing, the color intended for display. To differentiate between the colors in the image, we extracted the Scene Color, Final Color, and the color of the target pixel using an eyedropper tool.
The result showed that the Final Color and the target pixel color were almost identical, whereas the Scene Color was noticeably brighter. From this observation, we can infer that the processing with OpenColorIO pertains to the post-processing values. Therefore, when aiming to retrieve ACEScg's linear color space, it seems prudent to refer to the Final Color data.
Discussion: Why Does Scene Color Appear Brighter?
Having accomplished our main objective, let's now briefly ponder why the Scene Color appears brighter. In the past, during the UE4 era, there was talk of an issue where the color of a material appeared brighter than expected. One of the topics raised during this discussion was the influence of display gamma.
(Example)
https://forums.unrealengine.com/t/color-picker-different-to-color-in-engine/304047/2
https://forums.unrealengine.com/t/color-picker-shows-wrong-color-range-in-srgb-mode/306753
Display gamma in UE is a parameter that applies gamma correction to the entire screen being displayed. Let's adjust the display gamma and observe the changes in the Scene Color.
To change the display gamma, we use the Debug Tools. This tool can be found under Tools->Debug->Debug Tools. There you'll find an option to adjust the Gamma value, which is set to 2.2 by default. Adjust this value and observe the Scene Color.
The images above show the result when the display gamma is adjusted in the Scene Color. As anticipated, reducing the display gamma from 2.2 to 1.0 made the displayed image darker.
Furthermore, the image on the right displays the result when the display gamma is left at 2.2 and the ACEScg Raw information is output using OpenColorIO. The discovery here is that adjusting the display gamma to 1.0 made the Scene Color image closer to ACEScg Raw. It seems that the brighter appearance of the Scene Color is indeed influenced by the display gamma.
However, something else caught my attention. When adjusting the display gamma and using the Pixel Inspector, the Scene Color value hardly changes, but the Final Color value does. This indicates that the Final Color value reflects the display gamma.
Moreover, unlike a display gamma of 1.0, at 2.2 the Final Color and Scene Color values almost match. Looking at it this way, while Scene Color itself might represent pre-post-process RGB values, it seems that the display gamma is already factored into the Scene Color. I briefly glanced over UE's source code but couldn't pinpoint the exact cause, so this will remain a topic for future investigation.
Conclusion
I explored the method and examination of retrieving UE's ACEScg linear color space. Honestly, I still don't fully understand the behavior of the Scene Color, but thanks to OpenColorIO, it seems we can extract the Raw information. For now, I'd like to manage with this solution. Color is so complex!
Top comments (0)