Project Fortify, an experimental security project, has identified the following potential security issue. If you’re a feature owner CC-ed on this bug, please do your best to review these reports. Please see https://chromium.googlesource.com/chromium/src/+/main/docs/security/ai-generated-security-bugs-faq.md for more information.

Overview: A potential out-of-bounds write vulnerability in ANGLE’s D3D backend arises from a mismatch between shader variable collection and HLSL AST optimization. An active uniform that is optimized out of the AST is omitted from the register map, leading to a dereference of a map’s end iterator during linking. In release builds, this retrieves garbage register indices, causing an integer overflow during buffer allocation and a massive out-of-bounds offset calculation during pointer retrieval.

Affected files:

• third_party/angle/src/libANGLE/renderer/d3d/ProgramExecutableD3D.cpp
• third_party/angle/src/libANGLE/renderer/d3d/ShaderD3D.cpp
• third_party/angle/src/compiler/translator/tree_ops/hlsl/SeparateArrayConstructorStatements.cpp
• third_party/angle/src/compiler/translator/hlsl/TranslatorHLSL.cpp

Estimated timestamp from git blame: 2018-01-19

Technical Analysis

In ANGLE’s D3D11 backend, a potential vulnerability exists during program linking and uniform register assignment. The root cause is a layout mismatch where a uniform is initially classified as active, but is later stripped from the AST during compiler optimization passes.

1. Variable Collection vs. AST Optimization

When a shader is compiled, ANGLE first runs a variable collection pass via TCompiler::collectVariables (third_party/angle/src/compiler/translator/Compiler.cpp). If a uniform (e.g., u) is referenced in an array constructor (e.g., int[2](u, i++)), it is collected and marked as active = true.

Subsequently, during HLSL translation, SeparateArrayConstructorStatements (third_party/angle/src/compiler/translator/tree_ops/hlsl/SeparateArrayConstructorStatements.cpp) runs. This pass removes side-effect-free arguments from array constructor statements. Since referencing a uniform has no side effects, u is dropped from the AST, while the side-effecting expression i++ is preserved. However, the compiler does not re-run variable collection, meaning u remains marked as an active uniform in the compiled shader metadata.

2. Register Map Omission

During HLSL code generation, OutputHLSL traverses the AST to generate the shader output and registers uniforms in mUniformRegisterMap via ResourcesHLSL. Because u was removed from the AST, it is never visited, and no register is assigned to it.

3. Map End Iterator Dereference

During program linking, ProgramExecutableD3D::defineUniformBase (third_party/angle/src/libANGLE/renderer/d3d/ProgramExecutableD3D.cpp) iterates through the active uniforms. Finding u marked active, it calls:

unsigned int startRegister = shaderD3D->getUniformRegister(uniform.name);

In CompiledShaderStateD3D::getUniformRegister (third_party/angle/src/libANGLE/renderer/d3d/ShaderD3D.cpp):

unsigned int CompiledShaderStateD3D::getUniformRegister(const std::string &uniformName) const
{
    ASSERT(uniformRegisterMap.count(uniformName) > 0);
    return uniformRegisterMap.find(uniformName)->second;
}

In production Release builds, ASSERT is compiled out. Since "u" is missing from uniformRegisterMap, find() returns the end() iterator. Dereferencing the end() iterator (it->second) is undefined behavior. In Chrome’s libc++, this reads from the map’s internal anchor node, returning a semi-controlled garbage value G (reinterpreted from adjacent metadata/pointers).

4. Integer Overflow and Out-of-Bounds Pointer Derivation

• Corrupted Layout Encoding: G is passed to HLSLBlockEncoder::skipRegisters(G), which advances mCurrentOffset by G * 4 components. During type encoding, static_cast<int>(mCurrentOffset * kBytesPerComponent) overflows to a negative signed 32-bit integer (e.g., -4).
• Register Index Corruption: In BlockLayoutEncoder::GetBlockRegister, the negative offset is divided by kBytesPerComponent (which is size_t). Due to Usual Arithmetic Conversions, the negative offset is promoted to a massive unsigned size_t (e.g., 0xFFFFFFFFFFFFFFFC). The unsigned division and subsequent truncation result in an extremely large register index (e.g., 0xFFFFFFFF).
• Tiny Buffer Allocation: In initializeUniformStorage (ProgramExecutableD3D.cpp), the required size is calculated as (registerIndex + registerCount) * 16u. In 32-bit unsigned arithmetic, this multiplication overflows and wraps around to a very small size (e.g., 32 bytes), resulting in a tiny buffer allocation.
• Massive 64-bit Offset: When acquiring the uniform pointer via getDataPointer:

  size_t offset = ((registerIndex * 4 + registerElement) * sizeof(float));
  

  On 64-bit architectures, because sizeof(float) is a 64-bit size_t, the expression is promoted to 64-bit prior to multiplication. Consequently, the calculation does not wrap around and produces a massive offset (e.g., ~16 GB).

This results in a mismatch where the buffer allocated is tiny (e.g., 32 bytes), but the data pointer is assigned an address approximately 16 GB past the buffer. Updates to this uniform via standard GLES calls (e.g., glUniform*) write attacker-controlled data directly to this out-of-bounds memory location.

Potential Reproduction Steps

Note: Our tooling agent currently lacks the ability to execute code; therefore, these are potential steps to reproduce the issue.

1. Initialize a WebGL2 context on a system using the ANGLE D3D11 backend (e.g., Windows Chrome).
2. Compile a fragment shader that declares a uniform and references it in a side-effect-free manner inside an array constructor containing other side-effecting operations. For example:

   #version 300 es
   precision highp float;
   uniform int u;
   uniform UB { float pad; } ub; 
   out vec4 fragColor;
   void main() {
       int i = 0;
       int[2](u, i++); 
       fragColor = vec4(float(i) + ub.pad);
   }
   

3. Link the program. During linking, the browser process / GPU process will map the uniform register incorrectly.
4. Call glUniform1i on the uniform u to trigger the write to the corrupted address.

Suggested Fix

To resolve this issue, ProgramExecutableD3D::defineUniformBase must verify that the uniform actually has a valid mapping in the shader’s register map before attempting to retrieve it, similar to other lookup sites in the class.

Specifically, in ProgramExecutableD3D::defineUniformBase:

const SharedCompiledShaderStateD3D &shaderD3D = mAttachedShaders[shaderType];
if (!shaderD3D->hasUniform(uniform.name))
{
    // Skip or handle the missing uniform register gracefully
    return;
}
unsigned int startRegister = shaderD3D->getUniformRegister(uniform.name);

Evaluated with Chrome root at commit: b1520ef4a76878853a31f0943b565e42060edec8

Results so far have been promising, but there can be wrong deductions. Feel free to adjust as follows:

• If you are familiar with the severity guidelines, you may adjust the severity.
• If this is a false positive, and there’s no work to be done, please close as WAI.
• If there is work to do here but not a vulnerability, please change the issue type to Task/Bug/FR.

Data from false positives will be used to improve accuracy over time. And please feel free to reach out to me directly if you have concerns or feedback on the project.