Project Fortify, an experimental security project, has identified the following potential security issue. If you’re a feature owner CC-ed on this bug, please do your best to review these reports. Please see https://chromium.googlesource.com/chromium/src/+/main/docs/security/ai-generated-security-bugs-faq.md for more information.

Overview: A potential heap buffer overflow exists in libyuv::I010ToNV12 when processing 10-bit video on systems with AVX-512BW support. The function incorrectly selects an unmasked SIMD kernel based on the luma plane’s alignment but applies it to narrower chroma planes, leading to an out-of-bounds write in the renderer process.

Affected files:

• third_party/libyuv/source/convert.cc
• third_party/libyuv/source/row_gcc.cc
• media/video/mappable_shared_image_video_frame_pool.cc

Estimated timestamp from git blame: 2024-08-13

Summary

A potential heap buffer overflow (OOB write) has been identified in libyuv::I010ToNV12 when executed on CPUs supporting the AVX-512BW instruction set. The vulnerability arises from a mismatch in the SIMD kernel dispatching logic where an optimized ‘bare’ kernel (designed for 64-pixel aligned data) is selected based on the luma width but then invoked on chroma planes which have half the width. This results in the kernel processing more data than the destination buffer can hold.

Root Cause Analysis

In third_party/libyuv/source/convert.cc, the I010ToNV12 function converts 10-bit YUV 4:2:0 data to 8-bit NV12. It uses a local function pointer Convert16To8Row to handle the conversion of chroma planes ($U$ and $V$).

On AVX-512BW systems, the kernel selection logic is as follows (lines 708-715):

#if defined(HAS_CONVERT16TO8ROW_AVX512BW)
  if (TestCpuFlag(kCpuHasAVX512BW)) {
    Convert16To8Row = Convert16To8Row_Any_AVX512BW;
    if (IS_ALIGNED(width, 64)) {                  // <--- Dispatched on luma 'width'
      Convert16To8Row = Convert16To8Row_AVX512BW;
    }
  }
#endif

When the luma width is a multiple of 64 (e.g., 192), the bare Convert16To8Row_AVX512BW kernel is selected. This kernel processes data in unconditional 64-pixel iterations and does not handle partial remainders.

However, in the chroma processing loop (lines 781-788), this kernel is invoked using halfwidth (which is (width + 1) >> 1):

for (y = 0; y < halfheight; ++y) {
  Convert16To8Row(src_u, row_u, scale, halfwidth); // halfwidth = 96 for width = 192
  Convert16To8Row(src_v, row_v, scale, halfwidth);
  MergeUVRow(row_u, row_v, dst_uv, halfwidth);
  // ...
}

If width = 192, then halfwidth = 96. The bare AVX-512BW kernel will perform two 64-pixel iterations, attempting to write 128 bytes. The internal temporary buffer row_u is allocated using align_buffer_64(row_u, ((halfwidth + 31) & ~31) * 2), which for halfwidth = 96 provides only 192 bytes total for both $U$ and $V$ rows (96 bytes each). The final iteration of the kernel writes 32 bytes beyond the intended 96-byte boundary of each chroma row, resulting in a heap buffer overflow.

Reachability and Impact

This function is reachable in the Chromium renderer process via media/video/mappable_shared_image_video_frame_pool.cc (line 529). This occurs on Windows when converting 10-bit software-decoded video frames (e.g., AV1 or VP9 Profile 2) to NV12 for hardware-accelerated rendering.

An attacker who can provide a crafted video with specific dimensions (e.g., a coded width of 192 pixels) can trigger this overflow. The overflowed data consists of attacker-controlled chroma samples, providing a primitive for heap corruption and potential remote code execution (RCE). As the corruption occurs in a malloc’d buffer within third-party C code, it is not protected by MiraclePtr.

Suggested Fix

The dispatch logic in I010ToNV12 should be corrected to check the alignment of halfwidth instead of width when selecting the kernel for chroma row processing:

// Suggested fix in third_party/libyuv/source/convert.cc
#if defined(HAS_CONVERT16TO8ROW_AVX512BW)
  if (TestCpuFlag(kCpuHasAVX512BW)) {
    Convert16To8Row = Convert16To8Row_Any_AVX512BW;
    if (IS_ALIGNED(halfwidth, 64)) {  // Check halfwidth alignment for chroma
      Convert16To8Row = Convert16To8Row_AVX512BW;
    }
  }
#endif

Alternatively, using the _Any_ variant for chroma rows is a safer default if the width and halfwidth alignments differ.

Potential Steps to Reproduce

Note: These are potential steps as the vulnerability has been identified via code analysis.

1. Run Chromium on a CPU supporting AVX-512BW (e.g., Intel Ice Lake or newer).
2. Open a webpage that plays a 10-bit AV1 video with a coded width of 192 pixels.
3. Ensure the renderer uses the MappableSharedImageVideoFramePool for conversion to NV12.
4. The OOB write should occur during the I010ToNV12 conversion call.

Evaluated with Chrome root at commit: 29093e11cf509e3593f6229e4b1b075cca356049

Results so far have been promising, but there can be wrong deductions. Feel free to adjust as follows:

• If you are familiar with the severity guidelines, you may adjust the severity.
• If this is a false positive, and there’s no work to be done, please close as WAI.
• If there is work to do here but not a vulnerability, please change the issue type to Task/Bug/FR.

Data from false positives will be used to improve accuracy over time. And please feel free to reach out to me directly if you have concerns or feedback on the project.