Project Fortify, an experimental security project, has identified the following potential security issue. If you’re a feature owner CC-ed on this bug, please do your best to review these reports without the Chrome Security team. Please see https://chromium.googlesource.com/chromium/src/+/main/docs/security/ai-generated-security-bugs-faq.md for more information.

Overview: A logic error in libvpx’s vp9_int_pro_motion_estimation captures a reference stride before a potential buffer swap to a scaled frame. If scaling occurs, the function uses the stale, larger stride on the new, smaller buffer, leading to a potential heap out-of-bounds read in the renderer process when processing near frame boundaries.

Affected files:

• third_party/libvpx/source/libvpx/vp9/encoder/vp9_mcomp.c

Estimated timestamp from git blame: 2026-03-03

Description

A potential logic error in libvpx can cause a heap out-of-bounds (OOB) read during VP9 motion estimation. The issue arises because the function vp9_int_pro_motion_estimation captures a reference stride value before checking if the reference frame needs to be swapped for a scaled version. When a scaled reference is used (e.g., during dynamic resolution changes), the function incorrectly uses the original, larger stride to perform pointer arithmetic and memory accesses on the new, smaller scaled buffer.

Technical Details

In third_party/libvpx/source/libvpx/vp9/encoder/vp9_mcomp.c, the function vp9_int_pro_motion_estimation contains the following logic:

// line 2280: The stride is captured into a local const variable
const int ref_stride = xd->plane[0].pre[0].stride;
...
// line 2286: Check for a scaled reference
const YV12_BUFFER_CONFIG *scaled_ref_frame =
    vp9_get_scaled_ref_frame(cpi, mi->ref_frame[0]);
...
if (scaled_ref_frame) {
  ...
  // line 2298: The buffer pointer in xd->plane[0].pre[0] is updated to the scaled frame
  // This function ALSO updates xd->plane[0].pre[0].stride to the scaled stride.
  vp9_setup_pre_planes(xd, 0, scaled_ref_frame, mi_row, mi_col, NULL);
}

If scaled_ref_frame is true, vp9_setup_pre_planes correctly updates xd->plane[0].pre[0].buf to point to the scaled buffer and xd->plane[0].pre[0].stride to the new, smaller stride. However, the local variable ref_stride was declared as const at line 2280 and retains the original, larger stride.

The function then proceeds to use this mismatched ref_stride for subsequent memory accesses:

1. 1-D Column Scans: At line 2325, the starting pointer is calculated using ref_stride. During the loop (lines 2326-2329), the pointer ref_buf is advanced by ref_stride for every row. Because ref_stride is larger than the actual buffer’s stride, the pointer drifts further right and down than intended.
2. SAD Calculations: At lines 2305, 2350, 2360, and 2383, the sdf and sdx4df functions are called, passing ref_stride as the row advancement multiplier.

If the encoder is configured initially with a large resolution (yielding a large unscaled stride, e.g., 800), and then reconfigured to a smaller resolution (yielding a smaller scaled stride, e.g., 672), the difference between the strides is substantial. When processing blocks near the bottom boundary of the scaled frame, the accumulated drift from the over-sized stride causes the memory accesses to overshoot the 160-row VP9_ENC_BORDER_IN_PIXELS padding allocated in vpx_realloc_frame_buffer. This results in reading past the end of the YV12_BUFFER_CONFIG allocation into adjacent renderer process heap memory.

Potential Exploit Scenario

This vulnerability can potentially be triggered remotely by an attacker utilizing the WebCodecs API in a sandboxed renderer process.

1. An attacker configures a VideoEncoder for VP9 encoding with a large initial resolution (e.g., 478x755) and encodes a frame to establish a large reference buffer.
2. The attacker reconfigures the encoder to a smaller resolution (e.g., 341x655) and encodes an inter-frame (P-frame) that references the previous frame.
3. The motion estimation logic triggers the buffer swap to a scaled reference frame.
4. The stale, large stride is used during the SAD calculations, causing OOB reads of the renderer heap.
5. The out-of-bounds data influences the motion vectors selected by the encoder, which are subsequently encoded into the output VP9 bitstream.
6. The JavaScript receives the encoded chunk. By analyzing the motion vectors, the attacker might infer the contents of the OOB memory, leading to an information leak that could aid in bypassing ASLR.

Note: Our tooling agent does not have the ability to run code. These are suggested steps based on static analysis.

Suggested Fix

Update the local ref_stride variable after the potential buffer swap, ensuring it matches the stride of the buffer actually being used. This pattern is correctly implemented in vp9_pickmode.c (around line 2300).

// Remove the 'const' from the initial declaration
int ref_stride = xd->plane[0].pre[0].stride;
...
if (scaled_ref_frame) {
  // Swap out the reference frame...
  vp9_setup_pre_planes(xd, 0, scaled_ref_frame, mi_row, mi_col, NULL);
  // Update the local stride variable to match the new buffer
  ref_stride = xd->plane[0].pre[0].stride;
}

Evaluated with Chrome root at commit: 7353d249d9cacf9c7218e1d7b8a39cf39c72d646

Results so far have been promising, but there can be wrong deductions. If this proves to be a false positive, please close as WAI; data from false positives will be used to improve accuracy over time. And please feel free to reach out to me directly if you have concerns or feedback on the project.