Project Fortify, an experimental security project, has identified the following potential security issue. If you’re a feature owner CC-ed on this bug, please do your best to review these reports. Please see https://chromium.googlesource.com/chromium/src/+/main/docs/security/ai-generated-security-bugs-faq.md for more information.

Overview: An incomplete dimension check in MojoVideoEncodeAcceleratorService allows a compromised renderer to configure spatial layers with resolutions far exceeding the input visible size. This mismatch causes the GPU process to allocate an undersized VA-API encoder context and output bitstream buffer while executing the encode operation using the massive spatial layer resolutions, potentially leading to a heap out-of-bounds write inside the GPU process.

Affected files:

• media/mojo/services/mojo_video_encode_accelerator_service.cc
• media/gpu/vaapi/vaapi_video_encode_accelerator.cc
• media/gpu/vaapi/vp9_vaapi_video_encoder_delegate.cc

Estimated timestamp from git blame: 2021-06-25

Summary

A potential heap out-of-bounds (OOB) write vulnerability has been identified in the GPU process’s VA-API video encoder implementation. Due to missing validation checking that configured spatial layer resolutions are bounded by the input visible size (input_visible_size), an attacker in control of a compromised renderer process could supply mismatched configurations. This results in the allocation of an undersized VA-API encoder context and coded buffer while submitting frame encoding parameters using the much larger spatial layer resolutions, leading to potential driver-level heap corruption.

Root Cause Analysis

In media/mojo/services/mojo_video_encode_accelerator_service.cc, MojoVideoEncodeAcceleratorService::Initialize performs validation on the configured spatial layers:

for (const auto& spatial_layer : config.spatial_layers) {
  if (spatial_layer.width > limits::kMaxDimension ||
      spatial_layer.height > limits::kMaxDimension ||
      base::CheckMul<uint64_t>(spatial_layer.width, spatial_layer.height)
              .ValueOrDefault(...) > limits::kMaxCanvas) {
    // Reject
  }
}

While this validates spatial layers against absolute global bounds, it fails to enforce that each spatial layer’s resolution is bounded by config.input_visible_size (or the profile’s maximum resolution).

When a VP9 k-SVC configuration is initialized:

1. Context and Buffer Sizing: VaapiVideoEncodeAccelerator::InitializeTask in media/gpu/vaapi/vaapi_video_encode_accelerator.cc creates the VA-API context via vaapi_wrapper_->CreateContext(encoder_->GetCodedSize()) (line 436). The coded size is determined based on the small input_visible_size (e.g., 320x240), leading to an undersized VA-API context. The output coded buffer (VAEncCodedBuffer) size is similarly determined based on this small resolution.
2. Parameter Submission: In contrast, during the per-frame encoding loop, VP9VaapiVideoEncoderDelegate::SubmitFrameParameters (in media/gpu/vaapi/vp9_vaapi_video_encoder_delegate.cc) populates VAEncPictureParameterBufferVP9 with source and destination dimensions from the spatial layer (frame_width_src and frame_height_src), which can take unvalidated massive values up to 16384x16384 (copied from the configuration config via SVCLayers).
3. Mismatched Execution: The hardware-accelerated VA-API graphics driver is then requested to perform an encode operation with massive dimensions on an undersized context and output buffer, resulting in a potential heap out-of-bounds write on the driver heap inside the sandboxed GPU process.

Suggested / Potential Trigger Steps

Note: These are potential steps and have not been executed, as our tooling agent does not have code execution capabilities.

1. From a compromised renderer, bind the media.mojom.VideoEncodeAccelerator interface.
2. Call Initialize with the following parameters:
   • input_format: PIXEL_FORMAT_NV12
   • input_visible_size: 320x240
   • output_profile: VP9PROFILE_PROFILE0
   • storage_type: kGpuMemoryBuffer
   • inter_layer_pred: kOnKeyPic
   • spatial_layers: [{160x120, ...}, {320x240, ...}, {16384x16384, ...}]
3. The initialization succeeds. The GPU process allocates a VA context for 320x256 and an output coded buffer based on that resolution.
4. Call Encode with an NV12 GpuMemoryBuffer-backed VideoFrame of 320x240.
5. The GPU process executes SubmitFrameParameters with frame_width_src and frame_height_src set to 16384 against the small context and output buffer.
6. This causes the underlying VA-API driver to write past the allocated buffer bounds.

Suggested Fix

To remediate this issue, add validation in MojoVideoEncodeAcceleratorService::Initialize or VaapiVideoEncodeAccelerator::Initialize to explicitly reject configurations where any spatial layer resolution exceeds config.input_visible_size or the supported profile’s maximum resolution limit.

For example, in VaapiVideoEncodeAccelerator::Initialize:

for (const auto& spatial_layer : config.spatial_layers) {
  if (spatial_layer.width > config.input_visible_size.width() ||
      spatial_layer.height > config.input_visible_size.height()) {
    MEDIA_LOG(ERROR, media_log.get()) << "Spatial layer resolution exceeds input_visible_size";
    return {EncoderStatus::Codes::kEncoderInitializationError};
  }
}

Evaluated with Chrome root at commit: b1520ef4a76878853a31f0943b565e42060edec8

Results so far have been promising, but there can be wrong deductions. Feel free to adjust as follows:

• If you are familiar with the severity guidelines, you may adjust the severity.
• If this is a false positive, and there’s no work to be done, please close as WAI.
• If there is work to do here but not a vulnerability, please change the issue type to Task/Bug/FR.

Data from false positives will be used to improve accuracy over time. And please feel free to reach out to me directly if you have concerns or feedback on the project.