2feb5ed7bc  Validate identifiers when creating decoders and encoders in LibWebRTCCodecsProxy

The bug at a glance

LibWebRTCCodecsProxy runs in the GPUProcess and services createDecoder/createEncoder IPCs from web/content processes keyed by a caller-supplied VideoDecoderIdentifier/VideoEncoderIdentifier. Without validating that the identifier is not already in use, a malicious or buggy client could create a second decoder/encoder object under an identifier already mapped, producing IPC state confusion in the GPUProcess codec maps. The patch adds MESSAGE_CHECKs that reject duplicate identifiers before object creation, closing a state-confusion primitive in a privileged media process. It is a hardening/robustness fix against a compromised content process, hence high but bounded (no demonstrated corruption).

The angle is trust of a client-chosen identifier used as a map key in a privileged process. The GPUProcess kept m_decoders/m_encoders maps keyed by identifiers the untrusted caller supplies; nothing enforced uniqueness, so a duplicate create could overwrite or shadow an existing live codec’s map entry and desynchronize subsequent per-identifier operations from the object they think they address.

Root cause

LibWebRTCCodecsProxy is the GPUProcess-side receiver for WebRTC codec IPC. Content processes ask it to create video decoders and encoders, each identified by a VideoDecoderIdentifier or VideoEncoderIdentifier that the caller generates and then reuses for every follow-up message (decode, flush, set-format, release, etc.). The proxy stores live objects in m_decoders and m_encoders maps keyed by those identifiers, and dispatches later operations by looking the identifier up.

Before this patch, createDecoder and createEncoder ran on the work queue (assertIsCurrent(workQueue())) and immediately proceeded to build the codec without checking whether the identifier was already present in the map. A compromised or buggy content process could therefore send two createDecoder calls with the same VideoDecoderIdentifier (or reuse the identifier of a still-live decoder). The second create would clobber or race the existing map entry, leaving the GPUProcess’s notion of ‘which object this identifier names’ inconsistent with the client’s – classic IPC state confusion. Depending on downstream code, that can orphan an object (lifetime/UAF risk), route decode data to the wrong codec instance, or double-manage native codec resources.

The fix installs GPUProcess-standard MESSAGE_CHECKs. The file defines MESSAGE_CHECK(assertion) and MESSAGE_CHECK_COMPLETION(assertion, completion) bound to m_connection. In createDecoder it adds MESSAGE_CHECK_COMPLETION(!m_decoders.contains(identifier), callback(false)); and in createEncoder MESSAGE_CHECK_COMPLETION(!m_encoders.contains(identifier), callback(false));. Both checks run right after assertIsCurrent(workQueue()) and before any object is constructed. If the identifier is already mapped, the MESSAGE_CHECK treats the message as invalid: it invokes the completion with false and, per MESSAGE_CHECK semantics, flags the connection as having sent an invalid message (which leads to terminating the offending process). This enforces the invariant that an identifier must be fresh at creation time, eliminating the duplicate-key state-confusion window. The macros are #undef’d at end of file to avoid leaking into other translation units.

Key code

Reject a duplicate decoder/encoder identifier before creation (LibWebRTCCodecsProxy.mm)

#define MESSAGE_CHECK(assertion) MESSAGE_CHECK_BASE(assertion, m_connection)
#define MESSAGE_CHECK_COMPLETION(assertion, completion) MESSAGE_CHECK_COMPLETION_BASE(assertion, m_connection, completion)

// in createDecoder(VideoDecoderIdentifier identifier, ...):
    assertIsCurrent(workQueue());
    MESSAGE_CHECK_COMPLETION(!m_decoders.contains(identifier), callback(false));

// in createEncoder(VideoEncoderIdentifier identifier, ...):
    assertIsCurrent(workQueue());
    MESSAGE_CHECK_COMPLETION(!m_encoders.contains(identifier), callback(false));

Patch walkthrough

• Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm — Defines MESSAGE_CHECK / MESSAGE_CHECK_COMPLETION bound to m_connection; adds a duplicate-identifier check at the top of createDecoder (!m_decoders.contains(identifier)) and createEncoder (!m_encoders.contains(identifier)) before any codec object is created, calling back false on rejection; #undefs the macros at end of file.

Background

GPUProcess and LibWebRTCCodecsProxy — WebKit isolates media/GPU work in a separate GPUProcess. LibWebRTCCodecsProxy is the GPUProcess-side IPC endpoint that owns hardware/software video decoder and encoder instances on behalf of content processes, driving libwebrtc codecs. Because it runs in a more privileged process than the sandboxed content process, every identifier and parameter arriving over IPC is attacker-controllable input that must be validated.

Caller-supplied identifiers as map keys — VideoDecoderIdentifier and VideoEncoderIdentifier are generated by the client and used as keys into m_decoders/m_encoders. The proxy dispatches every subsequent operation for a codec by looking up its identifier. If the client can pick a key that already exists, it controls a collision in a privileged-process data structure, which is the root of the state-confusion risk.

MESSAGE_CHECK / MESSAGE_CHECK_COMPLETION — WebKit’s IPC hardening macro: MESSAGE_CHECK_BASE evaluates an invariant on an incoming message and, if it fails, records that the connection sent an invalid message – normally terminating the sender. MESSAGE_CHECK_COMPLETION additionally runs a supplied completion (here callback(false)) so an async reply is still delivered before the connection is torn down. Binding to m_connection ties the fatal action to the offending peer.

IPC state confusion — When a privileged receiver’s internal state (here the identifier->object maps) can be driven out of sync with the sender’s assumptions, later operations act on the wrong object. Duplicate creation under one key can overwrite a live entry, leak/orphan the previous object, double-free native resources, or misroute decoded frames – a family of bugs that can escalate to memory corruption in the GPUProcess.

Vulnerability window

1. Baseline — createDecoder/createEncoder build codec objects and insert them into m_decoders/m_encoders keyed by a caller-supplied identifier, with no uniqueness check.
2. Exposure — A compromised content process can send create requests reusing an identifier already mapped to a live codec, colliding a privileged-process map key.
3. Confusion — The duplicate create desynchronizes the GPUProcess’s identifier->object mapping from the client’s, enabling misrouted operations or resource mismanagement.
4. Hardening (this commit) — MESSAGE_CHECK_COMPLETION(!m_map.contains(identifier), callback(false)) is added to both create paths, rejecting duplicates before creation and terminating the misbehaving connection (bug 315019, rdar://177338999).
5. Coverage — The commit notes it is ‘Covered by existing tests’ – no new test; validation is a security strengthening of existing create paths.

Triggering

No PoC is included (‘Covered by existing tests’). Trigger: from a compromised content process, send two IPC::Messages::LibWebRTCCodecsProxy::CreateDecoder (or CreateEncoder) requests carrying the same VideoDecoderIdentifier/VideoEncoderIdentifier without releasing the first. Pre-patch the second create proceeds and collides the m_decoders/m_encoders entry; post-patch the second create fails the MESSAGE_CHECK, returns false, and the connection is flagged invalid.

Exploitation

1. Precondition — Attacker must already control a content/web process able to send LibWebRTCCodecsProxy IPC (i.e., a prior content-process compromise), the standard threat model for GPUProcess IPC hardening.
2. Primitive — Pre-patch, sending a create with an already-used identifier collides a map entry in the GPUProcess, potentially overwriting/orphaning a live codec object and desynchronizing subsequent per-identifier operations – a state-confusion primitive that, depending on downstream lifetime handling, could be developed toward UAF or resource double-management in the GPUProcess.
3. Post-patch outcome — The duplicate create is rejected before any object exists and the offending connection is treated as having sent an invalid message (process termination), removing the primitive and converting the attempt into a crash of the misbehaving client.
4. Honesty note — The diff proves only that duplicate creation was unchecked; it does not demonstrate a concrete corruption chain. The severity rests on the state-confusion risk in a privileged process, not a shown exploit.

Detection & hunting

For defenders and SOC / detection engineers:

• GPUProcess MESSAGE_CHECK terminations on codec create — Alert on content/web-process terminations triggered by LibWebRTCCodecsProxy MESSAGE_CHECK failures (duplicate identifier), which post-patch indicate a client sending malformed/duplicate create requests – a strong sign of a compromised or fuzzing content process.
• Anomalous createDecoder/createEncoder patterns — In IPC telemetry, watch for repeated create calls reusing identifiers or abnormally high create rates against LibWebRTCCodecsProxy from a single connection.
• GPUProcess crashes in codec dispatch — On unpatched builds, correlate GPUProcess crashes in libwebrtc codec paths with prior duplicate-identifier create sequences.

Audit directions

• Other caller-keyed maps in GPUProcess proxies — Audit RemoteMediaPlayer, RemoteImageDecoder, SampleBufferDisplayLayer and similar GPUProcess IPC endpoints for create/register paths that insert into identifier-keyed maps without a contains() uniqueness MESSAGE_CHECK.
• Lifetime on release paths — Confirm the corresponding releaseDecoder/releaseEncoder and error paths remove the identifier so a subsequent create is legitimately allowed, and that no window lets an in-flight operation outlive its map entry.
• Completion-before-terminate semantics — Verify MESSAGE_CHECK_COMPLETION callbacks (callback(false)) cannot themselves re-enter the proxy or touch state after the connection is invalidated.
• Identifier validation breadth — Check that all per-identifier operations (decode, encode, setFormat, flush) MESSAGE_CHECK the identifier’s presence, so the create-time uniqueness check is matched by lookup-time existence checks everywhere.