Pretty bad H264 performance after upgrading to iOS 15.4

[derMani]

Description

Hi everyone.

after upgrading our phones to iOS 15.4, we are experiencing a pretty bad H264 performance.
Versions before iOS 15.4 with the same source code are not affected by this provlem

I can provide a simple peer 2 peer example if needed

Console output looks normal.

Versions affected

• Cordova version 11
• Cordova iOS version: cordova-ios 5 and cordova-ios 6
• Plugin version 6 and 8.01
• iOS version 15.4
• Xcode version 13.3.1 (13E500a)

Steps to reproduce

Establish an iPhone to Chrome video connection via H264 and compare it with VP8.
A lot of jitter and bad FPS situations will occur, if you observe chrome://webrtc-internals

Both H264 profiles in the SDP with cordova iosrtc are affected

Expected results

Fluid video image, like in previous versions.
Maybe an upgrade of the webrtc lib could help?

Actual results

Stuttering video quality with H.264

[hthetiot]

Versions before iOS 15.4 with the same source code are not affected by this provlem

I don't think compilation of WebRTCLib will help here, and BTW, you don't need that plugin to use WebRTC using iOS 15.4.

Try without the plugin (comment register global) to see if Native iOS WebRTC also have the same performance issue.

[derMani]

Update: Without using the globals of cordova-iosrtc: H264 works just fine with "native" WebRTC of Webkit.

Pros:

• Good performance
• The Video is an actual html element and the streams are actual javascript objects
• One plugin less in your project

Cons:

• You get the camera prompt in every session. You can work around this behaviour in WKWebView with "requestMediaCapturePermissionForOrigin"
• Works only with iOS 15+
• You will run into some weird WKWebView WebRTC bugs

The plugin had a good run, but I think most users of this plugin can now move on to the "native" WebRTC of Webkit

Anyway: The libwebrtc bug remains for H264 performance. This also happens in non cordova based projects when using libwebrtc

[hthetiot]

Thank you @derMani

The plugin had a good run, but I think most users of this plugin can now move on to the "native" WebRTC of Webkit

I agree, since iOS 15+, you don't really need the plugin if you don't have to put speaker on or support old iOS versions. Still you will have to deal with know ios safari webrtc issues that the plugin handle to manage (auto play, speaker, background).

[hthetiot]

In any case I will leave this issue open to.see if updating to latest libwebrtc fix thr issues.

[hthetiot]

@derMani see webrtc-100 PR that update webrtc to v100 that pass the CI (include full build test https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/.github/workflows/main.yml#L59)

[shrhoads]

I think this project may still be useful if the H264 encoding issue can be fixed. It would be nice to be able to provide higher quality encoding than is available in the safari webrtc support.

[hthetiot]

@shrhoads

I think this project may still be useful if the H264 encoding issue can be fixed. It would be nice to be able to provide higher quality encoding than is available in the safari webrtc support.

Then, can you then try #748 to see if iosrtc rendering from h264 is better?

[hthetiot]

Ping @derMani @shrhoads

#744 (comment)

[dl4mmers]

@shrhoads

I think this project may still be useful if the H264 encoding issue can be fixed. It would be nice to be able to provide higher quality encoding than is available in the safari webrtc support.

Then, can you then try #748 to see if iosrtc rendering from h264 is better?

Hey @hthetiot,

I've tested PR #748 on iPhone 14 Pro (iOS 16.3.1) and it fixes the H264 performance and fps issues.