Twilio VideoJs adaptive simulcast undefined error, because of missing width and height properties from MediaTrackSettings

[ericzhao0825]

YOU MUST read first!

Please use Community Forum for general technical discussions and questions.

• I have used Google with the error message or bug in association with the library and Cordova words to make sure the issue I'm reporting is only related to iOSRTC.
• I have provided steps to reproduce (e.g. using sample app code https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample or updated extra/renderer-and-libwebrtc-tests.js file).
• I have provided versions of third party library name, ios, Xcode and plugin version and adapter.js version if used.

Note: If the checkboxes above are not checked (which you do after the issue is posted), the issue will be closed, removing this checkbox will result in automatic closed issue.

Versions affected

• Cordova version: 10.0.0
• **Cordova iOS version **: 5
• Plugin version: 8.0.3
• iOS version: 15.4
• Xcode version:13.3
• WebRTC-adapter version (e.g. 7.4.0):
• WebRTC Framework version (e.g. JSSip 3.1.2):

Description

width and height properties are missing from MediaStreamTrack.getSettings().

There is undefined error coming from twilio video js when setting preferredVideoCodecs to adaptive simulcast.
Twilio is checking var _a = track.getSettings(), width = _a.width, height = _a.height; var trackPixels_1 = width * height;

Steps to reproduce

set twilio preferredVideoCodecs to 'auto', then publish a local video

Expected results

able to publish a local video track with adaptive simulcast enabled

Actual results

twilio throws undefined error

[hthetiot]

MediaStreamTrack GetSettings is not implemented in iosrtc for now.

See source links bellow.

• cordova-plugin-iosrtc/js/MediaStreamTrack.js

  Line 109 in a502fcc

  MediaStreamTrack.prototype.getSettings = function () {

• cordova-plugin-iosrtc/js/MediaTrackSettings.js

  Line 8 in a502fcc

  data = data || {};

We can implement height and width, but you have to understand that this cordova plugin is only useful if you want to support iOS 14.5 and bellow, after that you don't need that plugin as ios webview support Webrtc after that.

Related:

• https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSettings

[RSATom]

We can implement height and width, but you have to understand that this cordova plugin is only useful if you want to support iOS 14.5 and bellow, after that you don't need that plugin as ios webview support Webrtc after that.

@hthetiot WebView's WebRTC implementation has some issues. For example it's not possible to manage AVAudioSession from application, since WebView is working in separate process on iOs. So iosrtc still has a meaning in some cases...

[samgabriel]

Actually we tried the the built in WebKit after 14.5 and it was horrible we had to revert back. it works fine on Safari but the performance was horrible in WkWebView.

[samgabriel]

Oh actually it was not the performance it was the fact that WebKit always prompts the user for permission for microphone and camera even after they just give it. The other problem is probably somewhere in twilio video there is a duplicate audio track causing an echo on the user side.

[RSATom]

@samgabriel workaround for repetitive permission request is mentioned in apache/cordova-ios#1166

[hthetiot]

Actually we tried the the built in WebKit after 14.5 and it was horrible we had to revert back. it works fine on Safari but the performance was horrible in WkWebView.

Glad to hear that our implementation is better on WkWebview, we have so much more Optimisations that stay in PR like #748 and #623 #427.

I will try to take a look this weekend and pass the resolution from MediaStream or tracks from swift to thr webview.

[hthetiot]

Note:

CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(captureFormat.formatDescription)

Source: https://chromium.googlesource.com/chromium/src/+/a71e470da08408312f674f095e715a3139130c8a/media/capture/video/apple/video_capture_device_avfoundation.mm