Skip to content

Commit b64841d

Browse files
jan-ivarjan-ivar
committed
Clean up algorithms (removing main thread streams)
1 parent 45066af commit b64841d

File tree

1 file changed

+128
-56
lines changed

1 file changed

+128
-56
lines changed

‎index.bs‎

Lines changed: 128 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -93,30 +93,31 @@ partial interface RTCRtpReceiver {
9393
};
9494
</pre>
9595

96+
This allows manipulation of encoded frames in the media pipeline
97+
between the processing steps of an {{RTCRtpSender}}'s underlying
98+
<dfn>encoder</dfn> and <dfn ignore=''>packetizer</dfn>, and/or
99+
between an {{RTCRtpReceiver}}'s underlying <dfn>depacketizer</dfn>
100+
and <dfn ignore=''>decoder</dfn>.
101+
96102
## Extension operation ## {#operation}
97103

98104
At the time when a codec is initialized as part of the encoder, and the
99105
corresponding flag is set in the {{RTCPeerConnection}}'s {{RTCConfiguration}}
100106
argument, ensure that the codec is disabled and produces no output.
101107

102-
103-
### Stream creation ### {#stream-creation}
104-
105108
At construction of each {{RTCRtpSender}} or {{RTCRtpReceiver}}, run the following steps:
106109
1. Initialize [=this=].`[[transform]]` to null.
107-
1. Initialize [=this=].`[[readable]]` to a new {{ReadableStream}}.
108-
1. <a dfn for="ReadableStream">Set up</a> [=this=].`[[readable]]`. [=this=].`[[readable]]` is provided frames using the [$readEncodedData$] algorithm given |this| as parameter.
109-
1. Initialize [=this=].`[[writable]]` to a new {{WritableStream}}.
110-
1. <a dfn for="WritableStream">Set up</a> [=this=].`[[writable]]` with its [=WritableStream/set up/writeAlgorithm=] set to [$writeEncodedData$] given |this| as parameter and its [=WritableStream/set up/highWaterMark=] set to <code>Infinity</code>.
111-
<p class="note">highWaterMark is set to Infinity to explicitly disable backpressure.</p>
112110
1. Initialize [=this=].`[[pipeToController]]` to null.
113111
1. Initialize [=this=].`[[lastReceivedFrameCounter]]` to <code>0</code>.
114112
1. Initialize [=this=].`[[lastEnqueuedFrameCounter]]` to <code>0</code>.
115113
1. [=Queue a task=] to run the following steps:
116114
1. If [=this=].`[[pipeToController]]` is not null, abort these steps.
117115
2. Set [=this=].`[[pipeToController]]` to a new {{AbortController}}.
118-
<!-- FIXME: Use pipeTo algorithm when available. -->
119-
3. Call <a href="https://streams.spec.whatwg.org/#readable-stream-pipe-to">pipeTo</a> with [=this=].`[[readable]]`, [=this=].`[[writable]]`, preventClose equal to true, preventAbort equal to true, preventCancel equal to true and [=this=].`[[pipeToController]]`'s [=AbortController/signal=].
116+
<!-- FIXME: Fix aborting. -->
117+
<p class="note">TODO: Specify details of how
118+
[=this=].`[[pipeToController]]` aborts streaming in the worker.</p>
119+
120+
### Stream processing ### {#stream-processing}
120121

121122
<p class=note>
122123
Streams backpressure can optimize throughput while limiting processing and memory consumption by pausing data production as early as possible in a data pipeline.
@@ -127,8 +128,6 @@ The User Agent is responsible for doing these adaptations, especially since it c
127128
For those reasons, streams backpressure is disabled in WebRTC encoded transforms.
128129
</p>
129130

130-
### Stream processing ### {#stream-processing}
131-
132131
The <dfn abstract-op>readEncodedData</dfn> algorithm is given a |rtcObject| as parameter. It is defined by running the following steps:
133132
1. Wait for a frame to be produced by |rtcObject|'s encoder if it is a {{RTCRtpSender}} or |rtcObject|'s packetizer if it is a {{RTCRtpReceiver}}.
134133
1. Increment |rtcObject|.`[[lastEnqueuedFrameCounter]]` by <code>1</code>.
@@ -345,7 +344,7 @@ The <dfn method for="SFrameTransform">setEncryptionKey(|key|, |keyID|)</dfn> met
345344
4. Return |promise|.
346345

347346

348-
# RTCRtpScriptTransform # {#scriptTransform}
347+
# Script Transform # {#scriptTransform}
349348

350349
In this section, the capture system refers to the system where media is sourced from and the sender system
351350
refers to the system that is sending RTP and RTCP packets to the receiver system where {{RTCEncodedFrameMetadata}} data is populated.
@@ -906,65 +905,30 @@ Their [=deserialization steps=], given |serialized|, |value| and |realm|, are:
906905
1. Set |value|'s metadata to the platform object representation of |serialized|.`[[metadata]]`
907906
1. Set |value|.`[[data]]` to the [=sub-deserialization=] of |serialized|.`[[data]]`.
908907

909-
## Interfaces ## {#RTCRtpScriptTransformer-interfaces}
910-
<pre class="idl">
911-
[Exposed=DedicatedWorker]
912-
interface RTCTransformEvent : Event {
913-
readonly attribute RTCRtpScriptTransformer transformer;
914-
};
915-
916-
partial interface DedicatedWorkerGlobalScope {
917-
attribute EventHandler onrtctransform;
918-
};
919-
920-
[Exposed=DedicatedWorker]
921-
interface RTCRtpScriptTransformer : EventTarget {
922-
// Attributes and methods related to the transformer source
923-
readonly attribute ReadableStream readable;
924-
Promise&lt;undefined&gt; generateKeyFrame(optional DOMString rid);
925-
Promise&lt;undefined&gt; sendKeyFrameRequest();
926-
// Attributes and methods related to the transformer sink
927-
readonly attribute WritableStream writable;
928-
attribute EventHandler onkeyframerequest;
929-
// Attributes for configuring the Javascript code
930-
readonly attribute any options;
931-
};
932908

909+
# <dfn interface>RTCRtpScriptTransform</dfn> interface # {#RTCRtpScriptTransform-interface}
910+
<pre class="idl">
933911
[Exposed=Window]
934912
interface RTCRtpScriptTransform {
935913
constructor(Worker worker, optional any options, optional sequence&lt;object&gt; transfer);
936914
};
937-
938-
[Exposed=DedicatedWorker]
939-
interface KeyFrameRequestEvent : Event {
940-
constructor(DOMString type, optional DOMString rid);
941-
readonly attribute DOMString? rid;
942-
};
943915
</pre>
944916

945-
## Operations ## {#RTCRtpScriptTransform-operations}
917+
## Constructor ## {#RTCRtpScriptTransform-constructor}
946918

947919
The <dfn constructor for="RTCRtpScriptTransform" lt="RTCRtpScriptTransform(worker, options)"><code>new RTCRtpScriptTransform(|worker|, |options|, |transfer|)</code></dfn> constructor steps are:
948-
1. Set |t1| to an [=identity transform stream=].
949-
2. Set |t2| to an [=identity transform stream=].
950-
3. Set |this|.`[[writable]]` to |t1|.`[[writable]]`.
951-
4. Set |this|.`[[readable]]` to |t2|.`[[readable]]`.
952920
5. Let |serializedOptions| be the result of [$StructuredSerializeWithTransfer$](|options|, |transfer|).
953-
6. Let |serializedReadable| be the result of [$StructuredSerializeWithTransfer$](|t1|.`[[readable]]`, « |t1|.`[[readable]]` »).
954-
7. Let |serializedWritable| be the result of [$StructuredSerializeWithTransfer$](|t2|.`[[writable]]`, « |t2|.`[[writable]]` »).
955921
8. [=Queue a task=] on the DOM manipulation [=task source=] |worker|'s global scope to run the following steps:
956922
1. Let |transformerOptions| be the result of [$StructuredDeserializeWithTransfer$](|serializedOptions|, the current Realm).
957-
2. Let |readable| be the result of [$StructuredDeserializeWithTransfer$](|serializedReadable|, the current Realm).
958-
3. Let |writable| be the result of [$StructuredDeserializeWithTransfer$](|serializedWritable|, the current Realm).
959-
4. Let |transformer| be a new {{RTCRtpScriptTransformer}}.
960-
5. Set |transformer|.`[[options]]` to |transformerOptions|.
961-
6. Set |transformer|.`[[readable]]` to |readable|.
962-
7. Set |transformer|.`[[writable]]` to |writable|.
923+
4. Let |transformer| be the result of [=RTCRtpScriptTransformer/creating=] a
924+
{{RTCRtpScriptTransformer}} with |transformerOptions|.
963925
8. [=Fire an event=] named <dfn event for="DedicatedWorkerGlobalScope">rtctransform</dfn> using {{RTCTransformEvent}} with {{RTCTransformEvent/transformer}} set to |transformer| on |worker|’s global scope.
964926

965927
// FIXME: Describe error handling (worker closing flag true at RTCRtpScriptTransform creation time. And worker being terminated while transform is processing data).
966928

967-
Each RTCRtpScriptTransform has the following set of [$association steps$], given |rtcObject|:
929+
## Algorithms ## {#RTCRtpScriptTransform-algorithms}
930+
931+
Each {{RTCRtpScriptTransform}} has the following set of [$association steps$], given |rtcObject|:
968932
1. Let |transform| be the {{RTCRtpScriptTransform}} object that owns the [$association steps$].
969933
1. Let |encoder| be |rtcObject|'s encoder if |rtcObject| is a {{RTCRtpSender}} or undefined otherwise.
970934
1. Let |depacketizer| be |rtcObject|'s depacketizer if |rtcObject| is a {{RTCRtpReceiver}} or undefined otherwise.
@@ -973,6 +937,97 @@ Each RTCRtpScriptTransform has the following set of [$association steps$], given
973937
1. Set |transformer|.`[[encoder]]` to |encoder|.
974938
1. Set |transformer|.`[[depacketizer]]` to |depacketizer|.
975939

940+
# <dfn interface>RTCRtpScriptTransformer</dfn> interface # {#RTCRtpScriptTransformer-interface}
941+
<pre class="idl">
942+
[Exposed=DedicatedWorker]
943+
interface RTCRtpScriptTransformer : EventTarget {
944+
// Attributes and methods related to the transformer source
945+
readonly attribute ReadableStream readable;
946+
Promise&lt;undefined&gt; generateKeyFrame(optional DOMString rid);
947+
Promise&lt;undefined&gt; sendKeyFrameRequest();
948+
// Attributes and methods related to the transformer sink
949+
readonly attribute WritableStream writable;
950+
attribute EventHandler onkeyframerequest;
951+
// Attributes for configuring the Javascript code
952+
readonly attribute any options;
953+
};
954+
</pre>
955+
956+
## Internal slots ## {#RTCRtpScriptTransformer-internal-slots}
957+
958+
A {{RTCRtpScriptTransformer}} object has the following internal slots.
959+
960+
<table class="data" dfn-for="RTCRtpScriptTransformer" dfn-type="attribute">
961+
<thead>
962+
<tr>
963+
<th>Internal Slot
964+
<th>Description (<em>non-normative</em>)
965+
</tr>
966+
</thead>
967+
<tbody>
968+
<tr>
969+
<td><dfn>`[[depacketizer]]`</dfn>
970+
<td class="non-normative">A [=depacketizer=] or undefined.
971+
</tr>
972+
<tr>
973+
<td><dfn>`[[encoder]]`</dfn>
974+
<td class="non-normative">An [=encoder=] or undefined.
975+
</tr>
976+
<tr>
977+
<td><dfn>`[[options]]`</dfn>
978+
<td class="non-normative">An optional {{Object}}, or null.
979+
</tr>
980+
<tr>
981+
<td><dfn>`[[readable]]`</dfn>
982+
<td class="non-normative">A {{ReadableStream}}.
983+
</tr>
984+
<tr>
985+
<td><dfn>`[[writable]]`</dfn>
986+
<td class="non-normative">A {{WritableStream}}.
987+
</tr>
988+
</tbody>
989+
</table>
990+
991+
<div algorithm="create|creating">
992+
To <dfn for="RTCRtpScriptTransformer" lt="create|creating">create</dfn> an
993+
{{RTCRtpScriptTransformer}}, given an |options| object, perform the following steps:
994+
1. Let |transformer| be a [=new=] {{RTCRtpScriptTransformer}}, with:
995+
: {{RTCRtpScriptTransformer/[[depacketizer]]}}
996+
:: undefined
997+
: {{RTCRtpScriptTransformer/[[encoder]]}}
998+
:: undefined
999+
: {{RTCRtpScriptTransformer/[[options]]}}
1000+
:: |options|
1001+
: {{RTCRtpScriptTransformer/[[readable]]}}
1002+
:: A new {{ReadableStream}}.
1003+
: {{RTCRtpScriptTransformer/[[writable]]}}
1004+
:: A new {{WritableStream}}.
1005+
1. [=ReadableStream/Set up=] |transformer|.{{[[readable]]}}. <p class="note">The
1006+
[$readEncodedData$] algorithm, given |this| as parameter, provides encoded frames to it.
1007+
</p>
1008+
1. Let |writeAlgorithm| be an action that runs [$writeEncodedData$] with |this| as
1009+
parameter and |frame| as input, given |frame|.
1010+
1. [=WritableStream/Set up=] |transformer|.{{[[writable]]}} with its
1011+
[=WritableStream/set up/writeAlgorithm=] set to |writeAlgorithm| and its [=WritableStream/set up/highWaterMark=] set to <code>Infinity</code>.
1012+
<p class="note">highWaterMark is set to Infinity to explicitly disable backpressure.</p>
1013+
1. Return |transformer|.
1014+
1015+
</div>
1016+
1017+
### Methods ### {#RTCRtpScriptTransformer-methods}
1018+
<dl dfn-for="RTCRtpScriptTransformer" class="dictionary-members">
1019+
<dt>
1020+
<dfn for="RTCRtpScriptTransformer" method>getMetadata()</dfn>
1021+
</dt>
1022+
<dd>
1023+
<p>
1024+
Returns the metadata associated with the frame.
1025+
</p>
1026+
</dd>
1027+
</dl>
1028+
1029+
## Operations ## {#RTCRtpScriptTransformer-operations}
1030+
9761031
The <dfn method for="RTCRtpScriptTransformer">generateKeyFrame(|rid|)</dfn> method steps are:
9771032
1. Let |promise| be a new promise.
9781033
1. Run the [$generate key frame algorithm$] with |promise|, |this|.`[[encoder]]` and |rid|.
@@ -1004,6 +1059,23 @@ The <dfn attribute for="RTCRtpScriptTransformer">onkeyframerequest</dfn> EventHa
10041059

10051060
## Events ## {#RTCRtpScriptTransformer-events}
10061061

1062+
<pre class="idl">
1063+
[Exposed=DedicatedWorker]
1064+
interface RTCTransformEvent : Event {
1065+
readonly attribute RTCRtpScriptTransformer transformer;
1066+
};
1067+
1068+
partial interface DedicatedWorkerGlobalScope {
1069+
attribute EventHandler onrtctransform;
1070+
};
1071+
1072+
[Exposed=DedicatedWorker]
1073+
interface KeyFrameRequestEvent : Event {
1074+
constructor(DOMString type, optional DOMString rid);
1075+
readonly attribute DOMString? rid;
1076+
};
1077+
</pre>
1078+
10071079
The following event fires on an {{RTCRtpScriptTransformer}}:
10081080

10091081
* keyframerequest of type {{KeyFrameRequestEvent}} - fired when the sink determines that a key frame has been requested.

0 commit comments

Comments
 (0)