Difference between revisions of "Feature:Audio/video communication"

From Kosmos Wiki
Jump to navigation Jump to search
(more on hyperchannel)
(IRC is the way to go for prototype)
Line 1: Line 1:
 
== Summary ==
 
== Summary ==
  
−
67P comes with integrated video- and audio chat capabilities. In the beginning, this is limited to 1on1 calls, but we will support conferencing and/or broadcasting at a later point. WebRTC (Web Real-time Communication) <http://webrtc.org> is used to enable this. WebRTC currently works in Firefox, Chrome, FF and Chrome on Android, Windows Opera and some lesser known browsers (e.g. Bowser on iOS). We might switch to ORTC <http://ortc.org> at some point, because that lower-level standard will be supported in Internet Explorer also.
+
67P comes with integrated video- and audio chat capabilities. In the beginning, this is limited to 1on1 calls, but we will support conferencing and/or broadcasting at a later point. WebRTC ([http://webrtc.org "Web Real-time Communication"]) is used to enable this. WebRTC currently works in Firefox, Chrome, FF and Chrome on Android, Opera and some lesser known browsers (e.g. Bowser on iOS). We might switch to ORTC ([http://ortc.org "Object RTC"]) at some point, which is a lower-level standard that will be supported in Internet Explorer also.
−
 
 
−
=== Points to Discuss (read on for more info) ===
 
−
 
 
−
* Text chat connection in stand-alone version
 
−
* How to do the signaling
 
  
 
== Features ==
 
== Features ==
  
−
67P RTC will be two things:
+
67P RTC is an integrated featuren in HyperChannel. Only direct 1on1 calls will be implemented for now.
  
−
* A stand-alone web application
+
=== HyperChannel Integration ===
−
* An integrated video/audio call feature in HyperChannel
 
  
−
Both implementations share a lot of their client-code and use the same webrtc backend stack. Only direct 1on1 calls will be implemented for now.
+
A video stream (or audio symbol for non-video) will be integrated into the 67P UI (maybe in an extra tab?) You can also "pop-out" this UI component, which makes working on multiple screens easier (and requesting full screen simpler).
  
−
=== HyperChannel Integration ===
+
=== Feature Ideas for Later™ ===
−
 
 
−
A video stream/audio symbol will be integrated into the 67P UI. You can also "pop-out" this UI component (which could be the same as the SA view or not), which makes working on multiple screens easier (and request full screen simpler).
 
  
 +
* Screensharing
 +
* Conferencing
 +
* Broadcasting
 +
* Call in to a WebRTC call via plain old phone system
  
−
=== Stand Alone Application ===
+
==== Stand Alone Application ====
  
 
The WebRTC Stand Alone Application will be needed in the following cases:
 
The WebRTC Stand Alone Application will be needed in the following cases:
Line 30: Line 26:
 
* Inviting people to an 1on1 chat that are not part of the current 67P organisation (maybe via email)
 
* Inviting people to an 1on1 chat that are not part of the current 67P organisation (maybe via email)
  
−
=== Feature Ideas for Later™ ===
+
The SA application displays the video stream of the other party, however, some kind of text communication channel is still needed (people want to exchange links etc) - This will be done via IRC or WebRTC's DataChannels.
−
 
 
−
* Screensharing
 
−
* Conferencing
 
−
* Broadcasting
 
−
* Call in to a WebRTC call via plain old phone system
 
  
 
== Technical Implementation ==
 
== Technical Implementation ==
  
 
'''Components:''' Client/UI Integration, Signaling Backend (might be IRC/SH), TURN Relay, SFU/MCU for Conferencing > 4
 
'''Components:''' Client/UI Integration, Signaling Backend (might be IRC/SH), TURN Relay, SFU/MCU for Conferencing > 4
−
 
−
=== Integration of Stand Alone App ===
 
−
 
−
The SA application displays the video stream of the other party, however, some kind of text communication channel is still needed (people want to exchange links etc). There are at least three possibile variants, how this could work:
 
−
 
−
* Create some anonymous IRC-Channel "on-the-fly" (Drawback: Has to be integrate with IRC Server, not really "stand-alone" anymore)
 
−
* No Text-Chat, but Button "promote this to HyperChannel session" (Drawback: No text chat initially)
 
−
* Provide P2P Text-Chat via WebRTC's DataChannels (Drawback: Alternative Text Chat implementation, no HC Enhancements, Logs, etc)
 
−
 
−
''Jan: Currently, my favorite option would be option 1 (if that's technically possible easily) or option 3''
 
  
 
=== Signaling ===
 
=== Signaling ===
Line 57: Line 38:
 
* Peers can exchange some messages w/ each other (like "offer: i can do this codec" or "ice: i have this kind of nat")
 
* Peers can exchange some messages w/ each other (like "offer: i can do this codec" or "ice: i have this kind of nat")
  
−
We already have some components in the stack that can exchange messages. But it might be better to let an external server/service handle this job. Here are three ways, this could work:
+
It is implemented on top of the existing IRC channels (via SockerHub) using special message formats, which will get parsed by the HC Client. For backwards compat (later), it could include a link to the SA app:
−
 
 
−
==== Option 1: Signaling via IRC ====
 
−
 
 
−
Peers exchange their signaling messages over IRC, using some "special" message formats. For backwards compat, it includes a link to the SA app:
 
  
 
     A → B [67p-rtc-invite] Here is your video call: https://playground.p67.io/rtc/bilfgneehlrieli
 
     A → B [67p-rtc-invite] Here is your video call: https://playground.p67.io/rtc/bilfgneehlrieli
Line 67: Line 44:
 
     A → B [67p-rtc-params] (Signaling messages)
 
     A → B [67p-rtc-params] (Signaling messages)
 
     ...
 
     ...
−
 
−
If B is not an HC client, it won't send the "accepted", so signaling will be done directly in the SA app, once the link is accessed. Option 3 would work very similar.
 
−
 
−
'''Advantages:'''
 
−
* No extra component/infrastructure required
 
−
* "Communication Channel" between peers already exists (at least in HC)
 
−
 
−
'''Disadvantages:'''
 
−
* Stand-alone app needs IRC connection
 
−
* Signaling must start from web app (Browser required) - cannot be done directly from IRC-Client
 
−
 
−
==== Option 2: Signaling via a SocketHub platform ====
 
−
 
−
SocketHub could have a WebRTC platform, that is used to find the peer and exchange the signaling messages. The verbs would probably be: send, join, leave, observe
 
−
 
−
'''Advantages:'''
 
−
* No extra component/infrastructure required
 
−
 
−
'''Disadvantages:'''
 
−
* SocketHub focus is more to wrap existing services, instead of handling accounts/peers directly. All peers would have be connected to the same SH to be able to send each other messages. The webfinger approach would be a way to work around this, but this is out of scope (we don't want to require each possible participant to have a webfinger record)
 
−
 
−
==== Option 3: External Signaling ====
 
−
 
−
Signaling could be done by an external component that does room based signaling ("signaling server") like palava.tv/signaling.io/smoke-signals/signalmaster/xmpp/whatever). Both, HyperChannel and the SA app integrate some code to connect this signaling component, and don't implement the code themselves. We don't have to decide on a specific server/service. Jan is currently working on a open source javascript library ("signaling.js"), that abstracts the signaling provider and offers the same api to the developer, who can use it togehter with a webrtc client library like "simple-peer". The prototype is implemented and it will be released soon. Another option would be, to a create a SocketHub webrtc platform, which wraps a signaling server/service.
 
−
 
−
'''Advantages:'''
 
−
* Same client code can be used in HC and in stand-alone
 
−
* Separation of concerns: main app does not need to know about webrtc-signaling-internals
 
−
 
−
'''Disadvantages:'''
 
−
* (Mostly) this means client establish an extra ws-connection
 
−
* More operations (or relying on 3rd party)
 
−
 
−
 
−
''Jan: I would suggest option 3 (since it seems most flexible), but also like option 1 a lot (especially if we go with always IRC in the stand-alone sessions)''
 

Revision as of 13:42, 4 December 2014

Summary

67P comes with integrated video- and audio chat capabilities. In the beginning, this is limited to 1on1 calls, but we will support conferencing and/or broadcasting at a later point. WebRTC ("Web Real-time Communication") is used to enable this. WebRTC currently works in Firefox, Chrome, FF and Chrome on Android, Opera and some lesser known browsers (e.g. Bowser on iOS). We might switch to ORTC ("Object RTC") at some point, which is a lower-level standard that will be supported in Internet Explorer also.

Features

67P RTC is an integrated featuren in HyperChannel. Only direct 1on1 calls will be implemented for now.

HyperChannel Integration

A video stream (or audio symbol for non-video) will be integrated into the 67P UI (maybe in an extra tab?) You can also "pop-out" this UI component, which makes working on multiple screens easier (and requesting full screen simpler).

Feature Ideas for Later™

  • Screensharing
  • Conferencing
  • Broadcasting
  • Call in to a WebRTC call via plain old phone system

Stand Alone Application

The WebRTC Stand Alone Application will be needed in the following cases:

  • One of the participants is not on HC, but using an IRC Client
  • Both parties are using IRC. The Server/Mod generates a room link on the SA app for them. Or maybe: Don't support this scenario?
  • Inviting people to an 1on1 chat that are not part of the current 67P organisation (maybe via email)

The SA application displays the video stream of the other party, however, some kind of text communication channel is still needed (people want to exchange links etc) - This will be done via IRC or WebRTC's DataChannels.

Technical Implementation

Components: Client/UI Integration, Signaling Backend (might be IRC/SH), TURN Relay, SFU/MCU for Conferencing > 4

Signaling

WebRTC requires "signaling" which serves two purposes:

  • Peers meet each other
  • Peers can exchange some messages w/ each other (like "offer: i can do this codec" or "ice: i have this kind of nat")

It is implemented on top of the existing IRC channels (via SockerHub) using special message formats, which will get parsed by the HC Client. For backwards compat (later), it could include a link to the SA app: 

   A → B [67p-rtc-invite] Here is your video call: https://playground.p67.io/rtc/bilfgneehlrieli
   B → A [67p-rtc-accept] Accepted
   A → B [67p-rtc-params] (Signaling messages)
   ...
Retrieved from "https://wiki.kosmos.org/index.php?title=Feature:Audio/video_communication&oldid=118"