Hangry blog

GSoC 2017: Log #2

28 Jun 2017 | Jitsi GSoC WebRTC

Previously, I have set up a RTCPeerConnection between the main jitis-meet window and the micro window for the video transmission.

However, it hard coded a lot of stuffs and honestly speaking, the code design was a total mess. There were both main window side’s code and micro window side’s code in one page, so it was vulnerable to unintended global variables usage and namespace conflicts. I decided to refactor the code into OOP-style one, so that it can be used in more general case. It is sort of a wrapper for the webkitRTCPeerConnection, suited for the inter-Electron BrowserWindow communication which does not require a STUN server for signalling.

API

Class: WindowPeerConnection

Parameter

  • windowName: the BrowserWindow’s name in main process

The WindowPeeerConnection wraps around webkitRTCPeerConnection class, which on construction in a renderer process, sets up a IPC listeners that receive message and data from other windows. All the message are relayed through the main process, and data are serialized before transmitted.

Method

  • attachStream ( streamToSend ): attaches a MediaStream object on the peer connection channel, which can be transmitted by sendStream method. Currently, it can only attach one stream per connection.
  • removeStream ( ): removes the MediaStream attached to this peer connection channel.
  • sendStream ( receiverName ): sends the attached MediaStream object to the target window specified the string ‘receiverName’. receiverName refers to the name of the variable assigned to the target BrowserWindow in the Electron’s main process.

Event Listener

  • onReceivedStream ( function(receivedStream) ): triggered when the WindowPeerConnection receives a MediaStream from a remote window.

Main Process: Data Relay Channel

A Javascript object that is called in the Electron’s main process. Since Electron does not support direct IPC between BrowserWindows, all messages and data must be relayed through the main process, using ipcMain. Each BrowserWindow is wrapped as a client in the data Channel.

Function

  • addClient ( client ): adds a client on the data relay channel. A client is a Javascript object with the following format: client = { window: browerWindowObject, name: “BrowserWindowName” }

  • removeClient ( clientName ): removes a client with the given window name from the data relay channel.

  • initChannel ( ): initiates the data relay channel with the current array of clients. Clients can be furthered added after the channel is initiated.

  • dispose ( ): disposes the data relay channel.

Example

Main process

main

Sender process

Sender

Receiver process

Receiver

Currently, it only supports transmission of MediaStream, but it could be extended to general Data transmission later on. I might be going to publish this code in npm, since I could find anything similar.


GSoC 2017: Log #1

22 Jun 2017 | Jitsi GSoC WebRTC

First of all, I need to make sure that the large video in the Main window’s Jitsi-meet iframe also appears in the Micro window. If I can’t even access the video elements in the Jitsi-meet iframe, I can’t proceed to anywhere.

There were three approaches I attempted:

  1. Import the video element directly from the render process, using ‘module require’.
  2. Use createObjectURL to generate URL of the video element and pass it to Micro mode’s window.
  3. Instead of creating a separate window for Micro mode, change the layout of main window when the window is minimized.

After trying all three, I realize that none of them actually works. The most important problem I missed out was that each Electron BrowserWindow is an separate, independent process, and Electron does not provides a Inter-Process-Communication(IPC) feature for media elements.

The reasons each approach failed are:

Node’s ‘module require’ does not support transferring of a HTMLMediaElement to another page. It became ‘undefined’ every time I attempted. Similar to the ‘require’ approach, the scope of URL create is limited within the same page. When I tried to receive the media element from the URL, I only got 404 response. This approach only works partially. I can restructure the layout every time the Micro mode is called, but that means the Normal mode and the Micro mode shares the same window. So when the user minimizes the (only) window, the Micro mode’s window gets minimized as well.

In the end, I ended up using webRTC’s RTCPeerConnection to set up a peer-to-peer connection between the main window and the micro window. By right, I was supposed to use a STUN/TURN server to do signalling between the main window and the micro window. But since they are both under the Electron’s main process, I used Electron’s IPC feature instead to pass necessary information for signalling.

After creating html and js files for the Micro mode, I added a module for the peer connection between the main window and the micro window.

The WebRTC connection is established in following order:

Main Window Micro Window
1. Sets up RTCPeerConnection
2. Sets up RTCPeerConnection
3. Attaches jitsi video stream to peer connection
4. Creates and sends connection offer
5. Receives the offer
6. Creates and sends answer
7. Receives the answer
8. Sends ICE candidate
9. Receives and adds ICE candidate
10. Receives the media stream

And then… Voila! rtc


GSoC 2017: Log #0

17 May 2017 | Jitsi GSoC WebRTC

This summer is for code. My proposal has been accepted to the Google Summer of Code(GSoC) 2017, so I would be working under Jitsi for next few months. I never thought I would be accepted as this was my first attempt to GSoC, but hell yeah.

gsoc

Project: Implementing ‘Micro Mode’ for Jitsi-Meet-Electron

URL: https://summerofcode.withgoogle.com/projects/#4757550437236736

This is a simple feature for Jitsi-Meet’s desktop app, which pops up a small video and set of GUIs when the user minimizes the main window. It is a similar one to the one in Skype, hope I can make it cooler than that.

So excited to start working on the project, and many thanks in advance to the mentors Hristo Terezov and Saúl Ibarra Corretgé who will be guiding(tolerating) me for the period of time. Hope I won’t fall behind.


Beginning of All Small Things

02 Apr 2017 |

beginning

I have lots of daily useless thoughts. I guess I can use this blog to pour them out, so I can empty my brain for more productive things. Should I write this blog in English or Korean? Well, apparently I am already writing in English…