Flutter (Dart)

Sariska Media provides powerful Dart API's for developing real-time applications.

You can integrate audio/video, live streaming cloud recording, transcriptions, language translation, virtual background and many other services on the fly.

This API documentation describes all possible features supported by sariska-media-transport which possibly covers any of your use cases.

Installation​

Dart:​

dart pub add sariska_media_flutter_sdk

Flutter:

 flutter pub add sariska_media_flutter_sdk

The command above will add this to the pubspec.yaml file in your project (you can do this manually):

dependencies:
    sariska_media_flutter_sdk: ^0.0.9

Setting up​

1. Initialize SDK​

After you install the SDK, perform initial setup tasks by running initializeSdk().

import 'package:sariska_media_flutter_sdk/Conference.dart';
import 'package:sariska_media_flutter_sdk/Connection.dart';
import 'package:sariska_media_flutter_sdk/JitsiLocalTrack.dart';
import 'package:sariska_media_flutter_sdk/JitsiRemoteTrack.dart';
import 'package:sariska_media_flutter_sdk/Participant.dart';
import 'package:sariska_media_flutter_sdk/SariskaMediaTransport.dart';
import 'package:sariska_media_flutter_sdk/WebRTCView.dart';

SariskaMediaTransport.initializeSdk();

2. Create Connection​

WebSockets are ideal to keep a single, persistent session. Unlike HTTPS, WebSocket requests are updated almost immediately. To start using the media services, the primary step is to create a Media WebSocket connection.

let token = {your-token}

let connection = new SariskaMediaTransport.JitsiConnection(token, "roomName", isNightly);

//  set isNightly true for latest updates on the features else build will point to stable version

connection.addEventListener("CONNECTION_ESTABLISHED", () {

});

connection.addEventListener("CONNECTION_FAILED", (error){
   if ( PASSWORD_REQUIRED === err ) {
        // token is expired
        connection.setToken(token) // set a new token
   }
});

connection.addEventListener("CONNECTION_DISCONNECTED", () {

});


connection.connect()

3. Create Conference​

Once you have your connection established, the next step is to create a conference. Sariska is backed by the Jitsi architecture.

let conference = connection.initJitsiConference();

conference.addEventListener("CONFERENCE_JOINED", () {
        for (JitsiLocalTrack track in localtracks) {
            conference.addTrack(track);
          }
});

conference.addEventListener("TRACK_ADDED", (track) {
        JitsiRemoteTrack remoteTrack = track;
        for (JitsiLocalTrack track in localtracks){
          if (track.getStreamURL() == remoteTrack.getStreamURL()){
              return;
            }
        }
        if (remoteTrack.getType() == "audio") {
          return;
        }
          streamURL = remoteTrack.getStreamURL();
          replaceChild(remoteTrack);
});

conference.addEventListener("TRACK_REMOVED", (track){
        // Do cater to track removal
});

conference.join();

Now, the conference object will have all events and methods that you would possibly need for any feature that you wish to supplement your application with.

4. Capture local streams​

A MediaStream consists of zero or more MediaStreamTrack objects, representing various audio or video tracks.

Each MediaStreamTrack may have one or more channels. The channel represents the smallest unit of a media stream, such as an audio signal associated with a given speaker, like left or right in a stereo audio track. Here we mostly talk about track.

Map<String, dynamic> options = {};
options["audio"] = true;
options["video"] = true;
options["resolution"] = 240; // 180,  240,  360, vga, 480, qhd, 540, hd, 720, fullhd, 1080, 4k, 2160

SariskaMediaTransport.createLocalTracks(options,(tracks) {
  localTracks = tracks;
});
//default value for framerates are already configured, You don't set all options.

5. Play local stream​

let videoTrack = localTracks[1];

@override
Widget build(BuildContext context) {
  return Container(
      child: Stack(
          children: [
              Align(
                 alignment: Alignment.topLeft,
                 child: WebRTCView(videoTrack.getStreamURL())
              )
          ]
      )
  );
}

//mirror = true or false,
//zOrder = 0, 1, 2
//objectFit = "cover" or "contain"

This will be your most basic conference call. However, we recommend following up with the two further steps to add customized features to enhance your experience.

Note: You don't any audio element to play sound as it is in conjunction with video the stream.

6. User Joined​

The moderator of the meeting controls and gatekeeps the participants. The moderator has exclusive control of the meeting.

If you wish to have a moderator, pass the moderator value as true while generating your token. Moderator has the following permissions:

  • Ability to add a password to a room

  • Ability to grant the moderator role to non-moderators

  • Ability to kick non-moderators or other moderators

  • Ability to mute participates

  • Ability to make everyone see the moderator video (Everyone follows me)

  • Ability to make participants join muted (Everyone starts muted)

  • Ability to make participants join without video (Everyone starts hidden)

  • Ability to enable/disable the lobby room

  • Ability to approve join/knocking requests (when the lobby is enabled)

  • When the moderator leaves, a new one is selected automatically.

7. Publish your stream to other peers​

Use the following code to now publish your call.

for (JitsiLocalTrack track in localTracks) {
    conference.addTrack(track);
}

8. Playing remote peers streams​

@override
Widget build(BuildContext context) {
  return Container(
      child: Stack(
          children: [
            Align(
                alignment: Alignment.topLeft,
                child: SingleChildScrollView(
                    scrollDirection: Axis.horizontal,
                    child: Row(
                        children: remoteTracks.map((track) =>
                        {
                          if (track.getStreamURL() == "video") {
                            Container(
                                width: 240,
                                height: 360,
                                child: WebRTCView(
                                  streamURL: track.getStreamURL(),
                                ))
                          }
                        }))
                )
            )
          ]
      )
  );
}

That's it you are done with a basic conference call, Follow the guide below for more features.

Analytics​

Sariska-media-transport comes with pre-configured top events used to help improvise your product and overall consumer experience.

Few popular events:

  • User left

  • User joined

  • Conference duration

  • Camera duration

  • Audio track duration

  • Video track duration

  • Recording started

  • Recording stopped

  • Transcription started

  • Transcription stopped

  • Local Recording started

  • Local Recording stopped

  • Speaker Stats

We will be updating the list of features soon.


// you can start tracking events just by listening 

conference.addEventListener("ANALYTICS_EVENT_RECEIVED",  (payload){

  // payload will have 

  // payload["name"] of event is string 
  // payload["action"] is string
  // payload["actionSubject"] is string
  // payload["source"] is string
  // payload["attributes"] , JSON of all extra attributed of the event

})

Features​

Active/Dominant Speaker​

You can easily detect the active or the dominant speaker. You could choose to stream only their video, thereby saving on the costs and better resolution to others. This is could be a use case for one-way streaming; such as virtual concerts.

conference.addEventListener("DOMINANT_SPEAKER_CHANGED", (id){
   // let id = as String;  dominant speaker id
});

Last N Speakers​

The idea is that we select a subset of N participants, whose video to show, and we stop the video from others. We dynamically and automatically adjust the set of participants that we show according to who speaks – effectively we only show video for the last N people to have spoken.

// to listen for last n speakers changed event

conference.addEventListener("LAST_N_ENDPOINTS_CHANGED", (leavingEndpointIds, enteringEndpointIds){
 // let leavingEndpointIds =  leavingEndpointIds as List<String>; //Array of ID's of users leaving lastN
 // let enteringEndpointIds = as enteringEndpointIds as List<String>; //Array of ID's of users entering lastN
});

// to set set last n speakers in mid or you can pass option during conference initialization

conference.setLastN(10)

Participant information​

Set Local Participant Property​

// to set local participant property 
conference.setLocalParticipantProperty(key, value);

// name is a string 
// value can be object string or object

// to remove local participant property 
conference.rempveLocalParticipantProperty(key)


// to get local participant propety 
conference.getLocalParticipantProperty(key)


// this notifies everyone in the conference with following PARTICIPANT_PROPERTY_CHANGED event

conference.addEventListener("PARTICIPANT_PROPERTY_CHANGED", (participant, key,oldValue, newValue){

});

Note: Local participant property can be used to set local particpants features example: screen-sharing, setting custom role or any other properties which help us group and identify participants by certain property.

Get participant count​

conference.getParticipantCount(); 

// pass boolean true if you need participant count including hidden participants

Note: Hidden participants are generally bots join the conference along with actual participants. For example: recorder, transcriber, pricing agent.

Get all participants in conference​

conference.getParticipants(); // list of all participants

Get all participants in conference without hidden participants​

conference.getParticipantsWithoutHidden();  // list of all participants without hidden participants

Pin/Select participant​

conference.selectParticipant(participantId)
//Elects the participant with the given id to be the selected participant in order to receive higher video quality (if simulcast is enabled).

Select/Pin Multiple Participants​

conference.selectParticipants(participantIds) // string array of participant Id's 
//Elects the participant with the given id to be the selected participant in order to receive higher video quality (if simulcast is enabled).

Access local user details directly from conference​

conference.isHidden()
confernece.getUserId()
conference.getUserRole()
conference.getUserEmail()
conference.getUserAvatar()
conference.getUserName()

Set meeting subject​

conference.setSubject(subject)

Remote/Local tracks​

Get all remote tracks

conference.getRemoteTracks();  // get all remote tracks

Get all local tracks​

conference.getLocalTracks()

Kick Out​

// notifies that participant has been kicked from the conference by moderator

conference.addEventListener("KICKED", (id){
  //let id = id as String;  id of kicked participant
});

// notifies that moderator has been kicked by other moderator

conference.addEventListener("PARTICIPANT_KICKED", (actorParticipant, kickedParticipant, reason){
   //let actorParticipant = actorParticipant as Participant;
   //let kickedParticipant = kickedParticipant as Participant;
   //let reason = reason as String; reason for kick
})

// to kick participant

conference.kickParticipant(id); // participant id


// to kick moderator 

conference.kickParticipant(id, reason); // participant id, reason for kick

Grant/Revoke Owner​

Grant Owner​

Except for the room creator, the rest of the users have a participatory role. You can grant them owner rights with the following code.

conference.grantOwner(id) // participant id


// listen for role changed event 

conference.addEventListener("USER_ROLE_CHANGED",( id, role){
    // let id = id as String;  id of participant
    //let role = role as String;  new role of user

    if (confernece.getUserId() === id ) {
        // My role changed, new role: role;
    } else {
       // Participant role changed: role;
    }
});

Revoke Owner​

To revoke owner rights from a participant, use the following code.

conference.revokeOwner(id) // participant id

Change Display Name​

// to change your display name

conference.setDisplayName(name); 


// Listens for change display name event if changed by anyone in the conference

conference.addEventListener("DISPLAY_NAME_CHANGED", (id, displayName){
 // let id = id as String;
 // let displayName = displayName as String;
});

Lock/Unlock Room​

Lock room​

A moderator can lock a room with a password. Use the code as follows.

//lock your room with password

conference.lock(password); //set password for the conference; returns Promise

Unlock room​

conference.unlock();

Subtitles​

// requesting subtitles

conference.setLocalParticipantProperty("requestingTranscription",   true);

// if u want request langauge translation also

conference.setLocalParticipantProperty("translation_language", 'hi'); // hi for hindi

// now listen for subtitles received event

conference.addEventListener("SUBTITLES_RECEIVED" , (id, name, text){
  // let id = id as String; id of transcript message
  // let name = name as String; name of speaking participant
  // let text = text as String; // spoken text
});


// stop requesting subtitle

conference.setLocalParticipantProperty("requestingTranscription",   false);

// supported list of langauge codes

// "en": "English",
// "af": "Afrikaans",
// "ar": "Arabic",
// "bg": "Bulgarian",
// "ca": "Catalan",
// "cs": "Czech",
// "da": "Danish",
// "de": "German",
// "el": "Greek",
// "enGB": "English (United Kingdom)",
// "eo": "Esperanto",
// "es": "Spanish",
// "esUS": "Spanish (Latin America)",
// "et": "Estonian",
// "eu": "Basque",
// "fi": "Finnish",
// "fr": "French",
// "frCA": "French (Canadian)",
// "he": "Hebrew",
// "hi": "Hindi",
// "mr":"Marathi",
// "hr": "Croatian",
// "hu": "Hungarian",
// "hy": "Armenian",
// "id": "Indonesian",
// "it": "Italian",
// "ja": "Japanese",
// "kab": "Kabyle",
// "ko": "Korean",
// "lt": "Lithuanian",
// "ml": "Malayalam",
// "lv": "Latvian",
// "nl": "Dutch",
// "oc": "Occitan",
// "fa": "Persian",
// "pl": "Polish",
// "pt": "Portuguese",
// "ptBR": "Portuguese (Brazil)",
// "ru": "Russian",
// "ro": "Romanian",
// "sc": "Sardinian",
// "sk": "Slovak",
// "sl": "Slovenian",
// "sr": "Serbian",
// "sq": "Albanian",
// "sv": "Swedish",
// "te": "Telugu",
// "th": "Thai",
// "tr": "Turkish",
// "uk": "Ukrainian",
// "vi": "Vietnamese",
// "zhCN": "Chinese (China)",
// "zhTW": "Chinese (Taiwan)"

Screen Sharing​

Start Screen Sharing​

A participant supports 2 tracks at a type: audio and video. Screen sharing(desktop) is also a type of video track. If you need screen sharing along with the speaker video you need to have Presenter mode enabled.

Map<String, dynamic> options = {};
options["desktop"] = true;
let videoTrack  = localTracks[1];
SariskaMediaTransport.createLocalTracks(options, (tracks){
    conference.replaceTrack(videoTrack, tracks[0]);
});

Send message​


conference.sendMessage("message"); // group 


conference.sendMessage("message", participantId); // to send private message to a participant 


// Now participants can listen message received event 

conference.addEventListener("MESSAGE_RECEIVED") , (message, senderId){
  // let message = message as String; message text 
  // let senderId = senderId as String; senderId 
});

Transcription​

Start Transcription​

 // at the end of the conference transcriptions will be available to download
conference.startTranscriber();

Stop Transcription​

conference.stopTranscriber();

// at the end of the conference transcriptions will be available to download

Mute/Unmute Participants​

Mute/Unmute Local participant​

track.mute() // to mute track

track.mute() // to unmute track

track.isMuted() // to check if track is already muted

Mute Remote participant​

The moderator can mute any remote participant.

conference.muteParticipant(participantId, mediaType)

// mediaType can be audio or video

Connection Quality​

// New local connection statistics are received. 
conference.addEventListener("LOCAL_STATS_UPDATED", (stats){ 

});


// New remote connection statistics are received.
conference.addEventListener("REMOTE_STATS_UPDATED",  (id, stats){ 

});

Internet Connectivity Status​

SDK is already configured to auto join/leave when internet connection fluctuates.

Peer-to-Peer mode​

Start peer-to-peer mode​

Sariska automatically switches to peer peer-to-peer mode if participants in the conference exactly 2. You can, however, still you can forcefully switch to peer-to-peer mode.

conference.startP2PSession();

Note: Conferences started on peer-to-peer mode will not be charged until the turn server is not used.

Stop peer-to-peer mode​

conference.stopP2PSession();

CallStats integration​

To monitor your WebRTC application, simply integrate the callstats or build your own by checking out the RTC Stats section.

Map<String, dynamic> options = {};
options["callStatsID"] = 'callstats-id';
options["callStatsSecret"] = 'callstats-secret';

let conference = connection.initJitsiConference(options);

Join Muted/Silent​

Join Silent( no audio will be sent/receive)​

join conference with silent mode no audio sent/receive

Map<String, dynamic> options = {};
options["startSilent"] = true;

let confernce = connection.initJitsiConference(options);

Join Muted​

To start a conference with already muted options.

// to join a conference with already muted audio and video 

Map<String, dynamic> options = {};
options["startAudioMuted"] = true;
options["startVideoMuted"] = true;

const conference = connection.initJitsiConference(options);

Live Streaming​

Stream to YouTube​

Map<String, dynamic> options = {}; 
options["broadcastId"] = "youtubeBroadcastID"; // put any string this will become part of your publish URL
options["mode"] = "stream"; // here mode will be stream
options["streamId"] = "youtubeStreamKey"; 
 
 // to start the live stream

conference.startRecording(options);

You can get the youtube stream key manually by login to your youtube account or using google OAuth API

Stream to Facebook​

Map<String, dynamic> options = {};
options["mode"] = "stream"; // here mode will be stream
options["streamId"] = "rtmps://live-api-s.facebook.com:443/rtmp/FB-4742724459088842-0-AbwKHwKiTd9lFMPy"; // facebook stream URL

 // to start live-stream

conference.startRecording(options);

You can get facebook streamId manually by login int to your facebook account or using Facebook OAuth API.

Stream to Twitch​

Map<String, dynamic> options = {};      
options["mode"] = "stream"; // here mode will be stream     
options["streamId"] = "rtmp://live.twitch.tv/app/STREAM_KEY";       
// to start live stream     
conference.startRecording(options);

Stream to any RTMP server

Map<String, dynamic> options = {};
options["mode"] = "stream"; // here mode will be stream
options["streamId"] = "rtmps://rtmp-server/rtmp"; // RTMP server URL

// to start live stream

conference.startRecording(options);

Listen for RECORDER_STATE_CHANGED event to know live streaming status

conference.addEventListener("RECORDER_STATE_CHANGED", (sessionId, mode, status){
   String sessionId = sessionId as String;  // sessionId of live streaming session
   String mode = mode as String;           // mode will be stream
   String status = status as String;       // status of live streaming session it can be on, off or pending
});

Stop Live Streaming​

conference.stopRecording(sessionId);

Cloud Recording​

// Config for object based storage AWS S3, Google Cloud Storage, Azure Blob Storage or any other S3 compatible cloud providers are supported. Login to your Sariska dashboard to set your credentials , we will upload all your recordings and transcriptios.


Map<String, dynamic> options = {};
options["mode"] = "file";
options["serviceName"] = "s3";


// config options for dropbox

Map<String, dynamic> options = {};
options["mode"] = "file";
options["serviceName"] = "dropbox";
options["token"] = "dropbox_oauth_token";


// to start cloud recording
conference.startRecording(options);

//listen for RECORDER_STATE_CHANGED event to know what is happening

conference.addEventListener("RECORDER_STATE_CHANGED", (sessionId, mode, status){
   String sessionId = sessionId as String;   // sessionId of  cloud recording session
   String mode = mode as String;             // here mode will be file
   String status = status as String;         // status of cloud recording session it can be on, off or pending
});

Stop Cloud Recording​

conference.stopRecording(sessionId)

PSTN​

Dial-in(PSTN)​


String phonePin = conference.getPhonePin();

String phoneNumber  = conference.getPhoneNumber();

// Share this Phone Number and Phone Pin to anyone who can join conference call without internet.

Dial-out(PSTN)​

conference.dial(phoneNumber)

// dialing someone to join conference using their phone number

Lobby/Waiting room​

To enable the feature for waiting room/lobby checkout API's below


// to join lobyy 

conference.joinLobby(displayName, email);

// This notifies everyone at the conference with the following events

conference.addEventListener("LOBBY_USER_JOINED", (id, name){
  //  let id = id as String;
  //  let name = name as String;
})

conference.addEventListener("LOBBY_USER_UPDATED",  (id, participant){
  //  let id = id as String;
  //  let participant = participant as Participant;
})

conference.addEventListener("LOBBY_USER_LEFT", (id){
  //  let id = id as String;

})

conference.addEventListener("MEMBERS_ONLY_CHANGED",  (enabled){
  //  let id = id as bool;
})

// now a conference moderator can allow/deny 

conference.lobbyDenyAccess(participantId); //to deny lobby access


conference.lobbyApproveAccess(participantId); // to approve lobby mode


// others methods

conference.enableLobby() //to enable lobby mode in the conference call moderator only 

conference.disableLobby(); //to disable lobby mode in the conference call moderator only 

conference.isMembersOnly(); // whether conference room is members only. means lobby mode is disabled

Video SIP Calling​

// start sip gateway session

conference.startSIPVideoCall("address@sip.domain.com", "display name"); // your sip address and display name

// stop sip call

conference.stopSIPVideoCall('address@sip.domain.com');

// after you create your session now you can track the state of your session

conference.addEventListener("VIDEO_SIP_GW_SESSION_STATE_CHANGED", (state){
// let state = state as String;
// state can be on, off, pending, retrying, failed
})

// check if gateway is busy

conference.addEventListener("VIDEO_SIP_GW_AVAILABILITY_CHANGED",  (status){
// let status = status as String;
// status can be busy or available 
})

One-to-one calling​

One to one calling is more of synchronous way of calling where you deal with things like

  • Calling someone even if his app is closed or background

  • Play a busy tone if a user is busy on another call or disconnected your call

  • Play ringtone/ringback/dtmftone

This is similar to how WhatsApp works.

  • Make an HTTP call to the Sariska server

URL: https://api.sariska.io/api/v1/media/poltergeist/create?room=sessionId&token=your-token&status=calling&user=12234&name=Kavi
Method: GET

where paramters are 
*  room: current session sessionId of the room you joined inviteCallee
*  token:  your jwt token 
*  status: calling 
*  user: callee user id 
*  name: callee user name 
*  domain: 'sariska.io'

  • Send push notifications to callee using your Firebase or APNS account

  • Callee now reads the push notification using ConnectionService or CallKit even if the app is closed or in the background

  • Callee can update his status back to the caller just by making an updated HTTP Call, no needs to join the conference via SDK

URL: https://api.sariska.io/api/v1/media/poltergeist/update?room=sessionId&token=your-token&status=accepted&user=12234&name=Kavi
Method: GET

where paramters are
*  room: current session sessionId of the room you joined inviteCallee
*  token:  callee jwt token
*  status: accepted or rejected
*  user: callee user id
*  name: callee user name
*  domain: 'sariska.io'

  • Since the Caller has already joined the conference using SDK he can easily get the status just by listening USER_STATUS_CHANGED event

conference.addEventListener("USER_STATUS_CHANGED", (id, status){
    let id = id as String; // id of callee
    let status = status as String; // status can be ringing, busy, rejected, connected, expired

    // ringing if callee changed status to ringing 
    // busy if callee is busy on ther call
    // rejected if callee has rejected your call
    // connected if callee has accepted your call
    // expired if callee is not able to answered within 40 seconds an expired status will be trigger by sariska 
});

  • After the callee has joined the conference rest of the steps are the same as the normal conference call

Calendar Sync​

Now you can programmatically start scheduling a meeting with google/microsoft calendar.

Slack integration​

This integration adds the /sariska slash command for your team so that you can start a video conference in your channel, making it easy for everyone to just jump on the call. The slash command, /sariska, will drop a conference link in the channel for all to join.

Mentioning one or more teammates, after /sariska, will send personalized invites to each user mentioned. Check out how it is integrated here.

RTC Stats​

Low-level logging on peer connection API calls and periodic getStats calls for analytics/debugging purposes. Make sure you have passed RTCstats WebSocket URL while initializing the conference. Check out how to configure RTCStats WebSocket Server here.

Logging​

TRACE
DEBUG
INFO
LOG
WARN
ERROR

Last updated