Java
Sariska Media provides powerful Java API's for developing real-time applications.
You can integrate audio/video, live streaming cloud recording, transcriptions, language translation and many other services on the fly.
This API documentation describes all possible features supported by sariska-media-transport which possibly covers any of your use cases.
Installation
Use pre-built SDK artifacts/binaries
In your project, add the Maven repository https://github.com/SariskaIO/sariska-maven-repository/raw/master/releases
and the dependency io.sariska:sariska-media-transport
into your build.gradle
files.
The repository typically goes into the build.gradle
file in the root of your project:
allprojects {
repositories {
maven {
url "https://github.com/SariskaIO/sariska-maven-repository/raw/master/releases"
}
google()
mavenCentral()
maven { url 'https://www.jitpack.io' }
}
}
In recent versions of Android Studios, allprojects{}
might not be found in build.gradle
. In that case, the repository goes into the settings.gradle
file in the root of your project:
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
maven {
url "https://github.com/SariskaIO/sariska-maven-repository/raw/master/releases"
}
maven {
url "https://maven.google.com"
}
}
}
Maven
Dependency definitions belong in the individual module build.gradle
files:
dependencies {
// (other dependencies)
implementation 'io.sariska:sariska-media-transport:5.4.9'
}
Setting up
1. Initialize SDK
After you install the SDK, perform initial setup tasks by running initializeSdk()
.
import io.sariska.sdk.SariskaMediaTransport;
import io.sariska.sdk.Connection;
import io.sariska.sdk.Conference;
import io.sariska.sdk.JitsiRemoteTrack;
import io.sariska.sdk.JitsiLocalTrack;
import io.sariska.sdk.Participant;
SariskaMediaTransport.initializeSdk();
2. Create Connection
WebSockets are ideal to keep a single, persistent session. Unlike HTTPS, WebSocket requests are updated almost immediately. To start using the media services, the primary step is to create a Media WebSocket connection.
String token = {your-token};
Connection connection = SariskaMediaTransport.JitsiConnection(token, "roomName", false);
// set isNightly true for latest updates on the features else build will point to stable version
connection.addEventListener("CONNECTION_ESTABLISHED" ()->{
});
connection.addEventListener("CONNECTION_FAILED", (error) -> {
if (error === "PASSWORD_REQUIRED") {
// token is expired
connection.setToken(token) // set a new token
}
});
connection.addEventListener("CONNECTION_DISCONNECTED", () -> {
});
connection.connect()
3. Create Conference
Once you have your connection established, the next step is to create a conference. Sariska is backed by the Jitsi architecture.
Conference conference = connection.initJitsiConference(options);
conference.join();
//Additional options initJitsiConference accepts to enable more feature
// startAudioMuted: true // to join audio muted
// startVideoMuted: true // to join video muted
// startSilent : true // to start conference in silent no audio recieve/send
// rtcstatsServer: “” // send data to rtcstats server
// callStatsID: “” // callStatsID to enble callstats
// callStatsSecret: “” //callstatssecret
// channelLastN: 10
4. Capture local streams
A MediaStream consists of zero or more MediaStreamTrack objects, representing various audio or video tracks.
Each MediaStreamTrack may have one or more channels. The channel represents the smallest unit of a media stream, such as an audio signal associated with a given speaker, like left or right in a stereo audio track. Here we mostly talk about track.
Bundle options = new Bundle();
options.putBoolean("audio", true);
options.putBoolean("video", true);
options.putInt("resolution", 240); // 180, 240, 360, vga, 480, qhd, 540, hd, 720, fullhd, 1080, 4k, 2160
// options.putBoolean("desktop", true); for screen sharing
// options.putString("facingMode", "user"); user or environment
// options.putString("micDeviceId", "mic_device_id");
// options.putString("cameraDeviceId", "camera_device_id");
// options.putString("minFps", 20);
// options.putString("maxFps", 24);
SariskaMediaTransport.createLocalTracks(options, tracks -> {
localTracks = tracks;
});
5. Play local stream
runOnUiThread(() -> {
for (JitsiLocalTrack track : localTracks) {
if (track.getType().equals("video")) {
WebRTCView view = track.render();
view.setObjectFit("cover");
mLocalContainer.addView(view);
}
}
})
// WebRTCView class provides additional methods to manage View styling
// view.setMirror( mirror)->true or false if you want to mirror your video to other participants
// view.setObjectFit("cover")—> Can be "contain" or "cover"
// view.setZOrderMediaOverlay(0)-> can be 0 or 1
// view.setZOrderOnTop(0)-> 0, 1, 2
This will be your most basic conference call. However, we recommend following up with the two further steps to add customized features to enhance your experience.
Note: You don't any audio element to play sound as it plays in conjunction with video stream.
6. User Joined
The moderator of the meeting controls and gatekeeps the participants. The moderator has exclusive control of the meeting.
If you wish to have a moderator, pass the moderator
value as true while generating your token. Moderator has the following permissions:
Ability to add a password to a room
Ability to grant the moderator role to non-moderators
Ability to kick non-moderators or other moderators
Ability to mute participates
Ability to make everyone see the moderator video (Everyone follows me)
Ability to make participants join muted (Everyone starts muted)
Ability to make participants join without video (Everyone starts hidden)
Ability to enable/disable the lobby room
Ability to approve join/knocking requests (when the lobby is enabled)
When the moderator leaves, a new one is selected automatically
conference.addEventListener("USER_JOINED", (id, participant)=>{
// String id = (String) id
// Participant participant = (Participant) participant
// joined Participant class has the following popular properties which you can use to maintain UI states
// avatar
// email
// moderator
// audioMuted
// videoMuted
// displayName
// role
// status
// hidden
// botType
// Generally participants are bots like transcriber, recorder
});
7. Publish your stream to other peers
Use the following code to now publish your call.
for (JitsiLocalTrack track : localTracks) {
conference.addTrack(track);
}
8. Playing remote peers streams
conference.addEventListener("TRACK_ADDED", (track) -> {
JitsiRemoteTrack track = (JitsiRemoteTrack) track;
runOnUiThread(() -> {
if (track.getType().equals("video")) {
WebRTCView view = track.render();
view.setObjectFit("cover");
mRemoteContainer.addView(view);
}
});
});
That's it you are done with a basic conference call, Follow the guide below for more features.
Analytics
Sariska-media-transport comes with pre-configured top events used to help improvise your product and overall consumer experience.
Few popular events:
User left
User joined
Conference duration
Camera duration
Audio track duration
Video track duration
Recording started
Recording stopped
Transcription started
Transcription stopped
Local Recording started
Local Recording stopped
Speaker Stats
We will be updating the list of features soon.
// you can start tracking events just by listening
conference.startTracking();
Features
Active/Dominant Speaker
You can easily detect the active or the dominant speaker. You could choose to stream only their video, thereby saving on the costs and better resolution to others. This is could be a use case for one-way streaming; such as virtual concerts.
conference.addEventListener("DOMINANT_SPEAKER_CHANGED", (id)=> {
//String id = (String) id; dominant speaker id
});
Last N Speakers
The idea is that we select a subset of N participants, whose video to show, and we stop the video from others. We dynamically and automatically adjust the set of participants that we show according to who speaks – effectively we only show video for the last N people to have spoken.
// to listen for last n speakers changed event
conference.addEventListener("LAST_N_ENDPOINTS_CHANGED", (leavingEndpointIds, enteringEndpointIds)=>{
// String[] leavingEndpointIds = ( String[] ) leavingEndpointIds; //Array of ID's of users leaving lastN
// String[] enteringEndpointIds = ( String[] ) enteringEndpointIds; //Array of ID's of users entering lastN
});
// to set last n speakers in mid or you can pass option during conference initialization
conference.setLastN(10)
Participant information
Set Local Participant Property
// to set local participant property
conference.setLocalParticipantProperty(key, value);
// name is a string
// value can be object string or object
// to remove local participant property
conference.rempveLocalParticipantProperty(key)
// to get local participant propety
conference.getLocalParticipantProperty(key)
// this notifies everyone in the conference of the following PARTICIPANT_PROPERTY_CHANGED event
conference.addEventListener("PARTICIPANT_PROPERTY_CHANGED", (participant, key,oldValue, newValue) => {
});
Get participant count
conference.getParticipantCount();
// pass boolean true if you need participant count including hidden participants
Note: Hidden participants are generally bots join the conference along with actual participants. For example: recorder, transcriber, pricing agent.
Get all participants in conference
conference.getParticipants(); // list of all participants
Get all participants in conference without hidden participants
conference.getParticipantsWithoutHidden(); // list of all participants
Pin/Select participant
conference.selectParticipant(participantId)
//Elects the participant with the given id to be the selected participant in order to receive higher video quality (if simulcast is enabled).
Select/Pin Multiple Participants
conference.selectParticipants(participantIds) // string array of participant Id's
//Elects the participant with the given id to be the selected participant in order to receive higher video quality (if simulcast is enabled).
Access local user details directly from conference
conference.isHidden()
confernece.getUserId()
conference.getUserRole()
conference.getUserEmail()
conference.getUserAvatar()
conference.getUserName()
Set meeting subject
conference.setSubject(subject)
Remote/Local tracks
Get all remote tracks
conference.getRemoteTracks(); // get all remote tracks
Get all local tracks
conference.getLocalTracks()
Kick Out
// notifies that participant has been kicked from the conference by modeator
conference.addEventListener("KICKED", (id)=> {
//String id = (String) id; id of kicked participant
});
// notifies that moderator has been kicked from the conference by another moderator
conference.addEventListener("PARTICIPANT_KICKED", (actorParticipant, kickedParticipant, reason)=>{
//Participant actorParticipant = (Participant) actorParticipant;
//Participant kickedParticipant = (Participant) kickedParticipant;
//String reason = (String) reason; moderator has to give reason for kick
})
// method to kick out a participant
confernece.kickParticipant(id) // participant id
Grant/Revoke Owner
Grant Owner
Except for the room creator, the rest of the users have a participatory role. You can grant them owner rights with the following code.
conference.grantOwner(id) // participant id
// listen for role-changed event
conference.addEventListener("USER_ROLE_CHANGED", (id, role) =>{
//String id = (String) id; id of participant
//String role = (String) role; new role of user
if (confernece.getUserId() === id ) {
// My role changed, new role: role;
} else {
// Participant role changed: role;
}
});
Revoke Owner
To revoke owner rights from a participant, use the following code.
conference.revokeOwner(id) // participant id
Change Display Name
// to change your display name
conference.setDisplayName(name);
// Listens for change display name event if changed by anyone in the conference
conference.addEventListener("DISPLAY_NAME_CHANGED", (id, displayName)=>{
// String id = (String) id;
// String displayName = (String) displayName;
});
Lock/Unlock Room
Lock room
A moderator can lock a room with a password. Use the code as follows.
//lock your room with a password
conference.lock(password); //set password for the conference; returns Promise
Unlock room
conference.unlock();
Subtitles
// requesting subtitles
conference.setLocalParticipantProperty("requestingTranscription", true);
// if you want to request langauge translation also
conference.setLocalParticipantProperty("translation_language", 'hi'); // hi for hindi
// now listen for subtitles received event
conference.addEventListener("SUBTITLES_RECEIVED", (id, name, text)=> {
// String id = (String) id; id of transcript message
// String name = (String) name; name of speaking participant
// String text = (String) text; // spoken text
});
// stop requesting subtitles
conference.setLocalParticipantProperty("requestingTranscription", false);
// supported list of language codes
// "en": "English",
// "af": "Afrikaans",
// "ar": "Arabic",
// "bg": "Bulgarian",
// "ca": "Catalan",
// "cs": "Czech",
// "da": "Danish",
// "de": "German",
// "el": "Greek",
// "enGB": "English (United Kingdom)",
// "eo": "Esperanto",
// "es": "Spanish",
// "esUS": "Spanish (Latin America)",
// "et": "Estonian",
// "eu": "Basque",
// "fi": "Finnish",
// "fr": "French",
// "frCA": "French (Canadian)",
// "he": "Hebrew",
// "hi": "Hindi",
// "mr":"Marathi",
// "hr": "Croatian",
// "hu": "Hungarian",
// "hy": "Armenian",
// "id": "Indonesian",
// "it": "Italian",
// "ja": "Japanese",
// "kab": "Kabyle",
// "ko": "Korean",
// "lt": "Lithuanian",
// "ml": "Malayalam",
// "lv": "Latvian",
// "nl": "Dutch",
// "oc": "Occitan",
// "fa": "Persian",
// "pl": "Polish",
// "pt": "Portuguese",
// "ptBR": "Portuguese (Brazil)",
// "ru": "Russian",
// "ro": "Romanian",
// "sc": "Sardinian",
// "sk": "Slovak",
// "sl": "Slovenian",
// "sr": "Serbian",
// "sq": "Albanian",
// "sv": "Swedish",
// "te": "Telugu",
// "th": "Thai",
// "tr": "Turkish",
// "uk": "Ukrainian",
// "vi": "Vietnamese",
// "zhCN": "Chinese (China)",
// "zhTW": "Chinese (Taiwan)"
Screen Sharing
Start Screen Sharing
A participant supports 2 tracks at a type: audio and video. Screen sharing(desktop) is also a type of video track. If you need screen sharing along with the speaker video you need to have Presenter mode enabled.
Bundle options = new Bundle();
options.putBoolean("desktop", true);
JitsiLocalTrack videoTrack = localTracks[1];
SariskaMediaTransport.createLocalTracks(options, tracks -> {
conference.replaceTrack(videoTrack, tracks[0]);
});
Send message
conference.sendMessage("message"); // group
conference.sendMessage("message", participantId); // to send private message to a participant
// Now participants can listen to message received event
conference.addEventListener("MESSAGE_RECEIVED", (message, senderId)=>{
// message sent by sender
// senderId
});
Transcription
Start Transcription
conference.startTranscriber();
Stop Transcription
conference.stopTranscriber();
// at the end of the conference transcriptions will be available to download
Mute/Unmute Participants
Mute/Unmute Local participant
track.mute() // to mute track
track.mute() // to unmute track
track.isMuted() // to check if track is already muted
Mute Remote participant
The moderator can mute any remote participant.
conference.muteParticipant(participantId, mediaType)
// mediaType can be audio or video
Connection Quality
// New local connection statistics are received.
conference.addEventListener("LOCAL_STATS_UPDATED", (stats)=>{
});
// New remote connection statistics are received.
conference.addEventListener("REMOTE_STATS_UPDATED", (id, stats)=>{
});
Internet Connectivity Status
SDK is already configured to auto-join/leave when the internet connection fluctuates.
Peer-to-Peer mode
Start peer-to-peer mode
Sariska automatically switches to peer peer-to-peer mode if participants in the conference exactly 2. You can, however, still, you can forcefully switch to peer-to-peer mode.
conference.startP2PSession();
Note: Conferences started on peer-to-peer mode will not be charged until the turn server is not used.
Stop peer-to-peer mode
conference.stopP2PSession();
CallStats integration
To monitor your WebRTC application, simply integrate the call stats or build your own by checking out the RTC Stats section.
Bundle options = new Bundle();
options.putString("callStatsID", 'callstats-id');
options.putString("callStatsSecret", 'callstats-secret');
Conference conference = connection.initJitsiConference(options);
Join Muted/Silent
Join Silent( no audio will be sent/receive)
join conference with silent mode no audio sent/receive
Bundle options = new Bundle();
options.putBoolean("startSilent", true);
Conference confernce = connection.initJitsiConference(options);
Join Muted
To start a conference with already muted options.
Bundle options = new Bundle();
options.putBoolean("startAudioMuted", true);
options.putBoolean("starVideoMuted", true);
Conference conference = connection.initJitsiConference(options);
Live Streaming
Stream to YouTube
Bundle options = new Bundle();
options.putString("broadcastId", "youtubeBroadcastID"); // put any string this will become part of your publish URL
options.putString("mode", "stream"); // here mode will be stream
options.putString("streamId", "youtubeStreamKey");
// to start live stream
conference.startRecording(options);
You can get youtube stream key manually by login to your youtube account or use google OAuth API
Stream to Facebook
Bundle options = new Bundle();
options.putString("mode", "stream"); // here mode will be stream
options.putString("streamId", "rtmps://live-api-s.facebook.com:443/rtmp/FB-4742724459088842-0-AbwKHwKiTd9lFMPy"); // facebook stream URL
// to start live stream
conference.startRecording(options);
You can get facebook streamId manually by login to your facebook account or use Facebook OAuth API.
Stream to Twitch
Bundle options = new Bundle();
options.putString("mode", "stream"); // here mode will be stream
options.putString("streamId", "rtmp://live.twitch.tv/app/STREAM_KEY"); // switch
// to start live stream
conference.startRecording(options);
Stream to any RTMP server
Bundle options = new Bundle();
options.putString("mode", "stream"); // here mode will be stream
options.putString("streamId", "rtmps://rtmp-server/rtmp"); // RTMP server URL
// to start live stream
conference.startRecording(options);
Listen for RECORDER_STATE_CHANGED event to know live streaming status
conference.addEventListener("RECORDER_STATE_CHANGED", (sessionId, mode, status)=>{
String sessionId = (String) sessionId; // sessionId of live streaming session
String mode = (String) mode; // mode will be stream
String status = (String) status; // status of live streaming session it can be on, off or pending
});
Stop Live Streaming
conference.stopRecording(sessionId);
Cloud Recording
// Config for object-based storage AWS S3, Google Cloud Storage, Azure Blob Storage or any other S3 compatible cloud providers are supported. Login to your Sariska dashboard to set your credentials , we will upload all your recordings and transcriptios.
Bundle options = new Bundle();
options.putString("mode", "file");
options.putString("serviceName", "s3");
// config options for dropbox
Bundle options = new Bundle();
options.putString("mode", "file");
options.putString("serviceName", "dropbox");
options.putString("token", "dropbox_oauth_token");
// to start cloud recording
conference.startRecording(options);
//listen for RECORDER_STATE_CHANGED event to know what is happening
conference.addEventListener("RECORDER_STATE_CHANGED", (sessionId, mode, status)=>{
String sessionId = (String) sessionId; // sessionId of cloud recording session
String mode = (String) mode; // here mode will be file
String status = (String) status; // status of cloud recording session it can be on, off or pending
});
Stop Cloud Recording
conference.stopRecording(sessionId)
PSTN
Dial-in(PSTN)
String phonePin = conference.getPhonePin();
String phoneNumber = conference.getPhoneNumber();
// Share this Phone Number and Phone Pin to anyone who can join a conference call without internet.
Dial-out(PSTN)
conference.dial(phoneNumber)
// dialing someone to join conference using their phone number
Lobby/Waiting room
To enable the feature for waiting room/lobby checkout APIs below
// to join a lobby
conference.joinLobby(displayName, email);
// This notifies everyone at the conference of the following events
conference.addEventListener("LOBBY_USER_JOINED", (id, name) => {
});
conference.addEventListener("LOBBY_USER_UPDATED", (id, participant)=> {
});
conference.addEventListener("LOBBY_USER_LEFT", id=> {
});
conference.addEventListener("MEMBERS_ONLY_CHANGED", enabled=> {
});
// now a conference moderator can allow/deny
conference.lobbyDenyAccess(participantId); //to deny lobby access
conference.lobbyApproveAccess(participantId); // to approve lobby mode
// other methods
conference.enableLobby(); //to enable lobby mode in the conference call moderator only
conference.disableLobby(); //to disable lobby mode in the conference call moderator only
conference.isMembersOnly(); // whether conference room is members only. means lobby mode is disabled
Video SIP Calling
// start sip gateway session
conference.startSIPVideoCall("[email protected]", "display name"); // your sip address and display name
// stop sip call
conference.stopSIPVideoCall('[email protected]');
// after you create your session now you can track the state of your session
conference.addEventListener("VIDEO_SIP_GW_SESSION_STATE_CHANGED", (state)=>{
// String state = (String) state;
// state can be on, off, pending, retrying, failed
});
// check if the gateway is busy
conference.addEventListener("VIDEO_SIP_GW_AVAILABILITY_CHANGED", (status)=>{
// String status = (String) status;
// status can be busy or available
});
One-to-one calling
One-to-one calling is more of the synchronous way of calling where you deal with things like
Calling someone even if his app is closed or background
Play a busy tone if a user is busy on another call or disconnected your call
Play ringtone/ringback/dtmftone
This is similar to how WhatsApp works.
Make an HTTP call to the Sariska server
URL: https://api.sariska.io/api/v1/media/poltergeist/create?room=sessionId&token=your-token&status=calling&user=12234&name=Kavi
Method: GET
where paramters are
* room: current session sessionId of the room you joined inviteCallee
* token: your jwt token
* status: calling
* user: callee user id
* name: callee user name
* domain: 'sariska.io'
Send push notifications to callee using your Firebase or APNS account
Callee now reads the push notification using ConnectionService or CallKit even if the app is closed or in the background
Callee can update his status back to the caller just by making an updated HTTP Call, no needs to join the conference via SDK
URL: https://api.sariska.io/api/v1/media/poltergeist/update?room=sessionId&token=your-token&status=accepted&user=12234&name=Kavi
Method: GET
where parameters are
* room: current session sessionId of the room you joined invite Callee
* token: callee jwt token
* status: accepted or rejected
* user: callee user-id
* name: callee user name
* domain: 'sariska.io'
Since, the Caller has already joined the conference using SDK he can easily get the status just by listening USER_STATUS_CHANGED event
conference.addEventListener("USER_STATUS_CHANGED", (id, status) => {
String id = (String) id; // id of callee
String status = (String) status; // status can be ringing, busy, rejected, connected, expired
// ringing if callee changed status to ringing
// busy if callee is busy on ther call
// rejected if callee has rejected your call
// connected if callee has accepted your call
// expired if callee is not able to answered within 40 seconds an expired status will be trigger by sariska
});
After the callee has joined the conference rest of the steps are the same as the normal conference call
Calendar Sync
Now you can programmatically start scheduling a meeting with google/microsoft calendar.
Slack integration
This integration adds the /sariska slash command for your team so that you can start a video conference in your channel, making it easy for everyone to just jump on the call. The slash command, /sariska, will drop a conference link in the channel for all to join.
Mentioning one or more teammates, after /sariska, will send personalized invites to each user mentioned. Check out how it is integrated here.
RTC Stats
Low-level logging on peer connection API calls and periodic getStats calls for analytics/debugging purposes. Make sure you have passed RTCstats WebSocket URL while initializing the conference. Check out how to configure RTCStats WebSocket Server here.
Logging
TRACE
DEBUG
INFO
LOG
WARN
ERROR
Last updated