Millicast Player Plugin

MillicastPlayer plugin for Unreal Engine 4

  • Supported UE version 4.27
  • Supported on Windows and Linux

This plugin enable to play real time stream from Millicast in your Unreal Engine game.
You can configure your credentials and configure your game logic using unreal object and then render the video in a texture2D.

Installation

You can install the plugin from the source code.
Follow these steps :

  • create a project with the UE editor.
  • Close the editor
  • Go at the root of your project folder (C:\Users\User\Unreal Engine\MyProject)
  • Create a new directory "Plugins" and move into it
  • Clone the MillicastPlayer repository : git clone https://github.com/millicast/millicast-player-unreal-engine-plugin.git MillicastPlayer
  • Open your project with UE

It will prompt you, saying if you want to re-build MillicastPlayer plugin, say yes.
You are now in the editor and can build your game using MillicastPlayer.

Note: After you package your game, it is possible that you will get an error when launching the game :

"Plugin MillicastPlayer could not be load because module MillicastPlayer has not been found"

And then the game fails to launch.
That is because Unreal has excluded the plugin.
If that is the case, create an empty C++ class in your project. This will force Unreal to include the plugin. Then, re-package the game, launch it, and it should be fixed.

Enable the plugin

To enable the plugin, open the plugin manager in Edit > Plugins.

Then search for MillicastPlayer. It is in the category "Media Players". Tick the "enabled" checkbox to enable the plugin. It will prompt you if you are sure to use this plugin, because it is in beta, just accept. After that Unreal will reboot in order to load the plugins.

If it is already enabled, just leave it like this.

Setup your stream in the editor

Basically, you have several Unreal objects to configure in order to view a stream in your game.

First, we will see how to configure a media source to setup your Millicast credentials. Then how to render the video in a texture 2D, by attaching it to an object in the World. Finally, implement the logic of the game using a blueprint class.

MillicastMediaSource

The media source object allow you to configure your credentials and is the source of the webrtc video stream.
To add a MillicastMediaSource object, add a new asset by clicking "Add/Import", and you will see the object in the "media" category.

After that, double click on the asset you just created so you can start configuring it.
The menu is divided in two categories. First, the millicast credentials :

  • the stream name you want to subscribe to
  • your account id
  • Whether you need to use the subscribe token or not. If you are using a secure viewer, enable it and enter your subscribe token. Otherwise, leave it to the default.
  • The subscribe api url, which usually is https://director.milllicast.com/api/director/susbcribe

The second categories is the Video Texture (MillicastTexture2D) object use to render the video frames. Either you create this asset directly here, or assign another object already created.

MillicastTexture2D

The MillicastTexture2D is a texture you can apply to an object in your World. Once it is created (from the MillicastMediaSource object for instance), drag'n drop it in the editor, on the object that will be used to render the video.
In the image below, I created a plane and apply the texture to it. You can see the plane is now black because no frame are being rendered right now. When you apply the texture, you will see that a material object is created in your assets.

Blueprint

We will now see how to implement the logic using the blueprint.
To do something really simple, the game will subscribe to millicast when it starts playing and unsubscribe when it ends.
To do that, the two important components are :

  • MillicastDirectorComponent : this object is used to authenticate to millicast using the credentials you provided in the millicast media source and get the websocket url and jwt.
  • MillicastSubscriberComponent: this object subscribes to the WebRTC stream by using the WebSocket url and the Jwt.

To do all this, first create a blueprint class. It will open a window, select actor.

Add it in your World (by drag'n drop, it is not visible when the game is playing) and double click on it.
Go in the event graph.
Add the Director and subscriber component.

Both component need the instance of the MillicastMediaSource object. Assign it by clicking one component and go in the panel on the right. Be careful and assign the one you already created. Do not create a new one.

Director Component

The important method here is "Authenticate". Drag'n drop your object in the graph and make a connection with this method. It returns a boolean to say if the request has been made or not.

If the request is successful, it will fire an event, "OnAuthenticated". That event forward a parameter, the Signaling information, which are the websocket url and the jwt.

If the request is not successful, it fires the event, "OnAuthenticationFailure" with the HTTP error code and the status message.

Connect the event "begin play" to the "Authenticate" method so the game will make a request to millicast when starting.

Subscriber Component

Connect the event "OnAuthenticated" with the signaling info to the method Subscribe. This method will subscribe to the Millicast stream and receive WebRTC audio and video tracks.

Then add a new event, "Event End Play" which is an event fired when the game ends, and connect it to the method "Unsubscribe" to unsubscribe from the stream.

Example

Here is an example of how to connect everything.

Launch the game

First broadcast some media to millicast from Millicast Studio, OBS, your browser ...
Note that for now, only VP8 and VP9 are supported by the plugin.
You can quickly test if everything works by playing the game in the editor.
You can subscribe to millicast and render the video track in the texture.
However, the audio will not play in the editor.
You need to launch the game on your platform, or package it to hear audio.

Setup the stream name dynamically

In the above example, the credentials were configured in the Media Source, but we couldn't change them directly from the game. The game was just launching and subscribing to Millicast with the stream name we configured from the editor.
Let's see now how to modify the stream name on a given event that happens in the game.
To illustrate this, we will walk through an example project, where when we shoot a cube, the game starts subscribing to a specific stream, and if we shoot on another cube, the game subscribes to another stream. Each cube is associated with a cube.

Basically, we must be able to detect a collision event on the cube, which will set the stream name on the Millicast Source and then call an event in our main blueprint to start the subscribing. To modify the Millicast Source object's stream name, we must get a reference on the object, either in the blueprints or in a C++ class. We will see both methods.

First let's see the main blueprint. Create a blueprint and add it in your game environment. Open the event graph.

The logic here is : we create a custom event "EventStartViewing". This event will be called by the blueprint associated with the cube, when a collision occurs. When it is fired, we check if we are already subscribed. If so, we stop subscribing and stop the viewer. Otherwise, we start it.
Let's see how to trigger a collision. Create a blueprint, or add a box trigger around the cube and replace the blueprint with a custom one. Check "Simulation Generates hit" to generate hit triggers, and "Block All" collision presets.

Now we can open the associated blueprint. Add an "Event Hit". Now let's see how to configure the Media Source object.

Using only blueprint

When using only the blueprint, add a public variable of type "MillicastMediaSource". Then, you can just set the stream name like the event graph below.

Close the event graph editor, in the settings of the box trigger, add the instance of the MillicastMediaSource object you already own, and the main blueprint instance. All public members are visible in the settings.

Using a C++ class

Create a C++ class (actor component in this example).
When Visual Studio opens, first add MillicastPlayer as a dependency in your Project.build.cs file.

PrivateDependencyModuleNames.AddRange(new string[] { "MillicastPlayer" });

Now, you can include and link against Millicast player public object.
The header file below has an instance of a UMillicastMediaSource object. It is set when Initialize is called.
ConfigureSource is the method we will call from the blueprint event graph to configure the stream name.

// USourceConfiguratorComponent Header

#pragma once

#include "CoreMinimal.h"
#include "Components/ActorComponent.h"
#include "MillicastMediaSource.h"


#include "SourceConfiguratorComponent.generated.h"

UCLASS( ClassGroup=(Custom), meta=(BlueprintSpawnableComponent) )
class MYPROJECT2_API USourceConfiguratorComponent : public UActorComponent
{
    GENERATED_BODY()

private:
    UPROPERTY(EditDefaultsOnly, Category = "Properties", META = (DisplayName = "Millicast Media Source", AllowPrivateAccess = true))
        UMillicastMediaSource* MillicastMediaSource = nullptr;

public: 
    // Sets default values for this component's properties
    USourceConfiguratorComponent();

    bool Initialize(UMillicastMediaSource* Source = nullptr);

    UFUNCTION(BlueprintCallable, Category = "Component", META = (DisplayName = "ConfigureSource"))
    void ConfigureSource(FString StreamName, FString AccountId);

    UFUNCTION(BlueprintCallable, Category = "Component", META = (DisplayName = "ConfigureSourceWithUrl"))
    void ConfigureSourceWithUrl(FString StreamName, FString AccountId, FString ApiUrl);

    UFUNCTION(BlueprintCallable, Category = "Component", META = (DisplayName = "ConfigureSecureSourceWithUrl"))
    void ConfigureSecureSourceWithUrl(FString StreamName, FString AccountId, FString ApiUrl, FString SubscribeToken);

protected:
    // Called when the game starts
    virtual void BeginPlay() override;

public: 
    // Called every frame
    virtual void TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction) override;

        
};
// USourceConfiguratorComponent source

bool USourceConfiguratorComponent::Initialize(UMillicastMediaSource* InMediaSource)
{
    if (MillicastMediaSource == nullptr && InMediaSource != nullptr)
    {
        MillicastMediaSource = InMediaSource;
    }

    return InMediaSource != nullptr && InMediaSource == MillicastMediaSource;
}

void USourceConfiguratorComponent::ConfigureSource(FString StreamName, FString AccountId)
{
    MillicastMediaSource->StreamName = std::move(StreamName);
    MillicastMediaSource->AccountId = std::move(AccountId);
}

Now, build your solution.
Return in unreal engine editor. Open the blueprint associated in the box trigger. Add the Source configurator component.

Don't forget to set up the MillicastMediaSource instance in the right panel. This is the step that will call our Initialize function.

Now, you can make the event graph :

Now, just launch the game, and shoot on one of the cube to start one stream or another.
This is a simple example, where we have hardcoded stream names in the blueprint event graph. But you can do something more complex, like some REST API call in your C++ class when the collision occurs and then configure the MediaSource object.


Did this page help you?