Introduction

This post describes a way to share (large amounts of) data between the UI layer and the RT (real time) layer of a VST3 plugin. Although it is not recommended to actually share data (messaging is the preferred approach in VST3), there are use cases where using pure messaging is very expensive.

For example, in my SAM-SPL 64 VST plugin, the user can load a sample to work with (which can be arbitrarily large). The UI layer needs the sample because it renders it as a waveform (including an editing tab where you can zoom in and out of the waveform). The RT layer needs the sample because it plays it. Using messaging requires copying the sample multiple times (at the very minimum UI → messaging, messaging → RT) and also keeping 2 distinct copies.

High level description

The main idea is to use a pair of std::shared_ptr pointing to the same (immutable) object when sharing. When one side updates its std::shared_ptr then they are pointing to different objects and a (very lightweight) message is sent to the other side so that it can update its own pointer to the new one: they now point to the same object again.

// start
- UI shared_ptr -> nullptr
- RT shared_ptr -> nullptr

// new object (A) in RT
- UI shared_ptr -> nullptr
- RT shared_ptr -> A | version 1
- RT sends "version 1" message to UI
- UI receives "version 1" notification and copies RT shared_ptr
- UI shared_ptr -> A | version 1

// new object (B) in UI
- RT shared_ptr -> A | version 1
- UI shared_ptr -> B | version 2
- UI sends "version 2" message to RT
- RT receives "version 2" notification and copies UI shared_ptr
- A is destroyed as a result (assuming the rest of the code is not still pointing to it)
- RT shared_ptr -> B | version 2 
As can be seen in this protocol:
* Only lightweight messages are exchanged (“version” is just a number)
* No object copy actually happens, we only copy shared_ptr

Implementation details

The SharedObjectMgr

The pair of std::shared_ptr is encapsulated in a class SharedObjectMgr along with a version so that we make sure that we don’t update to the wrong version (messaging is asynchronous so we can’t know for sure the order in which events will happen).

The class uses a SpinLock to ensure thread safety. Locking is very lightweight and the critical section only modifies shared pointers and version values which should never be an issue in practice. Also due to the notification mechanism, locking should happen only when something changes (in my case it is only when the user loads a sample, or uses sampling, which is very rare).

There are 2 primary methods on each side, prefixed by which side should call them:

  • uiSetObject and uiAdjustObjectFromRT for the UI side
  • rtSetObject and rtAdjustObjectFromUI for the RT side

Example of usage from RT:

  • RT has a new ObjectType to share with UI
  • RT calls mgr.rtSetObject and gets a version [v]
  • RT sends v to UI via messaging (RTJmbOutParam.broadcast(v) with Jamba)
  • UI receives v via messaging (simple callback with Jamba)
  • UI calls mgr.uiAdjustObjectFromRT(v) and gets the new ObjectType

Usage from UI follows the exact same pattern:

  • UI has a new ObjectType to share with RT
  • UI calls mgr.guiSetObject and gets a version [v]
  • UI sends v to RT via messaging (GUIJmbParam.broadcast(v) with Jamba)
  • RT receives v via messaging (RTJmbInParam.pop() with Jamba)
  • RT calls mgr.rtAdjustObjectFromUI(v) and gets the new ObjectType
On the RT side, it is recommended to extract and use the pointer directly instead of the std::shared_ptr because std::shared_ptr uses locks internally, and the object will never be deleted unless RT acts on it (mgr.rtSetObject or mgr.rtAdjustObjectFromUI(v)): each side manages its own (shared) pointer separately.

Sharing the SharedObjectMgr

In order to share the manager itself, RT creates it and send it to the UI via messaging:

  • serializing the pointer reinterpret_cast<uint64>(ptr) on the RT side
  • deserializing it on the UI side reinterpret_cast<SharedObjectMgr *>(ser)

In order for the host to keep RT and UI in the same process, thus making sharing possible, the plugin must be declared not distributable (the flag Steinberg::Vst::kDistributable must not be set).

There are a couple of details to pay attention to:

1. When using the editor to work on the GUI, the RT is not created or executed, and as a result the manager is never sent to the UI
2. It is hard to predict (and guarantee) the order in which the plugin will get initialized and when messages are actually exchanged, meaning the UI may need the manager prior to receiving it from RT.

It is thus recommended to create one in the GUI and discard it (and transfer its state) when the one from RT is received.

// Example on how to handle it in the UI

// With the following definitions
// using SharedSampleBuffersMgr32 = SharedObjectMgr<SampleBuffers<Sample32>, int64>;
// mutable std::unique_ptr<SharedSampleBuffersMgr32> fGUIOnlyMgr{};

// the code always call getSharedMgr() which does the right thing
SharedSampleBuffersMgr32 *SampleMgr::getSharedMgr() const
{
  auto sharedMgr = *fState->fSharedSampleBuffersMgrPtr;

  if(sharedMgr)
    return sharedMgr;

  // case when we have not received the manager yet
  if(!fGUIOnlyMgr)
    fGUIOnlyMgr = std::make_unique<SharedSampleBuffersMgr32>();

  return fGUIOnlyMgr.get();
}

// the callback gets registered to handle the manager from RT
void SampleMgr::registerParameters()
{
  // ...
  registerCallback<SharedSampleBuffersMgr32 *>(fParams->fSharedSampleBuffersMgrPtr,
    [this] (GUIJmbParam<SharedSampleBuffersMgr32 *> &iParam) {
      auto mgr = *iParam;
      if(fGUIOnlyMgr)
      {
        auto uiBuffers = fGUIOnlyMgr->uiGetObject();

        if(uiBuffers)
        {
          // there was a buffer that we need to transfer to the real time
          auto version = mgr->uiSetObject(uiBuffers);

          // we tell RT
          fGUINewSampleMessage.broadcast(version);
        }

        fGUIOnlyMgr = nullptr;
      }
    });
  // ...
}

Using Jamba

This section will show how to do it with Jamba since a lot of facilities are readily available.

Parameters

We need 3 parameters: one for the manager (pointer) and 2 for the version (one in each direction)

// declaration
JmbParam<int64> fUIToRTVersion; // used by UI to communicate new version to RT
JmbParam<int64> fRTToUIVersion; // used by RT to communicate new version to UI
JmbParam<SharedObjectMgr<MyType> *> fSharedObjectMgrPtr; // the shared mgr

// definition
fUIToRTVersion =
  jmb<Int64ParamSerializer>(EParamIDs::kUIToRTVersion, STR16 ("UI Version (msg)"))
    .guiOwned()
    .shared()
    .transient()
    .add();

fRTToUIVersion =
  jmb<Int64ParamSerializer>(EParamIDs::kRTToUIVersion, STR16 ("RT Version (msg)"))
    .rtOwned()
    .shared()
    .transient()
    .add();

fSharedObjectMgrPtr =
  jmb<PointerSerializer<SharedObjectMgr<MyType>>(EParamIDs::kSharedObjectMgrPtr, STR16 ("Shared Mgr (ptr)"))
    .transient()
    .rtOwned()
    .shared()
    .add();

Serializing the pointer

Implement the IParamSerializer and serialize the pointer into a uint64.

template<typename T>
class PointerSerializer : public IParamSerializer<T *>
{
public:
  using ParamType = T *;

  static_assert(sizeof(ParamType) <= sizeof(uint64), "Making sure that a pointer will fit");

  tresult readFromStream(IBStreamer &iStreamer, ParamType &oValue) const override
  {
    uint64 ptr;

    auto res = IBStreamHelper::readInt64u(iStreamer, ptr);
    if(res != kResultOk)
      return res;

    oValue = reinterpret_cast<ParamType>(ptr);

    return res;
  }

  tresult writeToStream(ParamType const &iValue, IBStreamer &oStreamer) const override
  {
    if(oStreamer.writeInt64u(reinterpret_cast<uint64>(iValue)))
      return kResultOk;
    else
      return kResultFalse;
  }
};

RT Side

On the RT side, we have the 3 parameters + the manager itself (which needs to be shared with the UI).

// definition (PluginRTState)
RTJmbInParam<int64> fUIToRTVersion;
RTJmbOutParam<int64> fRTToUIVersion;
RTJmbOutParam<SharedObjectMgr<MyType> *> fSharedObjectMgrPtr;

SharedObjectMgr<MyType> fSharedMgr{};
  
// Usage | sending the shared pointer to the UI
tresult SampleSplitterProcessor::setupProcessing(ProcessSetup &setup)
{
  // ...
  
  // sending the shared pointer to the UI
  fState.fSharedObjectMgrPtr.broadcast(&fState.fSharedMgr);
  
  // ...
}

// Usage | receiving a MyType from UI
tresult SampleSplitterProcessor::processInputs(ProcessData &data)
{
  // ...
  
  auto version = fState.fUIToRTVersion.pop();
  if(version)
  {
    auto myObject = fState.fSharedMgr.rtAdjustObjectFromUI(*version);

    if(myObject)
    {
      // store myObject.get() somewhere and use it in the rest of the code
    }

  }
  
  // ...
}

// Usage | updating MyType on the RT side
template<typename SampleType>
tresult SampleSplitterProcessor::genericProcessInputs(ProcessData &data)
{
  // ...
  
  auto myObject = std::make_shared<MyType>(...);
    
  // we store it in the mgr
  auto version = fState.fSharedMgr.rtSetObject(myObject);
  
  // and notify the UI of the new sample
  fState.fRTToUIVersion.broadcast(version);
  
  // store myObject.get() somewhere and use it in the rest of the code
  
  // ...
}

UI Side

On the UI side, we have the 3 parameters (and potentially a UI version of the manager, see warning section above to why).

// definition
GUIJmbParam<int64> fUIToRTVersion;
GUIJmbParam<SharedObjectMgr<MyType> *> fSharedObjectMgrPtr;
// no need to define fRTToUIVersion as we can use the one in Parameters instead

// retrieving manager
SharedObjectMgr<MyType> *getSharedMgr()
{
  // ... see above for more complicated implementation accounting for
  // no RT or non deterministic messaging 
  return *fSharedObjectMgrPtr;
}

// Usage | updating MyType on the UI side (from some view, controller, or listener)
void myFunction()
{

  // ....
  auto myObject = std::make_shared<MyType>(...);
    
  // we store it in the mgr
  auto version = getSharedMgr()->uiSetObject(myObject);

  // we tell RT
  fUIToRTVersion.broadcast(version);

}

// Usage | receiving a MyType from RT
void XXXController::registerParameters()
{
  // ...
  
  registerCallback<int64>(fParams->fRTToUIVersion,
                         [this] (GUIJmbParam<int64> &iParam) {
                           auto version = *iParam;
                           
                           auto myObject = getSharedMgr()->uiAdjustObjectFromRT(version);

                           if(myObject)
                           {
                             // ...
                           }
                         });
  
  // ...
}

Making the plugin not distributable

You can simply call the JambaPluginFactory::GetNonDistributableVST3PluginFactory API to make the plugin non distributable.

Conclusion

Implementing such a technique is definitely quite involved and complicated with multiple edge cases to deal with (like no RT when running the editor, messaging order, thread safety, etc…). In any case, I would not recommend using it unless you really want to share large amounts of data. If messaging can do the trick, just stick to it!

The latest version of SAM-SPL 64 implements this pattern/protocol and is the source of inspiration for this blog post.

This solution works quite well for my use case because what is shared only changes on user action (user loads a new sample, user uses sampling, user reloads the plugin).

One of the issue in this design is the fact that the critical section under lock may end up deleting an object which could affect RT. It is a non issue in my use case because of the fact that it can only happen on user action, not while the plugin is being used for rendering the sound. One improvement would be to have “delete” only happen on the UI side outside of the critical section under lock.