Introduction
This post describes a way to share (large amounts of) data between the UI layer and the RT (real time) layer of a VST3 plugin. Although it is not recommended to actually share data (messaging is the preferred approach in VST3), there are use cases where using pure messaging is very expensive.
For example, in my SAM-SPL 64 VST plugin, the user can load a sample to work with (which can be arbitrarily large). The UI layer needs the sample because it renders it as a waveform (including an editing tab where you can zoom in and out of the waveform). The RT layer needs the sample because it plays it. Using messaging requires copying the sample multiple times (at the very minimum UI → messaging, messaging → RT) and also keeping 2 distinct copies.
High level description
The main idea is to use a pair of std::shared_ptr
pointing to the same (immutable) object when sharing. When one side updates its std::shared_ptr
then they are pointing to different objects and a (very lightweight) message is sent to the other side so that it can update its own pointer to the new one: they now point to the same object again.
// start
- UI shared_ptr -> nullptr
- RT shared_ptr -> nullptr
// new object (A) in RT
- UI shared_ptr -> nullptr
- RT shared_ptr -> A | version 1
- RT sends "version 1" message to UI
- UI receives "version 1" notification and copies RT shared_ptr
- UI shared_ptr -> A | version 1
// new object (B) in UI
- RT shared_ptr -> A | version 1
- UI shared_ptr -> B | version 2
- UI sends "version 2" message to RT
- RT receives "version 2" notification and copies UI shared_ptr
- A is destroyed as a result (assuming the rest of the code is not still pointing to it)
- RT shared_ptr -> B | version 2
* Only lightweight messages are exchanged (“version” is just a number)
* No object copy actually happens, we only copy
shared_ptr
Implementation details
The SharedObjectMgr
The pair of std::shared_ptr
is encapsulated in a class SharedObjectMgr along with a version so that we make sure that we don’t update to the wrong version (messaging is asynchronous so we can’t know for sure the order in which events will happen).
The class uses a SpinLock
to ensure thread safety. Locking is very lightweight and the critical section only modifies shared pointers and version values which should never be an issue in practice. Also due to the notification mechanism, locking should happen only when something changes (in my case it is only when the user loads a sample, or uses sampling, which is very rare).
There are 2 primary methods on each side, prefixed by which side should call them:
uiSetObject
anduiAdjustObjectFromRT
for the UI sidertSetObject
andrtAdjustObjectFromUI
for the RT side
Example of usage from RT:
- RT has a new
ObjectType
to share with UI - RT calls
mgr.rtSetObject
and gets a version [v
] - RT sends
v
to UI via messaging (RTJmbOutParam.broadcast(v)
with Jamba) - UI receives
v
via messaging (simple callback with Jamba) - UI calls
mgr.uiAdjustObjectFromRT(v)
and gets the newObjectType
Usage from UI follows the exact same pattern:
- UI has a new
ObjectType
to share with RT - UI calls
mgr.guiSetObject
and gets a version [v
] - UI sends
v
to RT via messaging (GUIJmbParam.broadcast(v)
with Jamba) - RT receives
v
via messaging (RTJmbInParam.pop()
with Jamba) - RT calls
mgr.rtAdjustObjectFromUI(v)
and gets the newObjectType
std::shared_ptr
because std::shared_ptr
uses locks internally, and the object will never be deleted unless RT acts on it (mgr.rtSetObject
or mgr.rtAdjustObjectFromUI(v)
): each side manages its own (shared) pointer separately.Sharing the SharedObjectMgr
In order to share the manager itself, RT creates it and send it to the UI via messaging:
- serializing the pointer
reinterpret_cast<uint64>(ptr)
on the RT side - deserializing it on the UI side
reinterpret_cast<SharedObjectMgr *>(ser)
In order for the host to keep RT and UI in the same process, thus making sharing possible, the plugin must be declared not distributable (the flag Steinberg::Vst::kDistributable
must not be set).
There are a couple of details to pay attention to:
1. When using the editor to work on the GUI, the RT is not created or executed, and as a result the manager is never sent to the UI
2. It is hard to predict (and guarantee) the order in which the plugin will get initialized and when messages are actually exchanged, meaning the UI may need the manager prior to receiving it from RT.
It is thus recommended to create one in the GUI and discard it (and transfer its state) when the one from RT is received.
// Example on how to handle it in the UI
// With the following definitions
// using SharedSampleBuffersMgr32 = SharedObjectMgr<SampleBuffers<Sample32>, int64>;
// mutable std::unique_ptr<SharedSampleBuffersMgr32> fGUIOnlyMgr{};
// the code always call getSharedMgr() which does the right thing
SharedSampleBuffersMgr32 *SampleMgr::getSharedMgr() const
{
auto sharedMgr = *fState->fSharedSampleBuffersMgrPtr;
if(sharedMgr)
return sharedMgr;
// case when we have not received the manager yet
if(!fGUIOnlyMgr)
fGUIOnlyMgr = std::make_unique<SharedSampleBuffersMgr32>();
return fGUIOnlyMgr.get();
}
// the callback gets registered to handle the manager from RT
void SampleMgr::registerParameters()
{
// ...
registerCallback<SharedSampleBuffersMgr32 *>(fParams->fSharedSampleBuffersMgrPtr,
[this] (GUIJmbParam<SharedSampleBuffersMgr32 *> &iParam) {
auto mgr = *iParam;
if(fGUIOnlyMgr)
{
auto uiBuffers = fGUIOnlyMgr->uiGetObject();
if(uiBuffers)
{
// there was a buffer that we need to transfer to the real time
auto version = mgr->uiSetObject(uiBuffers);
// we tell RT
fGUINewSampleMessage.broadcast(version);
}
fGUIOnlyMgr = nullptr;
}
});
// ...
}
Using Jamba
This section will show how to do it with Jamba since a lot of facilities are readily available.
Parameters
We need 3 parameters: one for the manager (pointer) and 2 for the version (one in each direction)
// declaration
JmbParam<int64> fUIToRTVersion; // used by UI to communicate new version to RT
JmbParam<int64> fRTToUIVersion; // used by RT to communicate new version to UI
JmbParam<SharedObjectMgr<MyType> *> fSharedObjectMgrPtr; // the shared mgr
// definition
fUIToRTVersion =
jmb<Int64ParamSerializer>(EParamIDs::kUIToRTVersion, STR16 ("UI Version (msg)"))
.guiOwned()
.shared()
.transient()
.add();
fRTToUIVersion =
jmb<Int64ParamSerializer>(EParamIDs::kRTToUIVersion, STR16 ("RT Version (msg)"))
.rtOwned()
.shared()
.transient()
.add();
fSharedObjectMgrPtr =
jmb<PointerSerializer<SharedObjectMgr<MyType>>(EParamIDs::kSharedObjectMgrPtr, STR16 ("Shared Mgr (ptr)"))
.transient()
.rtOwned()
.shared()
.add();
Serializing the pointer
Implement the IParamSerializer
and serialize the pointer into a uint64
.
template<typename T>
class PointerSerializer : public IParamSerializer<T *>
{
public:
using ParamType = T *;
static_assert(sizeof(ParamType) <= sizeof(uint64), "Making sure that a pointer will fit");
tresult readFromStream(IBStreamer &iStreamer, ParamType &oValue) const override
{
uint64 ptr;
auto res = IBStreamHelper::readInt64u(iStreamer, ptr);
if(res != kResultOk)
return res;
oValue = reinterpret_cast<ParamType>(ptr);
return res;
}
tresult writeToStream(ParamType const &iValue, IBStreamer &oStreamer) const override
{
if(oStreamer.writeInt64u(reinterpret_cast<uint64>(iValue)))
return kResultOk;
else
return kResultFalse;
}
};
RT Side
On the RT side, we have the 3 parameters + the manager itself (which needs to be shared with the UI).
// definition (PluginRTState)
RTJmbInParam<int64> fUIToRTVersion;
RTJmbOutParam<int64> fRTToUIVersion;
RTJmbOutParam<SharedObjectMgr<MyType> *> fSharedObjectMgrPtr;
SharedObjectMgr<MyType> fSharedMgr{};
// Usage | sending the shared pointer to the UI
tresult SampleSplitterProcessor::setupProcessing(ProcessSetup &setup)
{
// ...
// sending the shared pointer to the UI
fState.fSharedObjectMgrPtr.broadcast(&fState.fSharedMgr);
// ...
}
// Usage | receiving a MyType from UI
tresult SampleSplitterProcessor::processInputs(ProcessData &data)
{
// ...
auto version = fState.fUIToRTVersion.pop();
if(version)
{
auto myObject = fState.fSharedMgr.rtAdjustObjectFromUI(*version);
if(myObject)
{
// store myObject.get() somewhere and use it in the rest of the code
}
}
// ...
}
// Usage | updating MyType on the RT side
template<typename SampleType>
tresult SampleSplitterProcessor::genericProcessInputs(ProcessData &data)
{
// ...
auto myObject = std::make_shared<MyType>(...);
// we store it in the mgr
auto version = fState.fSharedMgr.rtSetObject(myObject);
// and notify the UI of the new sample
fState.fRTToUIVersion.broadcast(version);
// store myObject.get() somewhere and use it in the rest of the code
// ...
}
UI Side
On the UI side, we have the 3 parameters (and potentially a UI version of the manager, see warning section above to why).
// definition
GUIJmbParam<int64> fUIToRTVersion;
GUIJmbParam<SharedObjectMgr<MyType> *> fSharedObjectMgrPtr;
// no need to define fRTToUIVersion as we can use the one in Parameters instead
// retrieving manager
SharedObjectMgr<MyType> *getSharedMgr()
{
// ... see above for more complicated implementation accounting for
// no RT or non deterministic messaging
return *fSharedObjectMgrPtr;
}
// Usage | updating MyType on the UI side (from some view, controller, or listener)
void myFunction()
{
// ....
auto myObject = std::make_shared<MyType>(...);
// we store it in the mgr
auto version = getSharedMgr()->uiSetObject(myObject);
// we tell RT
fUIToRTVersion.broadcast(version);
}
// Usage | receiving a MyType from RT
void XXXController::registerParameters()
{
// ...
registerCallback<int64>(fParams->fRTToUIVersion,
[this] (GUIJmbParam<int64> &iParam) {
auto version = *iParam;
auto myObject = getSharedMgr()->uiAdjustObjectFromRT(version);
if(myObject)
{
// ...
}
});
// ...
}
Making the plugin not distributable
You can simply call the JambaPluginFactory::GetNonDistributableVST3PluginFactory
API to make the plugin non distributable.
Conclusion
Implementing such a technique is definitely quite involved and complicated with multiple edge cases to deal with (like no RT when running the editor, messaging order, thread safety, etc…). In any case, I would not recommend using it unless you really want to share large amounts of data. If messaging can do the trick, just stick to it!
The latest version of SAM-SPL 64 implements this pattern/protocol and is the source of inspiration for this blog post.
This solution works quite well for my use case because what is shared only changes on user action (user loads a new sample, user uses sampling, user reloads the plugin).
One of the issue in this design is the fact that the critical section under lock may end up deleting an object which could affect RT. It is a non issue in my use case because of the fact that it can only happen on user action, not while the plugin is being used for rendering the sound. One improvement would be to have “delete” only happen on the UI side outside of the critical section under lock.