issues with BundleMessage
BundleMessage
has a quite fragile implementation. Some of the concerns are:
- messages of multiple types can be packed into the same bundle
- unpacking bypasses all checks for expected message types
As an example,
from bec_lib import messages
bm = messages.BundleMessage()
m1 = messages.DeviceMessage(signals={"samx": {}}, metadata={"test": "123"}).dumps()
m2 = messages.ScanMessage(point_id=1, scanID=2, data={"test2": 1}).dumps()
bm.append(m1)
bm.append(m2)
bmd = bm.dumps() # it packs!
If we try to unpack, expecting a completely different message type
msgs = messages.ScanMessage.loads(bmd) # it unpacks!
print(msgs)
[messages.DeviceMessage(**{'signals': {'samx': {}}}, metadata={'test': '123'}),
messages.ScanMessage(**{'point_id': 1, 'scanID': 2, 'data': {'test2': 1}}, metadata={})]
print(msgs[0].content)
{'signals': {'samx': {}}}
So far, it seems, it's only used with MessageEndpoints.device_read
in scan_bundler.py
(on the receiving side), where consumer expects to receive a list of messages. But having BundleMessage
as a general option, would naturally force all other receivers/readers of data on all endpoints to expect a list of messages too, which is not very convenient and, imho, should not be the case. At first sight, I can think of the following options to address this issue:
- try to replace
BundleMessage
with a redis pipeline + hiredis (the latter might help with the performance, as it was the reason to introduceBundleMessage
) - add new message types (or extend the existing ones), that can contain a list of content/metadata entries, and corresponding endpoints for that type of messages