Azure Service Bus

Azure Service Bus is an enterprise message broker which topics and queues can be used as Egress Sinks.

tip

See Consume Data in Your Systems page to learn more about configuring Egress Sinks and Routes.

Supported egress events:

Kind	Is supported
Messages	✅
Batch Completions	✅

When used as an Egress Sink, each logical Message or Batch Completion is routed to an Service Bus entity (queue or topic) as a physical Service Bus message. Functionality and configuration for routing in the topics and queues is identical.

The Azure Service Bus messages have two parts: body and properties. The body is a simple byte array, and the properties are a set of key-value pairs. In the context of Egress Sinks, the properties are used to transport the metadata about the Message/Batch Completion, such as stream-name. Depending on the Egress Route configuration and size of the routed Message, the body might contain the actual payload of the Message that arrived from a devicea. The exact format specification can be found below.

Configuration

In order to route a Message or a Batch Completion, the target Service Bus topic or queue must be configured as an Egress Sink and one or more Streams must have Egress Route targeting that Egress Sink.

Supported authentication methods are:

• Shared Access Key - The platform will use the provided connection string for entity-level shared access policy with at least Send permission/claim.
  • The connection string must contain SharedAccessKey and SharedAccessKeyName properties. Property SharedAccessKeySignature is not supported.
  • Example: Endpoint=sb://<service-bus-namespace-hostname>;SharedAccessKeyName=<key-name>;SharedAccessKey=<key>;EntityPath=<entity-name>;.
• Microsoft Entra ID - The platform will use the service principal of the Spotflow IoT Platform to access the Service Bus. This requires the platform service principal to have Azure Service Bus Data Sender role assigned.
  • This way of authentication is currently available only for dedicated instances.

The associated Egress Routes can be customized with the following parameters:

• Disable Payload Embedding: Flag that, when set to true, prevents Message payload to be embedded into the Service Bus message body. This can be used to optimize Service Bus throughput. Default is false which means that the payload is embedded if possible.
  • Cannot be used together with Storage Skipping.

Message format

If possible, body of Service Bus message contains embedded payload of the Message that arrived from a device. This is possible only if:

• The payload must be smaller than 240 KiB.
• The associated Egress Routes must not have Disable Payload Embedding flag set to true.

Otherwise, the body of the Service Bus message is empty.

note

When a stream has Storage Skipping enabled, compressed messages are delivered in their original format without being decompressed.

In any case, following properties are available on the Service Bus message:

Name	Type	Example	Description
kind	string	Message	Identifier that allows to distinguish amongst different kinds of events. For messages, this is always set to Message.
stream-group-name	string	group-a	Name of the stream group the message was sent into.
stream-name	string	telemetry	Name of the stream the message was sent into.
device-id	string	robot-125	Id of the device that sent the message.
batch-id	string	2023-12-19	Identifier of batch. It is provided by device or auto-filled by the platform (if configured).
batch-slice-id	string	logs	Identifier of batch slice (if it was provided by the device).
message-id	string	m00767	Identifier of the message. It is provided by device or auto-filled by the platform (if configured).
is-embedded-payload	boolean	true	Is true when the Message payload is embedded directly, otherwise false. If payload is not embedded, it can be downloaded via payload-sas-uri or payload-uri property.
payload-sas-uri	string		SAS URI to the Message blob in Stream Storage. Is present if: Stream Storage Skipping is not enabled. Shared Access Key authentication is used for the storage, otherwise payload-uri property should be used.
payload-uri	string		URI to the Message blob in Stream Storage. Is present if: Stream Storage Skipping is not enabled. Microsoft Entra ID authentication is used for the storage, otherwise payload-sas-uri property should be used.
workspace-id	string	69f09b3f-ec0d-4b9e-a5ec-87150b935296	Identifier of the Workspace that originating Device and Stream belong into. Formatted as GUID/UUID with 32 hexadecimal digits (lowercase) separated by hyphens.
ingress-enqueued-date-time	string	2023-12-19T11:25:56.1408925+01:00	Time when the Message was ingested by the platform. ISO 8601 format.
content-encoding	string	br	Content encoding of the payload. Can be present if Storage Skipping is enabled on the stream. The only possible value is br, which indicates that the payload is compressed with Brotli.

Batch completion format

Body of the Service Bus message is always empty and following properties are available:

Name	Type	Example	Description
kind	string	BatchCompletion	Identifier that allows to distinguish amongst different kinds of events. For Batch Completions, this is always set to BatchCompletion.
stream-group-name	string	group-a	Name of the stream group the message was sent into.
stream-name	string	telemetry	Name of the stream the message was sent into.
device-id	string	robot-125	Id of the device that sent the message.
batch-id	string	2023-12-19	Identifier of batch. It is provided by device or auto-filled by the platform (if configured).
blob-sas-uri	string		SAS URI to the blob in the Stream Storage that contains concatenated messages from given batch. Is present if: Stream has concatenation enabled. Batch slices are not used. Shared Access Key authentication is used for the stream storage, otherwise blob-uri property should be used.
blob-uri	string		URI to the blob in the Stream Storage that contains concatenated messages from given batch. Is present if: Stream has concatenation enabled. Batch slices are not used. Microsoft Entra ID authentication is used for the stream storage, otherwise blob-sas-uri property should be used.
blob-container-sas-uri	string		Container SAS URI that can be used to list and access messages stored as individual blobs, or to list and access batch slice blobs. Is present if: Stream has concatenation disabled or batch slices are used. Shared Access Key authentication is used for the stream storage, otherwise blob-container-uri property should be used.
blob-container-uri	string		Container URI that can be used to list and access messages stored as individual blobs, or to list and access batch slice blobs. Is present if: Stream has concatenation disabled or batch slices are used. Microsoft Entra ID authentication is used for the stream storage, otherwise blob-container-sas-uri property should be used.
blob-directory-path	string	group-a/telemetry/robot-125/2023-12-19	Name prefix for messages (stored as blobs) or for batch slices in the Stream Storage.
workspace-id	string	69f09b3f-ec0d-4b9e-a5ec-87150b935296	Identifier of the Workspace that originating Device and Stream belong into. Formatted as GUID/UUID with 32 hexadecimal digits (lowercase) separated by hyphens.