Datahub subscription

Overview   #

In today’s data-driven economy, organizations and third-party integrators require timely and accurate data to make informed decisions. This calls for a more efficient, robust, and real-time data delivery mechanism that reduces database load and enhances the responsiveness of integrated systems.

The Datahub Subscription (DHS) module redefines the way integrators and organizations access and utilize data from the SCALAR DataHub. By implementing an effective push-based data delivery system, the aim is to provide real-time, efficient, and scalable data integration solutions. This results in integrators receiving immediate updates, reducing database load, and enhancing the overall system performance. The DHS module therefore offers a seamless and automated data exchange ecosystem that supports the dynamic needs of modern businesses, enabling them to operate more effectively and respond to critical events with greater agility.

Data transmission process #

The Datahub portal captures and processes the data derived from SCALAR for events triggered such as assets, alarms, and sensors. The Datahub Subscription module stores and transmits this data to subscriber organizations. The administrator can set up the subscription in the Datahub Subscription module to start receiving data for the event groups defined.

Add subscription #

A subscription can be added from the Add subscription page, which includes the information specified below: 

• Subscription details
• Configuration (Webhook only)
• Subscription event groups

1. In the Datahub Portal module > click Datahub subscription . The Subscriptions page is displayed. 



2. Click Add subscription. The Add subscription page is displayed.
   
   
   
3. Enter details in the various sections as explained in this table. 
   

Section and fields	Action
Subscription details
Subscription name	Enter a name for the subscription.
Integrator name	Select the integrator from the list. Note: Data transmission from SCALAR integration is only accessible through an integrator. You must create an integrator before adding a subscription.
Subscription method	Select the subscription method from the list. The platform currently supports the Webhook and the FeedAPI method.
Batch size (Webhook only)	Enter the number of data points that can be transmitted in a batch.   For example, if the batch size is 10, the DHS module forms a batch of 10 data points to be transmitted at a time. If the batch size is 0, the DHS module transmits the data immediately. Note: The value is set to 250 by default.
Maximum waiting time (Webhook only)	Enter the waiting time before the system sends the available data, where the unit of measurement is milliseconds. The maximum waiting time should be between 5000ms (5s) to 1800000ms (30m). Note: The DHS module considers both the batch size and the maximum waiting time while transmitting data after determining which parameter value is less.  For example, if the batch size is 100 and the maximum waiting time is 5000, the DHS module transmits data to the customer every 5 seconds even if the batch size is less than 100.  The value is set to 10 by default. The minimum wait time for the batch size of 101 to 250 is 10 seconds.
Enable overflow (Webhook only)	Select to enable the feature that prevents data loss even if the webhook endpoint is unavailable for more than 30 minutes up to 24 hours. This feature ensures that data is not lost and can be delivered after the endpoint is restored. To mitigate data loss during webhook endpoint downtime, SCALAR offers an Overflow buffer mechanism. If a customer’s webhook endpoint is unavailable for more than 30 minutes, SCALAR will temporarily store the undelivered data for up to 24 hours. This buffer ensures that data is not lost and can be delivered once the endpoint is restored, providing resilience against short-term outages and improving reliability of data delivery.
Configuration (only available in case subscription method Webhook is selected)
Webhook URL	Enter the URL or the endpoint to which the data will be transmitted.
Webhook secret	The system generates a base64 string.  The data transmitted by the SCALAR DHS will be encrypted in the header using this secret. This ensures that the data is originating from a trusted source.   Note:  You must store the string for authentication. This key is required for encryption and decryption.
Enter the required custom header details (upto 5)	While setting up webhook subscriptions in SCALAR, you can configure up to five custom headers as key-value pairs. Click Add custom headers. The Add custom headers dialog is displayed. Enter values in the Key and Value boxes for custom headers, and then click Continue. The custom headers are added. Note: These custom headers will be included in the HTTP requests sent by SCALAR during data message delivery, enabling customers to implement custom authentication, routing, or tracking mechanisms at their endpoint. This is a representative image of a request header.
Test the webhook to verify before creating the subscription	Click Test webhook. If the connection succeeds, a ‘Webhook verified’ message is displayed. Note: This test is mandatory to create the subscription and avoid disruptions in data flow. For more information, see Test webhook.
Subscription event groups
Add event groups	Click Add event groups to subscribe to event groups. The Add event groups dialog is displayed.  Note:  A set of related events that are grouped together form an event group for the purpose of management and processing. The DHS module will transmit data when the selected events occur.  An integrator can have a maximum of two subscriptions with the same events.
Event group name	Select the event group from the list.  Note: Assets, sensors, and spot contracting are examples of event groups.
Version	Select the version from the list.
Custom event group options	Enter a JSON value    Example in JSON format  {“assets”:[{“assetId”:”9b11e94d-4215-4c75-afd2-445e21517d52″}, {“assetId”:”9a0154d4-673e-4509-b1d8-56de72489d4d”}, {“assetId”:”e47a2a15-6274-457d-9593-dadf621954f8″}]}   Note: This field is used to provide a JSON string that will have the ability to provide further functionality. For example, if Track and Trace event group wants to provide filtering on Asset IDs, this field can have an array of Asset IDs in JSON format.
Event names	For the selected event group, select the event from the list, for example, under the Asset event group, TPMS alarm and harsh braking events can be selected.  Note: You can enter multiple events specific to an event group.

4. Click Continue. The event groups subscribed are listed.

 

5. Click Save. The event groups subscribed to are saved. 




NOTE: The confirmation message indicating that the subscription is partially saved to the system, suggests that further steps such as verification are required before the subscription is fully active.

6. Click Continue. The subscription is added and listed on the Subscriptions page. The status is active by default.

View subscriptions #

The administrator can view subscription details, such as subscription configuration and event groups subscribed for. 
In the Datahub Portal module > click Datahub subscription . The Subscriptions page is displayed (see the image inserted in the previous section). Two tabs are available based on the configured subscription method: Webhook and Feed.

Webhook #

This tab provides an overview of all configured webhooks.

Available columns

Column	Description
Subscription name	Name of the subscription with the notification icon
Event group name	Event groups subscribed to, for example, assets, sensors, and spot contracting
Organization	Organization for which the subscription is being created
Batch size	Number of data points that can be transmitted in a batch
Maximum waiting time	Displays the milliseconds value of maximum waiting time since the last batch was sent
Integrator name	Name of the integrator
Created date	Date when the subscription was created
Overflow	Status type: Enabled or disabled NOTE: The overflow functionality ensures that data is preserved even if the Webhook endpoint is unavailable for more than 30 minutes up to 24 hours.
Status	Status type: Active, Inactive, or On hold
Actions	Click to view the menu, and then activate, deactivate, delete a subscription, and test the webhook

Feed #

This tab provides an overview of all configured FeedAPI’s.

Available columns

Column	Description
Subscription name	Name of the subscription with the notification icon
Event group name	Event groups subscribed to, for example, assets, sensors, and spot contracting
Organization	Organization for which the subscription is being created
Integrator name	Name of the integrator
Feed ID
Feed API endpoint
Created date	Date when the subscription was created
Status	Status type: Active, Inactive, or On hold
Actions	Click to view the menu, and then activate, deactivate and delete a subscription

Edit subscription  #

The administrator can edit subscription details, configuration, and subscription event groups.

1. On the Subscriptions page, click the subscription name in the Webhook or Feed tab.
NOTE:

If the subscription has a notification, the Notifications tab will be displayed. For more information, see View notifications.

2. On the Details tab, click Edit. The Edit subscription page is displayed.
   
   

 


3. Modify the details, and then click Save. The subscription details are updated.

Deactivate subscription #

An administrator can deactivate a subscription. Use this feature to suspend the subscription temporarily. 

1. On the Subscriptions page, in the Actions column, next to the required subscription name, click the menu , and then click Deactivate.
2. In the Deactivate subscription dialog, click Deactivate to confirm. The subscription is deactivated.

Reactivate subscription #

Reactivate a subscription if required.  

1. On the Subscriptions page, in the Actions column, next to the required subscription name, click the menu , and then click Activate.
2. In the Activate subscription dialog, click Activate to confirm. The subscription is activated.

Delete subscription #

An administrator can delete a subscription if the service is discontinued.

1. On the Subscriptions page, in the Actions column, next to the required subscription name, click the menu , and then click Delete. The Delete subscription dialog is displayed.

 


2. Click Delete to confirm. The subscription is deleted.

Note  

• You can also delete the subscription from the subscription details page. 
• Even if a subscription is deleted, it is not removed permanently. The record is archived.  
• To suspend the subscription temporarily, use the deactivate feature. 

Test webhook #

The webhook connection test feature enables users to validate the connectivity between their webhook endpoint and SCALAR during the subscription creation process. This test ensures that the endpoint is reachable and correctly configured to receive data. A failed connection test indicates potential delivery issues, which may prevent SCALAR from transmitting data messages. An administrator can test a webhook for a subscription. 

1. On the Subscriptions page, in the Actions column, next to the required subscription name, click the menu , and then clickTest webhook. The Test webhook dialog is displayed.
   
   

Note: The details of when the last test was run and the webhook connection was verified are displayed.

2. In the Test webhook dialog, click Run test. The test is run and the verification result is displayed.

View notifications #

SCALAR provides both in-app and email notifications to inform customers of events and changes related to their subscriptions. These notifications ensure that customers are given timely information of any operational or security-related changes affecting their data subscriptions.

1. On the Subscriptions page, click the subscription name link. The Notifications tab is displayed. 



2. Click the required notification. The notification details are displayed showing the severity level.
   
   
NOTE:
• The notification icon appears dimmed on the Notifications tab after it is read.
• After all the notifications are read, the notification icon appears dimmed on the Subscriptions page.

The table provides details of the events when the notifications are triggered.

Filter by notification status #

You can narrow search results by using the notification status filters.

1. In the Datahub Portal module > click Datahub subscription . The Subscriptions page is displayed.
2. To open the notification status filter pane, click All next to the Search box.



3. Select the required option. The results matching the selection criteria are displayed.

Filter subscriptions #

On the Subscriptions page, you can filter subscription records to narrow search results based on integrator name and status by using the Filters option.

1. In the Datahub Portal module > click Datahub subscription . The Subscriptions page is displayed.
2. To open the filter pane, click Filters.



3. Select the required option in the Integrator name and Status sections. The results matching the selection criteria are displayed.