Batching (for external feeds)

Platform-side batching for plugins

As of now, batching is only available for plugins of type AUDIENCE_SEGMENT_EXTERNAL_FEED

You may need to integrate with external partners that require batched data as input. However mediarithmics platform only provides unit updates to your external feed plugin, one UserPoint at a time (via /user_segment_update route).

This documentation explains you how to use the batching feature provided by the platform.

Create a batching configuration

1. Go to the Computing Console > Plugins

2. Find your plugin, select the version and go to the Batching configuration tab

3. Create the batching configuration

Batching configurations can also be defined using the following API.

Create a Plugin Version Batching Configuration

POST https://api.mediarithmics.com/v1/plugins/:plugin_id/versions/:version_id/batching_configurations

Request Body

Name	Type	Description
plugin_version_id*	String	The id of the Plugin Version that will be batched.
enable_batching*	Boolean	enable or disable the batching
batch_flush_max_size*	Integer	Set the maximum size of batches in number of lines.

Implement the batching feature in your external feed plugin

Design

You need to split your plugin logics in two parts :

1. Data formatting / gathering phase (/user_segment_update route implementation)

2. Batch delivery phase (/batch_update route implementation).

The platform will take care of aggregating the data returned by the /user_segment_update route into batches and then call back your plugin route /batch_update with the batches.

It's important to understand that the output of the call on /user_segment_update will be an element of /batch_update input.

Implementation

You need to use version 0.14+ of the plugin-nodejs-sdk

Rewrite your plugin to extend type BatchedAudienceFeedConnectorBasePlugin<T> (where T is the type of one batched line) provided by the SDK.

It will force your plugin to implement /batch_update route and force /segment_update response type to be BatchedUserSegmentUpdatePluginResponse<T>.

You can use the grouping_key field of the response if your data specifically need to be regrouped by a specific criteria.

For exemple, you may have an external partner that requires having only one identifier type per payload. To do so, you should use the identifier type as grouping_key.

mediarithmics platform won't interpret grouping_key, and will use it as an exact matching.