Activity analyzers

An Activity Analyzer is a Plugin that allows you to modify an activity on the fly before storing it. It runs as a part of the processing pipeline, for each activity of the channel it is associated with.

This feature is useful for:

• Reformatting data (adapting the ingestion data model to the datamart schema)

• Enriching events (for instance by fetching product information based on a product id)

• Improving data quality (filtering unwanted events, matching input values to standard catalogs, parsing URLs into categories etc.)

If you don't know what a plugin is, you can find the complete documentation in the specific section.

An activity analyzer is only executed for activities tracked in real time, e.g. via the user_activity API, javascript tag or pixel tracking (see real time user tracking guide). If you want to upload bulk activities, make sure they are already formatted before starting the upload as the activity analyzer won't run.

The standard group ID for an activity analyzer is {domain}.{organisation}.activity-analyzer, for example com.mediarithmics.activity-analyzer

Endpoints to implement

Activity analyzers have only one predefined endpoint to implement

Process an activity

POST myworker/v1/activity_analysis

This entry point is called any time an activity is processed by the platform. The activity analyzer receives an activity and responds to the request by returning a new activity. It cannot modify the identifiers that are passed in the incoming activities.

Request Body

Name	Type	Description
activity_analyzer_id	string	The ID of the activity analyzer instance that should be used to process the activity. Used to retrieve the instance properties.
datamart_id	string	The ID of the datamart
activity	object	The User Activity Object to analyze

200 In most cases, your plugin should respond with the modified user activity object that should be put into the platform.

{
    "status": "ok",
    "data": { 
        // New user activity object
    }
}

302 Here is what the plugin should respond if it encounters an unexpected error.

{
  "status": "error",
  "error": "Your error message"
}

See Plugin Instances to learn why you should use the activity_analyzer_id parameter to retrieve the instance properties.

Available outbound services

The code of the activity analyzer can call the following API endpoints to retrieve its instance context.

Retrieve the instance

GET https://api.mediarithmics.com/v1/activity_analyzers/:id

Use the activity_analyzer_id from the incoming request to retrieve the activity analyzer instance that has been called.

Path Parameters

Name	Type	Description
id	string	ID of the activity analyzer, typically the activity_analyzer_id from the incoming request.

200

{
  "status": "ok",
  "data": {
    "id": "1000",
    "name": "my analyzer",
    "organisation_id": "1000",
    "visit_analyzer_plugin_id": "1001",
    "group_id": "com.mediarithmics.visit-analyzer",
    "artifact_id": "default"
  }
}

Retrieve the instance properties

GET https://api.mediarithmics.com/v1/activity_analyzers/:id/properties

Get the properties associated with the activity analyzer instance

Path Parameters

Name	Type	Description
id	string	ID of the activity analyzer, typically theactivity_analyzer_id from the incoming request

200

{
  "status": "ok",
  "data": [
    {
      "technical_name": "debug",
      "value": { "value": false },
      "property_type": "BOOLEAN",
      "origin": "PLUGIN",
      "writable": true,
      "deletable": false
    },
    {
      "technical_name": "topic_properties",
      "value": { "value": "vertical" },
      "property_type": "STRING",
      "origin": "PLUGIN",
      "writable": true,
      "deletable": false
    }
  ],
  "count": 2
}

Creating an activity analyzer

See the plugins documentation to see how plugins are created and deployed.

An activity analyzer has the ACTIVITY_ANALYZER plugin type. Its group id should be {domain.organisation.activity-analyzer} (for example com.mediarithmics.activity-analyzer). Its artifact id should be the name of the activity analyzer, ie update-product-infos.

Use our Plugins SDK to create your activity analyzer in nodejs : the required routes are already defined and you only have to override specific functions.

You can find a sample activity analyzer in the examples folder of the plugins SDK.

We can provide you with a hello world project using our SDK. Please contact your Account manager in order to have access to it.

The project structure and files work as with every other plugin.

Interfaces to implement

Your should extend ActivityAnalyzerPlugin class and implement the instanceContextBuilder and onActivityAnalysisfunctions from the plugins SDK.

onActivityAnalysis function is called every time an activity runs through the activity analyzer. It is responsible for the activity transformation.

The instance context built in instanceContextBuilder is cached to improve performances. It should retrieve and store the plugin properties and configuration files used by the code.

Don't forget to catch your errors. You should log / respond with the appropriate message to facilitate debugging.

import { core } from "@mediarithmics/plugins-nodejs-sdk";
import { CustomInstanceContext } from "./interfaces/InstanceContextInterface";

export class ActivityAnalyzerPlugin extends core.ActivityAnalyzerPlugin {
    // Called to update a user activity
    // Uses the instance context built with instanceContextBuilder
    // to adapt to the properties and technical files
    protected async onActivityAnalysis(
        request: core.ActivityAnalyzerRequest,
        instanceContext: CustomInstanceContext)
            : Promise<core.ActivityAnalyzerPluginResponse> {

        try{
            const updatedActivity = request.activity;

            // Your code to modify the activity.
            // Exemple adding product infos in each event
            // If the technical configuration allows it
            if (instanceContext.technicalConfig.updateActivities){
                updatedActivity.$events.forEach(event => {
                    if (event.$properties && event.$properties.$items && event.$properties.$items.length > 0) {
                      event.$properties.$items.forEach((item: any) => {
                        var product = Products.find(p => p.$id == item.$id);
                        item.$name = product.$name;
                        item.categories = product.categories;
                        item.inStock = product.inStock;
                      });
                    }
                });
            }


            const response: core.ActivityAnalyzerPluginResponse = {
                status: "ok",
                data: updatedActivity
            };

            return Promise.resolve(response);
        }
        catch (err) {
          const errorResponse: core.ActivityAnalyzerPluginResponse = {
            status: 'error',
            data: request.activity
          };
          this.logger.error(`TRANSFORMATION ERROR while processing activity: ${JSON.stringify(request.activity)}`);
          return Promise.resolve(errorResponse)
        }
    }

    // Build the instance context
    // by fetching properties and configuration files
    protected async instanceContextBuilder(activityAnalyzerId: string)
        : Promise<CustomInstanceContext> {
        const baseInstanceContext = await super.instanceContextBuilder(activityAnalyzerId);
        try {

           // Retrieve a technical configuration file
           const validator = new Jsonschema.Validator();
           const technicalConfig: ITechnicalConfig = await this.validateJSONSchema(TECH_CONFIG_FILE, validator, technicalConfigurationSchema, activityAnalyzerId);

           // Retrieve a property from the plugin instance
           const eventExclusionList = baseInstanceContext.properties.findStringProperty("events_exclusion_list");

           // Return the completed instance context
            const result: CustomInstanceContext = {
                ...baseInstanceContext,
                event_exclusion_list: eventExclusionList,
                technicalConfig: technicalConfig
            };

            this.logger.debug(`Loaded InstanceContext with: ${JSON.stringify(result,null,4)}`);
            return Promise.resolve(result);
        } catch (err) {
            this.logger.error(`Something bad happened during the build of the Instance Context ${err}`);
            return Promise.reject(`Something bad happened during the build of the Instance Context ${err}`);
        }
    };
}

Your instance context interface should extend ActivityAnalyzerBaseInstanceContext

import { core } from "@mediarithmics/plugins-nodejs-sdk";

export interface CustomInstanceContext 
  extends core.ActivityAnalyzerBaseInstanceContext 
  {
    event_exclusion_list: string[];
    technicalConfig: ITechnicalConfig;
}

Creating an instance

Like other plugins, activity analyzer need to be instantiated. To create an instance, connect to Navigator and head toward Settings > Datamart > Activity Analyzers. You will get a list of existing instances and a button to create new ones.

Click on New Activity Analyzer.

Select the activity analyzer you want to instantiate.

Enter a name to easily recognize the instance, select an Error recovery strategy and fill Properties if you need to overwrite some of them. Save your modifications to create a new instance of your activity analyzer.

The error recovery strategy determines how the activity is processed when the plugin fails.

error_recovery_strategy	Failure reaction
STORE_WITH_ERROR_ID	The activity will be sent without any modification to the next activity analyzer.
STORE_WITH_ERROR_ID_AND_SKIP_UPCOMING_ANALYZERS	The activity will be saved without modification of the activity analyzer in failure. It doesn't be sent to the next plugin.
DROP	The activity won’t be saved

Linking an instance to a channel

Once your activity analyzer instance is created, you can link it to one or multiple channels. To do so, connect to Navigator and head toward Settings > Datamart > Channels and select the channel where you want your activity analyzer to be executed.

Go to the Activity Analyzers category.

Click on Add an Activity Analyzer and select your instance.

Several activity analyzers can be used on the same channel. In this case, they will process the same activity in a sequence of your choice: the second analyzer will process the activity as rendered by the first one and so on...

Currently, you can't get more than 5 activity analyzers. If you need more, please contact your Account manager.

Make sure to define the right order and error recovery strategies.

Debugging

Plugin logs and metrics

As activity analyzers are plugin, you can monitor them as you do with all plugins.

Verifying an activity

User activities that run through activity analyzers are generally aggregated into sessions. You won't see your user activity until it has been put into a session and gone through the whole activity processing pipeline. See how sessions are built to understand when you should see your activity or how you could fasten the process.

1. Go to the navigator > monitoring and search for the user point associated with the activity.

2. Click on the view json button on any activity on a timeline

3. You can check if all the properties are OK and if your activity analyzers processed the activity as expect

In case of problem, you can look at two properties added to the activity. processed_by will tell you if the activity has been processed by your activity analyzer, and $error_analyzer_id will give you an error ID if the activity analyzer returned an error response.

{
  "processed_by": "<YOUR_ANALYZER_ID>",
  "$error_analyzer_id": "<ERROR_ID>"
}