Activity analyzers

An Activity Analyzer is a Plugin that allows you to modify an activity on the fly before storing it. It runs as a part of the processing pipeline, for each activity of the channel it is associated with.

This feature is useful for:

• Reformatting data (adapting the ingestion data model to the datamart schema)

• Enriching events (for instance by fetching product information based on a product id)

• Improving data quality (filtering unwanted events, matching input values to standard catalogs, parsing URLs into categories etc.)

Endpoints to implement

Activity analyzers have only one predefined endpoint to implement

Process an activity

POST myworker/v1/activity_analysis

This entry point is called any time an activity is processed by the platform. The activity analyzer receives an activity and responds to the request by returning a new activity. It cannot modify the identifiers that are passed in the incoming activities.

Request Body

{
    "status": "ok",
    "data": { 
        // New user activity object
    }
}

{
  "status": "error",
  "error": "Your error message"
}

See Plugin Instances to learn why you should use the activity_analyzer_id parameter to retrieve the instance properties.

Available outbound services

The code of the activity analyzer can call the following API endpoints to retrieve its instance context.

Retrieve the instance

GET https://api.mediarithmics.com/v1/activity_analyzers/:id

Use the activity_analyzer_id from the incoming request to retrieve the activity analyzer instance that has been called.

Path Parameters

{
  "status": "ok",
  "data": {
    "id": "1000",
    "name": "my analyzer",
    "organisation_id": "1000",
    "visit_analyzer_plugin_id": "1001",
    "group_id": "com.mediarithmics.visit-analyzer",
    "artifact_id": "default"
  }
}

Retrieve the instance properties

GET https://api.mediarithmics.com/v1/activity_analyzers/:id/properties

Get the properties associated with the activity analyzer instance

Path Parameters

{
  "status": "ok",
  "data": [
    {
      "technical_name": "debug",
      "value": { "value": false },
      "property_type": "BOOLEAN",
      "origin": "PLUGIN",
      "writable": true,
      "deletable": false
    },
    {
      "technical_name": "topic_properties",
      "value": { "value": "vertical" },
      "property_type": "STRING",
      "origin": "PLUGIN",
      "writable": true,
      "deletable": false
    }
  ],
  "count": 2
}

Creating an activity analyzer

See the plugins documentation to see how plugins are created and deployed.

Use our Plugins SDK to create your activity analyzer in nodejs : the required routes are already defined and you only have to override specific functions.

We can provide you with a hello world project using our SDK. Please contact your Account manager in order to have access to it.

The project structure and files work as with every other plugin.

Interfaces to implement

Your should extend ActivityAnalyzerPlugin class and implement the instanceContextBuilder and onActivityAnalysisfunctions from the plugins SDK.

onActivityAnalysis function is called every time an activity runs through the activity analyzer. It is responsible for the activity transformation.

The instance context built in instanceContextBuilder is cached to improve performances. It should retrieve and store the plugin properties and configuration files used by the code.

import { core } from "@mediarithmics/plugins-nodejs-sdk";
import { CustomInstanceContext } from "./interfaces/InstanceContextInterface";

export class ActivityAnalyzerPlugin extends core.ActivityAnalyzerPlugin {
    // Called to update a user activity
    // Uses the instance context built with instanceContextBuilder
    // to adapt to the properties and technical files
    protected async onActivityAnalysis(
        request: core.ActivityAnalyzerRequest,
        instanceContext: CustomInstanceContext)
            : Promise<core.ActivityAnalyzerPluginResponse> {

        try{
            const updatedActivity = request.activity;

            // Your code to modify the activity.
            // Exemple adding product infos in each event
            // If the technical configuration allows it
            if (instanceContext.technicalConfig.updateActivities){
                updatedActivity.$events.forEach(event => {
                    if (event.$properties && event.$properties.$items && event.$properties.$items.length > 0) {
                      event.$properties.$items.forEach((item: any) => {
                        var product = Products.find(p => p.$id == item.$id);
                        item.$name = product.$name;
                        item.categories = product.categories;
                        item.inStock = product.inStock;
                      });
                    }
                });
            }


            const response: core.ActivityAnalyzerPluginResponse = {
                status: "ok",
                data: updatedActivity
            };

            return Promise.resolve(response);
        }
        catch (err) {
          const errorResponse: core.ActivityAnalyzerPluginResponse = {
            status: 'error',
            data: request.activity
          };
          this.logger.error(`TRANSFORMATION ERROR while processing activity: ${JSON.stringify(request.activity)}`);
          return Promise.resolve(errorResponse)
        }
    }

    // Build the instance context
    // by fetching properties and configuration files
    protected async instanceContextBuilder(activityAnalyzerId: string)
        : Promise<CustomInstanceContext> {
        const baseInstanceContext = await super.instanceContextBuilder(activityAnalyzerId);
        try {

           // Retrieve a technical configuration file
           const validator = new Jsonschema.Validator();
           const technicalConfig: ITechnicalConfig = await this.validateJSONSchema(TECH_CONFIG_FILE, validator, technicalConfigurationSchema, activityAnalyzerId);

           // Retrieve a property from the plugin instance
           const eventExclusionList = baseInstanceContext.properties.findStringProperty("events_exclusion_list");

           // Return the completed instance context
            const result: CustomInstanceContext = {
                ...baseInstanceContext,
                event_exclusion_list: eventExclusionList,
                technicalConfig: technicalConfig
            };

            this.logger.debug(`Loaded InstanceContext with: ${JSON.stringify(result,null,4)}`);
            return Promise.resolve(result);
        } catch (err) {
            this.logger.error(`Something bad happened during the build of the Instance Context ${err}`);
            return Promise.reject(`Something bad happened during the build of the Instance Context ${err}`);
        }
    };
}

Your instance context interface should extend ActivityAnalyzerBaseInstanceContext

import { core } from "@mediarithmics/plugins-nodejs-sdk";

export interface CustomInstanceContext 
  extends core.ActivityAnalyzerBaseInstanceContext 
  {
    event_exclusion_list: string[];
    technicalConfig: ITechnicalConfig;
}

Creating an instance

Like other plugins, activity analyzer need to be instantiated. To create an instance, connect to Navigator and head toward Settings > Datamart > Activity Analyzers. You will get a list of existing instances and a button to create new ones.

Click on New Activity Analyzer.

Select the activity analyzer you want to instantiate.

Enter a name to easily recognize the instance, select an Error recovery strategy and fill Properties if you need to overwrite some of them. Save your modifications to create a new instance of your activity analyzer.

The error recovery strategy determines how the activity is processed when the plugin fails.

Linking an instance to a channel

Once your activity analyzer instance is created, you can link it to one or multiple channels. To do so, connect to Navigator and head toward Settings > Datamart > Channels and select the channel where you want your activity analyzer to be executed.

Go to the Activity Analyzers category.

Click on Add an Activity Analyzer and select your instance.

Several activity analyzers can be used on the same channel. In this case, they will process the same activity in a sequence of your choice: the second analyzer will process the activity as rendered by the first one and so on...

Make sure to define the right order and error recovery strategies.

Debugging

Plugin logs and metrics

As activity analyzers are plugin, you can monitor them as you do with all plugins.

Verifying an activity

1. Go to the navigator > monitoring and search for the user point associated with the activity.

2. Click on the view json button on any activity on a timeline

3. You can check if all the properties are OK and if your activity analyzers processed the activity as expect

In case of problem, you can look at two properties added to the activity. processed_by will tell you if the activity has been processed by your activity analyzer, and $error_analyzer_id will give you an error ID if the activity analyzer returned an error response.

{
  "processed_by": "<YOUR_ANALYZER_ID>",
  "$error_analyzer_id": "<ERROR_ID>"
}