# Activity analyzers

An Activity Analyzer is a [Plugin](https://developer.mediarithmics.io/advanced-usages/plugins) that allows you to modify an activity on the fly before storing it. It runs as a part of [the processing pipeline](https://developer.mediarithmics.io/data-streams/data-ingestion/real-time-user-tracking/..#the-processing-pipeline), for each activity of the channel it is associated with.

This feature is useful for:

* Reformatting data (adapting the ingestion data model to the datamart schema)
* Enriching events (for instance by fetching product information based on a product id)
* Improving data quality (filtering unwanted events, matching input values to standard catalogs, parsing URLs into categories etc.)

{% hint style="success" %}
If you don't know what a plugin is, you can find the [complete documentation in the specific section.](https://developer.mediarithmics.io/advanced-usages/plugins)
{% endhint %}

{% hint style="info" %}
An activity analyzer is only executed for activities tracked in real time, e.g. via the user\_activity API,  javascript tag or pixel tracking (see [real time user tracking guide](https://developer.mediarithmics.io/data-streams/data-ingestion/real-time-user-tracking)). If you want to upload bulk activities, make sure they are already formatted before starting the upload as the activity analyzer won't run.

The standard group ID for an activity analyzer is {domain}.{organisation}.activity-analyzer, for example com.mediarithmics.activity-analyzer
{% endhint %}

## Endpoints to implement

Activity analyzers have only one predefined endpoint to implement

## Process an activity

<mark style="color:green;">`POST`</mark> `myworker/v1/activity_analysis`

This entry point is called any time an activity is processed by the platform. The activity analyzer receives an activity and responds to the request by returning a new activity. It cannot modify the identifiers that are passed in the incoming activities.

### Request Body

| Name                   | Type   | Description                                                                                                                         |
| ---------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------- |
| activity\_analyzer\_id | string | The ID of the activity analyzer instance that should be used to process the activity. **Used to retrieve the instance properties.** |
| datamart\_id           | string | The ID of the datamart                                                                                                              |
| activity               | object | The UserActivity Object to analyze                                                                                                  |

{% tabs %}
{% tab title="200 In most cases, your plugin should respond with the modified user activity object that should be put into the platform." %}

```javascript
{
    "status": "ok",
    "data": { 
        // New UserActivity object
    }
}
```

{% endtab %}

{% tab title="302 Here is what the plugin should respond if it encounters an unexpected error." %}

```javascript
{
  "status": "error",
  "error": "Your error message"
}
```

{% endtab %}
{% endtabs %}

See [Plugin Instances ](https://developer.mediarithmics.io/advanced-usages/plugins#instances)to learn why you should use the `activity_analyzer_id` parameter to retrieve the instance properties.

{% hint style="info" %}
If you need to create or update user profiles using the dedicated event `$set_user_profile_properties` , please refer to [Create / Update a UserProfile from within an activity](https://developer.mediarithmics.io/data-streams/data-ingestion/api#create-update-a-userprofile-from-within-an-activity).&#x20;

Specifically note that you must use properties `$compartment_id` and (optionnally) `$user_account_id`.

Properties `$set_user_profile_comp_token` / `$set_user_profile_user_account_id`  are reserved for the case when the event is [triggerred from a website via our javascript tag](https://developer.mediarithmics.io/data-streams/data-ingestion/web-events#userprofile-updates). They are translated into the above ones before the activity analyzer is called and should not be used in it.
{% endhint %}

## Available outbound services

The code of the activity analyzer can call the following API endpoints to retrieve[ its instance context](https://developer.mediarithmics.io/advanced-usages/plugins#instances).

## Retrieve the instance

<mark style="color:blue;">`GET`</mark> `https://api.mediarithmics.com/v1/activity_analyzers/:id`

Use the activity\_analyzer\_id from the incoming request to retrieve the activity analyzer instance that has been called.

#### Path Parameters

| Name | Type   | Description                                                                                  |
| ---- | ------ | -------------------------------------------------------------------------------------------- |
| id   | string | ID of the activity analyzer, typically the `activity_analyzer_id` from the incoming request. |

{% tabs %}
{% tab title="200 " %}

```javascript
{
  "status": "ok",
  "data": {
    "id": "1000",
    "name": "my analyzer",
    "organisation_id": "1000",
    "visit_analyzer_plugin_id": "1001",
    "group_id": "com.mediarithmics.visit-analyzer",
    "artifact_id": "default"
  }
}
```

{% endtab %}
{% endtabs %}

## Retrieve the instance properties

<mark style="color:blue;">`GET`</mark> `https://api.mediarithmics.com/v1/activity_analyzers/:id/properties`

Get the properties associated with the activity analyzer instance

#### Path Parameters

| Name | Type   | Description                                                                                |
| ---- | ------ | ------------------------------------------------------------------------------------------ |
| id   | string | ID of the activity analyzer, typically the`activity_analyzer_id` from the incoming request |

{% tabs %}
{% tab title="200 " %}

```javascript
{
  "status": "ok",
  "data": [
    {
      "technical_name": "debug",
      "value": { "value": false },
      "property_type": "BOOLEAN",
      "origin": "PLUGIN",
      "writable": true,
      "deletable": false
    },
    {
      "technical_name": "topic_properties",
      "value": { "value": "vertical" },
      "property_type": "STRING",
      "origin": "PLUGIN",
      "writable": true,
      "deletable": false
    }
  ],
  "count": 2
}
```

{% endtab %}
{% endtabs %}

## Creating an activity analyzer

See the plugins documentation to see [how plugins are created and deployed](https://developer.mediarithmics.io/advanced-usages/plugins).

{% hint style="success" %}
An activity analyzer has the `ACTIVITY_ANALYZER` plugin type. Its group id should be `{domain.organisation.activity-analyzer}` (for example com.mediarithmics.activity-analyzer). Its *artifact id* should be the name of the activity analyzer, ie `update-product-infos`.
{% endhint %}

Use our [Plugins SDK](https://developer.mediarithmics.io/resources/tools-and-libraries/plugin-sdk) to create your activity analyzer in `nodejs` : the required routes are already defined and you only have to override specific functions.

{% hint style="info" %}
You can find a sample activity analyzer [in the examples folder of the plugins SDK](https://github.com/MEDIARITHMICS/plugins-nodejs-sdk/tree/master/examples/activity-analyzer).
{% endhint %}

We can provide you with a hello world project using our SDK. Please contact your Account manager in order to have access to it.

The project structure and files work as [with every other plugin](https://developer.mediarithmics.io/advanced-usages/plugins/creation).

### Interfaces to implement

Your should extend `ActivityAnalyzerPlugin` class and implement the `instanceContextBuilder` and `onActivityAnalysis`functions from the plugins SDK.

`onActivityAnalysis` function is called every time an activity runs through the activity analyzer. It is responsible for the activity transformation.

The instance context built in `instanceContextBuilder` is cached to improve performances. It should retrieve and store the plugin properties and configuration files used by the code.

{% hint style="warning" %}
Don't forget to catch your errors. You should log / respond with the appropriate message to facilitate debugging.
{% endhint %}

```javascript
import { core } from "@mediarithmics/plugins-nodejs-sdk";
import { CustomInstanceContext } from "./interfaces/InstanceContextInterface";

export class ActivityAnalyzerPlugin extends core.ActivityAnalyzerPlugin {
    // Called to update a UserActivity
    // Uses the instance context built with instanceContextBuilder
    // to adapt to the properties and technical files
    protected async onActivityAnalysis(
        request: core.ActivityAnalyzerRequest,
        instanceContext: CustomInstanceContext)
            : Promise<core.ActivityAnalyzerPluginResponse> {

        try{
            const updatedActivity = request.activity;

            // Your code to modify the activity.
            // Exemple adding product infos in each event
            // If the technical configuration allows it
            if (instanceContext.technicalConfig.updateActivities){
                updatedActivity.$events.forEach(event => {
                    if (event.$properties && event.$properties.$items && event.$properties.$items.length > 0) {
                      event.$properties.$items.forEach((item: any) => {
                        var product = Products.find(p => p.$id == item.$id);
                        item.$name = product.$name;
                        item.categories = product.categories;
                        item.inStock = product.inStock;
                      });
                    }
                });
            }


            const response: core.ActivityAnalyzerPluginResponse = {
                status: "ok",
                data: updatedActivity
            };

            return Promise.resolve(response);
        }
        catch (err) {
          const errorResponse: core.ActivityAnalyzerPluginResponse = {
            status: 'error',
            data: request.activity
          };
          this.logger.error(`TRANSFORMATION ERROR while processing activity: ${JSON.stringify(request.activity)}`);
          return Promise.resolve(errorResponse)
        }
    }

    // Build the instance context
    // by fetching properties and configuration files
    protected async instanceContextBuilder(activityAnalyzerId: string)
        : Promise<CustomInstanceContext> {
        const baseInstanceContext = await super.instanceContextBuilder(activityAnalyzerId);
        try {

           // Retrieve a technical configuration file
           const validator = new Jsonschema.Validator();
           const technicalConfig: ITechnicalConfig = await this.validateJSONSchema(TECH_CONFIG_FILE, validator, technicalConfigurationSchema, activityAnalyzerId);

           // Retrieve a property from the plugin instance
           const eventExclusionList = baseInstanceContext.properties.findStringProperty("events_exclusion_list");

           // Return the completed instance context
            const result: CustomInstanceContext = {
                ...baseInstanceContext,
                event_exclusion_list: eventExclusionList,
                technicalConfig: technicalConfig
            };

            this.logger.debug(`Loaded InstanceContext with: ${JSON.stringify(result,null,4)}`);
            return Promise.resolve(result);
        } catch (err) {
            this.logger.error(`Something bad happened during the build of the Instance Context ${err}`);
            return Promise.reject(`Something bad happened during the build of the Instance Context ${err}`);
        }
    };
}
```

Your instance context interface should extend ActivityAnalyzerBaseInstanceContext

```javascript
import { core } from "@mediarithmics/plugins-nodejs-sdk";

export interface CustomInstanceContext 
  extends core.ActivityAnalyzerBaseInstanceContext 
  {
    event_exclusion_list: string[];
    technicalConfig: ITechnicalConfig;
}
```

## Creating an instance

Like other plugins, activity analyzer need to be instantiated. To create an instance, connect to Navigator and head toward Settings > Datamart > Activity Analyzers. You will get a list of existing instances and a button to create new ones.

![](https://4196284719-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MMuoqM-5hJ5JY0WnAKL%2Fuploads%2FWC7Vv0khEfdvQF2322lX%2FCapture%20d%E2%80%99e%CC%81cran%202022-07-05%20a%CC%80%2011.51.36.png?alt=media\&token=763e608a-d196-492c-8b4c-409bd1b7fe7b)

Click on **New Activity Analyzer**.

![](https://4196284719-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MMuoqM-5hJ5JY0WnAKL%2Fuploads%2FoNP6nVOUCsdJzMBucogY%2FCapture%20d%E2%80%99e%CC%81cran%202022-07-05%20a%CC%80%2011.53.30.png?alt=media\&token=2a7df553-f3ea-4f91-8476-066ee5784172)

Select the activity analyzer you want to instantiate.

![](https://4196284719-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MMuoqM-5hJ5JY0WnAKL%2Fuploads%2FnyDB3sksgFI9mX4hiOFz%2FCapture%20d%E2%80%99e%CC%81cran%202022-07-05%20a%CC%80%2011.56.42.png?alt=media\&token=31db4fa8-f3e1-4354-91bd-e0926c55c649)

Enter a **name** to easily recognize the instance, select an **Error recovery strategy** and fill **Properties** if you need to overwrite some of them. Save your modifications to create a new instance of your activity analyzer.&#x20;

The error recovery strategy determines how the activity is processed when the plugin fails.&#x20;

<table><thead><tr><th width="268.87910466607667">error_recovery_strategy</th><th>Failure reaction</th><th data-hidden></th></tr></thead><tbody><tr><td>STORE_WITH_ERROR_ID</td><td>The activity will be sent without any modification to the next activity analyzer.</td><td></td></tr><tr><td>STORE_WITH_ERROR_ID_AND_SKIP_UPCOMING_ANALYZERS</td><td>The activity will be saved without modification of the activity analyzer in failure. It doesn't be sent to the next plugin. </td><td></td></tr><tr><td>DROP</td><td>The activity won’t be saved</td><td></td></tr></tbody></table>

## Linking an instance to a channel

Once your activity analyzer instance is created, you can link it to one or multiple channels. To do so, connect to Navigator and head toward Settings > Datamart > Channels and select the channel where you want your activity analyzer to be executed.

Go to the **Activity Analyzers** category.

![](https://4196284719-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MMuoqM-5hJ5JY0WnAKL%2Fuploads%2FE7EYhtNTj5Hr8CLXwgNy%2FCapture%20d%E2%80%99e%CC%81cran%202022-07-05%20a%CC%80%2015.49.40.png?alt=media\&token=9e9ef113-36c4-4bf7-91d1-6f0d7276a633)

Click on **Add an Activity Analyzer** and select your instance.&#x20;

Several activity analyzers can be used on the same channel. In this case, they will process the same activity in a sequence of your choice: the second analyzer will process the activity as rendered by the first one and so on...

{% hint style="info" %}
Currently, you can't get more than 5 activity analyzers. If you need more, please contact your Account manager.
{% endhint %}

Make sure to define the right **order** and **error recovery strategies.**

## Debugging

### Plugin logs and metrics

As activity analyzers are plugin, you can monitor them[ as you do with all plugins](https://developer.mediarithmics.io/advanced-usages/plugins/monitoring).

### Verifying an activity

{% hint style="info" %}
UserActivity that run through activity analyzers are generally aggregated into sessions. You won't see your UserActivity until it has been put into a session and gone through the whole activity processing pipeline. See [how sessions are built](https://developer.mediarithmics.io/data-streams/data-ingestion/real-time-user-tracking/..#session-aggregation) to understand when you should see your activity or how you could fasten the process.
{% endhint %}

1. Go to the navigator > monitoring and search for the UserPoint associated with the activity.
2. Click on the **view json** button on any activity on a timeline
3. You can check if all the properties are OK and if your activity analyzers processed the activity as expect

In case of problem, you can look at two properties added to the activity. `processed_by` will tell you if the activity has been processed by your activity analyzer, and `$error_analyzer_id` will give you an error ID if the activity analyzer returned an error response.

```javascript
{
  "processed_by": "<YOUR_ANALYZER_ID>",
  "$error_analyzer_id": "<ERROR_ID>"
}
```