Cleaning rules

Cleaning rules define how each datamart manages the deletion of user data. They can apply to two different user objects: profiles and events.
When an object enters the datamart, mediarithmics computes the expiration timestamp of this object based on the different cleaning rules that can be applied. This expiration timestamp is part of the object properties, and can't be modified. That means cleaning rules are non-retroactive: changing the rules will only have an effect on future incoming events and profiles.
1.A user profile or user event enters the datamart
2.Which cleaning rules apply to it?
3.Based on the cleaning rules and the priority between them, compute the expiration timestamp of the object
4.Store the expiration timestamp in the object
An event or profile expires if it hasn't been modified before the expiration timestamp. In details:
For profile: last modification timestamp field ($last_modified_ts for profile) is checked against expiration timestamp field.
For event: since an event cannot be updated, creation timestamp field ($ts) is checked against expiration timestamp field.
When an object expires, it does not return in queries or in the interface. It still exists in the passive database, until it is removed by a scheduled operation.
All cleaning rules have a status (DRAFT, LIVE or ARCHIVED), an action (KEEP or DELETE), a duration and a set of filters.
KEEP action means the associated object should stay in the datamart for the duration.DELETE action means the associated object should be removed from the datamart after the duration has expired.
There are two types of cleaning rules: 
Event-based cleaning rules apply to events. They can apply to all events, or only to different channels, activity types or event names.
Profile-based cleaning rules apply to profiles. They can apply to all profiles or only to a compartment.
When a user point does not have any activity, nor profile, nor USER_LIST segment, it is removed from the platform. By cleaning activities and profiles, cleaning rules will clean your user points.
Cleaning rules can be easily edited in the navigator, in your datamart settings
Rules priority
KEEP cleaning rules have the priority on DELETE rules. 
Only LIVE cleaning rules are taken into account when computing the expiration timestamp of an object.
Example 1
The following cleaning rules apply to my incoming event :
- KEEP for 60 days
- KEEP for 180 days
- DELETE after 150 days
The event will be kept for 180 days then deleted.
Example 2
The following cleaning rules apply to my incoming event :
- KEEP for 60 days
- DELETE after 150 days
The event will be deleted after 150 days.
Example 3
The following cleaning rules apply to my incoming profile :
- DELETE after 10 days
- DELETE after 150 days
The profile will be deleted after 10 days.
Rules lifecycle
Cleaning rules are created with the DRAFT status. They can't be created LIVE or ARCHIVED. While DRAFT you can change the different properties of the rule, except its type.

You then change the rule status to LIVE by doing an edit request. Once live, a rule can't be edited anymore and only the status can be changed to ARCHIVED.
At any moment, there should always be a LIVE event based rule with a DELETE action. You'll need to create at least an other rule to be able to archive this one.
Once ARCHIVED, rules can't be changed anymore, nor deleted.
You can only delete DRAFT cleaning rules. Once LIVE, they'll have to be ARCHIVED 
Object reference
A cleaning rule has the following set of properties.
Property
Type
Description
id
string
The cleaning rule identifier
action
enum
KEEP or DELETE. 
​
KEEP means the cleaning rule will force data to be kept for the duration, while DELETE means the data should be deleted after the duration.
​
Only DELETE is allowed for cleaning rules of type USER_PROFILE_CLEANING_RULE
archived
boolean
Used to make a rule disappear from the UI. You can only set it to true if the status is already ARCHIVED
status
enum
ARCHIVED, DRAFT or LIVE.
​
A cleaning rule can only go from the DRAFT status to LIVE, and can only go from LIVE to ARCHIVED.
datamart_id
string
The ID of the associated datamart
type
enum
USER_EVENT_CLEANING_RULE or USER_PROFILE_CLEANING_RULE 
life_duration
ISO 8601 duration
The duration associated with the cleaning rule
channel_filter
string
USER_EVENT_CLEANING_RULE only.

Optional. If a cleaning rule only applies to a channel, specifies its ID.
​
channel_filter must be compatible activity_type_filter if set at the same time (for example you can't target an app channel if you set the activity type to DISPLAY_AD)
activity_type_filter
enum
USER_EVENT_CLEANING_RULE only.

Optional. If a cleaning rule only applies to a specific activity type, specifies its ID.
​
SITE_VISIT, APP_VISIT, TOUCH, DISPLAY_ADor EMAIL .
​
channel_filter must be compatible activity_type_filter if set at the same time (for example you can't target an app channel if you set the activity type to DISPLAY_AD)
compartment_filter
string
USER_PROFILE_CLEANING_RULE only.
​
Optional. If a cleaning rule only applies to a compartment, specified its ID.
Content filter
Cleaning rules of type USER_EVENT_CLEANING_RULE can be associated with a content filter :
Proprty
Type
Description
filter
string
Value of the filter
content_type
enum
Target of the filter. Only EVENT_NAME_FILTER allowed for now.
Samples
// KEEP User event cleaning rule with channel filter and activity type filter
{
    "id": "907",
    "action": "KEEP",
    "archived": false,
    "datamart_id": "1509",
    "channel_filter": "3844",
    "activity_type_filter": "SITE_VISIT",
    "life_duration": "P65D",
    "status": "DRAFT",
    "type": "USER_EVENT_CLEANING_RULE"
}
​
// DELETE User event cleaning rule without filters
{
    "id": "778",
    "action": "DELETE",
    "archived": false,
    "datamart_id": "1509",
    "channel_filter": null,
    "activity_type_filter": null,
    "life_duration": "P60D",
    "status": "LIVE",
    "type": "USER_EVENT_CLEANING_RULE"
}
// its associated content filter
​
​
// DELETE User profile cleaning rule with compartment filter
{
    "id": "905",
    "action": "DELETE",
    "archived": false,
    "datamart_id": "1509",
    "compartment_filter": "1606",
    "life_duration": "P365D",
    "status": "ARCHIVED",
    "type": "USER_PROFILE_CLEANING_RULE"
}
get
https://api.mediarithmics.com
/v1/datamarts/:datamartId/cleaning_rules
List cleaning rules in a datamart
get
https://api.mediarithmics.com
/v1/datamarts/:datamartId/cleaning_rules/:ruleId
Get a cleaning rule
get
https://api.mediarithmics.com
/v1/datamarts/:datamartId/cleaning_rules/:ruleId/content_filter
Get the content filter for a rule
post
https://api.mediarithmics.com
/v1/datamarts/:datamartId/cleaning_rules
Create a cleaning rule
​
put
https://api.mediarithmics.com
/v1/datamarts/:datamartId/cleaning_rules/:ruleId
Update a cleaning rule and/or change its status
You'll have an error if doing any update that is prohibited, like changing a property while the status is not DRAFT or 
delete
https://api.mediarithmics.com
/v1/datamarts/:datamartId/cleaning_rules/:ruleId
Delete a DRAFT rule
post
https://api.mediarithmics.com
/v1/datamarts/:datamartId/cleaning_rules/:ruleId/content_filter
Add or update a content filter for a rule
delete
https://api.mediarithmics.com
/v1/datamarts/:datamartId/cleaning_rules/:ruleId/content_filter
Remove a content filter for a rule
​

User choices

Exercise of user rights

Last modified 30d ago

Property	Type	Description
id	string	The cleaning rule identifier
action	enum	`KEEP` or `DELETE`. `KEEP` means the cleaning rule will force data to be kept for the duration, while `DELETE` means the data should be deleted after the duration. Only `DELETE` is allowed for cleaning rules of type `USER_PROFILE_CLEANING_RULE`
archived	boolean	Used to make a rule disappear from the UI. You can only set it to `true` if the status is already `ARCHIVED`
status	enum	`ARCHIVED`, `DRAFT` or `LIVE`. A cleaning rule can only go from the `DRAFT` status to `LIVE`, and can only go from `LIVE` to `ARCHIVED`.
datamart_id	string	The ID of the associated datamart
type	enum	`USER_EVENT_CLEANING_RULE` or `USER_PROFILE_CLEANING_RULE`
life_duration	ISO 8601 duration	The duration associated with the cleaning rule
channel_filter	string	`USER_EVENT_CLEANING_RULE` only. Optional. If a cleaning rule only applies to a channel, specifies its ID. `channel_filter` must be compatible `activity_type_filter` if set at the same time (for example you can't target an app channel if you set the activity type to DISPLAY_AD)
activity_type_filter	enum	`USER_EVENT_CLEANING_RULE` only. Optional. If a cleaning rule only applies to a specific activity type, specifies its ID. `SITE_VISIT`, `APP_VISIT`, `TOUCH`, `DISPLAY_AD`or `EMAIL` . `channel_filter` must be compatible `activity_type_filter` if set at the same time (for example you can't target an app channel if you set the activity type to DISPLAY_AD)
compartment_filter	string	`USER_PROFILE_CLEANING_RULE` only. Optional. If a cleaning rule only applies to a compartment, specified its ID.

Proprty	Type	Description
filter	string	Value of the filter
content_type	enum	Target of the filter. Only `EVENT_NAME_FILTER` allowed for now.