Advanced usages
Usage reports
Metrics are being introduced to allow you to monitor the use of resources on mediarithmics. The first metrics introduced will be linked to the use of the database: READ, WRITE, SCAN.
Usage reports are currently in Alpha Release. Get in touch with your mediarithmics contact if you want further information on how to enable it. Definition for the metrics might be subject to change.
READ metric
The READ metric is used to provide an insight into the number of reads to the database.
READ metrics apply to all contexts where an OTQL or GraphQL query is involved: exports, queries in the query tool, in dashboards or using the API.
Method of calculation
Given a OTQL or GraphQL request, the READ number will be equal to the number of distinct collections contained in the request (
SELECT clause for OTQL) multiplied by the number of objects queried.Collections here stand for mediarithmics standard object types as provided in the default schema section.
Examples
In OTQL
Let's consider the following OTQL query:
SELECT {profiles { name } activities { ts }} FROM UserPoint
Here the SELECT clause contains two distinct collections: profiles (which is typed as the standard object type UserProfile in the schema) and activities (UserActivity in the schema). The number of READ would then be 2 multiplied by the number of user points queried.Let's now consider the following query:
SELECT {profiles { name creation_ts }} FROM UserPoint
Here the SELECT clause contains only one collection: profiles. The number of READ would then be 1 multiplied by the number of user points queried.Let's finally consider the following query:
SELECT { id } FROM UserPoint
Here the SELECT clause is considered to contain one collection (the UserPoint). The number of READ would then be 1 multiplied by the number of user points queried.Similarly,
SELECT { id profiles { name }} FROM UserPoint would be counted as 2 multiplied by the number of user points queried as both UserPoint (through the id property) and UserProfile (through the name property) collections are in the SELECT clause.For
@count operations, the READ number is 0, since no collection is in the SELECT clause.In GraphQL
Let's consider the following GraphQL query:
1
query MyQuery {
2
user_point(user_point_id: "xxxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx") {
3
id
4
creation_ts
5
profiles {
6
gender
7
}
8
}
9
}
Copied!
For the same reason as in the
SELECT { id profiles { name }} FROM UserPoint example above, the READ number here would be counted as 2 multiplied by the number of user points queried (which is always 1 in the case of GraphQL queries, see below) as both UserPoint (through the id and creation_ts properties) and UserProfile (through the gender property) collections are in the query.For GraphQL queries the number of objects queried is always equal to 1 since this type of query is intended to be limited to a single user point data.
WRITE metric
The WRITE metric is used to provide an insight into the number of writes in the database.
WRITE metrics apply to all contexts where standard objects related to user points are created, modified, deleted: imports, live data ingestion, segment computation, automations, ...
Method of calculation
The WRITE number equals the number of objects of a standard object type that are created, modified, deleted by a query.
Examples
For example if segment A is computed for the first time and 1000 user points are found to be in segment A, the WRITE number for this operation would be 1000 because 1 object of type
UserSegment would have been created on each of those 1000 user points.Similarly if at the time of the next computation, 500 user points are removed from segment A and 1000 are added, the WRITE number for this operation would be 1500:
- 500 WRITEs for the deletion of
UserSegmentobjects on 500 user points - 1000 WRITEs for the creation of
UserSegmentobjects on 1000 user points
SCAN metric
The SCAN metric is used to provide an insight into the number of items scanned in the database.
SCAN metrics apply to all contexts that require the database to be scanned: exports, segment computation, audience feed initial loadings, queries in the query tool, in dashboards or using the API...
Method of calculation
Given a OTQL request, the SCAN measure will consider the
WHERE clause:- If the
WHEREclause is empty, the SCAN measure will be equal to the size of the collection in theFROMclause of the query - If the
WHEREclause is not empty and there is only one collection in theWHEREclause (or properties linked to the collection in theFROMclause), the SCAN measure will be equal to the size of the collection in theWHEREclause - If the
WHEREclause is not empty and there are several (distinct) collections in theWHEREclause- separated by an
ANDoperator, the SCAN measure will be equal to the smallest of collection sizes in theWHEREclause - separated by an
ORoperator, the SCAN measure will be equal to the sum of the (distinct) collection sizes in theWHEREclause - separated by a mix of
ANDandORoperators, the same rules will be applied as sub-operations in the order of operations defined by the query
Collection sizes used in the calculation of the SCAN metric are computed and stored daily.
For GraphQL queries the SCAN metric does not apply as this type of query is intended to be limited to a single user point data.
Examples
Let's consider the following OTQL query:
SELECT { id } FROM UserPoint
Here the WHERE clause is empty so the SCAN number will be equal to the size of the UserPoint collection.Let's now consider the following query:
SELECT @count{} FROM UserProfile WHERE gender = "W"
Here the WHERE clause is not empty and contains only a property directly linked to the collection in the FROM clause, so the SCAN number will be equal to the size of the UserProfile collection.Let's now consider the following query:
SELECT @count{} FROM UserPoint WHERE accounts{} AND NOT emails{}
Here the WHERE clause is not empty and contains several collections: UserAccount (through accounts) and UserEmail (through emails). Both are separated by an AND operator so the SCAN number will be equal to the smallest collection between the UserAccount and UserEmail collections.Let's now consider the following query:
SELECT @count{} FROM UserPoint WHERE accounts{} OR emails{}
Here the WHERE clause is not empty and contains several collections: UserAccount (through accounts) and UserEmail (through emails). Both are separated by an OR operator so the SCAN number will be equal to the sum of distinct collection sizes. UserAccount and UserEmail being distinct collections, the SCAN number will be the sum of their collection sizes.Let's now consider a more complex query:
SELECT {id} FROM UserPoint WHERE events {ts > "2020-06-20" AND name = "$basket_view"} OR ( scenarios {scenario_id="1785"} AND is_defined(emails) )
Here the WHERE clause is not empty and contains several collections: UserEvent (through events), UserScenario (through scenarios) and UserEmail (through emails). These collections are separated by a mix of AND and OR operators, so we have to consider the order of operations:- First we have to consider the sub-operation inside the brackets, where
UserScenarioandUserEmailare separated by anANDoperator, the SCAN number here will be equal to the smallest of collection size of the two. - Then we have the larger operation to consider between
UserEventand the sub-operation we've just considered. These are separated by anORoperator so the SCAN number will be the sum of theUserEventcollection size and the SCAN number we've found for the sub-operation between brackets.
Last modified 7mo ago