Basic Waaila functions

waaila.assert()

The assert() function is constructed for important checks where a failure signifies a serious problem, such as no sessions measured for yesterday or presence of completed orders with no products. The basic structure of the function without any messages or output tables is:

waaila.assert([passing condition], [score value]).break

When the condition in the function is not satisfied, assert() function fails (due to the inclusion of .break method) and the evaluation of the test ends. The result status is set to Failed. When the assert() function does not fail, that is when the condition is satisfied, the evaluation of the test continues. If there is no other assert() or warn() functions or if all other assert() and warn() functions do not fail as well, the result status is set to Passed.

If the assert() function does not fail, the score value is the score added to the total test score. The score is used for aggregate comparison of tests within one dataset or one depot. The realized score of tests is compared to the maximum score to calculate the performance of the data evaluation. The score needs to be a positive integer (zero score is not allowed).

Most often you also want to display a message showing what happened or display some data illustrating the situation. For example, if you test visits on largest hostname by calculating the hostname in the query and the test fails, you can display the name of the hostname in the assert message to verify it corresponds to your expectations. Moreover, if you compare to a threshold in the condition, you can output the value along with the threshold to see whether the threshold is missed only marginally or by a large degree.

For this purpose, you can specify a message or a table (or both) to be displayed in case of pass or failure (or both) by including further methods in a chain behind the main assert() function. These methods are optional. However, when you decide to include message or table in the chain, you need to specify if they should be outputted in case of pass or failure of the function's condition. You can do it by writing the .message() or .table() in the chain behind the .pass or behind .fail as you can see in the full example below. Just a note, you do not have to separate the code into new lines but it helps to improve legibility of the code.

waaila.assert([passing condition], [score value])
    .pass.message([string message in case condition passes]).table([normalized data displayed as table in case condition passes])
    .fail.message([string message in case condition fails]).table([normalized data displayed as table in case condition fails])
    .break

waaila.warn()

The warn() function has similar functionality to the assert() function but is intended for low risk tests or when something needs to be checked before the main evaluation. The basic structure of the function is:

waaila.warn([passing condition], [score value])

When the condition in the function is not satisfied, the result status is set to Warning. Unless the .break method is included in the end, the evaluation of the test continues. If there are other assert() or message() functions and all their conditions are satisfied, the status stays set to Warning. It can change only in case of assert() function failure (to Failed).

When the condition in the function is satisfied, the evaluation of the test continues as well and the result status is set to Passed. If there is some further assert() or warn() functions, the result status can change to Warning or Failed based on other functions' results.

The message and table outputs and the score value work in the same manner as for the assert() function which can be seen in the included example of the full set of options for the warn() function. The score needs to be a non-negative integer - contrary to the assert() function, zero score is allowed because the warn() function can be used for additional verifications that do not necessarily need to impact the quality of the data.

waaila.warn([passing condition], [score value])
    .pass.message([string message in case condition passes]).table([normalized data displayed as table in case condition passes])
    .fail.message([string message in case condition fails]).table([normalized data displayed as table in case condition fails])
    .break

waaila.message()

The message() and table() can be included both separately and as a method for the assert() or warn() function. For inclusion as a method for assert() or warn() function, refer to the specific sections. Below is described how they can be included separately. When no assert() or warn() function included, the inclusion of table() or message() functions sets the type of the test and the result status to Info.

The message() function provides the opportunity to output some values from the data loaded by the query. The structure of the function is very simple:

waaila.message([string message to be displayed])

waaila.table()

The table() function is an extended version of the message function. It allows you to display the data in the form of a table. This is very practical for an overview of the rows with the highest value, like top 10 hostnames. Alternatively, the table() function can be used to get an example of rows which do not satisfy a certain condition, so that you can decide whether it signifies a serious problem or only some minor misspecification in the data.

The structure of the function is the following:

waaila.table([normalized data input])

The table() function requires an appropriately structured JSON input which can be achieved by passing query results to normalizeGaResult() or normalizeAtiResult() function (based on the data source).

The table() function can be adjusted to display some of the columns color-coded to allow easier overview of the results. To display a column of a table color-coded, include a second parameter to the table() function as an array of objects for each column. Each object is defined for a unique column and includes a condition which if satisfied, colors given cell in green as an okay value, while otherwise the cells are colored in red as critical. The objects have the following structure:

• 'column' = the name of the column which should be color-coded
• 'condition' = condition to evaluate which consists of the type of the condition and the value for comparison
  • 'EQUAL': [value] (condition satisfied only if column is equal to given value)
  • 'GREATER_THAN': [value] (condition satisfied only if column is strictly greater than given value)
  • 'LESS_THAN': [value] (condition satisfied only if column is strictly less than given value)

const allowedDevicesArray = ['Desktop', 'Mobile Phone', 'Tablet'];
visits.forEach(function (row) {
    if (allowedDevicesArray.includes(row['Device - Type'])){
        row['isOK'] = 'OK'
    } else {
        row['isOK'] = 'NOK'
    }   
})
waaila.table(visits, 
             [{'column': 'visits', 'condition': {'GREATER_THAN' : 100000}},
              {'column': 'isOK', 'condition': {'EQUAL' : 'OK'}}]
            ); 

waaila.functions.normalizeGaResult()

The normalizeGaResult() function is constructed to normalize data from Google Analytics or Google Analytics 4 in order to simplify manipulation with the data and to prepare the data for transformation into a table (using the waaila.table() function). It makes the structure of the data more compact and thus clearer for extracting values from the data. The structure of the result, normalizedDataset, along with a particular example is described more in detail in the section Structure of the results.

The input of the normalizeGaResult is the data from a single query of a request to Google Analytics. In general, the function has the following format: normalizeGaResult(results['<id-of-query>'][<position-of-query>], <position-of-dateRange>). The parameter specifying the position of dateRange takes default value 0, i.e. the first dateRange available. Therefore if you only have one dateRange specified in the query, this parameter can be omitted. For illustration, for the first query of a request with id equal to 'sessions' and only one dateRange, the function can be specified as normalizeGaResult(results['sessions'][0]). This creates an Array of data rows where each contains key-value pairs for all dimensions and metrics.

waaila.functions.summarizeGaResult()

The function summarizeGaResult() allows you to extract summary of the data from the query output. The data are inputted in the same way as using the normalizeGaResult() function, i.e. the function has the following format: summarizeGaResult(results['<id-of-query>'][<position-of-query>], <position-of-dateRange>). The parameter specifying the position of dateRange takes default value 0, i.e. the first dateRange available. Therefore if you only have one dateRange specified in the query, this parameter can be omitted.

The output of the function is an Object with summary. In the summary there is the sum of all values (called 'total'), maximum and minimum value (called 'maximum' and 'minimum') and the type (marked as 'type') for every metric included in the data. The structure of the result, summarizedResult, along with a particular example is described more in detail in the section Structure of the results.

waaila.functions.normalizeAtiResult()

The normalizeAtiResult() function is constructed to normalize data from Piano Analytics (formerly AT Internet) in order to simplify manipulation with the data and to prepare the data for transformation into a table (using the waaila.table() function). It makes the structure of the data more compact and thus clearer for extracting values from the data. The structure of the result, normalizedDataset, along wtih a particular example is described more in detail in the section Structure of the results.

The input of the normalizeAtiResult is the data from a single query of a request. In general the function has the following format: normalizeAtiResult(results[<position-of-query>]). For illustration, for the first query of a request, the function can be specified as normalizeAtiResult(results[0]). This creates an Array of data rows where each contains key-value pairs for all dimensions and metrics.

waaila.functions.summarizeAtiResult()

The function summarizeAtiResult() allows you to calculate summary of the extracted data from the query output. The data are inputted in the same way as using the normalizeAtiResult() function, i.e. the function has the following format: normalizeAtiResult(results[<position-of-query>]).

The output of the function is an Object with summary. In the summary there is the sum of all values (called 'total'), maximum and minimum value (called 'maximum' and 'minimum') and the type (marked as 'type') for every metric included in the data. The structure of the result, summarizedResult, along with a particular example is described more in detail in the section Structure of the results.

waaila.functions.normalizedData()

There are several methods described below that can be applied on the normalized data once they are transformed into the unified structure of normalizedDataset. To use these methods on an object that was not created by the functions normalizeGaResult() or normalizeAtiResult() you can pass an object to the normalizedData() function. It needs to have the expected structure of an array of objects.

In the example below we extract the first part of the page path (expecting that the page path starts with /, e.g. from "/products/p01234/reviews" extracting "products"). Then we extract unique values of these page path starts, construct a normalized dataset and for each value calculate the sum of pageviews for all paths with given page path start. We can then use the methods .order() and .head() to display the 10 page starts with most pageviews (for more details refer to the following sections). Note that applying filter on the normalizedDataset produces also normalizedDataset.

const paths = waaila.functions.normalizeGaResult(results['input'][0]);
paths.forEach(row => {
    row.pathStart = row.pagePath.split('/')[1]; // expected path format: "/part1/..."
})

const pathStarts = paths.map(row => row.pathStart);
const pathStartsUnique = [... new Set(pathStarts)];
const pathsAggregJson = pathStartsUnique.map(val => {return {pathStart: val}});
const pathsAggreg = waaila.functions.normalizedData(pathsAggregJson);

pathsAggreg.forEach(rowAgg => {
    rowAgg.pageviews = paths.filter(row => row.pathStart === rowAgg.pathStart).reduce((sum, {pageviews}) => (sum + pageviews),0);
})

// display top 10 page path starts by the number of pageviews
waaila.table(pathsAggreg.order(['pageviews'], true).head(10));
// display top 10 page path starts by the number of pageviews (excluding empty string)
waaila.table(pathsAggreg.filter(row => row.pathStart !== '').order(['pageviews'], true).head(10));

normalizedDataset method .head()

There are several methods that can be applied on the normalized data once they are transformed into the unified structure of normalizedDataset. One of them is .head() method. It can be used to select only first several rows. Inside the parentheses you need to specify the number of rows from the top of the data that you want to select (the default is first 5 rows).

This is very helpful when you construct the test using the whole data and then you want to display a few top rows. For example, in case the test detects incorrect data, such as pagePaths including UTM parameters, you can display a sample of the problematic data (in the code below first 10 rows).

const paths = waaila.functions.normalizeGaResult(results['input'][0]);
const incorrectPaths = paths.filter(row => row.pagePath.includes('utm_'));
waaila.assert(incorrectPaths.length === 0, 100)
    .fail.message('There are paths including utm_')
        .table(incorrectPaths.head(10));

normalizedDataset method .filter()

The data in normalized dataset can be filtered in the same way as Javascript array. While you can also pre-filter your data already in the query, often you want to compare different parts of the data or perform a more difficult transformation. Note that applying filter on the normalizedDataset produces also normalizedDataset.

You can perform the filter on each row of the data (on each unique combination of dimensions with some metric values). In the code below we detect pagePaths including UTM parameters if they also have more than 100 page views.

const minIncorrectPageviews = 100;
const paths = waaila.functions.normalizeGaResult(results['input'][0]);
const incorrectPaths = paths.filter(row => {
    return row.pagePath.includes('utm_') && (row.pageviews > minIncorrectPageviews)
});
waaila.assert(incorrectPaths.length === 0, 100)
    .fail.message('There are paths including utm_')
        .table(incorrectPaths.head(10));

normalizedDataset method .select()

here are several methods that can be applied on the normalized data once they are transformed into the unified structure of normalizedDataset. One of them is .select() method. It allows you to select only a subset of columns to be included in the data. You specify the names of the columns inside square brackets.

This is useful for displaying a table when you have many metrics included for the evaluation of the test but you only need to display a subset of the columns. In the code below we detect pagePaths including UTM parameters. If any are detected a sample of 10 paths are displayed (without including the number of page views for the paths).

const paths = waaila.functions.normalizeGaResult(results['input'][0]);
const incorrectPaths = paths.filter(row => row.pagePath.includes('utm_'));
waaila.assert(incorrectPaths.length === 0, 100)
    .fail.message('There are paths including utm_')
        .table(incorrectPaths.select(['pagePath']).head(10));

normalizedDataset method .order()

There are several methods that can be applied on the normalized data once they are transformed into the unified structure of normalizedDataset. One of them is .order() method which allows you to sort the data by values of some column. You need to specify the name of the column to order by and the direction in which the column should be sorted (by specifying 'true' you are selecting the descending manner, i.e. from largest to smaller, and by specifying 'false' you are selecting the ascending manner).

While you can sort the data already in the query to the data source, you can use the method after filtering or when you want to display the top rows first by one column and then by another column.

const paths = waaila.functions.normalizeGaResult(results['input'][0]);
const incorrectPaths = paths.filter(row => row.pagePath.includes('utm_'));

// display top 10 incorrect paths by the number of pageviews
waaila.table(incorrectPaths.order(['pageviews'], true).head(10));
// display top 10 incorrect paths by alphabet
waaila.table(incorrectPaths.order(['pagePath'], false).head(10));