Use case 3 (Piano Analytics) – add and set a new test

In this use case we describe the whole process of creation of a new test using Piano Analytics (formerly AT Internet) data on an example of a check of visits on multi-domain website. In a multi-domain website, the traffic is split between several hostnames. However, if on there is no traffic on one of the required hostnames, there is either badly configured measurement or something unexpected is happening on the website (e.g. problem with access). In both cases, you want to be informed about it. Therefore, you want to check that all required hostnames are present in the data. If any are missing, the test results in a Failed status.

Main test

Follow the individual steps to proceed with creation of a test are described. If you need an overview how to create a dataset, where to find the "+ ADD TEST" button or where to locate the Query logic and Test logic, refer to the previous subsections.

Go to Depots/[MyDepotName]/[MyPianoDatasetName] or create a new Piano Analytics (AT Internet) dataset.
Select button "+ ADD TEST".
Set name to "Domains", select the Assertion template and set the score to 50.
In the Query logic insert the code below
1. columns contain visits by hostnames
2. the data are sorted by visits yesterday (from highest number to lowest)
3. the query uses the dataset code to locate the data or you can replace the string "[code]" with a different space parameter as a number (refer to Workspace setup to learn more about dataset code and how to find the space parameter)
4. there is a single period included (1 day ago)
5. the query loads the first 50 rows - if you expect more than 50 hostnames, increase the value for the max-results parameter (there is no need to decrease it if there are less rows)
```
{
  "columns": ["site_domain", "m_visits"],
  "sort": ["-m_visits"],
  "space": {
    "s": ["[code]"]
  },
  "period": {
    "p1": [
      {
        "type": "R",
        "granularity": "D",
        "offset": -1
      }
    ]
  },
  "max-results": 50,
  "page-num": 1
}
```

In the Test logic insert the normalizeAtiResult function to transform data to normalizedDataset and to extract the hostnames:

(results, waaila) => {
  const visits = waaila.functions.normalizeAtiResult(results[0]);

  // to display the whole data:
  waaila.table(visits);

  // to extract all hostnames as strings:
  const currentDomainsArray = visits.map(row => row['Site.domain']);
  waaila.message('all domains in data: "' + extractedDomains.join('", "') + '"');
}

Copy the extracted hostnames, delete any that are not necessary to be included in data every day, add any potentially missing and create a list of hostnames that is required to be present every day in the data. Your code should look like this:

(results, waaila) => {
  const visits = waaila.functions.normalizeAtiResult(results[0]);
  const necessaryDomainsArray = [
    "hostname1.cz",
    "hostname2.cz",
    "hostname3.cz"
  ];

  const currentDomainsArray = visits.map(row => row['Site.domain']);
}

To check that none of the necessary domains is missing, you need to add above the last curly bracket the following part. It contains extraction of all domains from the necessaryDomainsArray that are not included in the current data (by creating the missingDomainsArray). If there are any such, the assert fails and the message specified in fail_message_blackout is displayed.

// 1) none of the necessary domains disappeared
let missingDomainsArray = necessaryDomainsArray.filter(function (domain) {
  return currentDomainsArray.includes(domain) == false;
})
const fail_message_blackout = `There are some necessary domains that are missing in the data yesterday: ${missingDomainsArray.toString()}`;
waaila
  .assert(typeof missingDomainsArray[0] === 'undefined', 50)
  .fail.message(fail_message_blackout);

Run the test and check the results. The results should correspond to how you have set it in the previous step.
Save the test by using the either the "Save" button on the right, or the "Save and run" from the "Run" button drop-down if you want to run the test again
To exit the editor, navigate using the navigation line above the information about the test

Additional checks

There are several options to enhance the test for domains. You can check that no unexpected domain is present in the data by including an additional set of permitted domains (which includes all the necessary domains plus all that do not have to be present every day but that are still correctly measured). You can set individual limits on the share of some of your domains so that the test warns you if there is higher (or lower) share of all visits for given domain.

Alternatively, you can load comparison data for some previous period and observe whether the data are not changing relative to their previous state. If visits on any of your domains are increasing or decreasing rapidly, or even oscillating with a very high variation, this could indicate a problem with measurement or some other unexpected setting of your website. Of course, you need to be aware when the traffic increases due to a campaign - this will cause the test to provide a warning as well.

To evaluate relative change of the data, you can observe the visits on each of required hostnames and compare the current state (yesterday, "offset": -1) with the state a week ago ("offset": -8) to see that relative to weekly cycle the visits are within a certain threshold from the previously observed data. If you do not expect weekly cycles in your data, you can select a different comparison period (e.g. the day before yesterday, "offset": -2). You can set the comparison threshold experimentally or based on your previous observed week-to-week variation of visits.

You need to change the Query logic to the code below

metrics in columns are prefixed by the periods and additionally there is the difference between two periods (as defined in calculatedColumns)
the sorting is marked with a period prefix
there are two periods included (1 day ago and 8 days ago) - if you do not expect weekly cycles in your data, you can select a different comparison period p2 (e.g. selecting the day before yesterday, "offset": -2)

{
  "columns": [
    "site_domain",
    "p1.m_visits",
    "p2.m_visits",
    "dif_p2_p1_m_visits"
  ],
  "calculatedColumns": {
    "metrics": [
      {
        "key": "dif_p2_p1_m_visits",
        "op": "dif",
        "values": ["p2.m_visits", "p1.m_visits"]
      }
    ]
  },
  "sort": ["-p1.m_visits"],
  "space": {
    "s": ["[code]"]
  },
  "period": {
    "p1": [
      {
        "type": "R",
        "granularity": "D",
        "offset": -1
      }
    ],
    "p2": [
      {
        "type": "R",
        "granularity": "D",
        "offset": -8
      }
    ]
  },
  "max-results": 50,
  "page-num": 1
}

Next you need to specify the threshold for permitted change. You can set the comparison threshold experimentally or based on your previous observed week-to-week variation of visits. Here the parameter allows changes (both positive and negative) that are below 15 % of the comparison value (that is the absolute difference between visits in period p1 and p2 is smaller than 15 % of the number of visits in period p2).
```
const maxRatioChange = 0.15;
```

Then you can set evaluation of the changes (insert the code again inside the curly brackets of the test logic). In this code, data are extracted again so that you can filter them without changing the original dataset. Then the data are filtered to include only rows where the visits change does not satisfy the threshold and the domain is included among the required values. All domains which vary are extracted to varyingDomainsArray. If there are any such domains, the test fails, the fail_message_change message is displayed and additionally the whole data is displayed in a table with a new column 'Percent. change from (p2) to (p1)' that contains the relative change between p2 and p1 but in percents (i.e. value 25 means that visits have increased by 25 % of the number of visits in period p2).

const varyingDomainsVisits = visits.filter(function (row) {
    return (
        ((+row['Difference (m_visits (p2) - m_visits (p1))'] / +row['Visits (p2)'] <=
          -maxRatioChange) ||
         (+row['Difference (m_visits (p2) - m_visits (p1))'] / +row['Visits (p2)'] >=
          maxRatioChange)) &&
        (necessaryDomainsArray.includes(row['site.domain']))
    );
})
const varyingDomainsArray = varyingDomainsVisits.map(row => row['Site.domain']);
visitsMain = visits.filter(function (row) {
    return necessaryDomainsArray.includes(row['Site.domain']);
})
visitsMain.forEach(function (row) {
    row['Percent. change from (p2) to (p1)'] =
        Math.round((+row['Difference (m_visits (p2) - m_visits (p1))'] /
                    +row['Visits (p2)']) * 10000) / 100;
    delete row['Difference (m_visits (p2) - m_visits (p1))'];
})
console.log(JSON.stringify(visitsMain))
const fail_message_change =
      `Visits on some domains (${varyingDomainsArray.toString()}) have diverged more than ${maxRatioChange * 100}% from visits week before.`;
waaila
    .assert(typeof varyingDomainsArray[0] === 'undefined', 50)
    .fail.message(fail_message_change)
    .table(visitsMain);

Change the score to 100 - by going to the Edit option from the three dot options list at the test information on the top of the page.
Run the test and check the results. If the results additionally show a message that some domains have diverged, check the list and the corresponding values in the table to verify whether this is a change you want to be warned about. Experiment with the threshold maxRatioChange to get optimal results for your case.
Save the test by using the either the "Save" button on the right, or the "Save and run" from the "Run" button drop-down if you want to run the test again
To exit the editor, navigate using the navigation line above the information about the test.

Use case 2 (GA) – customize imported test

Use case 4 (GA 4) – add and set a new test

Use case 3 (Piano Analytics) – add and set a new test

Main test

Additional checks

On this page

One more click