Use case 1 (GA4) – customize imported tests

Simple customization

In this use case we describe the process of importing and customizing tests on a test for hostname traffic anomaly (Waaila code W4130). The purpose of this test is to ensure that every hostname is receiving expected traffic based on patterns from previous weeks. By verifying traffic consistency, the test can identify significant changes that might indicate potential measurement issues or problems with the website itself. The test consists of two parts: analyzing anomalies in the aggregate number of sessions and analyzing anomalies in sessions by hostname. If the traffic overall or for any specific hostname deviates significantly from expected patterns, it raises a red flag for further investigation. A configurable parameter allows you to set a minimal expected amount of sessions (e.g., 100) to be included in the results, ensuring the test focuses on hostnames with substantial traffic.

1. Go to the Template Gallery tab and select the General Daily Checks template for GA4 data.

2. Open selected template and click "Use tests"

2. Select an existing depot (folder) where the dataset should be located or create a new one by selecting the option "Create a new depot" - note: if you have no depot, this step is skipped automatically

3. Select the option to "Create a new dataset" (tests can also be added from a template to an existing dataset in this step if you already have some existing datasets) - note: if you have no dataset for given data provider, this step is skipped automatically

4. Select the Google account email where you have access to Google Analytics data that you want to test

5. From listed data sources, select a GA4 account and property that you want to use for testing, set a custom name to recognize the selected set of tests and confirm the configuration by clicking "Save"

6. Open the Hostname traffic anomaly test by clicking on its name.

7. Run the test (the Run button is located above the Query logic)

8. Check the output.

   • the first table shows if there is any unexpected increase or decrease in the total sessions (this test is stricter, by default only 10 % deviation from expected value is allowed)
   • the second table looks in detail at each hostname (this test allows by default 25 % deviation from expected value for each hostname)

9. At the top of the test logic there are two objects, anomalyDetectionConfigAgg and anomalyDetectionConfigHostname. In bothe there is a parameter sensitivity which governs the strictness of the test (the higher the sensitivity, the stricter the test). Sensitivity should be between 0 and 99. Adjust the sensitivity parameters if the results are alerting to changes that are no problematic, or are not alerting to changes that are problematic.

   • by default anomalyDetectionConfigAgg.sensitivity is set to 90, allowing only 10 % deviation from expected value
   • by default anomalyDetectionConfigHostname.sensitivity is set to 75, allowing 25 % deviation from expected value for each hostname

10. After you customize the test, save the test and run it again.

Test requiring configuration

In this use case we describe the process of importing and customizing tests on a test for hostname traffic anomaly (Waaila code W4130). The purpose of this test is to ensure that every domain is receiving expected traffic based on patterns from previous weeks. By verifying traffic consistency, the test can identify significant changes that might indicate potential measurement issues or problems with the website itself. The test consists of two parts: analyzing anomalies in the aggregate number of sessions and analyzing anomalies in sessions by hostname. If the traffic overall or for any specific domain deviates significantly from expected patterns, it raises a red flag for further investigation. A configurable parameter allows you to set a minimal expected amount of sessions (e.g., 100) to be included in the results, ensuring the test focuses on domains with substantial traffic.

1. Go to the Template Gallery tab and select the General Daily Checks template for GA4 data.

2. Create a dataset from the template

   • click on "Use in dataset" button

   • specify an existing depot (if you don't have any depots, create a depot before you start creating a dataset)

   • select "Create a new dataset"

   • select a provider account (email which has an access to your Google Analytics data)

   • you can specify custom name, description and tags for the dataset

     Confirming the settings creates the dataset and opens it at the selected location.

3. Open the Hostname traffic anomaly test by clicking on its name.

4. Check the Test configuration that is at the top inside the Test logic field and adjust it based on your expectations of domains in the data

   • necessaryDomainsArray should specify a list of domains that are required to be present in the data
   • permittedDomainsArray should specify list of all domains that are allowed to be present in the data (therefore it needs to include all domains specified in the necessaryDomainsArray)

5. Run the test (the Run button is located above the Query logic)

6. Check the output table.

   • If any domains that you specified in the necessaryDomainsArray are missing in the data, you will receive the following message along with a list of the missing domains: 'There are some necessary domains that are missing in the data'
   • If there are any domains not specified in the permittedDomainsArray, you will receive the following message along with a table including overview of all domains: 'There are some sessions on unexpected domains'

7. Adjust the configuration if the results are warning against states that are OK.

8. After you customize the test to fail if and only if there is some missing or unwanted domain, save the test.

9. To exit the editor, navigate using the navigation line above the information about the test.