A/B test and Split URL test report

Understanding A/B Test and Split URL Test Report

The Reports page appears after you have launched your experiment. It gives an overview, as well as a detailed analysis of the experiment's progress. 

Overview tab:

In the first section, all the major stats such as the visitor count, number of variants, number of goals, experiment start day, the day it is paused, and its active duration are displayed.

The Leaderboard shows the list of all goals assigned to the experiment with a star marking the primary goal.

Each goal has a table listing the following:

Rank: The rank of the variants, based on their performance, with respect to the corresponding goal 

Variant: A list of all the variants of your webpage, including the original 

Visitors: Number of unique visitors for the corresponding variation

Conversions: Number of unique instances of the visitor fulfilling the condition to achieve the goal

ConversionRate: This is calculated by dividing the Conversions by the number of unique Visitors. The conversion rate is represented by a percentage and can have any positive or negative value.

NOTE: For the pageviews goal, if the constraint type is an exact match and the constraint value is the same URL as the experiment URL, then the conversion rate will be 100%. This is because the number of unique visitors to the page will be same as reaching the goal: in this case a pageview. 


Detailed Analysis tab:

The first section is a chart that shows the performance trend of each goal over time. 

You can choose the type of graph by toggling between Time Range and Aggregate buttons.

Time Range: The performance over a period of time.

Aggregate: Direct comparison between performance of the variants or goals

Variants: A list of all the variants of your webpage including the original

Visitors: Number of unique visitors for the corresponding variation

Conversions: Number of unique instances of the visitor fulfilling the condition to achieve the goal

Conversion Rate: This is calculated by dividing the Conversion by the number of unique visitors. Conversion rate is represented as a percentage, with its values ranging from 0% to 100%.

Improvement: This refers to the percentage of improvement in conversions with reference to the original version of your webpage. The reference can be changed to any desired variation by clicking on the Baseline dropdown. 

The improvement percentage can have a positive or a negative value.

Significance: Achieving Significance or Statistical Significance means that the difference interval that is obtained for the conversion rate is not due to chance alone. For example, if the Statistical signficance is 95%, that means that there is a 5% or less chance that the improvement in the conversions from the changes you made in the variation is due to pure luck or chance. 

The second section consists of tables for each goal with in-depth and detailed stats. It contains the previously mentioned metrics with the addition of Difference Interval.

Difference Interval: This specifies the values between which the conversion rate lies. A 95% difference interval tells us that we can be 95% certain that the conversion rate lies somewhere between the lower and upper limits of the difference interval provided.

NOTE: If you had enabled heatmap for the experiment, a third tab titled Heatmap will appear alongside the exiting two. 

Share this post : FacebookTwitter

Still can't find what you're looking for?

Write to us: support@zohopagesense.com