The Wingify Web Experimentation report engine runs on the Bayesian statistical model that calculates the conversion rate within a range instead of an absolute number percentage. A range is a more accurate prediction of the conversion rate of an ongoing campaign where new data is collected and calculations are adjusted accordingly, while an absolute number helps in calculating the conversion rate of the data from the past.
You can use the Wingify date-range graph to see how the conversion rate of each variation has evolved over the duration of the test, and then derive the following insights:
- You can analyze the conversion trends of all variations in the test, leading up to the current state of the campaign.
- Identify any sudden changes in the conversion trends during the campaign.
- Compare how overlapping in conversion rates of different variations has changed over time. You will notice that the test starts with a huge overlap in the conversion rate across the variations. The overlap may change as Wingify collects more data for the test. The calculation of Winner/SmartDecision depends on the extent of overlap across variations, so looking at the overlap trends can make you dive deep into reasons why Smart Decisions/Winners are declared.
- A narrow range means that there is a lesser probability of the extremes to happen and a much higher probability for the median to be the actual conversion rate.
How the Calculation Works
Instead of expressing the number of conversions as a percentage, the Wingify Report calculates a conversion range within which the true conversation rate lies with 99% probability.
So, if a reported conversion rate ranges from 3.98% to 4.34%, it means that there is a 99% probability of the actual conversion rate lying in the range 3.98–4.34%. The more data your test collects, the narrower the range becomes with the highest likely values within it.
For example, if the conversion range is in the range 6.23–10.23% and the median conversion rate is 8.23%, it means that there is a 99% chance that the conversion rate lies in this range. And in that range, 8.23% has the highest probability.
The more data your test collects, the smaller this range gets, so we can be sure of the median value.