Note: This article covers SmartStats, Wingify’s Bayesian-powered statistics engine, which applies only to:
- Accounts on the non-Data360 framework, and
- Campaigns created before an account was migrated from non-Data360 to Data360.
For new campaigns (created after migration) or all campaigns in accounts created directly on the Data360 framework, refer to Campaign Reports in Data360, Wingify’s upgraded statistical engine that offers deeper insights and improved control on statistics.
You can use our Test Duration Calculator to find out the statistical significance required for your business needs. Alternatively, you can download this excel spreadsheet to calculate the data yourself.
Note: The Test Duration Calculator and the excel spreadsheet uses the Frequentist Method to calculate data. However, the statistical significance used in Frequentist Method does not apply to A/B and Split tests in Wingify, which are based on Bayesian statistics.
You may observe that the ‘significance value’ is different in the calculator and excel spreadsheet. The reason is that the excel spreadsheet shows the exact ‘p-value’, while the significance calculator will give ‘p-value’ between 1-p if the p-value comes out to be more than 0.5. If the p-value is less than 0.5, both the duration calculator as well as excel sheet will show the same value.
USE CASE 1: IF THE P-VALUE IS THE SAME IN BOTH CALCULATORS
USE CASE 2: IF THE P-VALUE IS DIFFERENT IN BOTH CALCULATORS
Refer to our article on Bayesian statistics and the mathematics behind calculating SmartStats. For revenue goals, download this excel file to understand the statistical significance.