NOTE: The concepts explained in this article mainly concern Testing campaigns. However, they’re parallelly applicable for Personalize campaigns, as well.
Access Graphs
To access the graphs:
- Navigate to any running or concluded test campaign
- Go to the Reports tab and scroll down on the report screen.
Based on the metrics you select, the report displays different graphs. Refer to the subsequent section to learn more about different types of graphs.
Types of Graphs
Based on your campaign report data, Wingify provides the following graphical formats:
Date Range The Date Range graph gives you the real-time status of the conversions, revenue, conversion rate, and the number of visitors that have arrived at your campaign URLs. The following are the Key Performance Indicators (KPIs) represented in the graph:
| X-axis | Represents the dates, as specified in the date range filter in your report. |
|---|---|
| Y-axis | Represents the count of one of the following parameters based on your selection from the KPI dropdown: For binary metrics Conversion Rate (%) - Represents the median of the expected conversion rate in percentage. You can select the Show Ranges checkbox to view the ranges of the expected conversion rates of the selected versions. For continuous metrics Total value - This is exclusive for revenue-like metrics. It represents the sum of metric values for all visitors. Common parameters Unique Conversions - Represents the number of unique conversions. Visitors - Represents the number of visitors that arrived at your campaign URLs. NOTE: Other than Conversion Rate(%), all the other KPIs are, by default, displayed as Day-wise graphs. You can switch to the cumulative view by selecting the Cumulative option from the View dropdown. |
Expected Conversion Rate Graph This Expected Conversion Rate curve in the shape of a bell depicts the prediction for your campaign, giving you the probable scope of the conversion rate (for binary metrics) or Metric value per visitor (for continuous metrics). It only features two parameters for this—the Expected conversion rate of the metric on the X-axis and the Probability Density on the Y-axis. Note: You can only view the Expected value per visit curve for the Revenue Metric.
Let’s look at the following example of a Probability Density curve of a binary metric representing our estimate of the conversion rate of the control version in a campaign.
The curve progresses upwards from around 44% and reaches the peak at 50%. It finally touches the ground at 56%. This means that the conversion rate range for your campaign is likely to fall between 44% and 56%, with 50.00% being the highest probability. This range is generated by accumulating data points, which will keep the curve shrinking towards the centre with the incoming visitors. This means the nearness to certainty keeps increasing with more data. In the below graph, the violet curve represents the control version, and the orange curve represents the variation. Here, the violet curve (Control) is narrower than the orange curve (variation), indicating that the violet curve (Control) has had more visitors as compared to the orange one (variation).
Note: NOTE: The PD curve keeps changing with the accumulation of data as the conversion rate is susceptible to change with it
For a campaign that has several variations, you will be able to see the PD graph with a curve for the control and each of the variations. This brings the following scenarios based on the overlap of the curves.
No Overlap
When the curves appear so far apart from each other that there is no overlap between them, then the curve at the right would be the one with the higher conversion rate. This means that the one on the right side has a greater scope of becoming a winner. The following graph shows the control (blue curve) and a Variation 3 (yellow curve), which do not overlap each other, where the variation has a higher probability density as compared to control.
The Control's conversion rate is just 50%, while the variation's is 84.16%. In this case, the variation is a clear winner.
With Overlap
You may encounter scenarios where these curves overlap. The overlap of the curves means that the probability of conversion rate is shared between the variations. The area of overlap plays a significant role in deciding the winner.
Case 1: If most curves fall within the overlapping area, the winning chances are equally spread between the two. This leaves no room for a conclusion. The following graph features two variations - blue and yellow.
Here, the median conversion rates are much closer to each other, and so are the probabilities ranges. In such cases, declaring a winner is almost impossible.
Case 2: If the overlapping area is small, the candidate with the higher probability density and conversion rate is more likely to be declared the winner. However, it doesn’t always happen that both these parameters belong to the same variant. Sometimes, the curve on the left has more certainty and a smaller range than the one on the right. This would probably lead to a situation where declaring a winner could be complex, and you would rather settle for declaring a variant as the smart decision.
Case 3:
Chances are more likely that you will run your campaign with more variations than just one. Overlaps, here, can provide a good indication of which variants are superior to others. You can choose to select only those variations whose trends you want to read for a detailed comparison.
Note: NOTE: The concept of overlap with probability to beat the baseline is only applicable when comparing a variation to the baseline.
In the below graph, the Violet curve indicates Control, the Orange curve indicates Variation 1, and the Green curve indicates Variation 2.
As you can see, the Green curve does not overlap with the Violet or Orange curves. Therefore, you can declare the Green curve (Variation 2) the winner, as it outperforms the others.
Expected Improvement Curve
The expected improvement curve represents a specific variation's expected improvement over the baseline. It only features two parameters for this - the Expected improvement relative to the baseline on the X-axis and the Improvement Density on the Y-axis. The value marked on the curve represents the most probable improvement you can expect to achieve after deployment. Let’s look at the following example of an EI curve of a variation relative to the baseline in a campaign.
The curve progresses upwards from around -10% and reaches the peak at 7.14%. It finally touches the ground at 25%. This means that the expected improvement for your campaign is likely to fall between -10% and 25%, with 7% being the most likely. This range is generated by accumulating data points, which will keep the curve shrinking towards the centre with the incoming visitors. This means the nearness to certainty keeps increasing with more data. The real strength of improvement distribution lies in its ability to provide a probabilistic understanding of potential improvements. Upon hovering over the curve the improvement distribution is divided into three distinct regions:
- Worse: (-infinity to -ROPE): The proportion of the Expected improvement curve in this region is the probability of that variation performing worse than the baseline.
- Equivalent: (-ROPE to +ROPE): The proportion of the Expected improvement curve in this region is the probability of that variation performing equivalent to the baseline.
- Better: (+ROPE to infinity): The proportion of the Expected improvement curve in this region is the probability of that variation performing better than the baseline.
Did you know? These proportions directly translate to the decision probabilities in the table above (in in-depth review mode).
Funnel Graphs
Funnel Graphs provide a powerful visual representation of how users move through a set of steps on your website or app. They show where people drop off or stop before completing a goal, like making a purchase or signing up. This helps you identify friction areas and make improvements to increase your conversions.
What You'll See in a Funnel Graph
- Step-by-Step Progression: Each stage of your funnel is clearly depicted, showing the number of unique conversions at each step you have defined in the funnel. This allows you to visualize the flow and identify bottlenecks for both - Variation and Control variants. You can compare any two variants simultaneously at a given time. Use the dropdowns to select the variants you want to compare.
- Drop-off Identification: The funnel's visual representation immediately highlights where users abandon the process. This information guides your efforts to improve specific stages.
- Variation Performance at a Glance: When analyzing a campaign, the Funnel Graph displays the performance of each variation side-by-side, making it easy to compare and understand which flow is most effective in guiding users to conversion.
At a high level, Funnel Graphs simplify the analysis of end-user journeys by providing a clear, visual breakdown of user behavior across multiple steps. It helps you with:
- A Complete Picture: Unlike metrics focusing only on one page, a Funnel Graph shows the entire user journey, from the first interaction to the final action.
- Step-by-Step Insights: Each step in the funnel is tracked separately, so you can see how each part is performing individually.
- Easy Comparison: You can view the performance for both, Control and Variation or any two variations simultaneously. This lets you instantly evaluate which version is doing better. In the image above, Control has a 25% conversion rate, while Variation 1 has 35%. That means Variation 1 is guiding more users through the funnel.
In short, Funnel Graphs make it easier to spot friction points, assess test performance, and make confident, data-backed decisions to optimize your user journey.