Interpreting Wingify Personalize Campaign Reports

This article covers the following:

• Overview of Personalize Reports
• Access Personalize Reports
• Filter Personalize Reports
• Understand the Report Table
• Understand the Graph View

Overview of Personalize Reports

Wingify recommends that while configuring the campaign, choose the mode that aligns with your analysis goals. If you want to compare individual experiences, disable Holdback. If you want to measure the overall impact of personalization, enable Holdback.

Access Personalize Reports

To access your Personalize campaign report, go to Personalize, and select the campaign for which you want to view the report. You can view the report for your campaign under the Reports tab. It features three sections - Reports that highlights the outcomes of your Personalize campaign, Graphs that present the data graphically, and Filters that can be applied to the results for targeted study.

Note: If you have enabled multi-target mode in the campaign, you will notice a label stating Visitor qualification evaluated on every visit. This indicates that Wingify assesses the visitor’s targeting conditions and applies qualification rules every time a visitor accesses the campaign URL.

Filter Personalize Reports

As an initial step, you must set up your filter to view the corresponding report. You can set up the date range, segment of visitors, and visitor dimensions to fetch the data accordingly.

Understand the Report Table

Wingify Personalize campaign reports offer four distinct structures based on the combination of the Holdback setting configuration and Group By selection. You can use the Group By dropdown to group the report data by:

• Experiences Use this view to get a high-level overview of how each experience is performing across all your target segments. This is helpful for identifying which experiences are generally performing well or poorly. For example, if you have three experiences (A, B, and C) running across two targets (Target 1 and Target 2), the Experiences view will show you the combined performance of Experience A across both Target 1 and Target 2, the combined performance of Experience B across both targets, and so on. This aggregated view is helpful for quickly identifying winning or losing experiences without getting bogged down in segment-specific details.
• Targets Use this view to drill down into the performance of each experience within a specific target segment. This is helpful for understanding how different variations resonate with different audiences and for identifying opportunities for optimization within each segment. Using the same example as above, the Targets view will show you the performance of Experience A, B, and C specifically within Target 1, and then the performance of those same experiences specifically within Target 2. This segmented view is crucial for understanding how different variations perform with different audiences and for tailoring your personalization strategy accordingly.

This flexibility allows you to analyze your campaign performance from various perspectives. Here's a breakdown of each report structure and its parameters:

Holdback: Disabled and Group By: Experiences When Holdback is off, the report compares the performance of individual experiences within your campaign. As the report is grouped by Experiences, the report provides an aggregate overview of how each experience performs across all targets.

• Purpose: Compare the performance of different experiences across all target segments.
• Focus: Identify the best-performing experience overall.
• Data Displayed:
  • Experiences: Lists each experience in the campaign.
  • Current Traffic Split: Percentage of traffic allocated to each experience.
  • Total Value (Unique Conversions/Visitors): Total value, unique conversions, and visitors for each experience.
• Visualizations: Date Range Graph and Traffic Split.
• Example: Compare the performance of Experience 1, Experience 2, and Experience 3 across all target segments combined.

Holdback: Disabled and Group By: Targets

When Holdback is off, the report focuses on comparing the performance of individual experiences within your campaign. As the report is grouped by Targets, this view provides a more granular breakdown of campaign performance by target segment.

• Purpose: Compare the performance of different experiences within each target segment.
• Focus: Understand how different variations resonate with specific audiences.
• Data Displayed:
  • Targets: Lists each target segment.
  • Experiences: Lists each experience within each target.
  • Current Traffic Split: Percentage of traffic allocated to each target.
  • Total Value (Unique Conversions/Visitors): Total value, unique conversions, and total visitors for each target and each experience within the target.
• Visualizations: Date Range Graph and Traffic Split.
• Example: Compare the performance of Experience 1, Experience 2, and Experience 3 specifically within Target 1 and then separately within Target 2.

Holdback: Enabled and Group By: Experiences

When Holdback is on, Wingify creates a control group, providing a baseline against which to measure the impact of your personalized experiences. As the report is grouped by Experiences, Wingify groups all your personalized experiences together for comparison. This allows you to assess the overall impact of your personalization strategy by measuring the combined performance of all experiences against the baseline of the Holdback group. In addition, this report shows the expected value per visitor for each experience and the holdback group, the overall expected improvement in the primary metric compared to the holdback group, and the probability that the experiences will outperform the holdback group. These additional metrics provide statistical significance to your results, helping you confidently determine whether personalization drives positive change.

• Purpose: Compare the combined performance of all experiences against the holdback (control) group.
• Focus: Measure the overall impact of personalization.
• Data Displayed:
  • Holdback: Represents the control group.
  • Experiences: Groups all experiences together.
  • Current Traffic Split: Percentage of traffic allocated to the holdback group, the combined experiences, and the individual experiences.
  • Unique Conversions/Visitors: Total value, unique conversions, and visitors for the holdback group, the combined experiences, and the individual experiences.
  • Expected Conversion Rate: Expected conversion rate for combined experiences, each experience, and the holdback group.
  • Expected Improvement: Expected improvement in the primary metric for the combined experiences compared to the holdback group.
  • Probability of Better: Probability that the combined experiences will outperform the holdback group.
• Visualizations: Date Range Graph, Traffic Split, and Expected Improvement Graph.
• Example: Compare the combined performance of Experience 1, Experience 2, and Experience 3 against the holdback group to determine the overall impact of personalization.

Holdback: Enabled and Group By: Targets

When Holdback is on, Wingify creates a control group, providing a baseline against which to measure the impact of your personalized experiences. As the report is grouped by Targets, Wingify provides a granular breakdown of your campaign performance by target segment, including a Holdback group for each target. This allows you to measure the impact of personalization within specific audience segments and understand how different variations resonate with different audiences.

• Purpose: Compare the performance of different experiences within each target segment, including the holdback group for that target.
• Focus: Measure the impact of personalization within specific audience segments.
• Data Displayed:
  • Targets: Lists each target segment.
  • Holdback: Represents the control group within each target.
  • Experiences: Lists each experience within each target.
  • Current Traffic Split: Percentage of traffic allocated to each target.
  • Unique Conversions/Visitors: Total value, unique conversions, and visitors for the holdback group, each target, and experience within each target.
  • Expected Conversion Rate: Expected conversion rate for each experience and the holdback group within each target.
  • Expected Improvement: Expected improvement in the primary metric for each experience compared to the holdback group within each target.
• Visualizations: Date Range Graph, Traffic Split, and Expected Improvement Graph.
• Example: Compare the performance of Experience 1, Experience 2, and Experience 3 against the holdback group, specifically within Target 1 and then separately within Target 2. This allows you to see the impact of personalization on each segment individually.

The report table displays the data collected for each experience in the campaign. You can disable the experiences or preview the experiences to verify your campaign settings using Disable Experience and Preview Experience options, respectively from the vertical ellipsis icon (⋮) present at the right end of the table row.

Understand the Graph View

The graphs provide information on the conversion rate, conversions, and visitors over the configured period of time. To view the graph for a different period of time, visitor segment, or visitor dimension, modify the report filters appropriately.

The graph section contains the following four parts:

• Date Range - This feature shows the conversion rates, conversions, visitors, and revenue per visitor data.
• Traffic Split - This features the split of the overall traffic for each experience.
• Expected Improvement Graph - This is generated only if Holdback is enabled in the campaign. It represents a specific variation's expected improvement over the baseline.
• Funnel Graphs - This feature provides a powerful visual representation of how users move through a set of steps on your website or app.

Interpreting the Date Range Graph

Using the Date Range graph you can read the conversion rates, conversions, visitors, and the revenue per visitor by configuring them in the dropdown at the top-left of the graph.

In case of selecting the Conversion Rate (%) option, the Show ranges option appears at the top-right of the graph. The Show ranges option displays the median, the best case and the worst case values of the expected conversion rate corresponding to the experience. Through this, you can also figure out the uncertainty overlap an experience may share with the others.

Interpreting the Traffic Split

The overall traffic to your campaign is represented by the donut that is divided into different portions based on the number of experiences. The size of each portion represents the percentage of the amount of traffic that receives that experience.

For more information on how to interpret the graphs, see Leverage Graphs to Analyze Your Campaign Report.

Expected Improvement Graph

The Expected Improvement Graph compares different variations (experiences) against a baseline (Target 1) to show their likely impact.

• X-axis (Improvement): Percentage change in performance vs. the baseline.
  • Left side (red zone) = Not better performance
  • Right side (green zone) = Better performance
• Y-axis (Improvement Density): Probability of seeing the level of improvement.
• Each curve represents a variation’s distribution of possible improvements, with the dotted vertical line marking its most likely improvement.

At a high level, the farther right and taller a curve is, the more likely that variation will outperform the baseline.

Funnel Graph

The funnel graph provides a step-by-step comparison of how visitors progress through your conversion process for Control vs. Variations. It also highlights the drop-offs and overall performance differences.

You can leverage the Target selection dropdown to compare the performance of multiple experiences configured within a specific Target.

For more details on how to read and interpret funnels, see Funnel Graphs.

Need more help?

For further assistance or more information, contact Wingify Support.