When we talk about hypothesis testing in the CRO world, we typically mean testing different ideas against an observed problem or opportunity. It is unlikely that only one or two ideas exist for testing. In reality, the number of potential ideas can be extensive, and testing each of them requires time and resources.
With this being said, one might have the following questions- Which hypotheses should be tested? Which ones should be deprioritized or just thrown away? Thus, you must figure out the potential hypothesis to test first and spend more time on, i.e., you need to identify the order in which they will be tested. In other words, you must prioritize your hypothesis.
In this article, we will explore how you can prioritize your hypotheses in Wingify.
In Wingify, once you have created an observation and formed a hypothesis around it, the next step is to prioritize each hypothesis.
Prioritizing helps you scientifically sort your hypotheses. The closer a hypothesis is to the top of the list, the higher is its priority.
To prioritize your hypothesis, Wingify provides built-in prioritization frameworks such as CIE (Confidence, Importance, Ease), PIE (Potential, Importance, Ease), and ICE (Impact, Confidence, Ease), that you can select directly when creating a hypothesis. Each parameter is scored on a scale of 1 (lowest) to 5 (highest). The final prioritization score is calculated as the average of these parameters.
Refer to the table below for additional information.
| Framework | Components | Description | Best For |
|---|---|---|---|
| CIE | Confidence, Importance, Ease | This framework prioritizes hypotheses based on your confidence in the expected outcome, the importance of the traffic (or pages) involved, and the ease of implementation. | Teams that want to prioritize based on evidence and reduce uncertainty. |
| PIE | Potential, Importance, Ease | A balanced framework focusing on potential uplift, business importance, and implementation effort. | Teams looking for a simple, well-rounded approach to prioritization. |
| ICE | Impact, Confidence, Ease | Measures the potential impact of the change, your confidence in that impact, and how easy it is to implement the change. | Teams looking for quick prioritization and sorting of a large backlog of ideas. |
The prioritization framework streamlines the scoring process, allowing your team to adopt a standardized, industry-recognized model with a single click, ensuring all ideas are evaluated consistently.
While you create a hypothesis, the option to rate it appears in the Prioritization Score section. You will see the Prioritization Framework dropdown, which defaults to the CIE (Confidence, Importance, Ease) framework.
To score a hypothesis:
- Click the dropdown to view the available frameworks.
- Select the framework that best suits your requirement. For reference in this procedure, let’s select ICE (Impact, Confidence, Ease). Notice that the scoring parameters below the dropdown automatically update to reflect the chosen framework (Impact, Confidence, Ease).
- Assign a score from 1 to 5 for each parameter by clicking the desired value.
- Once you have filled in all the required fields and assigned your scores, click Create.
Your new hypothesis appears in the Backlog column with its calculated prioritization score. The prioritization score is a simple average of the values you assign to each parameter. For example, if you score Impact=3, Confidence=4, and Ease=5 for the ICE framework, the final score will be (3 + 4 + 5) / 3 = 4.0.
To review the score details, click on the hypothesis card. A panel will open on the right, displaying the breakdown of the scores you assigned.
You can rate a hypothesis based on the following parameters:
- Confidence: On a scale of 1 to 5 (1 being the lowest, and 5 being the highest), select how confident you are about achieving the expected improvement through the hypothesis?
- Importance: On a scale of 1 to 5 (1 being the lowest, and 5 being the highest), select how crucial the visitor landing on the test pages (for which the hypothesis is created) is.
- Ease: On a scale of 1 to 5 (1 being the most difficult, and 5 being the easiest), select the hypothesis's complexity. Rate how difficult it’ll be to implement the changes identified for the hypothesis.
The purpose of prioritizing is to ensure that you are running the most relevant tests for the business and getting the best ROI on your tool and people. However, there are other factors too that you must consider when prioritizing tests:
- How confident are you of achieving the uplift?- Prototyping the user persona you are targeting can help you in determining the potential of a hypothesis. With a sound understanding of your audience, you can make an educated assumption on whether the hypothesis will address the users’ apprehensions and doubts and nudge them to convert.
- How valuable is the traffic you are running this test for?- Your website may be attracting visitors in large numbers, but not all visitors become buyers. Not all convert. For example, a hypothesis built around the checkout page may hold a higher importance than the one built around the product features page.
- How significant will the impact be on your business goals? Impact measures the direct effect the change will have on your key business metrics if the hypothesis is successful. For example, a test on your homepage that affects all visitors will have a much larger impact than a change on a low-traffic blog post. Consider how a winning test will influence macro goals such as revenue, user sign-ups, or lead generation.
- What is the potential for improvement on the page? This parameter evaluates how much room for improvement exists on the pages you are testing. For example, a page with a high bounce rate, significant drop-offs in the funnel, or clear usability issues has high potential for uplift. Conversely, a page that has already been heavily optimized and performs well has lower potential for uplift. Before scoring, analyze the page's performance to make an informed judgment.
- How easy is it to implement this test? Next comes determining the ease of implementing your test. Try to answer some questions: Would it require a lot of strategizing on your part to implement the hypothesis? What is the effort needed in designing and developing the solution proposed by the hypothesis? Can the changes suggested in the hypothesis be implemented using just the Visual Editor, or does it warrant adding custom code?