Balancing between anecdotal and statistical data in problem research

Which kind of data is which?

Anecdotal data is evidence through real stories. It is based on true experience, and the experience can involve either an individual or group. It often leans heavily on what a person thought or felt through whatever happened.

Statistical data is information you can quantify with numbers. You usually get it by asking a specific question or observing something. And although you can gather it manually, one piece at a time, contemporary tools like artificial intelligence and survey platforms mean that you can collect and understand statistical information involving many people, products or events incredibly quickly.

The bias misconception

People often view statistical data as infallible in the sense that 1 = 1 = 1. But statistical data often has hidden bias. Whereas the numbers involved might be irrefutable, the questions and data used to gather and aggregate those numbers in the first place can be based in entrenched subconscious or conscious beliefs. The way these biases are finding their way into modern tools is also a major concern for today's founders and is causing professionals to think critically about the way they design digital products.

At the same time, people also intentionally present the numbers so that they come across a certain way. For example, you can say that 4 million out of 10 million citizens participated in an election (the highest ever!) compared to 3.8 million the previous year, or you can say that just 40 percent of citizens cast a ballot (democracy is doomed!). Both statements are factually accurate. But people won't perceive them identically. This is known as “spin”, and people always use it with a specific purpose or end goal in mind.

Emotions rather than facts

Looking at anecdotal evidence, because anecdotal data is based in what actually happened, it does include facts. But the tendency is to see the data as less valuable because it is based on the perception of just one individual or group and typically is emotion-centered. However, emotion and relatability is what helps people connect and feel invested. When we hear anecdotal evidence, the information often can seem more “real” as a result. We pay attention and take action, because we transfer their experience to ourselves and feel like it involves or impacts us personally.

So both anecdotal and statistical data are imperfect. People can gather or present either one with bias.

Neither type is better than the other in this sense.

Balance is important

In real life settings, leaning too much on statistical data can make a researcher seem objectively authoritative at the expense of relatability. And conversely, leaning too heavily on anecdotes can make a person or group easier to connect with at the expense of conveying objective authority. The solution is to include both personal stories and statistical data, and to link them in a way that feels smooth, logical and relevant to the problem that's going to be solved by a product.

There is no hard and fast room for what constitutes balanced presentation, because a few good numbers can balance a single, longer and dramatic story and vice versa. Many founders and researchers find the right sweet spot for different circumstances or goals quite naturally. But others struggle. They might turn too much to statistics, for example, because they're more skilled with numbers, are shy about proactively finding people with stories or have the outdated, mistaken perception that it's not OK to mix real life and feelings with business. And some people err on the opposite side, such as if they don't know how to do statistical research or how to interpret numbers well in multiple contexts.

Never stop researching

You can never know what works and what doesn't until you try it in real life. It is especially important for digital products, who's founders often neglect the end users experience and behavior in favor of the product's perfectness. Only constantly iterating over research and implementation of the ideas based on the collected data and insights you can start seeing real influence. Never forget about the balance and start dealing with both numbers and real people!