Surveys Have No Place in Entrepreneurship Classes

Surveys Have No Place in Entrepreneurship Classes

Gathering information from customers is the most valuable skill an entrepreneur can practice.

Two common methods for collecting that information are surveys and customer interviews. Customer interviews are, hands down, more valuable for entrepreneurs than surveys because they:

  • Provide the depth of insight to validate problem hypotheses
  • Provide emotionally driven marketing copy from the customer’s perspective
  • Identify high potential marketing channels
  • Identify realistic competitors, and competitive advantages
  • Provide potential pivot opportunities, by eliciting alternative problems to solve if hypothesized problem is not one customers are seeking a solution to

The qualitative nature of interview-based research gives entrepreneurs the chance to dive deeply into the problems and emotions a potential customer is feeling. It’s those feelings that the entrepreneur will ultimately resolve that will lead to their success.

Surveys in entrepreneurship classes, on the other hand, largely avoid addressing customers’ underlying emotional needs, because few, if any, potential customers will complete a survey about their feelings. Instead, customer surveys in entrepreneurship classes often use leading questions in an attempt to do the impossible – predict future customer behavior:

  • Would you use a product that does ______________?
  • How often would you use a product that does ________________?
  • How much would you pay for a product that does ______________?

The result of these surveys is that students either confirm their bias that there’s high demand for their product without discovering the emotional ways customers describe their problems, or they conclude there isn’t sufficient demand, leaving them without any actionable next steps either way.

Validation surveys provide no actionable marketing strategy if demand is “confirmed”, and no potential pivots if demand is “invalidated.”

While surveys have the allure of producing statistically significant data, statistically significant data on people’s predictions of their own behavior aren’t worth anything – especially in terms of business model validation. If we really want to answer questions like how much customer will pay for a product, there are far more effective ways of doing that than surveys, for example, selling pre-orders.

If we believe interviews are a far more powerful tool than surveys for business model validation, the question becomes:

How do we show students interviews are more powerful than surveys?

In our Surveys vs. Interviews Lesson Plan, we provide an experience that will demonstrate to your students just how much more effective interviews are than surveys, by having them complete both experiences, and compare them.

As a part of our Experiential Entrepreneurship Curriculum (ExEC), we recommend that before this lesson, students complete the following lessons:

  • Emotionally Intelligent Innovation. Here they learn that customer problems are the most effective place to look for value propositions, and
  • Idea Generation. Here they hypothesize the customers for whom they are uniquely suited to solve problems, and they hypothesize the problems they are uniquely suited to solve

With this background, they begin figuring out how to test those hypotheses.

Step 1: Problem Survey

Before class, ask your students to complete a Challenges Survey (find a sample in the lesson plan). Your students will be asked questions about the problems they face and how they have tried solving those problems.

In ExEC, we provide results from thousands of students at the universities using the curriculum so you can highlight how difficult it is to validate hypotheses about problems students face using a survey. What we find, and what your students will likely produce, are:

  • Low volume of responses
  • Short answers, with little emotional depth
  • Some responses aren’t even comprehendible

Step 2: Surveys vs. Interviews

Start class discussing with students the pros and cons of asking customers about their problems using surveys and using interviews. Each method of validation has pros and cons, as highlighted below. After the discussion, show this table and highlight any relevant points. Let students know they will now experience these differences.

Surveys Pros Surveys Cons Customer Interview Pros Customer Interview Cons
Fast Difficult to get responses to open-ended questions Higher quality information Takes longer to facilitate than surveys
Can produce statistically significant results Don’t provide insights on an emotional level Significant emotional depth Results aren’t statistically significant
Difficult to probe/ask follow-up questions Probe as deeply as necessary by asking follow-up questions
Often expensive (in time and money) to collect enough quantitative data to be statistically significant Can explore multiple problems

 

Step 3: Discuss Their Survey Experience

In the lesson plan, we guide you through a conversation with your students about this surveying experience. First, discuss why some students did not complete it. Then transfer those reasons to customers from whom they want to gather information. Next discuss what it felt like completing the survey, and how much emotional depth they provided.

Step 4: Interview Experience

We then guide you through introducing your students to customer interviewing. In groups, students will experience being interviewed, interviewing, and taking notes/observing. In these groups, students will ask and answer the same exact same questions from the survey, but in a format that’s much more conducive to problem validation.

Step 5: Compare their Survey vs Interview Experiences

The lesson ends with a discussion, focused on two key points:

  • Comparing the quality and depth of information gathered through each method, and
  • Comparing the ability to validate problem hypotheses using the information gathered through teach method

This is where the magic happens, as you reveal that in both their survey and interviews, they answered the exact same questions. As professor Emma Fleck told us after this lesson:

“I genuinely feel that this was a light bulb moment in my class. Students were frustrated and angry about this survey and didn’t see the point. However, 2 days later, when we did this as customer interviews, I was able to illustrate to them how much I could learn from using a different format with customers. They really started to understand as many of them had taken marketing research classes and were convinced that all of their customer learning would come from surveys!! Great exercise.”

Key Takeaways

This is a powerful lesson for students as they begin their entrepreneurial journey. It engages them in two important methods for gathering information to validate aspects of their business model. But more importantly, it offers two benefits:

  • Students feel the benefit of interviewing as a hypothesis validation tool.
  • Students practice customer interviewing. They learn how to be able to talk to anyone about their problems, so they can put themselves in a position to solve them.

 

Below is the complete lesson plan of the Surveys vs. Interviews exercise.


Get the “Surveys vs. Interviews” Lesson Plan

We’ve created a detailed lesson plan for the “Surveys vs. Interviews” exercise to walk you, and your students, through the process, step-by-step.

Get the Lesson Plan

 

It’s free for any/all entrepreneurship teachers, so you’re welcome to share it.

 


Get our Next Free Lesson Plan

We email new experiential entrepreneurship lesson plans regularly.

Subscribe here to get our next lesson plan in your inbox!

Join 4,000+ teachers. Get new lesson plans via email.
USASBE 2019 – See You in Florida!

USASBE 2019 – See You in Florida!

Are you going to the USASBE conference in St. Pete Beach, FL in January? You should – we got so much value from last year’s conference.

This conference is an incredible few days of entrepreneurship educators and folks planning entrepreneurship programs sharing their work and ideas.

If you’re going, we’ll see you there!

Friday Night Party

Let’s get the important stuff out of the way first. We’re hoping to host our second-annual happy hour party Friday night after the conference activities. We’ve got room for 100 USASBE Conference attendees to join us, so register here if you’d like to attend.

3 Talks + A Competition

We’ll be leading a handful of sessions during the conference:

  • Mechanical Pencil Challenge: Defining “Early Adopters” And Where To Find Them (Fri. @9:30 am in the Banyan room & again Sun. @9:30 am in Blue Heron).

This exercise uses mechanical pencils, and a 10-minute competition between attendees, to introduce Early Adopters. We contrast them with Early Majority and Late Majority customers, and demonstrate where and how to find a business model’s Early Adopters.

  • A Better Toothbrush: Testing Assumptions Via Customer Observations (Fri. @9:30 am in the Banyan room & again Sun. @ 9:30 am in Blue Heron).

This fun and high energy exercise utilizes children’s toothbrushes to help attendees see how easily they can make hidden assumptions that hinder the success of a project. We introduce attendees to customer observations, a tool to mitigate the consequences of hidden assumptions. Overall, this is an engaging Design Thinking exercise that encourages attendees to assume less, and observe more.

  • Rigorously Assessing Experiential Courses: Transparent Grading Using Check-Ins, Mini-Cases, And Reflections (Sat. @ 9:30 am in Snowy Egret).

During this session, attendees feel how frustrating it is to be evaluated by vague/subjective criteria. Attendees learn five tools for transparently evaluating experiential courses and then brainstorm ways they can incorporate these techniques in their course. Attendees leave with session with a set of detailed sample rubrics that will enable them to both teach more experientially and assess more objectively.

The Mechanical Pencil Challenge and the Better Toothbrush exercise

…are finalists for the 3E Experiential Entrepreneurial Exercises Competition!

Come cheer us on at 9:30am on Friday in Banyan!

Hope to see you there!

Justin, Doan and Federico

Which Customers Should Students Interview?

Which Customers Should Students Interview?

Click play above for the video version of this post.

Helping Your Students With Customer Interviews

In our last article, we used the business model canvas to describe why students should interview their customers. We also talked about how to motivate your students to actually conduct those interviews.

If you haven’t read that article yet, please do that now.

If you have, let’s talk about some of the common problems your students will experience when they get out of the building to talk to customers.

  • customer interviewingThey’ll have trouble getting people to agree to interviews
  • They won’t find a pattern among the problems they’re hearing from people they interview
  • They won’t hear anything about the problems they want to solve

All of these problems are common and are…

The consequence of simply interviewing the wrong customers.

Which Customers To Interview?

Effective entrepreneurs interview their early adopters, so we need to teach our students who early adopters are and how they can find theirs.

To define early adopters, we’ll leverage definitions by Rogers, Moore, and Steve Blank, but with a twist to make the definition more actionable. You can start by reminding your students that…

Customers don’t buy products. Customers buy solutions to problems.

Your students shouldn’t think about early adopters in terms of their relationship to a product. We want them to think about Early Adopters in relationship to a problem.

early adopter problemsEarly adopters are people who have the problem that your students want to try and solve, know they have that problem, and. . .

Early Adopters are actively seeking a solution to their problem.

These customers, who are seeking a solution to their problem, are the ones you want your students to interviews.

Focus Customer Interviews on Early Adopters

If your students can find, and interview, their early adopters, they will have accomplished the single-most important aspect of finding Product-Market Fit.

Customer interviews validate almost half of a business model canvasThat’s because during their interviews with early adopters, your students are going to validate their:

  1. Customer segments
  2. Value proposition
  3. Customer relationships and
  4. Channels

All told, interviewing early adopters will validate almost half of your students’ business models.

Plus, these interviews will form the basis of their experiments for the rest of the business model canvas.

On the other hand, if your students can’t find early adopters, they won’t have anyone to provide social proof to the early majority. That means they’re not going to find anyone who’s going to bring on the late majority or the laggards.

If they can’t find early adopters, it is very unlikely your students are going to find Product-Market Fit.

Can't find Early Adopters? Can't find Product-Market Fit.If your students can’t find people seeking a solution to a problem, it doesn’t mean the problem doesn’t need solving. It doesn’t mean that the business idea is bad. It means the time isn’t right to solve the problem.

If your students can’t find anyone seeking a solution to this problem, now is not the opportune time to try solving that problem. Your students could be too early to solve this problem, or they could be too late. We know that now is not the right time.

Early Adopters are the Product-Market Fit litmus test

By trying to interview early adopters, your students can form the basis of their business model if they find them. If your students don’t find them, that’s helpful news as well, because they can pivot with confidence. If your students can’t find people seeking a solution to the problem, it’s better to know now than later, so they can another problem to solve that’s more likely to lead to their success.

Finding Early Adopters

When you teach your students how to find early adopters, you may find it easiest to contrast early adopters with the early majority, late majority, and laggards, especially if you can a case study to do it, like Airbnb.

Laggards don't have the problem you're trying to solveStart by describing the problem your case study company solves. In this case, the problem Airbnb was trying to solve when it got started was that it was too hard to find cheap hotel rooms during a conference.

Next, describe the concept of laggards, people who literally don’t have the problem the entrepreneurs are trying to solve. Because they don’t have the problem, they don’t know they have the problem, and they’re not seeking a solution for that problem.

An example laggard for Airbnb’s early days might be someone attending a conference but their company pays for their room. Or someone who can expense any hotel costs they have, so they don’t worry about the cost.

Late Majority don't know they have the problem you want to solveContrast that to the late majority. This is someone who has the problem the entrepreneurs want to solve, but doesn’t know it; they are not aware they have the problem. If your students are ever trying to convince someone they have a problem, they are likely talking to someone in the late majority. These are some of the worst people sell to, because they are not aware they have the problem. Someone who doesn’t know they have a problem is rarely willing to talk about solving that problem, and if someone won’t talk about solving a problem, they certainly won’t pay to solve it. It’s important that you educate your students about the late majority, otherwise they’ll try to “educate” all their customers to convince them they have a problem, and won’t make any traction.

In AirBnb’s example, a member of the late majority might be someone who charges the hotel room to a credit card, even though it’s too expensive for their budget. They may simply think this is the cost of doing business and not even realize they’re getting charged exorbitant fees for a room in high-demand.

Early Majority aren't actively seeking a solution to the problem you want to solveCompare the late majority to the early majority. These are customers who know they have the problem, but are not seeking a solution to it. Maybe they have experienced the problem, and acknowledged it’s an annoyance, but they haven’t been so disturbed by it that they sought a solution. Or maybe they did seek a solution, and either found one that was good enough, or they didn’t, and assumed the problem wasn’t solvable. No matter what, a member of the early majority isn’t actively seeking a solution now (but will jump on one if they hear about it from an early adopter).

In the Airbnb example, a member of the early majority might be someone who skips the conference because they can’t find a cheap hotel room. They know rooms are too expensive. They searched for cheaper rooms online, but they couldn’t find something to fit their budget. They had other problems to solve so maybe they gave up and simply decided not to attend the conference.

Early adopters are actively seeking a solution to the problem you want to solveThe last, and most important, group is our early adopters. These customers not only know they have the problem the entrepreneurs want to solve, but are seeking a solution to that problem.

In the Airbnb example, an early adopter might be someone posting on the conference discussion group asking to share a room to lower their costs. Or maybe they’re searching the hostels in the area to find an affordable room.

To find these all-important early adopters, your students should brainstorm behaviors that indicate someone is seeking a solution to the problem. In the Airbnb example, the behavior would be “posting on a forum for a room share”, so to find those early adopters, the founders would simply look on the design forum.

Only Interview Early Adopters

Interviewing non-early adopters is worse than a waste of timeYour students should avoid interviewing anyone who is not an early adopter for the problem they want to solve.

That’s because if your students interview non-early adopters, they will discover problems entirely unrelated to the problem they are trying to solve – and problems few people actively seeking solutions for.

Imagine your students asking a late majority, laggard, or early majority the hardest part about going to a conference in the Airbnb example. Because these non-early adopters customers are not aware of, or seeking a solution to the problem the founders want to solve, the customers will describe completely unrelated problems like…

  • The food isn’t very good
  • The presentations are boring
  • The tickets to the conference are too expensive

We don’t want your students getting distracted by these other problems – we want them to validate, or invalidate their current problem hypothesis.

To do that, your students’ best bet is to focus their attention on their early adopters. Your students can use their customers’ solution-seeking behavior to tell them where their early adopters are.

In the Class

finding early adopter customersAnother case study that’s fun to use is Uber. Have your students think about the early days of Uber. The problem they were solving was the difficulty finding a cab in a big city like San Francisco.

Ask your students to describe a laggard in the Uber example. Maybe it’s someone who doesn’t take cabs at all – maybe they ride their bike everywhere.

Next, ask your students to describe an member of Uber’s original late majority. An example example could be someone who takes cabs but is often late. This segment, the late majority, take it for granted and don’t think cabs could be faster. To them, it’s part of their daily routine and they don’t think it’s a problem.

Now ask your students to identify behaviors exhibited by an early majority customer. Remind them this is someone who knows they have the problem. Maybe they have a black cab service on speed dial. They don’t want to use regular cabs because they’re too slow, so they’ll pay the extra price for a black cab service. The early majority is someone who has a solution that’s good enough for now.

Finally ask your students to identify behaviors exhibited by an early adopter. Remind them that early adopters are seeking a solution. They could be reading reviews on Yelp to find the fastest cab service in San Francisco, or they could be leaving reviews complaining about the slow response time for certain cab companies.

Remember: students should use early adopters’ solution-seeking behavior to find them for interviews.

For more details, take a look at the complete lesson plan we’ve provided below.

Get the Who are Early Adopters Lesson Plan

We’ve created an experiential, 45-minute, Who are Early Adopters Lesson Plan to help you teach your students who to interview. It encapsulates everything we’ve talked about above.

Get the lesson plan


It’s free for any/all entrepreneurship teachers. Please feel free to share it.

All we ask is that you leave us some feedback on it the comments below so we can improve it!


Better Customer Interviews

better customer interviewsIn this article we described who early adopters are and how to find them. That will help your students conduct better interviews.

  • They will get more interviews.
  • Your students will find consistent problem patterns because they’re talking to people who are trying to solve that problem.
  • And Your students will find problems they want to solve because they’re not talking to late majority or laggards.

If they interview their early adopters, your students will form the basis of their business model. If they can’t find early adopters to interview, they’ll know isn’t the right time to solve the problem they hypothesized and they will have the confidence to pivot (to a backup idea they generated through their problem generation process).

What’s Next?

In future articles, we’ll talk about who your students should target for interviews, and what to ask during them. If you’d like those lesson plans, subscribe here to get them in your inbox.

Join 4,000+ teachers. Get new lesson plans via email.