4 Steps to Assessing an Experiential Class
Experiential teaching is arguably the best way to engage entrepreneurship students. At the same time, classes without textbooks are notoriously difficult to assess:
- No multiple choice tests means objective grades are hard to come by
- Team & project based-grades cause stress and conflict between students
- Grading written reflections is subjective, and doesn’t provide the “grade defensibility” more traditional assignments do
- Grade distributions can be difficult to achieve when the focus is on skills as opposed to scores
We’ve spent the last year developing a robust assessment strategy for our textbook-replacing Experiential Entrepreneurship Curriculum (ExEC).
With the new academic year upon us, we wanted to share our strategies in time for you to incorporate them.
ExEC’s Assessment Philosophy
Assess Process. Not Progress.
What that Means
- Ensuring students understand how to create businesses that fulfill customers’ emotional needs (e.g. solve problems, achieve desires, etc.) via an iterative process consisting of devising and executing experiments to validate assumptions.
- As teachers, we have very limited time with students – one, maybe two terms. The businesses they build during their time in school are not going to be their best/last chance at success. Students’ time with us is best spent developing a mindset that prepares them for creating future ventures.
Why We Believe it
A focus on process encourages:
- Skill development (not syllabus gaming)
- Meaningful learning about the market, customers, problems, etc. (not inflating/falsifying numbers/results)
- An experimental entrepreneurial/intrapreneurial mindset they can leverage regardless of where their career takes them
How We Achieve it
Evaluating students’ understanding and implementation of the business model validation process through:
- Out-of-class exercises
- Written reflections
- Presentations
- Small-group meetings with instructors
What Not To Assess:
- Achieving “Product-Market Fit” or “Problem Validation.” Often times the best outcomes for business model experiments is determining the model isn’t worth pursuing in its current design. Students should be rewarded, not penalized, for invalidating their assumptions, even if it means they don’t validate a problem during their interviews, or generate revenue during their demand tests.
- Number of interviews conducted. While students conducting very few interviews (e.g. < 5) aren’t demonstrating an understanding of the business model validation process, a high number of interviews doesn’t correlate to high comprehension of the process. In fact, in many cases, not being able to find customers to interview is a great way to invalidate assumptions. Avoid assessing students on the number of interviews they conduct, and instead, focus on the process they used to try to acquire their interviews, what they learned during their interviews, and how that informed their future hypotheses.
- Number of paying customers or revenue generated. Putting emphasis here will incentivize students to alter the results of their experiments. Instead, we want to encourage students to run objective experiments, and report out on their actual results, even, and especially, if that means their experiments “fail.” Emphasizing their process, over their progress, will decrease students’ fear of failure, and encourage a more risk-tolerant and innovative mindset.
- The originality or innovativeness of the idea. Assessing originality and innovativeness can be extremely subjective. Moreover, the focus of ExEC is to show students a process they can use to create successful businesses that solve problems. The solutions do not necessarily have to be original or innovative to solve a problem or teach students a process.
What To Assess:
Student’s ability to:
- Effectively recruit prospective customers for business model validation experiments.
- Design and execute business model validation experiments like demand testing, customer interviewing, etc.
- Conduct interviews to understand the emotional perspective of their customers.
- Use information from business validation experiments to devise and iterate possible solutions to their customer’s problems.
- Assess the financial viability of their solution.
- Describe their validation journey and understanding of the process.
An overview on how we implement our philosophy is below.
Assignments and Rubrics
There are four steps to the ExEC assessment model, all of which are graded on the following scale:
- Full Credit: means the student demonstrates a consistent and complete understanding of, and ability to apply, the validation principles underlying the assignment.
- Partial Credit: is given when students demonstrate an incomplete or inaccurate understanding of the underlying principles, or difficulty applying the principles.
- No Credit: is given when students demonstrate a lack of willingness to learn, or apply, the underlying principles of the exercise.
Instructors are given the freedom to implement this scale as they see fit (e.g. a points system, A-F grades, etc.). Details on the specific steps of ExEC’s assessment model below:
Step 1: Exercises
Written assignments students complete outside of class that help them design and execute their business model validation experiments.
There are 29 exercises students do outside of class during a typical 15-week ExEC course. So as to not overwhelm our instructors, we recommend they assess most exercises with a simple complete/incomplete scale, based on good faith effort. We do however call out four exercises that are worth assessing thoroughly:
- Business Plans vs Business Experiments: A written, or recorded, reflection on their Tower Building Challenge, where students educate a fictitious friend about the dangers of hidden assumptions and the power of experimentation and iteration. Why we grade this thoroughly:
- First exercise of the class.
- Demonstrates students’ understanding of the pros and cons of business planning versus business model validation.
- Underscores the importance of experimentation, which they will be assessed on repeatedly throughout the class.
- Your Early Adopters: This exercise demonstrates the difference between Early Adopters, Early Majority, et al., and helps students identify who they should conduct problem interviews to increase the efficacy of their outreach. Why grade this thoroughly:
- Basis for student interviews. If they don’t get this right, much of the rest of their exercises will falter.
- Will highlight the importance of empathizing with customers, which they will be assessed over and over throughout the course.
- Key principle of entrepreneurship.
- Customer Interview Analysis & Interview Transcripts: In these exercises, students record and transcribe (via automated transcription tools) each of their customer interviews, and build affinity maps to highlight the patterns in their qualitative data. Why we grade this thoroughly:
- Demonstrates students’ ability to conduct customer interviews.
- Demonstrates students’ ability to empathize with customers.
- Demonstrates students’ ability to do qualitative analysis.
- Will determine future experiments.
- Experiment Design Template: This exercise asks students to design an experiment to test their Business Model’s riskiest assumption, including how they’ll execute the experiment, how long it will take to execute, what the success and failure metrics are, and what their next steps are based on the potential outcomes of the experiment. Why we grade this thoroughly:
- Demonstrates students’ ability to identify the riskiest assumptions of their business model.
- Demonstrates students’ understanding of effective success metric definition.
- Demonstrates students’ ability to design and execute experiments that test falsifiable hypotheses.
Step 2: Validation Check-Ins
Short, 10 minute meetings between our instructors and individual teams where instructors assess a team’s understanding and application of the validation process, and help them overcome specific challenges they’re facing designing/executing their experiments.
Each check-in’s assessment focuses on four elements:
| Preparedness: students completed and brought all required materials. |
| Empathy: students were able to understand the emotions driving their customers’ pains/gains, and utilize that understanding to effectively resolve their customers’ needs. |
| Experimentation: students effectively hypothesized falsifiable assumption and design, and implement experiments to test those assumptions. |
| Overall Process Execution: students effectively demonstrates an awareness of why they are taking a given step in the validation process, understand how it will lead to their next validation step, and execute those steps effectively |
Step 3: Business Model Journal
A collection of Business Model Canvas iterations, and written reflections, detailing each student’s business model assumptions, experiments, and learnings throughout the course.
Unlike courses that produce a single Business Model Canvas at the end, ExEC students iterate their canvas upwards of 10 times throughout a course based on the experiments they run. Each iteration of their canvas is accompanied by a short reflection describing:
- What hypothesis the students tested this week
- The experiment they ran to test the hypothesis
- The results of that experiment
- How those results influence the experiment they’ll run next
Instructors can use this written history of each student’s validation journey, to assess how well the student understands and applies the validation process individually – independently of the contributions of their teammates.
Step 4: Process Pitch
A presentation of each team’s validation journey during the course, including all of their (in)validated assumptions, emphasizing their ability to execute the validation process, more than the final outcome of the business.
Students wrap up the ExEC course with a pitch, but not a traditional product-centric, Shark Tank-style pitch; this pitch is process-centric.
More important than the outcome of any single experiment, or grade on any one assignment, is helping students learn an entrepreneurial mindset – a process they can use to repeatedly use to solve problems of the people they want to serve.
This pitch not only helps instructors assess how well students understand the validation process, it will reinforcement one more time, the most important principles of that process:
- Empathy
- Experimentation
Exams
ExEC does not include any exams, choosing instead to focus student efforts on out-of-class projects. ExEC is however compatible with exams when appropriate or required by an institution.
For midterm or final exams, we recommend presenting students with a scenario and asking them to describe what should be done next. For example:
- Illustrate the focal venture’s business model using the BMC. Students can also be asked to create different version of the BMC based on changes in a key aspect of the business model (e.g., customer segment).
- Creating an interview guide (who to interview, where to find them, what to ask)
- Identify the riskiest assumption of the focal venture’s business model and design an experiment to test it (what assumption to test, specifics of the experiment design, metric to track success)
For possible scenarios/cases that can be used for an exam, consider the following:
- EcoWash: A business opportunity worth pursuing?
- An episode of the podcast “The Pitch.” In each episode of this podcast, real entrepreneurs are pitching their ventures to real investors.
- A news article about a newly-opened venture started by a local entrepreneur. (As an illustration, here is an article about an entrepreneur who started a shoe cleaning service). The business page of the local newspaper is a great source for possible scenarios.
Thoughts? Feedback?
That is the overview of the ExEC experiential assessment model. If you have any feedback, or suggestions on how to improve it, we’re all ears. Please leave a comment below.
We’d love to hear how you structure assessment in your experiential class.
On the other hand, if you…
Want Structured Assessment in your Class?
If you like the engaging of power of experiential teaching, and are looking for a structured approach to assessment, request your preview of ExEC today.
It only takes a couple days to get a feel for the material, and get your course set up to use it. If you’d like to try ExEC for your upcoming term, take a look today.
Want More Student Engagement?
