2) Experiments in design
Why experiments?
Educate! recently re-branded Design and Training to Design and Experiments (D&X). The main reason was to emphasize our cultural tenet of 'always learning'.
Designers are now fully set up to innovate more freely and adapt more rapidly. Our contribution as D&X is to interpret data to identify problems/opportunities and run quick and timely tests to find solutions that drive impact.
In design we run experiments to answer 'how might we ....' questions. Sometimes these experiments can be small, we make prototypes (tangible or abstract) and ask feedback from end-users. Other times these experiments can be big; we run a variation of our program in selected schools.
However, experiments are ALWAYS geared towards better design, for more meaningful impact.
Note: In the module 'Human-istic' you will learn lots of frameworks that will help you in learning processes related to design.
In this session you will learn:
- What types of experiments we run in E!
- How to set a good guiding question for an experiment
The 'safe to fail' principle
Traditionally experiments have aimed at establishing cause and effect relationships. If we do A then B will happen. Sometimes we throw in 'intervening variables'. If we do A, and C is in place, then B will happen.
In educational change however, the causal relationships we are trying to influence are much more complicated than this. We don't always know which factors influence the change and how they will affect change.
As we seek the best design solutions to complex problems, we encourage designers and leaders in E! to create a culture of 'safe to fail'.
Why? Because the best way of knowing what works is trying it out. Trying out something new can either fail or succeed. And that is fine, as long as: 1) no one experiences negative effects of it in the long-run and 2) it generates learning that matters. The beauty of trying out things is that it can unearth new insights into benefits and losses, spark of new possibilities etc.
The safe to fail principle is embraced by an increasing number of global leaders who accept that their working environment is increasingly complex, and tolerance for failure is becoming a must!
Check out more about this in the links below:
Three main types of experiments
Prototyping
The smallest form of an experiment: build and test a prototype. A prototype is a rough version or representation of your final design product. This can be used to prompt responses from end-users or colleagues.
The best prototypes:
- Test key design assumptions (what will work how?)
- Do not take a lot of time to design or create.
Examples:
- Describe 3 group activities that can be done in a Skills Lab and ask mentors or teachers which they would prefer and why?
- Create a story board to explain the course of action in a module, check with colleagues if the sequencing is impactful.
Build, Measure, Learn (BML)
Get real-time experiences and feedback by rapidly testing your design (assumptions).
Inspired by the Lean Start-Up Approach, BMLs are used in Educate! to quickly learn about solving design problems.
The best BMLs:
- Are quick! Something is built and tested within 1 school term.
- Test out different ideas and compare which is best.
Examples:
- BML in Kenya to test how students can be motivated to attend consistently.
- BML in Rwanda to try out if District Monitoring Letters can be used as an incentives for schools and teachers.
Pilots
When we have tested out elements of our designs, we can choose to run a pilot. This is a real-life, full-on test-run of a program. This program is run by mainstream staff, in real schools and under 'normal' conditions.
The best pilots:
- Are as close to the final, workable design as possible.
- Can be iterated between terms.
Examples:
- Skilled Learning Program in Uganda. This program was piloted in several variations in 1 community unit. In 2019, the pilot was extended to 5 community units.
Guiding questions for experiments
A good experiment requires a good guiding question. And a good guiding question starts with purpose!
1. What is the design problem you are trying to solve?
Write down a general statement about your design problem or goal.
- You might be seeing a gap in what you are trying to achieve and what you are achieving (e.g. attendance rates are lower than expected).
- Or you have introduced a new goal to the program (e.g. running the E! Exchange through Government structures).
- Maybe you are foreseeing changes in the future that may have implications for the program (e.g. the product making skill you used to run is no longer marketable)
2. Why is this important? What other goals/strategies does this align to?
Determine whether this is a problem worth solving.
- If we solved this problem would we create more meaningful impact? (e.g. according to theory of change)
- Are other stakeholders expecting us to solve this problem? (e.g. donor, head teachers, government)
- Would this help us achieve our sustainability goals as an organization?
3. Formulate a guiding question for your experiment
Turn your statement into a design question starting with, 'How might we...'
How might we increase attendance to 100 students across masterclasses?
How might we utilize involve the district in ensuring accountability on pedagogical change?
4. Design the experiment
Brainstorm different answers to the question and identify which ideas are best to test! If your question does not evoke many ideas, you may have to rephrase your question!
Assignment #2 Set a guiding question for an experiment
Tip: Read through this example BML design from Kenya.
- Identify a problem or change your team will design for this month (e.g. incentivizing teachers, blending e-learning in YBE, etc.)
- Explain WHY it is important to make this change/solve this problem.
- Formulate a 'How Might We...' question for this problem
- Explain which type of experiment you can use to answer this question and why you chose this type.