Introduction
For the final PfPC article of the year I’m going to completely swerve away from the theme of the rest of this academic year’s articles, and am instead going to offer some personal reflections on an aspect of my recent practice that didn’t go as well as I would have ideally liked. In so doing I will draw upon the spirit and messages of Black Box Thinking, Matthew Syed’s engaging and inspirational book on the importance of systematically learning from failures.
Black Box Thinking
In Black Box Thinking, Matthew Syed observes that as a society we can be prone to unhealthy attitudes towards failure, seeing it as something to be ashamed of, a threat to our professionalism (or ego), or an indication of fault or weakness. He notes that this can lead to a number of counterproductive behaviours, such as: finding it difficult to accept our own failures; becoming prickly if we have underperformed; seeking to blame or scapegoat others for our shortcomings; or rationalizing mistakes as ‘just one of those things’ and failing to reflect upon them properly afterwards. As a point of contrast, Syed argues that we can all learn much from the aviation industry, where:
“Every aircraft is equipped with two, almost indestructible black boxes… If there is an accident, the boxes are opened, the data is analysed, and the reason for the accident excavated. This ensures that procedures can be changed so that the same error never happens again” (p. 9).
As such, aviation is characterised by a progressive attitude to failure that drives forward key improvements in practice and systems. For Syed, this is a mindset that we all need to incorporate into our everyday lives if we are to unleash our potential and move forward. His challenge to us is clear:
“…We need to redefine our relationship with failure, as individuals, as organisations, and as societies. This is the most important step on the road to a high-performance revolution… Only by redefining failure will we unleash progress, creativity and resilience” (p. 14).
Indeed.
I will now apply some Black Box Thinking to a real-life scenario from my own practice.
“Every aircraft is equipped with two, almost indestructible black boxes… If there is an accident, the boxes are opened, the data is analysed, and the reason for the accident excavated. This ensures that procedures can be changed so that the same error never happens again” (p. 9).
As such, aviation is characterised by a progressive attitude to failure that drives forward key improvements in practice and systems. For Syed, this is a mindset that we all need to incorporate into our everyday lives if we are to unleash our potential and move forward. His challenge to us is clear:
“…We need to redefine our relationship with failure, as individuals, as organisations, and as societies. This is the most important step on the road to a high-performance revolution… Only by redefining failure will we unleash progress, creativity and resilience” (p. 14).
Indeed.
I will now apply some Black Box Thinking to a real-life scenario from my own practice.
The context: Head Teachers’ conference
In March of 2017, I had the privilege of being invited to speak at a local Head Teachers’ conference. The focus of my session was: How coaching can contribute to enhanced performance, development & wellbeing in schools. As you may be aware, this is the subject of my passion, my area of specialism, the focus of my book and a key strand of the APS mission and goals. So, a great opportunity. I enlisted the support of some colleagues to support me in co-delivering the session (each of whom would bring something different to the session, including two schools who were willing to talk through their experiences of using coaching), and set about preparing the content & structure. On the day of the conference itself, as I finished my talk and sat down, I heard one of the delegates at one of the nearby tables say: “Inspirational.” Thank-you, Bristol, and goodnight. Job done.
Or was it?
Or was it?
The importance of feedback
While I was glad to hear that positive comment, anyone who delivers training will know how important it is to get the full picture of participant reactions before drawing conclusions about an event. That’s why, for me, gathering evaluation feedback is such a crucial process (in fact, in APS, we evaluate more-or-less every piece of work undertaken, but that’s another story). On this occasion, participants had been asked by the conference organizers to rate the sessions on a scale from 1-5, with 1 being ‘Excellent’ and 5 being ‘Very poor’.
The feedback (as regards the session on coaching) from the 71 forms returned was as follows:
The feedback (as regards the session on coaching) from the 71 forms returned was as follows:
Uh-oh. Now, there are some positive ratings there, certainly, and that’s not to be ignored. However, the positive comment I’d heard at the end of the session was clearly not representative of the views of the majority. Furthermore, the average rating for the session was 2.73 – just on the positive side of the centre hump, but not by much, and a long way short of the standards I want to achieve (and, moreover, the standard I want APS clients to experience). Regrettably, there was also a good percentage (15%) of very dissatisfied participants – an unusual experience for me, and a telling statistic. Most notably, the mode response was 3. Average. Not a disaster, certainly – in aviation terms we’re not talking mid-air collision, here; however, continuing the analogy for a moment, it wasn’t a great flight and there was a close shave with another plane on the way into the airport. It was certainly far removed from what I would have liked to achieve at the session, and was a situation that merited further analysis.
So: What went wrong?
This is where Black Box Thinking comes in.
So: What went wrong?
This is where Black Box Thinking comes in.
Black Box Thinking
The Black Box mentality is characterised by two key questions: What happened? And what can we learn from that analysis to inform future practice? In this case, to address these questions, we need to turn to the qualitative data from the participant evaluations. So as not to risk anyone’s comments being identified, I’m just going to summarize the key messages from the feedback here:
Positive aspects
How did the session fall short?
All of which is fair enough, and all are messages that I accept.
The question is: What to do with that feedback?
Positive aspects
- The case studies from the two schools were consistently valued.
- Some participants (34%) found the session, or elements of it, helpful. It’s important not to lose sight of this, and I'm pleased those people were catered for.
How did the session fall short?
- It was too long (two hours was too much for a session delivered in this way at this time).
- For this audience, it needed to be more practical, and less theory/research-evidence focused, instead exploring how to use coaching in schools (as per the title!).
- The participants needed longer talk opportunities (we did include several of these, which helped the session, but they were only five minutes each in duration and didn’t allow for collective feedback/discussion afterwards).
- The material wasn’t pitched correctly for the majority of the audience. Many were further along the line in their knowledge and development as regards coaching, so they saw it as nothing new to them.
- The session wasn’t interactive enough. I didn't engage with the audience or take feedback/questions.
- The delivery was rather dry & factual, and too focused on reading from a script/PowerPoint slides.
- While I have every confidence in the actual material, some of it was perhaps not the right selection for this audience, in this forum, on a Friday afternoon.
All of which is fair enough, and all are messages that I accept.
The question is: What to do with that feedback?
Learning to inform future practice
Here are what I believe to be the key learning points to emerge from the experience, and what I would do differently if given the opportunity to run the session again with the same audience:
Interestingly, when I look back at those learning points, none of them is particularly revelatory to me. Indeed, I’ve been providing training since 2003, and those are all lessons that I learned relatively early on in my career (I think it’s fair to say I wouldn’t be developing a successful career as a trainer without having learned them!).
So what went wrong on this occasion?
Maybe it was as simple as failing to apply existing learning in a new context. Maybe I allowed my own interest in the material to interfere with a proper consideration of the needs of the participants. Maybe I allowed assumptions about what is usually expected from the context (a conference talk) to override my principles about what constitutes effective learning. Maybe my perception of the event as a potentially high-stakes occasion led me to prepare defensively in ways that don’t come naturally to me (for example, I never usually have a written script when training, so it was an interesting decision for me to adopt that particular strategy on this occasion).
I think possibly all of those reasons could have played a part to some extent, and that is some very useful learning to take forward into future endeavours.
- More acknowledgement of, respect for, and recognition of the audience’s starting point with respect to the material, with the pitch and learning methods adjusted accordingly (for example, there were parts of this session where it would’ve been better to facilitate a collaborative discussion and elicit experiences from the participants, rather than simply delivering input with slides).
- Less detail/facts, more stories/heart to bring the material to life.
- Less input, more interaction between and with the audience.
- More of a focus on practical implementation.
Interestingly, when I look back at those learning points, none of them is particularly revelatory to me. Indeed, I’ve been providing training since 2003, and those are all lessons that I learned relatively early on in my career (I think it’s fair to say I wouldn’t be developing a successful career as a trainer without having learned them!).
So what went wrong on this occasion?
Maybe it was as simple as failing to apply existing learning in a new context. Maybe I allowed my own interest in the material to interfere with a proper consideration of the needs of the participants. Maybe I allowed assumptions about what is usually expected from the context (a conference talk) to override my principles about what constitutes effective learning. Maybe my perception of the event as a potentially high-stakes occasion led me to prepare defensively in ways that don’t come naturally to me (for example, I never usually have a written script when training, so it was an interesting decision for me to adopt that particular strategy on this occasion).
I think possibly all of those reasons could have played a part to some extent, and that is some very useful learning to take forward into future endeavours.
Reflection questions
- What opportunities will you have in the coming days or weeks to seek feedback about an aspect of your practice?
- What can you learn from an experience that has gone less well recently?
- How will that learning shape your future practice?
- Who may benefit from that learning being applied?
References
Syed, M. (2015). Black Box Thinking: Marginal Gains and the Secrets of High Performance. London: John Murray.