A stand-up comic, a poker player & an actor walk into a bar…
TLDR: Certain jobs tend to have high frequency but very noisy feedback (such as stand-up comics, poker players and actors). The frequency and quality of feedback has a large impact on how people learn, and what attributes allow them to thrive.
A stand-up comedian, a poker player and an actor walk into a bar - not the start of a joke, instead a few weeks ago I met these three people at a friend's birthday party. Pretty quickly I became (probably unsocially) curious about each of their jobs. As it turns out, all three of them had one thing in common: they frequently received feedback on their work, but the quality of the feedback was very noisy and unreliable.
These three conversations led to me going down an unusually deep rabbit hole about learning, feedback and personal attributes that lead to success. But to understand how we get there, we should spend a little bit of time with each of these three people. Also as a warning, you will see lots of versions of this 2x2 grid in the course of this post:
Let’s start with our poker player. He had two main objectives: understand to what extent he was a winning player, and understand how much certain types of studying & training improved his performance. He also gets a lot of feedback. You get information every session you play based on whether you win or lost money. During each session, you get more information every hand you play as opposing players reveal what cards they had. And on an even more granular level you get information based on what people bet on each “street”. But that information is extremely noisy for a few key reasons: players would play a “range” of similar cards in the same way, regardless of cards they actually have and you may only be able to beat some of the cards in this “range”. Noise also comes from the luck involved in which cards end up on the board. And people are fallible, so you often make small mistakes, which collectively add a lot of noise. All of these things mean the information we spoke about is hard to interpret.
The consensus in the poker community is that it takes 10,000 hands to work out if you’re a winning player and around 60,000 - 100,000 to work out the margin that you are winning by to any degree of accuracy. When non-poker players hear this number they are often shocked. Unlike Malcolm Gladwell’s famous 10,000 hours to become an expert, poker players have to play around 1000 hours just to measure how much of an expert they are. That doesn’t mean you don’t learn anything in each individual hand. In fact, the best players can thoughtfully analyse how they played with the help of poker solvers (computer models which look at optimal decisions in specific scenarios). Regardless, the problem poker players face is that their outcomes are partially luck-driven, so they face large amounts of uncertainty about whether they are doing well (or if they are getting luck) every day that they play.
The level of noise in the feedback a poker player gets is very extreme, but noisy feedback is not uncommon. The stand-up comedian I spoke to said that she gets a read on the audience every time she tells a joke, but so many variables may influence the crowd: if the last comedian was good, it’s much easier to get a laugh; if there are audience members laughing a lot, the overall crowd can be more forgiving; and the location of the crowd, time of the day, type of venue and even temperature of the room all contribute to how much they laugh at any given joke. As a result, she spoke about how she tests out the same joke many times with different audiences (through the open mic circuit). An immovable issue for her is that “the funny-ness” of a joke is subjective and context-specific.
Subjectivity also impacts the actor I spoke to. She has been performing in a large volume of auditions to try to land roles. The quality of her acting performance is hard to assess in any objective way, and it’s even harder for her to assess her own work without the distance that watching someone else offers. A second problem she faces is that a large amount of information is private: the casting people and directors have a specific vision of characters in mind, and sometimes you might perform well but not be what they are looking for. Every audition would only result in a “yes, you get to go to the next stage” or a “no”. When she’s unsuccessful, there isn’t a robust way to understand why she gets a “no”.
In all three of these areas, the people struggle with regular failure and rejection: losing money, bombing a set, not getting a call-back. That failure is inevitable given the poor signal to noise ratio of the feedback they get, but it doesn’t make it much easier to stomach. All three of these people emphasised the importance of resilience in the face of the failure and importance of forcing themselves to “zoom out” and look at the longer term patterns.
How do other jobs compare?
But, we all get feedback all the time. And different jobs have different frequencies and varying quality of feedback. So what kind of jobs would fill the other parts of this 2x2 matrix? I’ve added three jobs which I think are archetypical of each of the other boxes. [Disclaimer: these are all subjective and I imagine some readers may disagree with them]
Professional golfers repeat a similar movement every stroke they take, and they get high quality feedback every time because they can see where the ball goes. The variables that influence the feedback, such as wind or interactions with hazards, are well understood and easily forecastable before taking each swing. So they get a lot of feedback and that feedback tends to be pretty consistently helpful. Unsurprisingly a sport like golf has started to become more like a science over time. Athletes use software to analyse their swing, they optimise their diet and gym routine to hit further, and there is a drive for caddies to get more scientific about the impact of weather to inform the golfers they work with.
Tax accountants have to work carefully to put together financial submissions and at the end of that complex, lengthy process they may get audited. Audits are not very frequent, but when they happen they are extremely detailed and (we should hope) the quality of the results is consistently high. Finding issues in an audit is particularly painful for the accountant - it comes with reputational risk and legal jeopardy for their clients. This also means that, while the feedback is infrequent, the stakes are high. Similar examples can be found in any job which requires costly and detailed inspection.
Venture capital firms make investments in early stage companies. It takes years to see if those investments work out, and this outcome is often not simply a function of the quality of the product or service the company offers. Similarly to the poker player the outcome is to some degree probabilistic, and similarly to the stand-up the context of the market has a big impact on valuations. There is a lot these VCs can do to reduce the variance, but the bets they make will by their nature always be high variance.
In each of these cases the feedback environment is structural. It is hard to influence the frequency or quality of the feedback. But most companies don’t face set structures, instead they can influence the amount of feedback and learning they get by changing their processes and being more deliberate. This is true at a company level, when frequent pilots and A/B testing can boost frequency and quality of learning. But it is also true at an individual level, where some deliberate changes in career development processes can boost individual learning. Higher frequency feedback can be achieved through normalising ad-hoc informal feedback, by introducing more mentorship and by putting regular learning moments in place (e.g., team retrospectives). Higher quality feedback can be achieved through deliberate moments of self reflection and goal setting, more objective performance measurement and creating personalised learning plans. As a result, in most circumstances leaders can move their organisation up and right on this grid to improve learning outcomes.
What can individuals do?
If you are an individual, there are also some lessons from these examples: the individual attributes you hold and develop can be adapted to suit the company environment you find yourself in:
If you are lucky enough to find yourself in a company that offers high frequency and high quality feedback, then learning is almost entirely a function of time and practice. Like the golfer, you should think about how you can invest time in the “driving range” and practise the skills you want to improve. Recommended reading: Outliers by Malcolm Gladwell
If on the other hand, you find yourself in a tax-accountant-like-situation then you will be rewarded for diligence and patience. Developing tools that will help you retain focus, and finding ways to introduce “light versions” of feedback will be key (e.g., introduce red-team reviews, where a peer can assess the work you’ve done). Being aware of cognitive biases is also key, because making mistakes is particularly costly. Recommended reading: Thinking fast and slow by Daniel Kahneman
If you’re in a similar place to our actor or comedian or poker-player, then two attributes are important: a deep well of grit or personal resilience, and an ability to step back and try to find the bigger patterns in results. Recommended reading: Grit by Angela Duckworth
Finally, those who are in the bottom-left box may find themselves incentivized to “fake it until they make it”. It’s often hard to objectively assess performance, and by the time performance is visible it’s often too late to adjust. In this case, I would recommend trying to build a toolbox of relevant experiences, where you have opportunities to learn lessons which you can apply to this particularly ambiguous environment. Having confidence and “faith” in the core thesis that you are pushing is also important, because similar to the above example there will be lots of noise that you will have to filter out. Recommended reading: Range by David Epstein
Summary and final thoughts
Learning is an ongoing process, where we respond to information about how we’re doing and adjust how we work to improve. This is all the more complex when we get information infrequently or when we have to filter the information because we can’t trust the quality of the signals we’re receiving. Companies have quite a bit of control over the frequency and quality of inputs people receive, but some kinds of jobs have structural realities which impose hard limits. In different learning environments, some individuals with different attributes tend to thrive more than others but we can be deliberate about developing the relevant skills to meet the challenges of our specific context.
This all matters because mastery is key to how much we enjoy our jobs and it is key to driving high performance in companies. We often see policy makers lament the lack of productivity growth, but we have the ability to control one of the biggest levers to productivity growth: deliberate and thoughtful learning to acquire human capital. Companies striving to implement pilots and experiments will have a competitive advantage, and people who try to create opportunities to learn more effectively will be most able to adapt to changes in work (such as the integration of AI). So helping people learn more effectively really is a win-win for everyone involved. This often isn’t easy, but each of us can put more effort into more frequent and high quality feedback.
