The curse of Cook
Six opening partners tried and rejected in three years – a turnover of one opening batsman per Test match season. The inability to find a player to rise to the challenge of opening the innings alongside Alastair Cook is the most prominent of the selection problems besetting England. This post does not venture a solution (although I have provided a mathematical response), it looks at the impact on those six selected, then rejected batsmen.
The impact of playing with Cook and then being dropped is assessed in a narrow, statistical fashion. The first graph shows the ten innings batting average in first class matches before and after each player’s brief career as a Test opener. (NB Adam Lyth’s post-Cook average is based on the six first-class innings he has played to date)
Across the group, there is a reduction in batting average of 40 runs per completed innings (56%). Joe Root has the sharpest reduction. He and Trott are the only members of the group who played Test cricket before opening with Cook; and Root is the only member of the group who played Test cricket after opening with Cook.
Three of the players (Compton, Root, Robson) may have harboured hopes that their Test opening careers would continue when they returned to first class cricket. Compton, for example, played four innings (including a century and a fifty) before his supplanting by Joe Root was made clear by the selection of an England side for a warm-up match. Selecting ten innings from his return to the Somerset side or from his official relegation from the England side makes little difference to this ten innings average (48.2 v 47.6).
We should not be surprised that players’ first class averages drop after a tough period as rookie Test match openers. They had been picked as form players – all six had short-term averages exceeding their career average when brought into the team – and their strong form had been interrupted by the stiffer challenge of Test cricket. In Trott’s case, his return to first-class cricket involved more than just re-finding form with the bat, but psychological health, too. The fall away in their performance, however, is noteworthy for its abruptness and consistency across the group.
To test whether it is a short-term effect, I have also compared their batting average for the last full season of first-class cricket before their selection as Cook’s partner and the first full season of first-class cricket following their demotion from the captain’s sidekick. In all cases except Root’s, the seasons assessed were England county seasons.
In this analysis the average fall in batting performance is less severe and is less consistent across the group. Root, the only player to remain in the Test team, maintained his pre-selection season average and Carberry’s varied downwards by fewer than five runs per completed innings.
All of the six players struggled for most of the innings they opened alongside their captain in Tests matches. Once out of the team (or in Root’s case, batting lower in the order), they were unable to regain their earlier productivity.
Alex Hales is strongly favoured to be Cook’s next opening partner. His current 10 innings first-class batting average is 36.0 – lower than all of his predecessors (although Hales may have further innings in the County Championship and in the UAE to improve on this before the Tests against Pakistan). Hales will, of course, be aiming to repel the curse of Cook that leaves batsmen under-performing when dropping back into county cricket. The surest way of doing this is by scoring so many runs for England that he stays in the team, opening alongside the captain.
Quick single: Maths and the opening partner problem
England have tried: the leading scorer in county cricket, the fresh graduate of every ECB age group team, an Australian, a man who survived a medical emergency, England’s most prolific number 3 batsman in decades and a bald Yorkshireman. With so many options tried, but no solution found, they could do worse than look now to maths. There’s a formula that could help find Alastair Cook an opening partner.
It’s the solution to the Optimal Stopping Problem. Its role is to assist in situations that feature the following characteristics:
- An agent has many options to choose from but can test only one option at a time.
- Once an option has been tested and discarded it’s very difficult to go back to it later.
- If an early choice is selected for ‘keeps’ then the agent would remain ignorant of what all the other options could have offered and whether they would have been superior.
- But if the agent keeps testing more and more options in search of a better one, the best option may get discarded.
The method is also known as the ‘secretary problem’, recalling a time in the last century when recruiting the right personal assistant was the sort of issue that bedevilled business men. For not unconnected reasons, it is now talked about as an aid to singletons trying to find a life partner.
Alastair has Alice as his life partner, but Straussy has long gone (from whites and track suits, anyway) and the search is on to find an opening partner. The Optimal Stopping Problem solution says that the agent (A Cook) should estimate the total number of partners he would be likely to try out in his (post-Straussy) Test career. In 29 Tests since the former captain’s retirement, Cook had, on average a new partner every six Tests. If his career continues for another five years, he could appear in 60 more Tests. That would equate to 10 more opening partners at the current attrition rate – and 15 in total.
The next step of the solution is to identify the number of partners that should be tested in order to get a feel for the quality of the field. Research has shown that the square root of the total number of potential partners (3.87) gives a strong probability of getting that feel but, to be certain, the agent should divide the total number of potential partners by 1/e (ie 35% of 15 = 5.25). So, Cook should test five partners and, following the theorem, identify the best of those and then keep changing partners until another one matches that standard.
When Cook opened with Trott, he completed the testing phase. His task now is to identify the best of the five and find another partner to match that standard. Who of Compton, Root, Carberry, Robson and Trott was the best? None, of course, made a compelling case, but with two Test centuries and one 50 in nine matches and an average of 32, I think Nick Compton has the edge. The mathematical solution for Cook is that when he comes across another opening partner who can emulate Compton’s record, he should toss aside his promiscuity and settle into a long term opening partnership.
In (at least) one respect, selecting an opening partner differs from the classic Optimum Stopping Problems: it is of course possible to reselect a previously discarded partner. That provides a very neat solution to Cook and England’s dilemma: call up Nick Compton.
Addendum: I am grateful to Seamus Hogan for this contribution:
@seamus_hogan: @chrisps01 Drawing on a paper by Weitzman (1979), you could add that ECB should try high-variance openers first!
I interpret this to mean that Alex Hales should be given a run in the Test team.
_____________
For more on the Optimum Stopping Problem, listen to the interview with Matt Parker in this episode of the BBC’s ‘More or Less‘.
Quick single: Jonathan Trott walking out by himself
Jonathan Trott’s short and uncomfortable innings on the final day of the 2nd Test led to a discussion on Test Match Special about the difference between opening and batting in the middle order. Michael Vaughan took the declarative approach initially: “It’s just different”. But pressed by Ed Smith, Vaughan revealed how he didn’t like having to wait to bat when first playing for England as a middle order batsman, with a background for Yorkshire as an opener (my recollection is that he didn’t have to wait long to bat on his debut against South Africa). Then, thinking of Trott’s move in the other direction, Vaughan suggested that he might not like sharing the walk out to the middle, as an opener does, as distinct from the lone walk of any other batsman.
It all sounded pretty trite.
Ed Smith ventured an explanation based on technique. He likes, he said, to see openers keeping their heads still when on strike. Trott at the start of his innings in this series had not just engaged a trigger move but his head was in motion as the ball was delivered, Smith observed.
Smith sounded insightful.
I have previous in this area. I wrote a piece over two years ago, What is an opening batsman? I looked at the conventional definition (orthodox technique, etc) and the performance of openers in recent years. There appeared to be no correlation between effectiveness and a match to the conventional definition. I concluded that four things made a batsman suitable for opening in Test matches:
- Experience of the role
- Complement to the opening partner (the weakest factor)
- Not the best batsman in the team
- Wants the job.
Smith was making a useful technical observation, but one no less relevant to a middle order batsman than to an opener. Vaughan, struggling to articulate a reason and sounding trite, was I believe closer to the truth and an understanding of Trott’s lack of success specifically in the role as opener: he doesn’t have experience opening and would probably prefer to bat somewhere else.
What is an opening batsman?
Nick Compton (above right) became the 868th player and 218th English cricketer to open the batting in a Test match in the recent Ahmedabad Test. He scored 46 runs from 181 balls, batting in a conventional opener’s fashion – even though the experience was unconventional with a spinner opening the bowling in both innings. Compton continued to impress in his second Test match with his solid defence, patience and judgement of when to play or leave deliveries.
In this third article on openers in Test cricket, I will examine the view that to face the new ball successfully a batsmen must necessarily have a conventionally correct technique and be risk averse. What is clear from the first two articles in this series is the importance, right now, of opening batsmen: the last two years have seen a slump in opening partnership performance, yet a partnership at the top of the order is not just a platform, but a springboard for the innings.
At the end of the English summer, with discussion around who would take Andrew Strauss’ place at the top of the England order, I asked a friend to detail what makes a batsman peculiarly an opening batsman. His answer I will refer to as the orthodox view:
Solid defence… Good judge of the ball to leave… Strong concentration… Expert against the short ball.
However, the orthodox view is neither true only of openers, nor is it true of all successful openers.
Batsmen whose games are built around a strong defence and the careful accumulation of runs may gravitate to the top of the order, but many of the finest exponents bat lower down. In recent years, Kallis, Dravid, Trott and Chanderpaul have been prolific and in the most part, measured scorers – perhaps benefiting from the efforts of their opener colleagues. Having a secure defence and a mind attuned to making big scores helps wherever in the order a batsmen is placed. If wickets fall early (as they very often do in a form of the game where the median first wicket score is 24), middle order batsmen need to display the same adhesiveness as the openers. Old ball (aka reverse) swing and the availability of a second new ball can also demand of the middle order circumspection and committed defence.
Nick Compton spent his first day of Test cricket watching Virender Sehwag (top left) roaring to a run-a-ball century, driving, flaying and carving shots to balls that many openers would watch carefully as they pass unimpeded past the off-stump to the wicket-keeper.
Sehwag, as an opening batsman, is a unique talent (described so well on Different Shades of Green) and yet not alone as an unorthodox player at the top of the order. Jayasuryia, Gayle, Sehwag, Hayden, Dilshan, McCullum and Warner have recently all prospered at the top of the order with an aptitude for quick scoring. What each of these shares is the ability to take the attack to the bowling, rather than their defensive prowess.
I would argue that the orthodox definition of an opening batsman is one that 
preoccupies England more than any other major cricket nation. English cricket’s inability to embrace the unorthodox is exposed in a different context – the career of Maurice Holmes, mystery spinner, by The Spin in the Guardian.
The records of the most successful Test openers of the last ten years show that a high scoring rate is associated with strong performance, when measured by batting average. Each square in the chart below represents one batsmen with over 1,000 runs in the last ten years as an opener. England batsmen are indicated by the red squares, with Marcus Trescothick having the highest strike-rate of the four.
Batsmen with below sample average scoring rates are distributed across almost the full range of batting averages. Openers with above average scoring rates have middling and high batting averages, with Virender Sehwag recording the highest on both measures. Amongst opening batsmen with strike rates below the sample average only Neil McKenzie, Alastair Cook and Simon Katich have batting averages over 45.
This picture of success as an opener being associated with fast scoring is reinforced when the aggregate performance of opening batsmen for the eight major Test teams over the last ten years is graphed (NB this is based upon opening batsmen, not opening partnerships). Higher scoring rates are associated with higher averages. The line of best fit proposes that for every additional run per 100 balls, the batting average rises by 0.9 runs.
Moving to a comparison of strike rate by each position in the batting order for this period shows there is no common pattern across the Test teams. India and South Africa’s opening batsmen score faster than numbers 3-6. New Zealand’s innings pick up speed with the middle-order.
I posed myself the question, what is an opening batsman? The answer does not seem to lie in either orthodoxy of technique or attitude to risk. To try to pinpoint what is distinctive about opening batsmen, it’s important to consider exactly what they do that is different to others.
Opening batsmen begin each innings against the opposition’s bowling at its freshest. They face the new ball – firm and gleaming, which makes it bouncier and liable to movement off the seam and in the air. One in two Test matches they are the first to bat on the pitch prepared for the match. It can have moisture that makes the bowling lively, but it is unblemished and should offer predictable bounce. The start of an innings very nearly always brings tension: the first overs of a Test match, the start of a reply to a large total, the initial stages of a run chase or match-saving rearguard. Batsmen in other places in the order will have occasions when the match situation contributes little to the nerves they feel when starting an innings.
From these statements of fact, I propose the following as the elements that make a successful batsman an effective opener:
- Experience of the role – the player who is brought up opening the batting and continues that role in the first-class game is well-placed to handle the pressure that goes with the role, at a greater intensity, in Test cricket.
- Complement to their partner – left hand/right hand combinations are common because of the disruption it can cause to bowlers trying to establish the optimum line and length at the start of an innings. Another complementary pairing might be fast and slow scorer. Overall, I consider this to be the weakest factor of the four described here.
- Not the best batsman in the team – the state of the bowling attack, ball and pitch all make opening the batting risky. It is in the team’s interest that its best batsmen are, if possible, held back until the initial push from the bowling side has abated.
- Want the job – to rise to the challenge, the batsman should be willing. A reluctant opening batsman may cope less well with the pressure.
That is what I have concluded gets closest to defining an opening batsman. It is about having experience of the role, being a good fit to the other opener, being of the required standard but not the team’s star player and above all, an opening batsman needs to be a batter who wants to be an opener.
Opening: platform or spring-board?
On England’s last tour to India in 2008, Alastair Cook and Andrew Strauss recorded a century opening partnership in their first innings of the first test in Chennai then came down to earth with a blob in their first innings of the next match at Mohali. In the game at Chennai, England were bowled out for 316. In the next game, without the leg-up of any sort of opening partnership, England totaled 302.
Observing these anomalies in the relationship of an opening partnership to the innings total had led me to research how influential the performance of the opening pair really is. By convention, the batsmen opening the innings represent one of the most crucial combinations in the sport. They face the opposition’s attack at its freshest and have the opportunity to set the tone for the innings and match.
This post is the second in a short series on opening partnerships and batsmen, which began with a review of the recent decline – since 2011 – in first wicket partnership scores.
I have selected as my sample for this exercise all Test matches since January 2003. I have considered only each team’s first innings – that is, the first and second innings of the match. In most cases, first innings have run their course, allowing a comparison to be made of the situation with one wicket lost and at the innings close. Innings three and four are much more likely to be curtailed by declaration, time or achievement of victory, making the required comparison misleading. 826 opening partnerships and innings totals have been analysed.*
The first chart shows the distribution of first wicket partnerships. The score with the highest frequency is 0 (7% of total). 29% of partnerships were broken by 10; 45% by 20. At the other end of the scale, 12% reached three figures. The mean score was 43 and the median was lower at 25.
Chart 1
The second chart shows the average innings total for every first wicket partnership score recorded from 0 to 415. The horizontal axis is not linear as it misses out scores for which no opening partnership was made (the lowest was 92). The darker, lower columns depict the opening partnership value.
Chart 2
To gain a perspective on how overall innings totals vary with opening partnerships scores, I split the data into three based on opening partnerships of 0-50, 51-100 and 100-250. In this last case, because of the lower frequency of scores, the analysis was based on ranges of 10 (e.g. 100-109, 110-119, etc). Lines of best fit have been added to the charts to show the relationship between the two figures.
Partnership 0-50: the average innings total where the first wicket was lost at 0 is close to 300. While many opening partnership scores have lower average innings totals than this, there is a clear association of increasing opening partnership and higher innings totals. The formula for the line of best-fit shows that for every run that the opening partnership increases, the innings total increases by more than two runs.
Chart 3
Partnership 51-100: across this range of scores there is also a positive association, although it is slightly less pronounced, with each additional run scored by the opening pair associated with an increase to the innings total of 1.8 runs.
Chart 4
Once the opening partnerships exceed 100, the addition of ten runs is associated with an increase to the innings total of nine runs. The association between opening pair and total score is weaker at this level of partnership.
Chart 5
It seems clear that the Cook and Strauss examples cited at the head of this post were exceptions from the general trend observed in the data. In its early stages, the opening partnership appears to have the potential to be a springboard for the innings as each run scored is associated with an increase to the innings total of over two runs. The association weakens as the partnership reaches three figures, so that from 100 upwards the additional runs scored by the opening pair see a lower number of runs ultimately added to the innings total.
I have tried to be cautious with my language in describing these data. But the title to the post does assume that there is a causal relationship between opening partnership and innings total. The logic to that argument is well understood; crudely: by blunting the opening attack the first wicket pair establish an ascendancy over the fielding side which the rest of the team can capitalise upon. However, as statistics textbooks insist, we need to recognise that:
evidence of correlation does not imply causation.
How could this be?
The size of the opening partnership and of the innings total may both be influenced by third factors, rather than themselves be causally linked. There may be more runs scored later in the innings, not because the openers got the innings off to a good start, but because the whole innings benefited from factors such as: a weak or out-of-form bowling attack; poor standard of fielding; favourable weather and pitch conditions. In this interpretation, the opening partnership is merely a platform on which the rest of the innings will be built, consistent with its context of opposition and environment. Whether this is the case, or whether the opening partnership directly influences the fortunes of the batsmen that follow is difficult to unpick in a statistical review.
However, what numerical analysis might not reveal, cricketing commonsense can discern. The opening partnership is acting as a springboard when, for example:
- the opening pair, perhaps on the first morning of the game, battle through to lunchtime against challenging new ball bowling on a lively pitch. By protecting the rest of the line-up from a potent bowling attack, the openers will have increased their teammates’ run-making potential.
- one or both openers takes the attack to a quick bowler who has the potential to be a threat. By forcing a bowler out of the attack, the openers have managed to disrupt the fielding captain’s plans, undermined that bowler’s confidence, with the potential to reduce his and his colleagues’ effectiveness.
Perhaps these are exceptions, which have the impact that all opening partnerships aim for, but in most innings must settle for providing a platform.
Are there examples you can identify where a test match opening pair have acted as a springboard for their team’s innings?
__________________
Note
* The sample includes 125 innings concluded by declaration. The average opening partnership and innings total were higher for this sub-sample.
I excluded a small number of innings from the analysis where I decided the circumstances would not shed light on the relationship of opening partnership to innings total: one of the openers retired injured and so the opening partnership involved more than two batsmen (3); the innings was closed with the opening partnership unbroken (1); the innings was closed by the end of the game (8).
Acknowledgement: thanks to Michael Wagener, stats sage, for reading a draft of this article.
Opening: downturn and upheaval
England will very shortly select a new opening batsman to partner Alastair Cook. India have been under pressure to jettison one or both of their openers who are without a century in over two years of test cricket. Australia keep a brave face on the situation five years after Langer and Hayden last opened together. New Zealand and West Indies can only call on their dominant forces at the top of the order when franchise cricket allows.
I listed these topical examples of downturn and upheaval in Test match opening batting off the top of my head. It persuaded me to a little research, which has spawned a short series of articles. This first post tests whether there is anything afoot with the top of the order in Test cricket right now. It will be followed by an analysis of how important the opening partnership is to a Test match innings and then my true quarry in this chase: what is an opening batsman?
The opening partnership can be the foundation of an innings. If it can blunt the keenest blade of the opposing attack, fairer batting conditions can be created for those that follow. It’s obvious, but I’ll state it nonetheless: the opening partnership is where a match and also a series begins. It has the potential to set the tone, establish an early advantage. How much of this is true, or even a reasonable expectation, I may address later. My starting point is that the opening partnership is important and that it is a weakness right now in Test cricket.
In 2011, the average opening partnership in Test cricket was 31. It follows ten years when the average never fell below 35 and pushed 45 in 2009 and 2010. In the 66 years of Test cricket since the second world war, the average for 2011 is the lowest bar only five years, the most recent being 2000 and the only one significantly below the figure for 2011 was 1963 (24.26). The figure for the first 10 months of 2012 has recovered a little, to 34.43. The average for the post-war period is a shade under 38.
A possible explanation could be that across Test cricket, run-scoring has become more difficult in the last two years. Looking at the same statistic – average completed partnership – for wickets two to five confirms that that is not the case. The second wicket average fell in 2011 but five of the previous 20 years were lower still. Each of the other wickets had an average partnership above the 20 year norm for at least one of 2011 and 2012.
Averages can conceal as much as they reveal. Every opening batsman from school cricket upwards knows that their principle task on taking guard is to see off the new ball (I am excluding limited overs cricket from my analysis and narrative). The effectiveness of a new ball wanes at different rates according to the ground and air conditions, make up of the bowling attack and brand of ball. I have set a score of 70 as the mark at which Test match opening partners can allow themselves a moment of satisfaction to reflect on a job well done, on the basis that more than 20 overs are likely to have been bowled and the best part of a session elapsed. Since World War 2, around one in six opening partnerships has reached 70. The chart below shows this percentage for each year since 1992.
This perspective suggests that 2011 and 2012 have not been very far from the norm. The most striking aspect of the chart is the halving of the proportion of opening partnerships reaching 70 between 2010 and 2011.
This piece concludes with a team-by-team review of opening partnerships since January 2011. Three aspects of performance are considered: stability (average tests per partnership), productivity (average partnership score) and delivery (% partnerships greater than or equal to 70).
Only Australia and South Africa have more productive opening partnerships 
than the post-war average. In terms of delivery, South Africa and Zimbabwe better the post-war proportion of opening partnerships seeing off the new ball (i.e. scoring 70 or more). Pakistan, by some distance, have had the most stable opening pair – Mohammad Hafeez and Taufeeq Umar – although their record is unremarkable.
The data appear to support my anecdote and observation: the opening partnership in Test cricket is at a low ebb, though certainly not in crisis. Few teams have been able to count on stability in the recent past and perhaps only South Africa and Pakistan have strong reason to believe that their established opening partners will remain in post this time next year.
In looking for a cause, the obvious place to start is with the opening batsman’s direct opponent – the opening bowler. South Africa, England and Australia have all fielded strong fast bowling units in the last two years. But only one of these has not faltered in recent times and fast-bowling stocks are lower elsewhere in Test cricket.
Could the pitches be contributing? Experts reporting from these matches often criticise pitches for being too slow. Could groundsmen be compensating by leaving grass on the pitch? Even if true, this might only account for lower opening partnerships in the first innings of a match.
I wonder about T20 cricket. I have already mentioned how it has deprived West Indies and New Zealand of Chris Gayle and Brendon McCullum [see comment below, which corrects my misapprehension that McCullum has missed tests for T20]. Could it also have eroded the virtues of batsmanship associated with effective opening batting in Test cricket?
What do you think could account for the downturn? Will opening pairs bounce back?
Declaration Game cricket blog 