The NRL is one of the more volatile leagues in the world, where from one year to the next a team’s fortunes can flip dramatically.
In 2019 the Broncos finished 8th, only to find themselves last in 2020. Penrith finished 9th in 2019 only to win the minor premiership the next year. In 2019 we saw the wooden spoon holding Eels go from 16th to 5th, the Raiders from 10th to 4th and the Dragons from 7th to 15th. Every year we see dramatic shifts in team performance and ladder position so much so that outside of a couple of outliers, its often a crapshoot trying to determine a team’s fortunes each year.
Which brings me to the important question – can we accurately project team performance from year to year? Or is it just randomness – are team fortunes linked to a bunch of events that are nearly impossible to predict like a key injury, or a boom rookie appearing out of nowhere?
The answer is that projections in the traditional sense are not that useful for the NRL, or at least any projection that I’ve attempted in the last 4+ years of trying to use data to map the NRL. The reason why is likely again due to the erratic nature of the league, where there is so much volatility from year to year that most projections with any relevant confidence intervals will still spit out ranges up to 12 ladder positions wide. Would anyone be interested in knowing that Newcastle is likely to finish somewhere between 2nd and 13th? Probably not.
While there is often limited use for wide projection ranges, there are some valuable evidence-based metrics that can inform us of the likelihood of a team improving or tanking heading into the season. I will spare some of the more nitty-gritty mathematical elements of the indicators being used, but if you are particularly sick in the head and interested, feel free to message me on Twitter and I will happily share.
The general theory being applied to these metrics is simple – a teams win total is not the best way to gauge how good they are. There are a bunch of reasons why this is the case, but the two largest in my opinion are that;
- Win total does not factor in a team’s opponents, and;
- By how much they have won or lost
The NRL season is not a complete home and away fixture, some teams face an opponent twice, and others once. As a result, there’s advantages and disadvantages littered in the schedule, where a team like Penrith for instance has only played the Melbourne Storm once in every regular season for the last 5 consecutive seasons. Quantifying by how much a team may have been advantaged or disadvantaged in the schedule can help accurately assess just how good that team is.
The other component is the binary nature of a win; a 1-point win is the same as a 50-point win. Alternatively, a 1-point loss will still hand you the same L as a 50-point loss. Again, this can create situations where a teams win total is not a completely accurate reflection of their quality – a team could eke out a bunch of close wins while suffering huge losses and end up with the same win total as a team who has suffered an unlucky number of close losses while winning some games by a lot. An easy example might be the Eels from last season, finishing 3rd by win total despite limping into the finals with a for/against differential that would rank them 7th, and subsequently being shelled in straight sets in the finals. Their 7th place point differential ranking was likely more of an accurate indicator of their quality than their win total would suggest.
This is the essence of what we’re attempting to capture with metrics – quantifying these exact types of influences which aren’t captured with traditional statistics.
The three metrics of interest we’re using to assess a team’s quality in this case are;
- Pythagorean Wins; a measure of how many wins expected from a team based on their For/Against differential
- Luck-adjusted Wins; a teams Pythagorean wins adjusted for their schedule difficulty
- Schedule-adjusted-rating (SAR); A teams raw per game point differential adjusted for strength of schedule (SOS).
Skipping over the more boring elements of how these are calculated, here are every teams 2020 metrics, sorted by SAR;
Lots of numbers, but the gist is that ideally these metrics provide a ladder more accurately reflecting a team’s quality. You will also notice that there are differences between the metrics; SAR for instance has Melbourne as the best team, Pythagorean Wins and Luck Adjusted Wins have Penrith as the leader.
The differences between these are not as interesting, we’re more focused on how we can use these to project a team’s performance in the following year. To do this we look at how these metrics compare to a team’s actual win total; did teams overperform expectation and bag more wins than the metrics think they should’ve? Or did they underperform the metrics expectation and win less games?
For the 2020 season, this looks like the following, where a negative value indicates that a team has won more games than they should’ve or ‘overperformed’ , and a positive value indicates they underperformed their win total;
How we apply these values is based on simple regression. If a team overperformed (negative value) their win total, in theory they are more likely to regress in the next season. A team who has underperformed (positive value) their expectation is more likely to improve in the next season. An easier way might be to call it luck, which teams were lucky and won more games than they should’ve, and which teams were unlucky.
This is an overly complicated and boring way to say if a team has a green value, that metric thinks they’ll be better next season, and red if they think they’ll be worse.
I’m sure that’s mildly interesting to some people, and less interesting to others – but the real revelation is just how accurate these indicators are.
Here are the last 3 seasons, comparing a teams indicated trajectory (LESS or MORE wins) based on the prior years metrics, compared to whether the team won more or less games than in the year it is projecting; with a green bar indicating the projection was accurate, and red indicating it was incorrect.
In total across the whole data-set the indicators predicted a teams trajectory 74.4% of the time. The Pythagorean indicator was 81.25% accurate, the Luck-Adjusted indicator was 72.9% accurate and the SAR indicator was 68.75% accurate. When all 3 indicators aligned and indicated the same trajectory, that team followed the indicators 78% of the time.
That’s…. pretty amazing.
Without looking for a second at a teams player roster, whether a key player is injured or whether a team has a new coach – over the last 3 seasons these indicators have projected whether a team is going to win more or less games at nearly 75% accuracy… just by looking at nothing else but their past years performance.
Suddenly the crapshoot that is year to year performance in the NRL isn’t looking so crappy anymore, and we have something that can capture which direction a team is likely headed with some level of confidence.
So how does this look for 2021?
The indicators have a consensus of 7 teams likely to improve their win-total from last season, being;
The Tigers, Rabbitohs, Roosters, Bulldogs, Warriors, Dragons and Cowboys
and 4 teams being handed all red indicators;
the Raiders, Panthers, Titans and Eels
For everyone else the indicators are conflicted, more than any other year since tracking started in 2017 with 5 teams having some indicators suggesting improvement and another suggesting the opposite. Does this mean this season will be extra unpredictable? Probably not, it could be more an indication that fewer teams wildly under or overperformed their expectation last season, potentially due to fewer games or other issues like rule changes.
For some teams, these indicators might line-up with observational expectation, and for others not so much. The Titans for example have undergone a significant transformation on the playing roster since last season and are penciled in for a big improvement in the eyes of most. Even for a team like the Titans though these indicators can still be valuable – people who are using last years Titans win percentage to project their performance this year might be using faulty data – the Titans win percentage last season was an overperformance for their overall quality based on those metrics, and they may need an even bigger improvement to meet their expectation this year.
At the top end of the ladder these indicators might be capturing some bigger moves. Top 4 hopefuls like the Eels, Panthers and Raiders being projected to win less games is troublesome, and only exacerbated by the indicators bullishness on teams like the Roosters and Rabbitohs who were already quality outfits being projected to improve. The indicators also failed to give a unanimous determination on teams like the Sharks and Knights, who could potentially drop out of the 8, especially with teams like the Tigers and Cowboys who have finals aspirations being given green lights.
These indicators might be a bit rugged and less than perfect, but given the lottery that is the NRL from season to season having some evidence-based measure of projecting a team’s future performance is a blessing, especially one shooting at a 75% clip.
While these metrics are not a death sentence or golden ticket for each team, they have shown an ability to confidently capture the likelihood of a team improving, and with little else in the analytical sphere to guide decision-making, they’re significant.
Also, if you are looking for something neat and perfect, you are probably following the wrong sport.