In recent days, stories on various local news websites have flagged places like Plymouth, Hull, Stoke-on-Trent and Stockport as having the “highest number of empty shops”, according to a new survey.
Very sad https://t.co/iWy7tbi2jk
— Hull Live (@hulllive) December 15, 2021
So, let’s have a look at this in more detail, and do the one thing that often gets overlooked – scrutinising and interrogating the data on which these claims are based.
Who’s the source, and how credible are they?
The first thing to ask is who’s behind the data, and, hence, get a sense of how reliable that data is. Articles cite the research as being conducted by a “tech firm”, Property Inspect.
The company’s commercial director is cited in some articles. He’s mentioned in various places on the website at propertyinspect.com/uk, which helps confirm we’ve got the right source.
When the media cites any survey, I like to find the original source, to better understand the data and methodology. You’d expect there to be a press release on the Property Inspect website, but nothing shows up.
There’s nothing on Property Inspect’s own social media on Twitter, Facebook or LinkedIn either.
The company claims to “produce professional property reports, periodic inspections, building audits, risk assessments & more using our award winning inspection software”, though it’s not clear whether its own software has been used at all in this survey.
Based on how the software is described, it doesn’t seem to be a tool for analysing empty shops data.
Rather, we know from the articles that Property Inspect has “used retail property listings from Rightmove, Zoopla and Savills to determine the average number of central properties up for rent”.
It sounds like these sources have been scraped, though there’s nothing to suggest that Rightmove, Zoopla and Savills participated in the research directly.
So, it’s probably reasonable to assume that Property Inspect has compiled this data to help it gain coverage and brand awareness in the commercial property space, which is fair enough.
Its Twitter fanbase (79 followers since 2014) is currently pretty modest, so you can see the potential thought-leadership value of generating a fair amount of press coverage.
What methods were used to collect the data?
So what about the data then? We know that Property Inspect has used data on shops for rent – from Rightmove, Zoopla and Savills – as a proxy for measuring the number of empty shops in different locations.
We don’t know how the boundaries of these places have been defined, and the survey obviously relies on the data scraped being accurate and up to date.
In reality, I know from my own work that premises can often still appear as to let online even once a new tenant has been agreed.
Equally, there are many empty shops to let that won’t appear on any of the three featured platforms. So, the data gives a partial snapshot of what’s happening, which – as ever – a survey like this tends to underplay.
In fact, recording empty shops is notoriously hard. Even a methodology that visits every place, rather than scraping online data, can make mistakes. For example, a shop can appear empty but be in the process of being let, or temporarily vacant while upper floors are repurposed.
From first-hand experience, I know that the only proper way to keep track of vacancy is to have feet on the ground, talking to councils, landlords and agents.
Is the analysis of the data fair and robust?
Back to this particular research, though, the next problematic bit is how the survey then uses the data it has scraped to plot the number of empty high street shops in each location against the number of residents in that place.
The articles variously refer to the number of empty shops per 10,000 or, in some cases, 100,000 residents. This discrepancy seems like an error, but with no original source to refer to, we can’t verify which is right.
Either way, the measure of the number of vacant units per 10,000/100,000 residents is used to rank all the locations. That’s where #Hull is ranked the “worst” (this Bedfordshire Live article features the full league table).
But I know from visiting Hull that one of its biggest challenges is its very tightly drawn local authority boundaries. In this survey, we assume that the number of residents is based on local authority area, but without a methodology we can’t be sure.
In Hull’s case, though, the boundaries mean that the city serves a much bigger catchment than actually live within the Hull City Council area. But this appears to challenge the whole premise of the survey.
People don’t live their lives within the confines of the particular local authority in which they reside. So, given all the people it serves in neighbouring East Riding too, we might be singling out Hull unfairly.
Is there a fundamental methodological flaw?
In fact, looking at the top ten “worst” places, pretty much all are large towns and cities that form part of a larger conurbation, so serve a community wider than their own local authority.
Perhaps there’s a bigger methodological issue at play here? Without the methodology being transparent, however, we can only make a best guess. So, what conclusion can we draw?
We appear to have a tech firm scraping partial data about units to let, and marrying this up with questionable assertions about catchment, to produce the headline “Hull has more empty shops than anywhere else in UK”.
Like any company, Property Inspect is entitled to conduct and publish whatever surveys it likes. But we are also entitled to question that data, especially when it risks perpetuating – seemingly unfairly – negative and unhelpful place perceptions.
You only have to read people’s council-bashing comments in response to the coverage to see the unhelpful – and often poorly informed – reactions that stories like this stir up.
It’s frustrating for place leaders who are working hard, often behind the scenes in dealing with issues such as absentee landlords, to improve their town and city centres.
But it’s especially frustrating for the (often independent) shops and other businesses that are straining every sinew to give their places something good, only for those towns to keep getting bashed at every opportunity, in the name of a headline.
Does the press add any of its own critique?
The one positive of coverage like this is that it gives place leaders an opportunity to challenge the narrative – as long as the local press invites them to. This Hull article unfortunately just cites the survey data without question.
As does this Plymouth article, where the slightly odd quote from the survey authors talks about wearing in masks in shops but doesn’t say anything meaningful about vacant units.
The Manchester Evening News coverage is much better, including a lengthy quote from Stockport Council.
Rightly, the spokesperson notes that “correlating vacancy rates against resident population gives a skewed picture of vacant units”, and highlights recent openings and future plans.
Conclusion
So, the long and short is to always understand who’s behind the data, what their intentions are, and what methodology has been employed. Empty shops are clearly a challenge in many places – which those places are fully aware of, and in many cases tackling.
However, league tables of the “worst” places only undermine those efforts – especially when they neglect local on-the-ground knowledge, and employ methods that don’t bear up to scrutiny.
A version of this blog post was originally published as a Twitter thread, which you can follow here.
