Airbnb’s Problem With Good Reviews

In the era of online marketplaces and the sharing economy, it is unsurprising that users’ reviews are important. Depending on which study one favors, between 83 and 88 percent of consumers use online reviews as a tool to guide their purchasing decisions. Uber drivers who average a rating of below 4.6 stars are kicked out of the system and not allowed to drive for the company anymore. Consumers looking at hotels with a three “star” rating (from consumers) are three times less likely to book those rooms than those with five stars. In the restaurant business, bad online reviews are known as the silent killer – while it is hard to quantify how many reservations it is costing exactly, restaurants with low consumer scores don’t stay open for all that long.

While we could go on forever, in short, in an economy going increasingly digital and mobile, the danger that travels with a bad online review can hardly be overstated.

But what about the dangers of good reviews? Granted it’s not a problem that most businesses worry about too much – it is hard to imagine a hotelier logging on and saying “Drat – another day of customers reporting their exceptional level of satisfaction” – but when reviews are trending too high, consumers also may start to get suspicious.

“Airbnb wants a system that works and [in] order for it to work, it has to have credible reviews,” Davide Proserpio told MPD CEO Karen Webster in a recent conversation.

Proserpio is a Ph.D Candidate in Computer Science at Boston University and the co-author of a new paper with a Boston University professor that takes a closer look at the role reviews play in consumer choice in the sharing economy.

User reviews are an important topic of conversation for all of the reasons noted, plus the fact that not everyone leaves a review. About 20 percent of people do, and generally when they’ve had either a really good – or really bad – experience.

It seemed Airbnb had a peculiar issue – its reviews on average were notably more positive – by about 14 percent -than those posted on sites like TripAdvisor – even when the reviews were actually about the same property. Webster noted that this seemed to imply that either people on vacation were in a state of bliss that made them want to give the world and all its people a good review, or the hosts on Airbnb – possibly knowing a review is coming and the importance of that review – are working that much harder to please their guests.

Proserpio agreed that there may be some truth in that – but that there was also another factor in play with sharing economy services like Airbnb where users and providers regularly interact – social pressure.

“The fact is that you are exchanging a service where there is a personal relationship between the guest and the host,” Proserpio noted. “So even if the stay is not amazing, but the host is a nice person, consumers don’t want to give a bad review, they give a good review in any case. There is a bias that I would call social bias,” he explained.

This, he noted, is a difficulty with bilateral review systems where both users and providers are reviewed and why many notable companies have actually dropped that sort of system. Users rate a service or product higher if they get a higher rating in return – and vice versa.

“EBay actually switched from a bilateral system to a unilateral system.  Now only the buyer rates the seller and not vice versa, as it was until 2006.”

Other players, like Uber, have stuck with the bilateral system, but one that is not transparent to all parties – users can see driver reviews, drivers can see user reviews, but users cannot see their own reviews. While that does perhaps have a salutary effect on social bias, Proserpio noted, it does leave the door open to another, fairly obvious, problem.

“For example, a few weeks ago I was taking an Uber and the driver told me I had a low rating, and I didn’t know about it … which is interesting, because I cannot fix a problem if I don’t know there is a problem.”

Because, while someone’s problems as a passenger are obvious ….

“You know sometimes you know. I mean if it is late at night and you are drunk, it’s probably pretty clear why you got the rating,” he offered as an example.

But other times, the problem is not obvious. Webster pointed out that some Uber drivers mark down passengers who simply aren’t “chatty” enough.

So, Airbnb came up with a different solution for the problem – a two week, double blind review period.

“Both hosts and guests may worry that if they leave an honest review that includes praise and criticism, they might receive an unfairly critical review in response, Proserpio noted. “To address this concern, the reviews period is shortened to 14 days, after which a review cannot be left anymore. At the end of the 14-day review period,  reviews will be revealed to hosts and guests simultaneously.”

With some of that social pressure removed, the lovefest on Airbnb calmed down some – the averages went down – but on the whole the Airbnb’s reviews were still higher. And that, noted Proserpio, might just come down to a difference in user base between TripAdvisor and Airbnb.

“Airbnb may attract a different kind of user, and they may be more positive than the typical user of TripAdvisor.”

With 80-90 percent of consumers using reviews to make purchasing decisions, they are simply not something that can be reasonably ignored. And while gaming or faking reviews is increasingly popular – Yelp is dealing with a controversy around this, one likely to dent consumer confidence some – it is unlikely to be broken. Especially with work like Proserpio’s, that aims to help to put those customer reviews in an even more useful context.