Spotify is one of Google Cloud’s cornerstone customers, representing a step change towards Google’s enterprise ambitions. It is also a lucrative one. In its IPO filing in 2018, Spotify revealed it was paying Google €365 million across three years to do so – which equates to just over €10m (£8.78m) per month.
But what is behind those figures? A new podcast from Spotify has shed light on how the company chose Google Cloud for its migration, as well as pain points and misconceptions around the process.
One misunderstood aspect of cloud migrations, argued Gustav Söderström, Spotify chief R&D officer, was the additional cost and complexity. There is no magical switch where everything changes over. The old system has to run in parallel with the new one.
Emil Fredriksson, operations director, noted there was a “huge hill you have to climb”, both mentally and in terms of workload. Fredriksson added that Spotify ordered ‘millions and millions of dollars’ of servers during the migration process because of this overlap. “It doesn’t feel great doing that, knowing these are going to be in use for a short period of time, but this is what you needed to do,” he added.
Spotify addressed certain aspects of the journey with frankness. Fredriksson was one of the earliest hires, having joined Spotify as a senior network engineer four months after the company was founded. He explained that ‘very early on’, he had tried to move the recommendation system, built on Hadoop clusters, to the cloud.
“I remember thinking that ‘this is the future, this is where we have to go, it’s going to make our life so much easier’ – and it failed miserably,” he said. “It was immensely expensive to do the calculations, it was so slow that we couldn’t use it at all, and we just had to roll back and build our own large in-data-centre analytics cluster to do that.”
Fredriksson admitted there was immaturity in the product and the use case was not a direct fit. Yet the title of the podcast – ‘when your winning bet becomes your losing bet’ – encapsulated the decisions made. “We knew… it was a matter of time before this was going to happen, but it wasn’t the time then,” added Fredriksson of cloud migration.
Spotify also admitted that its culture hampered progress. Being engineering-heavy, and confident in its teams, any procurement discussions usually resulted in build rather than buy. “We didn’t really have any decision-making frameworks or guidelines for the buy versus build decision at all,” noted Tyson Singer, VP technology and platform.
This was a tough lesson the company learned. “It’s the sign of a seasoned engineering or product development manager, or a more mature company, to know when to take a step back and ask – how will building this particular tool move the needle on our core strategy?” noted Söderström.
The decision to migrate was therefore made – but not without disagreement, in another frank admission. “DevOps, which were very powerful constituents at Spotify at the time, were actually quite against the cloud altogether,” said Spotify CEO Daniel Ek. “Part of that was fear of losing their job, and part was because they were hired to build a cool tech stack.
“It took quite some persuasion to clear people from saying ‘look, you’re not actually going to lose your job over this,'” Ek added.
The further decision to choose Google Cloud as Spotify’s cloud provider came with benefits for both sides – and makes for interesting analysis in vendor and customer approaches. Spotify noted a ‘really strong fit in engineering culture’ in terms of synergy between the two companies, while it also did not want to be seen as ‘just another customer.’ This meant that, as a big ticket item, Spotify was able to make a mark on Google Cloud’s roadmap.
Circa 2016, Google Cloud was certainly in the infancy of its journey. It is interesting to note that when Evernote announced its move to Google in September of that year, Spotify was already mentioned as a customer. But alongside Coca-Cola, Philips and Snapchat at that time, the cupboard was relatively bare for blue chip names.
Singer noted the risk. “[Google] did not have the maturity of a lot of enterprise level capabilities, but one of the things that working with Google gave us the advantage is they were working higher up in the stack; they were working on managed services,” said Singer. “If you can pick a cloud provider where you can influence the direction of those managed services so it fits your business needs more substantially, then it becomes a much better proposition.”
In some instances, Singer recalled, Google products were better at scale. NoSQL database service BigTable is an example; ‘the variability and latency was way too high’ at low RPS (request per second), argued Singer, but Spotify wanted a ‘standardised NoSQL-type solution that fitted in with the BigTable model.’
It is a two-way street, however. Urs Holzle, SVP technical infrastructure, noted Google solved a big problem around accuracy of billing through its work with Spotify. For those who have seen Corey Quinn of The Duckbill Group breaking down an Amazon Web Services (AWS) bill, then one can understand the significance.
Ultimately, Singer noted an experience of ‘understanding’ the mutual benefits between the two companies. “Because Spotify had been a small company, as a big company with a large amount of purchase power, we could influence the roadmap and feature set of other companies, and that really played out in the context of our cloud provider,” said Singer.
“We came in and scaled up really rapidly, and our spend scaled up really rapidly, and so all of a sudden we had this tremendous leverage and influence over what was then a pretty small scale cloud service,” he added.
You can listen to the full Spotify podcast here.
Want to find out more about topics like this from industry thought leaders? The Cloud Transformation Congress, taking place on 13 July 2021, is a virtual event and conference focusing on how to enable digital transformation with the power of cloud.
