Game of Thrones Season 8 is well underway, with ferocious dragons, an abundance of questionable sexual-tension, and more exciting plot-twists than ever.
On April 14th, viewers across the globe tuned-in to see the latest action of Westeros, with HBO estimating that 17.4 million people watched this season’s premiere on their streaming and linear platforms.
But how does a streaming service ,like HBO go, cope with almost as many viewers as the population of the celtic nations combined, flooding their platforms at the same time?
An increasing amount of Streaming Platform Developers are opting to embrace containerisation-platforms.
At the 2017 KubeCon, Illya Chekrykin, Senior Staff Engineer at HBO, explained how their team went from not having a single service running inside a container, to hosting the entire GoT season 7 on Kubernetes.
Cherkrykin noted that they previously relied on API services running in Node.js, to run their streaming platform.
But was this the most effective solution?
Wasteful
The team ran into a multitude of problems with the current setup. They found Node.js + EC2 to be extremely wasteful, with 50+% of CPU unutilised. They also found that the current solution was slow to react to spikes of traffic, leading to over-provisioning of their deployments to accomodate for unprecedented traffic patterns.
Too Many Elastic Load Balancers!
Every service required an ELB (Elastic Load Balancer). Even internal communications! This meant that the team were quickl
y running out of resources and facing reaching quota limits.
Ironic considering as they were under-utilising more than 50% of CPUs, yet running out of all other resources.
Whilst running API services in Node.js is a fine solution for general, low-demand cases, HBO required a far more robust solution.
Enter: Kubernetes
HBO were able to leverage the power of Kubernetes to improve utilisation and introspection capabilities. It was also seen as the fastest and safest option for them.
With the addition of Terraform, HBO were able to manage and deploy their Kubernetes clusters through custom Terraform templates.
“For us, we found that many problems with our services were not caused by Kubernetes. They were there all along and Kubernetes just made them more visible,”
Illya Chekrykin – 2017 KubeCon
https://m.youtube.com/watch?v=7skInj_vqN0
Netflix
Netflix is widely regarded as the king of streaming services. With over 118 Million Subscribers, the streaming service is miles ahead of competitors such as Amazon Prime and Hulu.
The streaming giant began their container journey in the early days of Kubernetes, when orchestration platforms were scarce. Because of this, they had to decide whether to adopt an existing platform or develop their own.
Netflix decided to build a dedicated container orchestration platform called Titus. The Netflix Technology blog claims that most organizations look to write greenfield applications on new container platforms such as Kubernetes.
However, its team wanted to consider existing applications as well. Consequently, Netflix chose to build their Titus container management system on top of Mesosphere.
Titus powers critical aspects of the Netflix business, from video streaming, recommendations and machine learning, big data, content encoding, studio technology, internal engineering tools, and other Netflix workloads. Titus offers a convenient model for managing compute resources, allows developers to maintain just their application artifacts, and provides a consistent developer experience from a developer’s laptop to production by leveraging Netflix container-focused engineering tools.”
– Netflix Technology Blog
The project has proved so successful for Netflix, that they are sharing its power with the container community by making Titus completely open-sourced.
You can learn more about how Titus works by reading it’s documentation here.
BBC iPlayer
BBC iPlayer allows viewers to watch BBC TV “whenever, wherever, and however they want”. They claim that their audience watch over 3,600 hours of on demand BBC TV content on over 10,000 different devices. On top of that, they have over 170 hours of live streams everyday.
Source: BBC Design Engineering Medium
So how does BBC iPlayer handle over 140 million API requests everyday to encode and distribute content while balancing quality, performance and cost?
Surprisingly, the BBC are yet to embrace container technology to power the BBC iPlayer.
Like HBO did previously, BBC iPlayer uses an API to serve a consistent product to all of its clients.
The iPlayer Business Layer (iBL) is built using a microservices architecture of over 70 different components, with the majority built using Node.js.
As the demand for online streaming increases rapidly, surely it’s only a matter of time that the engineers begin to see under-utilization of their deployed resources?
“We are continuously iterating to improve the user experience… This is a big challenge for us from ‘how do we measure the impact of our changes?’ to ‘how do we scale our services technically to remain performant whilst also being cost efficient?’.”
Kit Wong – BBC.
It sounds like change is coming, and with the BBC successfully using container-based deployments and Google Cloud as part of their BBC+ App stack, maybe it’s only a matter of time before the rest of the business follows suit.
If you want to learn more about Kubernetes and container orchestration, read our blog post. And if your business is interested in embracing containers, why not give Mobilise a look?
