SBC News Voice of the Sports Betting Industry
With the Grand National behind us, and the Champions League final fast approaching, we wanted to know how bookmakers keep up with demand. To find out, we sat down with the experts at Erlang Solutions – CEO Stuart Whitfield and Francesco Cesarini, Founder and Technical Director.
____________________________________
SBC: bet365 are widely respected as the leader in online betting – what role does Erlang play in in the firm’s day to day running?
Stuart Whitfield: Prior to our work with them, bet365’s in-play betting platform was developed in Java, and initially it met their needs well. However, exponential growth saw them host up to two million users at any time, generating thousands of updates per second.
This put scalability, latency, and reliability at risk. With increasing scalability requirements, traditional relational databases would also become problematic.
Francesco Cesarini: Erlang perfectly met bet365’s requirements. Erlang is a robust, compact language which gives you actor style concurrency, immutability and error handling semantics. Together with the OTP middleware you get a programming model that greatly relieves these concerns.
We have supported bet365 to migrate infrastructure from Java to Erlang, which they found a much better fit than the original Java-based solution. There is much more insight about our partnership and results we achieved in our webinar with bet365.
SBC: What do you find the biggest challenge is for online bookmakers compared to other industries?
FC: There’s a little something we like to call the “innovation-scale dilemma”. Bookmakers must not only meet the demands of high frequency requests and traffic bursts, but also keep the tech-savvy, multi-channel customers engaged.
To meet user demand, the industry continuously creates and improves on new types of games and bets. This is when tech departments hit upon the dilemma. Over a certain number of users, the ability to innovate quickly is lost. Precious time is spent rewriting software so that it runs to scale, slowing release time.
SW: To break this innovation-scale dilemma, the most innovative industry players – such as bet365 and William Hill – have chosen Erlang (or Elixir, a language based on the same underpinnings as Erlang) as their core technology. We also support global payment companies, like Klarna, messaging companies like ooVoo, and IoT companies such as Intel – all of whom face similar technical challenges to online bookmakers.
SBC: Why is the Erlang language and ecosystem so well suited to betting operators?
SW: Erlang was built with telecoms in mind, but its strengths map perfectly to betting operators’ needs. With support for millions of lightweight processes, Erlang allows you to build massively concurrent and fault tolerant applications very quickly.
Erlang’s extreme fault-tolerance isolates errors and system crashes, allowing the system to quickly recover without the user ever noticing that it had failed!
FC: Its functional paradigm makes program outputs easier to predict, debug and analyse. This reduces the complexity of the code base significantly, increasing product delivery speed and reducing time to market. It also reduces maintenance and operational costs, as it requires less people and infrastructure to run on.
Erlang excels in handling request explosions and multiplexing, cascading requests from a single event to span hundreds of servers while maintaining request delivery order. I’ve seen Erlang systems connect ten times more users than competing Java solutions and support data changing at four times the previous rate.
SBC: Can you elaborate on the work you did with William Hill?
SW: Our most recent work with William Hill’s revolved around Erlang-based Riak KV, a highly performant key value data store solution for bookmakers.
Most legacy systems are based on database-centric architecture: relational databases with fixed capacities. These databases don’t meet the requirements of massive scalability, speed, and fault tolerance needed to run a modern betting infrastructure. Left as they are, they force operators to rely on additional hardware to vertically scale. This is not a sustainable nor a cost-effective approach.
FC: With Riak KV, William Hill can scale elastically, as and when capacity is required. Riak KV is a distributed NoSQL database with a key/value design and advanced local and multi-cluster replication that guarantees reads and writes, even in the event of hardware failures or network partitions.
We’ve been able to scale out, up and down predictably, deal with unavoidable server network or data center failures, and achieve lower latency than the systems we have replaced.
SBC: What can operators do to improve their systems in the short term?
FC: If you are already using Ruby, start programming in Elixir. Elixir is an emerging programming language from the Erlang ecosystem providing the same guarantees of performance, scalability and resilience. It shares its syntax with Ruby, making it more palatable to developers who are already familiar with Ruby.
Pinterest and Moz are among Elixir adopters, and we’ve supported Bleacher Report, the world’s second largest sports website, in deploying Elixir to handle their surges in traffic.
SW: I would also recommend evolving your system into a distributed architecture, which can be easily done by plugging into Erlang-based products which are quick to deploy and scale elastically.
SBC: What’s the future for Erlang and Elixir? What’s upcoming?
SW: In today’s world, we’re all always on. Apps are increasingly concurrent. For the most part, the traditional way of achieving concurrency using threads, shared memory and locks will not scale. Erlang and Elixir offer a solution.
The open source BEAM virtual machine, on which Erlang and Elixir operate, is a beacon of reliability, scalability, and concurrency. It’s highly optimised for real time, massively concurrent and transactional systems. As long as there is a demand for these systems, BEAM, and Erlang and Elixir, will continue to have an upward trajectory, especially in telecoms, messaging, online betting, and IoT.
FC: While consumers are always on, so are their service providers. Always evolving, deploying something new. High level programming languages like Erlang and Elixir allow development teams to refactor the code and move it around painlessly.
It’s because of this that Erlang has seen adoption by businesses with large volumes of concurrent, active users who transact a massive amount of data in real-time, such as Klarna, Vocalink, and EE. I’m constantly in conversation with companies who have depended on Ruby, but want to exploit BEAM’s power to reduce their operational, server and infrastructure costs.
____________________________________
Profiles:
Francesco Cesarini
Francesco Cesarini is the founder and technical director of Erlang Solutions. He has used Erlang on a daily basis since 1995, starting as an intern at Ericsson’s computer science laboratory, the birthplace of Erlang. He moved on to Ericsson’s Erlang training and consulting arm working on the first release of OTP, applying it to turnkey solutions and flagship telecom applications. In 1999, he founded Erlang Solutions.
Stuart Whitfield
Stuart Whitfield is CEO at Erlang Solutions and has been its Commercial Director since 2010. With his background in the commercial law, Stuart supports Erlang Solutions’ clients in getting the best solution for them, he is also well versed in the challenges bookmakers face.
Erlang Solutions
Erlang Solutions specialises in giving business truly scalable solutions through the creation, integration, delivery, and lifetime support of products and services based on the Erlang programming language. Four of the world’s top bookmakers currently look to Erlang Solutions for technical know-how. Click here to learn more about Erlang Solutions’ work in the online gambling & betting industry.
