Hi, I am Saša Jurić, a software developer with 10+ years of professional experience in programming of web and desktop applications using Elixir, Erlang, Ruby, JavaScript, C# and C++. I'm also the author of the upcoming Elixir in Action book. In this blog you can read about Erlang and other programming related topics. You can subscribe to the feed, follow me on Twitter or fork me on GitHub.

Erlang based server systems

| 2 comments
One of the biggest advantages of Erlang is that you can use it to implement and run your entire server system. More specifically, you can develop following parts of your system in Erlang:
  • web server
  • shared and/or persistent state
  • jobs and scheduled tasks
  • process monitoring and restarting
  • distributed systems running on multiple machines
In fact, distributed systems aside, you can run everything inside exactly one OS process using only a handful of OS threads. In addition, that one OS process can run arbitrary number of independent server systems (e.g. multiple web servers).

Web server

Erlang can be used to build completely standalone web server, serving both dynamic and static content. Even without an HTTP server such as Nginx or Apache in front of it, an Erlang web server will work efficiently on its own, and be able to to handle large number of concurrent requests. Typical Erlang web server uses one Erlang process per request. Since Erlang processes are lightweight, you can create a large number of them, so there's no fear that you will run out of available web request handlers.

Additionally, Erlang's preemptive scheduler will ensure that long running requests don't block the rest of the system, regardless of whether the processing is I/O or CPU bound.

Finally, the server will usually be vertically scalable, so it will be able to use all available CPU resources, allowing you to handle increased load by adding more hardware power.

Shared and/or persistent state

In a typical server, you often need to manage state which extends beyond the context of a single request, and/or is shared among different requests, or even users, for example caches, user session data, any global server data, etc. Erlang gives you a couple of built-in mechanisms for doing this, thus eliminating the need for external components such as memcached, Redis or external databases.

Typical approach is to create one or more separate Erlang processes, which run for a long time (possibly forever) and manage state. This is often referred to as the Actor model, where an actor is a concurrent entity (in Erlang case a process) which encapsulates state, can receive messages from other actors and modify its state accordingly, or send parts of that state to other actors. Consequently, from your web request handler processes, you can communicate with the state related processes, retrieving the current state or providing some new data which must be incorporated into it.

In addition, Erlang offers a fast in memory "mutable" key-value structure, called ETS, with concurrent read/write access, meaning that it can be shared and simultaneously used from multiple running processes.

On top of this, Erlang comes with a nosql database called Mnesia which offers typical database services such as transactions, flexible queries, indexes, and gives you the full control over which data will be kept only in memory, and which will be persisted to disc. Therefore you can use Mnesia not only to keep in memory state, but also to persist that state in order to recover after OS process or machine restarts.

Jobs and scheduled tasks

Typical servers often perform some amount of background processing, and you can implement these entirely in Erlang, without having to resort to approaches such as cron jobs, daemons, services, Resque, delayed_job, etc. When you need to do a background job, simply start a separate Erlang process and perform your task in that process constantly in an infinite loop, periodically in regular time intervals, irregularly depending on some external event, or once, depending on your needs. Again, the preemptive scheduler will ensure that CPU resources are fairly used, i.e. that background tasks don't block the rest of your system.

Process monitoring and restarting

The standard Erlang framework (OTP) provides a facility called supervisor, the Erlang process which monitors (supervises) other processes (workers), restarting them if they crash. You can use supervisors to ensure that all parts of your system are working. In addition, Erlang has the so called "heart" service: a separate OS process which monitors and, if required, restarts the Erlang VM. This removes the need for solutions such as monit or god.

Distributed systems running on multiple machines

Erlang offers simple, yet powerful primitives of communication between different Erlang VM instances. Instead of combining restful or SOAP based web services, and implement json or xml (de)serialization of your data model, you can send messages and invoke functions in a completely strongly typed manner, and in this way distribute your system over multiple machines. I've touched the topic briefly in this post, so I won't repeat myself here. I will only add that Mnesia has built in support for replication across nodes, giving you an easy way of sharing state between multiple Erlang VM instances.

Multiple servers in one OS process

It is easy to start multiple independent Erlang applications inside a single instance of an Erlang VM, and at the same time separately deploy, stop and restart those applications. In this way, Erlang is sort of an OS inside an OS, with similar characteristics: multiple applications can run (in)dependently in an Erlang system and each application is divided into multiple (in)dependent processes. A crash of one application will not impact another (unless it explicitly depends on it). However, a malicious application can intentionally kill the other ones, consume all hardware resources, and even crash the entire system. Therefore, use this approach only for applications coming from trusted sources (e.g. in-house developed, or official ones coming from Ericsson).

Final thoughts

Implementing entire system with one technology offers many advantages. It provides a uniform development platform, promotes code reuse, while simplifying operational tasks, such as environment setup, deployment, monitoring, testing, scaling, balancing, etc. Erlang gives you that option, and in this aspect it outshines every other development platform I am familiar of. In addition, none of the mentioned approaches is an improvisation or a hack. Instead, they are all mechanisms and services developed by Ericsson to fulfill exactly those needs, and have been used in production in large systems for two decades.

Every presented Erlang based approach should suffice for small to medium uses, and many of them will also excel under heavy load. Still, not all are full fledged substitutes for corresponding mainstream technologies. For example, when serving large number of static files, I would resort to Nginx, Apache or CDN. When dealing with large data quantity, I would prefer a traditional (R)DBMS to Mnesia. In systems, consisting of components implemented in many different technologies, I would use e.g. Redis, or some message queue for data sharing. In addition, Erlang distributed model lacks a serious security model, so it is appropriate only in a trusted environment.

Nevertheless, Erlang based approaches will often suit your needs, and when they don't, you have the option of using something else. With many other development platforms, you must turn to external technologies simply because there's no alternative provided in the platform.

2 comments:

  1. http://www.javalimit.com/2011/05/erlang-is-not-a-concurrent-functional-programming-language.html

    How true is this?

    ReplyDelete
  2. I agree with most of what is said there, although it took me a while to grasp this. I also initially regarded Erlang as a cool platform for ultra concurrency. While this is definitely true, such view treats concurrency as a goal, rather than a tool.

    We don't want concurrency for concurrency's sake. Instead, we want to develop systems that run and provide fast and reliable service regardless of run-time errors, load increases, server outages, or any other problems that happen in production. Such systems are then highly-available.

    Concurrency plays an important role in reaching these goals, meaning it is a tool that is used to make systems more available and reliable.

    I talked about this in more details in first chapter of my upcoming book Erlang in Action (http://manning.com/juric/). This chapter is free, so you don't need to buy the book to read it :-)

    ReplyDelete