Hi, I am Saša Jurić, a software developer with 10+ years of professional experience in programming of web and desktop applications using Elixir, Erlang, Ruby, JavaScript, C# and C++. I'm also the author of the upcoming Elixir in Action book. In this blog you can read about Erlang and other programming related topics. You can subscribe to the feed, follow me on Twitter or fork me on GitHub.

Why Elixir?

| 19 comments
It's been about a year since I've started using Elixir. Originally, I intended to use the language only for blogging purposes, thinking it could help me better illustrate benefits of Erlang Virtual Machine (EVM). However, I was immediately fascinated with what the language brings to the table, and very quickly introduced it to the Erlang based production system I have been developing at the time. Today, I consider Elixir as a better alternative for the development of EVM powered systems, and in this posts I'll try to highlight some of its benefits, and also dispell some misconceptions about it.

The problems of Erlang the language

EVM has many benefits that makes it easier to build highly-available, scalable, fault-tolerant, distributed systems. There are various testimonials on the Internet, and I've blogged a bit about some advantages of Erlang here and here. In addition, benefits of both Erlang and Elixir are presented in chapter 1 of my upcoming book Elixir in Action.

Long story short, Erlang provides excellent abstractions for managing highly-scalable, fault-tolerant systems, which is particularly useful in concurrent systems, where many independent or loosely-dependent tasks are constantly running.

I've been using Erlang in production for more than three years, to build a long-polling based HTTP push server that in peak time serves over 2000 reqs/sec (non-cached). Never before have I written anything of this scale nor have I ever developed something this stable. The service just runs happily, without me thinking about it. This was actually my first Erlang code, initially burdened with anti-patterns, and bad approaches. And still, EVM proved to be very resilient, and the server ran efficiently from the very beginning. Most importantly, it was fairly straightforward for me to work on such complex problem, in large part owing to Erlang concurrency mechanism.

However, despite some great properties, I never was (and I'm still not) quite comfortable programming in Erlang. The coding experience somehow never felt very fluent, and the resulting code was always burdened with excessive boilerplate and duplication. The problem was not the language syntax. I did a little Prolog back in my student days, and I liked the language a lot. By extension, I also like Erlang syntax, and actually think it is in many ways nicer and more elegant than Elixir. And this is coming from an OO developer who spent most of his coding life in languages such as Ruby, JavaScript, C# and C++.

The problem I have with Erlang is that the language is somehow too simple, making it very hard to eliminate some frequent cases of boilerplate and structural duplication. Consequently, the resulting code gets a bit messy, being harder to write, analyze, and modify. After coding in Erlang for some time, I thought that functional programming is inferior to OO, when it comes to efficient code organization.

What Elixir is (not)

This is where Elixir changed my opinion. After I've spent enough time with the language, I was finally able to see benefits and elegance of functional programming more clearly. Now I can't say anymore that I prefer OO to FP. I find the coding experience in Elixir much more pleasant, and I'm able to concentrate on the problem I'm solving, instead of dealing with the language's shortcomings.

Before discussing some benefits of Elixir, there is an important thing I'd like to stress: Elixir is not Ruby for Erlang. It is also not CoffeeScript, Clojure, C++ or something else for Erlang. Relationship between Elixir and Erlang is unique, with Elixir being often semantically very close to Erlang, but extending the original concept further by borrowing approaches from various other languages. The end result may on surface look like Ruby, but I find it much more closer to Erlang, with both languages completely sharing the type system, and taking the same functional route.

So what is Elixir? To me, it is an Erlang-esque language with improved code organization capabilities. This definition differs from what you'll see on the official page, but I think it captures the essence of Elixir, when compared to Erlang.

Ability to organize the code and to flush out duplication and boilerplate is in my opinion an important role of every programming language. If developers have to constantly duplicate some code structure (such as this one), without being able to eliminate this duplication reasonably easy, then the language doesn't succeed when it comes to code organization. It is in this department where Elixir improves on Erlang, by providing additional tools to organize the code, making developers more efficient in writing production-ready, maintainable code.

Ingredients

Much has been said about Elixir on the Internet, but I especially like two articles from Devin Torres which you can find here and here. Devin is an experienced Erlang developer, who among other things wrote a popular poolboy library, so it's worth reading what he thinks about Elixir.

Here, I'll try to focus on some of the most important tools that help us organize the code efficiently.

Metaprogramming

Metaprogramming in Elixir comes in a couple of flavors, but the essence is the same. It allows us to devise elegant and concise constructs that seems as if they're integral part of the language. These constructs are in compile-time then transformed into a proper code.

On a more specific level, metaprogramming helps us remove structural duplication - a situation where two pieces of code share the same abstract pattern, but differ in various details. For example, a following snippet presents a sketch of a module that models a User record:

defmodule User do
  #initializer
  def new(data) do ... end

  # getters
  def name(user) do ... end
  def age(user) do ... end

  # setters
  def name(value, user) do ... end
  def age(value, user) do ... end
end

Some other type of record will follow this pattern, but with different fields. Instead of constantly repeating this pattern, we can use Elixir defrecord macro:

defrecord User, name: nil, age: 0

Based on the given definition, defrecord generates a dedicated module with utility functions for manipulating our User record. Thus, the common pattern is stated in a single place (the code of defrecord macro), while the particular code is relieved of implementation mechanics, remains concise and easy to understand.

Elixir macros are nothing like C/C++ macros. Instead of working on strings, they are something like compile-time Elixir functions that are called in the middle of parsing, and work on the abstract syntax tree (AST), which is a code represented as Elixir data structure. Macro can work on AST, and spit out some alternative AST that represents the generated code. Since macros are executed in compile-time, the performance of a running system will not be affected.

Owing to macros, most of Elixir, is actually implemented in Elixir, including constructs such as if, unless, or unit testing support. Unicode support works by reading UnicodeData.txt file, and generating the corresponding implementation of Unicode aware string function such as downcase or upcase. All of this makes it easier for developers to contribute to Elixir, since for most feature they don't need to be familiar with another language.

Macros also allow 3rd party library authors to provide internal DSLs that naturally fit in language. Ecto project, that provides embedded integrated queries, something like LINQ for Elixir, is my personal favorite that really showcases the power of macros.

I've seen people sometimes dismissing Elixir, stating they don't need metaprogramming capabilities. While extremely useful, metaprogramming can also become very dangerous tool, and it is advised to carefully consider its usage. That said, there are many features that are powered by metaprogramming, and even if you don't write macros yourself, you can still enjoy many of these features, such as aforementioned records, Unicode support, or integrated query language.

Pipeline operator

This seemingly simple operator is so useful, I actually "invented" its Erlang equivalent even before I was aware it exists in Elixir (or other languages for that matter).

Let me first describe the problem it solves. In Erlang, there is no pipeline operator, and furthermore we can't reassign variables. Therefore, typical Erlang code will often follow this pattern:

State1 = trans_1(State),
State2 = trans_2(State1),
State3 = trans_3(State2),
...

This is a very clumsy code that relies on intermediate variables, and correct passing of the last result to the next call. I actually had a nasty bug in production because in one place, I accidentally used State6 instead of State7.

Of course, we can go around this by inlining function calls:

trans_3(
  trans_2(
    trans_1(State)
  )
)

Notice how the code now must be read "inside-out", the most inner function being called first. This style, known as staircasing, can soon get ugly, and the problem is often aggravated when functions receive additional arguments, and the number of functions in the chain increases.

The pipeline operator makes it possible to combine various operations without using intermediate variables:

state
|> trans_1
|> trans_2
|> trans_3

The code can easily be followed by reading it from top to bottom. We pass the state through various transformations to get the desired result, each transformation returning some modified version of the state. This highlights one of the strengths of FP, where we regard functions as data transformations that are combined in various ways to achieve the desired result.

For example, the following code computes the sum of squares of all positive numbers of a list:

list 
|> Enum.filter(&(&1 > 0))       # take positive numbers
|> Enum.map(&(&1 * &1))         # square each one
|> Enum.reduce(0, &(&1 + &2))   # calculate sum

The pipeline operator works extremely well because the API in Elixir libraries follows the "subject (noun) as the first argument" convention. Unlike Erlang, Elixir takes the stance that all functions should accept the thing they operate on as the first argument. So String module functions accept string, while Enum module functions accept enumerable as the first argument.

Polymorphism via protocols

Protocols are the Elixir way of providing something roughly similar to OO interfaces. Initially, I wasn't much impressed with them, but as the time progressed, I began to understand the benefits they bring.

Protocols allow developers to create a generic logic that can be used with any type of data, assuming that some contract is implemented for the given data. An excellent example is the Enum module, that provides many useful functions for manipulating anything enumerable. For example, this is how we iterate an enumerable:

Enum.each(enumerable, fn -> ... end)

Enum.each works with different types such as lists, or key-value dictionaries, and of course we can add support for our own types by implementing corresponding protocol. This is resemblant of OO interfaces, with an additional twist that it's possible to implement a protocol for a type, even if you don't own the type's source code.

One of the best example of protocol usefulness is the Stream module, which implements a lazy, composable, enumerable abstraction. A stream makes it possible to compose various enumerable transformations, and then generate the result only when needed, by feeding the stream to some function from the Enum module. For example, here's the code that computes the sum of squares of all positive numbers of a list in a single pass:

1
2
3
4
list 
|> Stream.filter(&(&1 > 0))
|> Stream.map(&(&1 * &1))
|> Enum.reduce(0, &(&1 + &2))   # Entire iteration happens here in a single pass

In lines 2 and 3, operations are composed, but not yet executed. The result is a specification descriptor that implements an Enumerable protocol. Once we feed this descriptor to some Enum function (line 3), it starts producing values.

It's worth mentioning that laziness is in no special way supported by Elixir compiler. The general protocol support, and first-class functions are all it takes to implement lazy enumerables.

The mix tool

The final important piece of puzzle is the tool called mix that helps us create and build projects, and manage their dependencies. The tool does its job in a simple and easy way. With a single command line you can create an OTP application skeleton, that consists of only 7 files (this includes .gitignore and README.md). There's not much to talk about mix - it does its job well, and makes it easy to quickly dive into proper development of EVM powered systems.

Other goodies

The list doesn't stop here, and there are many other enhancements Elixir provides, such as support for variable rebinding, optional parentheses, implicit statement endings, nullability, short circuits operators, ...

In general, I like many of the decisions made in this department. It's nice to be able to write if without obligatory else. It's also nice that I don't have to consciously think which character to use for statement ending. Still, I don't find these enhancements to be of crucial importance. They are nice finishing touches, but if this was all Elixir had to offer, I'd probably still use pure Erlang.

Wrapping up

Much has been said in this article, and yet I feel that the magic of Elixir is far from being captured. The language preference is admittedly something subjective, but I feel that Elixir really improves on Erlang foundations. With more than three years of production-level coding in Erlang, and about a year of using Elixir, I simply find coding experience in Elixir to be much more pleasant. The resulting code seems more compact, and I can be more focused on the problem I'm solving, instead of wrestling with excessive noise and boilerplate.

And yet, we get to keep all the benefits of EVM. The underlying concurrency mechanisms makes it significantly easier to tackle complexity of a highly-loaded concurrent system that must constantly provide service and perform many simultaneous tasks.

Personally, I think that both Elixir and EVM raise the abstraction bar, and help me tackle complex problems with greater ease. This is why I would always put my money behind Elixir/EVM combination as the tools of choice for building any server-side system that is at least moderately complex. YMMV of course.

19 comments:

  1. The pipe operator is just a foldl. lists:foldl( fun(Func, Acc) -> Func(Acc) end, Initial, Funcs ).

    Elixir's almost-hygenic macros don't hold a candle to parse transformations.

    Polymorphism via protocols is just pattern matching in the receive.

    Your four line sum of the squares of a list is lists:sum([ J*J || J <- List ]).

    Mix is not as convenient or as powerful as reltool or rebar.

    Elixir records are not as complete as Erlang records, let alone -types.

    EVM isn't a thing. BEAM is not the only Erlang virtual machine.

    "without being able to eliminate this duplication reasonably easy, then the language doesn't succeed when it comes to code organization. It is in this department where Elixir improves on Erlang"

    Usually when someone criticizes a language, it's because they haven't learned the language well enough.

    ReplyDelete
  2. Thank you for your thoughts. Some responses are below:


    1. Pipe operator is not implemented in terms of fold. Please do your research before making such bold claim. Of course, pipe can be simulated with fold. That is, if you prefer:

    lists:foldl(
    fun(Fun, Acc) -> Fun(Acc) end,
    Initial,
    [
    fun(X) -> foo(..., x) end,
    fun(X) -> bar(..., x) end,
    fun(X) -> baz(..., x) end
    ]
    )

    to Elixir's

    initial
    |> foo(...)
    |> bar(...)
    |> baz(...)

    I've tried the fold approach in my early Erlang days, and really disliked it. I can respect if you have different aesthetic preferences.


    2. Parse transforms may be more powerful, but I find macros much simpler to use, they are compiled during the same compilation cycle as the source using them, they are properly documented, and can even do some things not possible from within parse transforms (can you define multiple modules from a parse transform?).


    3. Protocols have nothing to do with cross-process messages and pattern matching in receive. Again - please research before making such claims.


    4. Of course we can use lists:sum, or Elixir's Enum.sum, but that was not the point. I might have used a more complex example, but chose to stick with something simple.


    5. After using both in production, I find mix easier and simpler to use than rebar. YMMV of course.


    6. Elixir records are tagged tuples just like Erlang records. In fact, it is possible to import Erlang records from hrl files. Elixir also has support for typespecs.


    7. I prefer to use EVM since the term BEAM is itself overloaded (it also denotes a compiled file). Alternative VMs usually have special names (Erjang, LING). Note that Joe Armstrong himself used the term EVM when he discussed Elixir.


    8. I never claimed to be an expert on Erlang, but I have been using it in production for more than three years now, and I'm using it at my current job. While there are many things I like, there are also some I dislike. I find that Elixir helps, without losing benefits of the underlying VM, or introducing some other problems. Of course you can think that all of us favoring Elixir over Erlang are some bunch of hipster noobs and/or non-talented programmers who were not able to grasp Erlang. Alternatively, it is possible that we are people who have had some issues with Erlang and find Elixir much more pleasant to use for our everyday work. How you decide to regard us is of course up to you.

    ReplyDelete
  3. Thanks for this. It's got me excited about learning Elixir again - really appreciate it.

    ReplyDelete
    Replies
    1. I'm glad you liked the article, and hope you'll enjoy Elixir as well! Thanks for the feedback!

      Delete
  4. Hey Sasa, great intro. Tried erlang a few months ago, I was writing a web sockets server. I found the language "beautifully ugly". After reading your post it seems Elixir does some stuff to change that so I think I'm gonna give it a shot.

    ReplyDelete
    Replies
    1. Hi Or,

      I'm glad the article sparked your interest. Personally, I find Elixir really exciting, and more approachable than Erlang. That said, you should be prepared that as a functional language it's still going to feel "different". This means that it will take some effort to "get it right" (but I think it's also the fun part). On the positive side, I think Elixir is probably the most approachable and easiest of functional languages to learn, so it's a great opportunity to learn another paradigm (this is assuming you're coming from OO).

      Most importantly, I find underlying Erlang platform (the virtual machine) extremely useful and powerful for development of scalable, fault-tolerant, distributed systems. And this is coming from my own professional experience. I use it now on a daily basis and I'm very pleased with it.

      In any case, I hope you'll enjoy your Elixir journey. If you get stuck, feel free to ask on elixir-lang-talk (https://groups.google.com/forum/#!forum/elixir-lang-talk) or #elixir-lang on IRC.

      Happy programming!

      Delete
  5. Nice article, I heard about Elixir when researching about functional languages, I should definitely give it a try.

    ReplyDelete
    Replies
    1. Thanks for the comment, and hope you'll enjoy Elixir! It's very close to 1.0 so it's much more stable (and also feature rich) than at the time of writing the post.

      Delete
  6. Great article! It was a delightful read. I have recently been enlightened in the importance of paradigmatic diversity. My initial attraction to Object-Oriented languages of the C flavor (C++, C#) and programming in primarily those languages over the past four years have given me a very formulated Object-Oriented mindset. But after a few programmers, of whom I admire for their experience and skill, pointed out how they think all programmers should expand their horizons, and delve into each of the paradigms, I have decided that I would like to expose myself to such types of languages before I begin college, as I have just recently finished high school.

    Would you say that Elixir would suffice as a first Functional language?

    I'm not sure if learning Elixir as my first Functional approach would impose the title of being a "fake" Erlang programmer. Furthermore, speaking in a "career" sense, would you say that knowledge of Elixir would be beneficial in the near future, over other Functional languages?

    Frankly, I do prefer the syntax of Elixir very much over the primitive Erlang syntax. but that's just preference! :)

    Thanks!
    - Shane

    ReplyDelete
    Replies
    1. Hi,

      Thank you for the positive feedback, and I'm glad you liked the article.

      I definitely think Elixir is one of the simpler functional languages, and as such is a good start for making the transition. The language seems to strike a good balance between simplicity and expressiveness, and it doesn't require learning "advanced" concepts such as monads, functors, etc. As a long time OO programmer myself, I think it's not that hard to transition, even more so if you've already used first-class and higher-order functions. That said, there will definitely be some gotchas, most notably you have to get used to working with immutable data.

      It's worth pointing out that Elixir, just like Erlang, has a strong focus on concurrency, which in turn allows you to build highly available systems that are able to constantly provide service, even when faced with unexpected errors. I've written more about in chapter 1 of my upcoming book (http://manning.com/juric/). The chapter is free for download, so you don't need to buy the book to read it :-)

      With that in mind, I believe Elixir has a promising career potential. It's of course hard to predict what will happen, but given that Erlang already has some impressive references (w.g. WhatsApp, Heroku, Chef, ...), and that Elixir has gained some momentum, I am optimistic about the language future. I've already seen a couple of mentions of people using the language in production, and there were even a couple of job adds (http://www.reddit.com/r/elixir/comments/2aa0iv/relixirs_q3_2014_hiring_thread/).

      I wouldn't worry much about the "fake Erlang" label. My impression is that there are only a couple of extreme Erlang zealots who like to sneer down on everything Elixir. Otherwise, Erlang community seems approving of Elixir. They invite Elixir people to give talks on Erlang conferences, and even a co-inventor of Erlang, Robert Virding, gave a talk on the first ever Elixir conference. At the end of the day it's all about usefulness of the tool. In my mind, Elixir improves on some deficiencies of Erlang by providing some elegant language features and including a nice tooling support. However, for the most part, and especially so in the concurrency department and OTP, there will be no differences and you'll need to adapt to the same mindset.

      Delete
    2. Shane, I think if you're just learning functional programming, you may wish to stick to a "simpler" language. Scheme is an excellent pedagogic language and the Racket family is particular well-suited. The language was developed by Matthias Felleisen who has an explicit interest in teaching first year CS students how to program and so co-designed Racket with that aim in mind. Otherwise, I would begin with Lisp and perhaps then venture to something like Haskell or OCaml. These languages share many similar ideas.

      Delete
  7. I liked Elixir so far the only thins I dont like about it is the assignment operator ( = ). I know underneath there is no variable and everything is emutable but in Elixir you feel that you have mutable data type and can change the data anytime.
    you can assign a=2 and then have a=3 without any problem.

    ReplyDelete
    Replies
    1. Yes, this is a controversial property, either loved or hated :-) I fall in the first camp, and actually like the ability to reuse the same variable for different data.

      It becomes easier to reason about if you separate a concept of data and variable. In Elixir, data is immutable, while variable can be rebound to point to a different data.

      It's also worth mentioning that in Elixir, there's not much need for this (in my experience), owing to the pipe operator (|>). It happens that frequently we can just pipe intermediate results through a chain of functions, and then we don't even need to use temporary variables.

      Delete
    2. thanks for you reply. the |> operator is perfect and very handy and I agree with you that by using it, there is less need to rebound variable.
      I'm waiting for your book to be published and I do believe Elixir will be a lot more popular.

      Delete
    3. Yeah, I think Elixir has a good chance. It definitely offers a great combination of productivity tools on top of battle tested Erlang VM. Hope you'll enjoy the book!

      Delete
  8. Would you mind if I publish the Japanese translation of this great article? It must be useful not only for English speaking people but for Japanese people.

    ReplyDelete
    Replies
    1. Sure, feel free to do it, and let me know when it's online so I can tweet about it. Glad you liked the article :-)

      Delete
    2. Thank you very much. The link is here: http://qiita.com/mathhun/items/a3042628f770030b1811

      Delete