Is Julia fast?

Contents

With this notebook, you will learn

• Two basic julia concepts related performance:
  • type-inference
  • JIT compilation
• Some Julia syntax
• Some useful Julia packages

Using Jupyter notebooks in Julia

We are going to use Jupyter notebooks in this and other lectures. You provably have worked with notebooks (in Python). If not, here are the basic concepts you need to know to follow the lessons.

Tip: Did you know that Jupyter stands for Julia, Python and R?

How to start a Jupyter nootebook in Julia

To run a Julia Jupyther notebook, open a Julia REPL and type

julia> ]
pkg> add IJulia
julia> using IJulia
julia> notebook()

A new browser window will open. Navigate to the corresponding notebook and open it.

Warning: Make sure that the notebook is using the same Julia version as the one you used to launch `IJulia`. If it is not the same, go to Kernel > Change Kernel and choose the right version.

Running a cell

To run a cell, click on a cell and press Shift + Enter. You can also use the "Run" button in the toolbar above.

1+3
4*5

As you can see from the output of previous cell, the value of the last line is displayed. We can suppress the output with a semicolon. Try it. Execute next cell.

1+3
4*5;

Cell order is important

Running the two cells below in reverse order won't work (try it).

foo() = "Well done!"

foo()

REPL modes

This is particular to Julia notebooks. You can use package, help, and shell mode just like in the Julia REPL.

] add MPI

? print

; ls

How fast is Julia code?

NB. Most of the examples below are taken from the lecture by S.G. Johnson at MIT. See here: https://github.com/mitmath/18S096/blob/master/lectures/lecture1/Boxes-and-registers.ipynb

Example

Sum entries of a given array $a = [a_1,a_2,...,a_n]$

$$s = \sum_{i=1}^n a_i$$

Hand-written sum function

function sum_hand(a)
    s = zero(eltype(a))
    for ai in a
        s += ai
    end
    s
end

Test it

The Julia macro @test which is provided in the Test package is useful to write (unit) tests in Julia.

using Test

a = rand(5)

@test sum_hand(a) ≈ sum(a)

Benchmarking

In Julia, the most straight-forward way of measuring the computation time of a piece of code is with the macro @time.

a = rand(10^7);

@time sum_hand(a)

Note that @time also measures the compile time of a function if it's the first call to that function. So make sure to run @time twice on a freshly compiled function in order to get a more meaningful result.

A part of getting rid of compilation time, one typically wants to measure the runtime several times and compute sole. To do this we can call our code in a for-loop and gather the runtimes using the Julia macro @elapsed. This measures the runtime of an expression in seconds, just as the @time macro, only @elapsed discards the result of the computation and returns the elapsed time instead.

@elapsed sum_hand(a)

BenchmarkTools

The BenchmarkTools extension package provides useful macros for sampling runtimes automatically.

using BenchmarkTools

First of all, the @benchmark macro runs the code multiple times and gives out a lot of details: the minimum and maximum time, mean time, median time, number of samples taken, memory allocations, etc.

bch_sum_hand = @benchmark sum_hand($a)

For quick sanity checks, one can use the @btime macro, which is a convenience wrapper around @benchmark. It returns only the minimum execution time and memory allocations.

@btime sum_hand($a)

Similar to the @elapsed macro, BenchmarkTool's @belapsed discards the return value of the function and instead returns the minimum runtime in seconds.

@belapsed sum_hand($a)

As opposed to @time and @elapsed, @btime and @belapsed run the code several times and return the minimum runtime, thus eliminating possible compilation times from the measurement.

Built-in sum function

bch_sum = @benchmark sum($a)

Hand-written sum in Python

using PyCall

py"""
def sum_py_hand(A):
    s = 0.0
    for a in A:
        s += a
    return s
"""
sum_py_hand = py"sum_py_hand"

@test sum(a) ≈ sum_py_hand(a)

bch_sum_py_hand = @benchmark sum_py_hand($a)

Numpy sum

using Conda

numpy = pyimport("numpy")
sum_numpy = numpy["sum"]

@test sum_numpy(a) ≈ sum(a)

bch_sum_numpy = @benchmark sum_numpy($a)

Sumary of the results

timings = [bch_sum_hand,bch_sum,bch_sum_py_hand,bch_sum_numpy]

methods = ["sum_hand","sum","sum_py_hand","sum_numpy"]

using DataFrames

df = DataFrame(method=methods,time=timings)

Improving the hand-written sum in Julia

# ✍️ Exercise 3
function sum_hand_fast(a)
    s = 0.0
    @simd for ai in a
       s += ai
    end
    s
end

@test sum_hand_fast(a) ≈ sum(a)

@benchmark sum_hand_fast($a)

Conlcusions so far

• Julia code (for loops) are much faster than in Python
• Julia code can be as fast as optimized C code

Why Julia is fast?

• Julia is a compiled language (like C, C++, Fortran)
• Julia is JIT compiled (C, C++, Fortran are AOT compiled)
• Type declarations are optional in Julia

Conclusion: Why we use Julia in this course

• Julia code is fast (it can be as fast as C)
• Julia is a high-level language with simpler syntax than C
• Julia supports different parallel programming models

We will look into the third point in a later section of this course.

Solution to the exercises

Solution to Exercise 1

function sum_hand(a)
    s = 0.0
    for ai in a
        s += ai
    end
    s
end

Solution to Exercise 2

using Statistics

a = rand(10^7)
num_it = 15
runtimes = zeros(num_it)
for i in 1:num_it
    runtimes[i] = @elapsed sum_hand(a)
end
@show mean(runtimes) 
@show std(runtimes)
@show minimum(runtimes)
@show maximum(runtimes);

# ✍️ Exercise 3
function sum_hand_fast(a)
    s = 0.0
    @simd for ai in a
       s += ai
    end
    s
end