{
"cells": [
{
"cell_type": "markdown",
"id": "7606d30a",
"metadata": {},
"source": [
"
\n",
"\n",
"### Programming large-scale parallel systems"
]
},
{
"cell_type": "markdown",
"id": "4ac1e5d9",
"metadata": {},
"source": [
"# Distributed computing with MPI"
]
},
{
"cell_type": "markdown",
"id": "a341be2e",
"metadata": {},
"source": [
"## Contents\n",
"\n",
"\n",
"In this notebook, we will learn the basics of parallel computing using the Message Passing Interface (MPI) from Julia. In particular, we will learn:\n",
"\n",
"- How to run parallel MPI code in Julia\n",
"- How to use basic collective communication directives\n",
"- How to use basic point-to-point communication directives\n",
"\n",
"For further information on how to use MPI from Julia see https://github.com/JuliaParallel/MPI.jl\n"
]
},
{
"cell_type": "markdown",
"id": "8862079b",
"metadata": {},
"source": [
"## What is MPI ?\n",
"\n",
"- MPI stands for the \"Message Passing Interface\"\n",
"- It is a standardized library specification for communication between parallel processes in distributed-memory systems.\n",
"- It is the gold-standard for distributed computing in HPC systems since the 90s\n",
"- It is huge: the MPI standard has more than 1k pages (see https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf)\n",
"- There are several implementations of this standard (OpenMPI, MPICH, IntelMPI)\n",
"- The interface is in C and FORTRAN (C++ was deprecated)\n",
"- There are Julia bindings via the package MPI.jl https://github.com/JuliaParallel/MPI.jl"
]
},
{
"cell_type": "markdown",
"id": "99c6febb",
"metadata": {},
"source": [
"### What is MPI.jl ?\n",
"\n",
"- It is not a Julia implementation of the MPI standard\n",
"- It is a wrapper to the C interface of MPI\n",
"- You need a C MPI installation in your system\n",
"\n",
"\n",
"MPI.jl provides a convenient Julia API to access MPI. For instance, this is how you get the id (rank) of the current process.\n",
"\n",
"```julia\n",
"comm = MPI.COMM_WORLD\n",
"rank = MPI.Comm_rank(comm)\n",
"```\n",
"\n",
"Internally, MPI.jl uses `ccall` which is a mechanism that allows you to call C functions from Julia. In this, example we are calling the C function `MPI_Comm_rank` from the underlying MPI installation.\n",
"\n",
"```julia\n",
"comm = MPI.COMM_WORLD \n",
"rank_ref = Ref{Cint}()\n",
"ccall((:MPI_Comm_rank, MPI.API.libmpi), Cint, (MPI.API.MPI_Comm, Ptr{Cint}), comm, rank_ref)\n",
"rank = Int(rank_ref[])\n",
"```\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "82e6e98f",
"metadata": {},
"source": [
"### Installing MPI in Julia\n",
"\n",
"The Jupyter Julia kernel installed by IJulia activates the folder where the notebook is located as the default environment, which causes the main process and the worker processes to not share the same environment. Therefore, we need to set the environment as the global environment."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "705c2439-c805-4818-be73-342182f7b7a0",
"metadata": {},
"outputs": [],
"source": [
"] activate"
]
},
{
"cell_type": "markdown",
"id": "e99c7676-989e-4e91-b65e-ebca2d5626a4",
"metadata": {},
"source": [
"MPI can be installed as any other Julia package using the package manager."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0b44409e",
"metadata": {},
"outputs": [],
"source": [
"] add MPI"
]
},
{
"cell_type": "markdown",
"id": "abc6f017",
"metadata": {},
"source": [
"
MPI.Wait() before modifying the send buffer or using the receive buffer.\n",
"