{
"cells": [
{
"cell_type": "markdown",
"id": "7606d30a",
"metadata": {},
"source": [
"
\n",
"\n",
"### Programming large-scale parallel systems"
]
},
{
"cell_type": "markdown",
"id": "4ac1e5d9",
"metadata": {},
"source": [
"# Distributed computing with MPI"
]
},
{
"cell_type": "markdown",
"id": "a341be2e",
"metadata": {},
"source": [
"## Contents\n",
"\n",
"\n",
"In this notebook, we will learn the basics of parallel computing using the Message Passing Interface (MPI) from Julia. In particular, we will learn:\n",
"\n",
"- How to run parallel MPI code in Julia\n",
"- How to use basic collective communication directives\n",
"- How to use basic point-to-point communication directives\n",
"\n",
"For further information on how to use MPI from Julia see https://github.com/JuliaParallel/MPI.jl\n"
]
},
{
"cell_type": "markdown",
"id": "8862079b",
"metadata": {},
"source": [
"## What is MPI ?\n",
"\n",
"- MPI stands for the \"Message Passing Interface\"\n",
"- It is a standardized library specification for communication between parallel processes in distributed-memory systems.\n",
"- It is the gold-standard for distributed computing in HPC systems since the 90s\n",
"- It is huge: the MPI standard has more than 1k pages (see https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf)\n",
"- There are several implementations of this standard (OpenMPI, MPICH, IntelMPI)\n",
"- The interface is in C and FORTRAN (C++ was deprecated)\n",
"- There are Julia bindings via the package MPI.jl https://github.com/JuliaParallel/MPI.jl"
]
},
{
"cell_type": "markdown",
"id": "99c6febb",
"metadata": {},
"source": [
"### What is MPI.jl ?\n",
"\n",
"- It is not a Julia implementation of the MPI standard\n",
"- It is a wrapper to the C interface of MPI\n",
"- You need a C MPI installation in your system\n",
"\n",
"\n",
"MPI.jl provides a convenient Julia API to access MPI. For instance, this is how you get the id (rank) of the current process.\n",
"\n",
"```julia\n",
"comm = MPI.COMM_WORLD\n",
"rank = MPI.Comm_rank(comm)\n",
"```\n",
"\n",
"Internally, MPI.jl uses `ccall` which is a mechanism that allows you to call C functions from Julia. In this, example we are calling the C function `MPI_Comm_rank` from the underlying MPI installation.\n",
"\n",
"```julia\n",
"comm = MPI.COMM_WORLD \n",
"rank_ref = Ref{Cint}()\n",
"ccall((:MPI_Comm_rank, MPI.API.libmpi), Cint, (MPI.API.MPI_Comm, Ptr{Cint}), comm, rank_ref)\n",
"rank = Int(rank_ref[])\n",
"```\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "82e6e98f",
"metadata": {},
"source": [
"### Installing MPI in Julia\n",
"\n",
"MPI can be installed as any other Julia package using the package manager."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0b44409e",
"metadata": {},
"outputs": [],
"source": [
"] add MPI"
]
},
{
"cell_type": "markdown",
"id": "abc6f017",
"metadata": {},
"source": [
"
MPI.Wait() before modifying the send buffer or using the receive buffer.\n",
"