Adding mpi collectives notebook

This commit is contained in:
Francesc Verdugo committed 2024-08-23 12:41:21 +02:00
1 parent 0cee9268b5
commit 7b3b632b4a
1 file changed
+592
+592
View File
@@ -0,0 +1,592 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "7606d30a",
"metadata": {},
"source": [
"<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/39/VU_logo.png/800px-VU_logo.png?20161029201021\" width=\"350\">\n",
"\n",
"### Programming large-scale parallel systems"
]
},
{
"cell_type": "markdown",
"id": "4ac1e5d9",
"metadata": {},
"source": [
"# Intro to MPI (collectives)"
]
},
{
"cell_type": "markdown",
"id": "a341be2e",
"metadata": {},
"source": [
"## Contents\n",
"\n",
"\n",
"In this notebook, we will continue learning the basics of parallel computing using the Message Passing Interface (MPI) from Julia. In particular, we will learn:\n",
"\n",
"- How to use basic collective communication directives\n",
"- How to use MPI communicators\n"
]
},
{
"cell_type": "markdown",
"id": "c87b3c82",
"metadata": {},
"source": [
"## Collective communication\n",
"\n",
"MPI provides collective communication functions for communication involving multiple processes. Some usual collective functions are:\n",
"\n",
"\n",
"- `MPI_Barrier`: Synchronize all processes\n",
"- `MPI_Bcase`: Send the same data from one to all processes\n",
"- `MPI_Gather`: Gather data from all processes to one\n",
"- `MPI_Scatter`: Scatter data from one to all all processes\n",
"- `MPI_Reduce`: Reduce data from all processes to a single one\n",
"- `MPI_Scan`: Scan (or prefix) reduction\n",
"- `MPI_Allgather`: Like a `MPI_Gather` but all processes receive the result\n",
"- `MPI_Allreduce`: Like `MPI_Reduce` but all processes receive the result\n",
"- `MPI_Alltoall`: Exchange data from all to all processes\n",
"\n",
"\n",
"We will discuss some of them in greater detail in this notebook."
]
},
{
"cell_type": "markdown",
"id": "4ffa5e56",
"metadata": {},
"source": [
"## Why collective primitives?\n",
"\n",
"Point-to-point communication primitives provide all the building blocks needed in parallel programs and could be used to implement the collective functions described above. Then, why does MPI provide collective communication directives? There are several reasons:\n",
"\n",
"- Ease of use: It is handy for users to have these functions readily available instead of having to implement them.\n",
"- Performance: Library implementations typically use efficient algorithms (such as reduction trees).\n",
"- Hardware support: Hardware vendors can optimize the MPI library for their machine and even develop hardware specially design to perform MPI operations efficiently. For instance, [IMB Blue Gene](https://en.wikipedia.org/wiki/IBM_Blue_Gene) was equipped with auxiliary networks for MPI collectives.\n"
]
},
{
"cell_type": "markdown",
"id": "53a0f5fa",
"metadata": {},
"source": [
"## Semantics of collective operations\n",
"\n",
"\n",
"- Completeness: All the collective communication directives above are *complete* operations. Thus, it is safe to use and reset the buffers once the function returns.\n",
"- Standard mode: Collective directives are in standard mode only, like `MPI_Send`. Assuming that they block is erroneous, assuming that they do not block is also erroneous.\n",
"- Synchronization: Completion of a call does not guarantee that other processes have completed the operation. A collective operation may or may not have the effect of synchronizing all processes, the only exception is `MPI_Barrier` of course.\n",
"\n",
"\n",
"<div class=\"alert alert-block alert-info\">\n",
"<b>Note:</b> Recent versions of the MPI standard also include non-blocking (incomplete) versions of collective operations (not covered in this notebook). A particularly funny one is the non-blocking barrier `MPI_Ibarrier`.\n",
"</div>"
]
},
{
"cell_type": "markdown",
"id": "ba473544",
"metadata": {},
"source": [
"## MPI_Barrier\n",
"\n",
"This function is used to synchronizes a group of processes. All processes block until all have reached the barrier. It is often invoked at the end of for loops to make sure all processes have finished the current loop iteration to move to the next one. We will see an example later in another notebook when studying the traveling sales person problem (TSP).\n",
"\n",
"In Julia:\n",
"```julia\n",
"MPI.Barrier(comm)\n",
"```\n",
"\n",
"In C:\n",
"```c\n",
"int MPI_Barrier(MPI_Comm comm)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "db74fbf4",
"metadata": {},
"source": [
"### Example\n",
"\n",
"In this example the ranks sleep for a random amount of time and then they call barrier. It is guaranteed that the message \"Done!\" will be printed after all processes printed \"I woke up\" since we used a barrier. Try also to comment out the call to `MPI.Barrier`. You will see that the message can be printed in any order in this case."
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "fdd99ed7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[rank 0] I woke up\n",
"[rank 1] I woke up\n",
"[rank 2] I woke up\n",
"[rank 3] I woke up\n",
"[rank 0] Done!\n",
"[rank 1] Done!\n",
"[rank 2] Done!\n",
"[rank 3] Done!\n"
]
}
],
"source": [
"code = quote\n",
" using MPI\n",
" MPI.Init()\n",
" comm = MPI.COMM_WORLD\n",
" rank = MPI.Comm_rank(comm)\n",
" sleep(rand(1:3))\n",
" println(\"[rank $rank] I woke up\")\n",
" MPI.Barrier(comm)\n",
" println(\"[rank $rank] Done!\")\n",
"end\n",
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
]
},
{
"cell_type": "markdown",
"id": "97dc2886",
"metadata": {},
"source": [
"## MPI_Gather\n",
"\n",
"Each rank sends a message to the root rank (the root rank also sends a message to itself). The root rank receives all these values in a buffer (e.g. a vector). This function assumes that the amount of data sent from each rank is the same. The root process can be any process and it is rank 0 by default in Julia.\n",
"\n",
"In Julia:\n",
"```julia\n",
"MPI.Gather!(sendbuf, recvbuf, comm; root=0)\n",
"```\n",
"\n",
"In C:\n",
"```c\n",
"int MPI_Gather(const void *sendbuf, int sendcount, MPI_Datatype sendtype,\n",
" void *recvbuf, int recvcount, MPI_Datatype recvtype, int root,\n",
" MPI_Comm comm)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "197d6908",
"metadata": {},
"source": [
"### Example\n",
"\n",
"Each process sends an integer to rank 0.\n"
]
},
{
"attachments": {
"g13884.png": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABFEAAANWCAYAAAA2uxORAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13mGRlnf7/d01iYMg5BxUlKCiggiImDKiYV9c16xpXZY3o6q7u18SuusZVMcvq+jMHRFcRTBgQDICIiGSQnAcmz/3745xmTp+uTtXV/VR3v1/XVVdPV9c5dVd1z3PO+dQTOkiSJM24LAG2AbZufd0I2AxYCmxc/3sRsBWwGNi0vn9p/bPFwJaTfPKh/dwKrANWACuBNcDy+jE31V+X1/evrB+3DrgFuLl+zM317ZbGv2+GzqpJZpIkSbNAp3QASZI0F2QpsAuwM7A7sC0bCiNDRZJtG//erEzOGbOCYUUVbgauB64G/gZcA1wJXFt97dxWKKckSZoEiyiSJGkcWUZVGNmlvu1Wf921vu0MbF8sHqwC7pjkNquBJfW/N6XqnVLSHcBVVEWWdqHlKuBi4BLorC4VUJIkWUSRJEkAZBtg7/p298a/7wps0ecnuwm4ob7d2Pq6iqrXxhrgNjYMoxkaVnMTsLb+2QrorOxfrAwNIVoIbF7fuQWwANiEaqjRkvpnWzZuW7W+b96W0D/rqYorF49yuxI66/v4fJIkqcUiiiRJ80Y2Z0NxpF0s2XoKO14FXEF1gX9Z/fVquhdJbphfF/rZhA0Fle2oevBsT9V7Z0dgp/q2A9Vwp6lYDVzKhqLKRcB5wJ+oerGsm+L+JUma9yyiSJI052QRVYHkAODA+usBVENvJusOqgvzK+vb5fXXK7izcNK5tg+hRTaiKrDs0vi6A7AHsFd924Wqp8xkrQT+XN/OZUNx5a/QWTPl6JIkzRMWUSRJmtWyNXBvNhRKDgD2p1q9ZqJWAhfUt782/n0BdP7W17iaoiymmp9mr1Fuk52bZg3V7/pPjCiwWFyRJKnNIookSbNGtgUOrW8HURVMdpngxuuoCiR/YViRhAuAy6GTvsdVAVnGhoLK3YF9gf3qr5NZCno1cA7wu8bt7P7OQSNJ0uxjEUWSpIGUhcA+wMHAA4HDqS6EJ3LsvgX4I1Wvgj8BvwV+B53JrmCjOSVbUfVS2q/x9S71bSLWUhXhflvfzq2+dm7qf1ZJkgaTRRRJkgZCdqTqYXJY/fUQqhVhxrIWOJ+qx8BZ9e0c6FwxjUE152RbNvRW2Q+4J1VPp4n0XFlHVVgZ6q3yG+BMe6xIkuYqiyiSJBWRPYCHAw8FHkQ1eeiYG1BdrJ4O/Lr+ei50Vk1nSs1nuQtVMaV5224CG64GzgR+Cfyi+urkw5KkucEiiiRJMyLbAQ8BjqQamrPfOBssp+pZchrVheivoXPddCaUxpedqYaYDd2GhgSN5yo2/C2fBvx+fi11LUmaKyyiSJI0LbIF8GCq3iYPo5qDYrTj7nqquUtOB35F1dPkPC8yNTtkR6peKodQDUc7DNhinI1upPpb/0V9OwM6K6YzpSRJ/WARRZKkvsgiqglgH01VNDkYWDjag6km5TwVOAX4GXRunomU0vTLAqqi4eHAA6j+X+w1zkarqIoqp9S3M6CzdjpTSpLUC4sokiT1LMuoCiaPA54A7DDGgy9iw1CG7zn5q+aX7Ajclw0rTR0CbDTGBrdTFVV+VN9+5zLckqRBYBFFkqRJyd2Bo4HHUk0Iu2iUB15J1dOkvnUum5l80myQTdhQVHkAcASw2RgbXMWGnls/gs7l0x5RkiRJkjRZWQR5COS9kPMhGeW2EvIDyCsh9yidWppdsgjyAMi/Qn4CWTXG/7XU/xc/CnlKPf+QJEmSJKmMbAR5POQLkBvHuJC7BvLZ+kJurE/RJU1KNoE8CvKfkN9C1o3x/3A15BTIqyF7l04uSZIkSfNAFkIOh3wQct0YF2wX1o85sp5MVtK0y2b1/7njIGeO00vlQsjxkKMhS0onlyRJkqQ5IgshD6svuK4f5YJsBeT7kJdD9iidWBJAdoc8H/JlyM1jFFRuqh/zbMi2pVNLkiRJ0iyTBXWPkw9DrhpjfpNvQ/4BsmnpxJLGksV1MfR9kL+MUVBZCzkN8kbIvUqnliRJkqQBlgPri6zLx5hX4XuQ50K2LJ1WUq9yd8hrIKfW/6/HGvZzHOSQ0oklSZIkaQBkC8iL60+fu11Erat/dgxk+9JpJfVbltVzoxwP+dsYBZVL67mODod0SqeWJEmSpBmSTt21/wuQO0YpnPy0nuNkh9JpJc2ULIQ8EPJuyLljFFQuhrwHcj8LKpIkSZLmqOwG+de6i363C6M/Q46F7FI6qaRBkP0gb4WcM0ZB5RLIeyGHWlCRJEmSNMtlCeTvqFbPWdvlAug2yGeqLvqSNJrsC/k3yNnjDPn5r6qgIkmSJEmzRvaA/AfkulEudn4BeSFks9JJJc022QfyFshZYxRU/lIXXfYqnVaSJEmSRpGDISdA1nS5qLmhnjzygNIpJc0V2aueeHq0yakDObN+zDal00qSJEma97IU8gLIH7pcvKyFfAfyBMji0kklzWW5O+TNkPNGKabcAfkS5DGQRaXTSpIkSZpXsiPkbaMM2bmZajnSPUunlDQfZX/IcZBrRimoDPWMcz4mSZIkSdPpziE7q7tcmJxfd5tfVjqlJFEtm3xk3WbdPkpB5dy6ILxH6bSSJEmS5oQsgDytnlugfQGyDvItyMNKp5Sk0WUryEvria27FVPWQk6EPK4qvkiSJEnSpGQx5DmjzDFwS90dfp/SKSVpcrI75FjIBaMUVK6shwPZO0WSJEnSeLIx5J8gl3S5uPgT5GUO2ZE0+6UDeRDkk5Bbu7R3a+qedkdVPfIkSZIk6U7ZtJ7T5MouFxO/r3ul2M1d0hyUpZC/Y/Tlkq+oe6fsXjqpJEmSpKKyed21/YYuFw6nQY4unVCSZk4Orocr3jbK3Ckn1wUXi8qSJEnS/JEdIe/pcqGwHvJdyANKJ5SkcrJ5PRnt70fpnXIh5E2QHUonlSRJkjRtshXVkp7tOQDW1atTHFI6oSQNljF7p6yCfAVy/9IpJUmSJPVNNqmH7dzYugBYDTnBlXYkaTzZHPLiMXqnnOZQH0mSJGlWy5J6tZ2ruhRPPu5EiZLUixwG+Z+6J0q7mHIB5FWQzUqnlCRJkjQhWVB/InpBlzlPvgLZu3RCSZr9skM9RPK6LsWUWyEfhOxZOqUkSZKkUeXIUbqbnww5qHQ6SZp7shHVUvBnd2l7h+acOrJ0SkmSJEl3yoMhv+xyAv9zyOGl00nS3JcO5BGQk+qef+32+AzIM6uhlpIkSZIKyF6Qb3Q5Wf895DGl00nS/JR7QD4KWd6lfb4C8hrIpqVTSpIkSfNENqnH4q9onZxfXK8g4QoRklRcNoccA7mkSzHlFshxkK1Lp5QkSZLmqHQgz4Bc3joZvx7yMsji0gklSW1ZBHk65PRRJqF9D2Sn0iklSZKkOST3gfysdfK9BnI8ZNvS6SRJE5HD68lm28WUVZATIHcvnVCSJEmaxbJ1vVTm2tYJ96mQA0qnkyT1Iveuiybttn1oRZ+DSyeUJEmSZpEsquc3ua51gn15tZymJGn2y93qHoVrWm39+rqYcljphJIkSdKAy4Mg57ZOqJdD3gJZWjqdJKnfsle9ok97wvBAToEcWTqhJEmSNGCyRT10Z13rBPpEyB6l00mSplu2r1dfu6lLMeWXkIeXTihJkiQNgDwJcmXrhPn31SSEkqT5JVtB/rXLkM5AfuQwH0mSJM1T2Rny9dYJ8h2QN1TzokiS5q8sg7wGck2XYspJTkArSZKkeSKdaoLY3NA6Kf4p5B6l00mSBkk2gRwDubpLMeVkyH1KJ5QkSZKmSe5WTxTYPAm+qV6Np1M6nSRpUGVTyLFd5kxZD/mKRXhJkiTNIVlUn/y2V184EbJL6XSSpNkim9XHk5tbx5N1dTFl79IJJUmSpCnIAZCzWie7l0IeWzqZJGm2ynaQ99VzaTWPL6shx0N2K51QkiRJmoQsgLwWsrL1SeEHq08SJUmaquwM+XDrWJP6+/dBti6dUJIkSRpHdoOc2jqhPQ9yaOlkkqS5KLtDPln3RGkee26sC/oblU4oSZIkdZFntCb+Ww/5CGTj0skkSXNd7go5oe752CymXAT5eycxlyRJ0oDIFvWJa/Ok9WrI40onkyTNN9kP8t0uyyKfAXlw6XSSJEma1/IAyIWtE9WvQ7YtnUySNJ/lSMgfuhRTTobcs3Q6SZIkzStZDHlbq9v0rZAXl04mSVIlCyB/B7mkVUhZU6/ks2PphJIkSZrzsl+XpYtPg+xVOpkkSSNlGeTfILe1jl23QP4FsknphJIkSZqT8szWSejq+gR0YelkkiSNLTtCPl73RGkWUy6HPK/quSJJkiRNWTaCfKzL0sUHl04mSdLkZD/IiV3mSzkTcmjpdJIkSZrVsjvk160TzW9Uq/JIkjRb5TDIL1rHt/X1inM7lE4nSZKkWSePh9zUOLlcCXl56VSSJPVHOpBnQa5oFVNuhhwDWVQ6oSRJkgZeFnVZfedSuzlLkuambFIf91a2iil/hjyqdDpJkiQNrGwP+VHrJPJEyNalk0mSNL2yN+SkLvOlnAjZo3Q6SZIkDZQ8BPK3xknj2vqTOVcskCTNIzkaclGrkHJ7fUxcWjqdJEmSissb6qLJ0Mni3yBHlE4lSVIZ2bgumtzRKqb8BfKY0ukkSZJURDaCfLZ1gvgTyE6lk0mSVF52rVfs6TbE5y6l00mSJGnGZHvIaa2TwndDFpZOJknSYMmj6olmm8fMFZB/gywpnU6SJEnTKveCXNxavvi5pVNJkjS4srhe+viWLkN8Hlo6nSRJkqZFHg+5rXHydzXksNKpJEmaHbIL5H9bhZR1kI9BtiidTpIkSX2TY+oTvaGTvrMhe5ZOJUnS7JOHQM5vFVOugjyndDJJkiRNSdcJLine truncated
}
},
"cell_type": "markdown",
"id": "4bdf9c02",
"metadata": {},
"source": [
"<div>\n",
"<img src=\"attachment:g13884.png\" align=\"left\" width=\"350\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "1f8a70c6",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[rank 1] I am sending [30]\n",
"[rank 2] I am sending [40]\n",
"[rank 0] I am sending [20]\n",
"[rank 0] I received [20, 30, 40]\n",
"[rank 1] I received nothing\n",
"[rank 2] I received nothing\n"
]
}
],
"source": [
"code = quote\n",
" using MPI\n",
" MPI.Init()\n",
" comm = MPI.COMM_WORLD\n",
" rank = MPI.Comm_rank(comm)\n",
" msg = 10*(rank+2)\n",
" root = 0\n",
" sndbuf = [msg]\n",
" if rank == root\n",
" nranks = MPI.Comm_size(comm)\n",
" rcvbuf = zeros(Int,nranks)\n",
" else\n",
" # This can be anything as only the root process\n",
" # Will receive data\n",
" rcvbuf = nothing\n",
" end\n",
" println(\"[rank $rank] I am sending $sndbuf\")\n",
" MPI.Gather!(sndbuf, rcvbuf, comm; root)\n",
" println(\"[rank $rank] I received $rcvbuf\")\n",
"end\n",
"run(`$(mpiexec()) -np 3 julia --project=. -e $code`);"
]
},
{
"cell_type": "markdown",
"id": "31c6a70e",
"metadata": {},
"source": [
"## MPI_Gatherv\n",
"\n",
"This function is similar to `MPI_Gather`, but it is used when there is a different amount of data sent from each process."
]
},
{
"cell_type": "markdown",
"id": "10a22e69",
"metadata": {},
"source": [
"### Example\n",
"\n",
"Each processes sends `rank+1` integers to rank 0."
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "8c0893a1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[rank 1] I am sending [10, 10]\n",
"[rank 2] I am sending [20, 20, 20]\n",
"[rank 3] I am sending [30, 30, 30, 30]\n",
"[rank 0] I am sending [0]\n",
"[rank 0] I received [0, 10, 10, 20, 20, 20, 30, 30, 30, 30]\n"
]
}
],
"source": [
"code = quote\n",
" using MPI\n",
" MPI.Init()\n",
" comm = MPI.COMM_WORLD\n",
" rank = MPI.Comm_rank(comm)\n",
" sndbuf = fill(10*rank,rank+1)\n",
" root = 0\n",
" if rank == root\n",
" nranks = MPI.Comm_size(comm)\n",
" # Define the amount of data to be received\n",
" # from each rank\n",
" counts = [ rank+1 for rank in 0:(nranks-1)]\n",
" # Total amount of receive data\n",
" ndata = sum(counts)\n",
" data = zeros(Int,ndata)\n",
" rcvbuf = MPI.VBuffer(data,counts)\n",
" else\n",
" # This can be anything as only the root process\n",
" # Will receive data\n",
" rcvbuf = nothing\n",
" end\n",
" println(\"[rank $rank] I am sending $sndbuf\")\n",
" MPI.Gatherv!(sndbuf, rcvbuf, comm; root)\n",
" if rank == root\n",
" println(\"[rank $rank] I received $data\")\n",
" end\n",
"end\n",
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
]
},
{
"cell_type": "markdown",
"id": "bba52493",
"metadata": {},
"source": [
"## How to get the amount of data to be received?\n",
"\n",
"Remember that `MPI_Probe` allows you to query the amount of data to be received with `MPI_Recv`. Unfortunately, this mechanism does not work for MPI collectives. A workaround is to first communicate the amount of data to be gathered, allocate the receive buffer accordingly, and then communicate the actual message.\n"
]
},
{
"cell_type": "markdown",
"id": "3b894795",
"metadata": {},
"source": [
"### Example\n",
"\n",
"Each process sends a random amount of integers to rank 0."
]
},
{
"cell_type": "code",
"execution_count": 39,
"id": "2d90800b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[rank 1] I am sending [1, 1]\n",
"[rank 2] I am sending [2]\n",
"[rank 3] I am sending [3, 3, 3, 3]\n",
"[rank 0] I am sending [0]\n",
"[rank 0] I received [0, 1, 1, 2, 3, 3, 3, 3]\n"
]
}
],
"source": [
"code = quote\n",
" using MPI\n",
" MPI.Init()\n",
" comm = MPI.COMM_WORLD\n",
" rank = MPI.Comm_rank(comm)\n",
" # Get number of ints to send\n",
" count = rand(1:4)\n",
" # Generate data to send\n",
" sndbuf = fill(rank,count)\n",
" # Communicate counts\n",
" root = 0\n",
" if rank == root\n",
" nranks = MPI.Comm_size(comm)\n",
" counts = zeros(Int,nranks)\n",
" else\n",
" counts = nothing\n",
" end\n",
" MPI.Gather!([count], counts, comm; root)\n",
" # Allocate receive buffer\n",
" if rank == root\n",
" ndata = sum(counts)\n",
" data = zeros(Int,ndata)\n",
" rcvbuf = MPI.VBuffer(data,counts)\n",
" else\n",
" rcvbuf = nothing\n",
" end\n",
" # Communicate data\n",
" println(\"[rank $rank] I am sending $sndbuf\")\n",
" MPI.Gatherv!(sndbuf, rcvbuf, comm; root)\n",
" if rank == root\n",
" println(\"[rank $rank] I received $data\")\n",
" end\n",
"end\n",
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
]
},
{
"cell_type": "markdown",
"id": "8b4254d1",
"metadata": {},
"source": [
"## MPI_Scatter\n",
"\n",
"This is the inverse operation of `MPI_Gather`. The root rank sends a distinct message to each rank.\n",
"\n",
"In Julia:\n",
"```julia\n",
"MPI.Scatter!(sendbuf, recvbuf, comm; root=0)\n",
"```\n",
"\n",
"In C:\n",
"```c\n",
"int MPI_Scatter(const void *sendbuf, int sendcount, MPI_Datatype sendtype,\n",
" void *recvbuf, int recvcount, MPI_Datatype recvtype, int root,\n",
" MPI_Comm comm)\n",
"```\n",
"\n",
"<div class=\"alert alert-block alert-info\">\n",
"<b>Note:</b> `MPI_Scatter` sends the same amount of data to each rank. To send a variable amount of data there is `MPI_Scatterv` (`MPI.Scatterv!` in julia), which works similar to `MPI_Gatherv`. \n",
"</div>\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "e65e85f1",
"metadata": {},
"source": [
"### Example\n",
"\n",
"Rank 0 sends a unique integer to each rank. "
]
},
{
"attachments": {
"g13389.png": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABFEAAANWCAYAAAA2uxORAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13nGV1ff/x190Ou5SlI10FKQpSBaLYsBB7xW6sUSMhxoJGjRiNIbGXRIk1qD8TuwI2FBsWBAsoICC91wUWtu++f3+cM+yZs3fq3pnvnZnX8/G4j9l7555z33Nn9nvO+dxv6SBJkjTpMg/YGtiq9XU+sBmwANik/vccYDEwF1hUP76g/t5cYMsxvvjAfu4C1gLLgRXAauDu+jlL6q9314+vqJ+3FrgTuKN+zh317c7Gv++AzsoxZpIkSVNAp3QASZI0HWQBsBNwH2BXYBvWF0YGiiTbNP69WZmck2Y5g4oq3AHcCtwIXA/cBFwH3Fx97SwtlFOSJI2BRRRJkjSCLKQqjOxU33apv+5c3+4DbFcsHqwElo1xm1XAvPrfi6h6p5S0DLiBqsjSLrTcAFwBXAmdVaUCSpIkiyiSJAmAbA3sWd/2avz7fsAWPX6xJcBt9e321teVVL02VgNLWT+MZmBYzRJgTf295dBZ0btYGRhCNBvYvH5wC2AWsCnVUKN59fe2bNwWt+43b/PonXVUxZUrhrhdB511PXw9SZLUYhFFkqQZI5uzvjjSLpZstRE7XglcS3WBf3X99Ua6F0lum1kX+tmU9QWVbal68GxH1XtnB2DH+rY91XCnjbEKuIr1RZXLgYuAC6l6sazdyP1LkjTjWUSRJGnayRyqAsn+wAH11/2pht6M1TKqC/Pr6ts19ddrubdw0rm5B6FF5lMVWHZqfN0e2A3Yo77tRNVTZqxWAH+ubxewvrjyF+is3ujokiTNEBZRJEma0rIV8GDWF0r2B/ajWr1mtFYAl9a3vzT+fSl0ru9pXG2kzKWan2aPIW5jnZtmNdXv+kI2KLBYXJEkqc0iiiRJU0a2AQ6vbwdRFUx2GuXGa6kKJJcwqEjCpcA10EnP46qALGR9QWUvYB9g3/rrWJaCXgX8Efhd43Z+b+egkSRp6rGIIklSX8psYG/gYOCvgIdSXQiP5th9J/Anql4FFwK/BX4HnbGuYKNpJYupeint2/h63/o2GmuoinC/rW8XVF87S3qfVZKk/mQRRZKkvpAdqHqYHFF/PYRqRZjhrAEupuoxcF59+yN0rp3AoJp2sg3re6vsCzyQqqfTaHqurKUqrAz0VvkNcK49ViRJ05VFFEmSishuwKOBRwIPo5o8dNgNqC5WzwZ+XX+9ADorJzKlZrLcl6qY0rxtO4oNVwHnAr8EflF9dfJhSdL0YBFFkqRJkW2BRwBHUw3N2XeEDe6m6llyFtWF6K+hc8tEJpRGlvtQDTEbuA0MCRrJDaz/Wz4L+P3MWupakjRdWESRJGlCZAvg4VS9TR5FNQfFUMfddVRzl5wN/Iqqp8lFXmRqasgOVL1UDqEajnYEsMUIG91O9bf+i/p2DnSWT2RKSZJ6wSKKJEk9kTlUE8A+nqpocjAwe6gnU03KeSbwI+Bn0LljMlJKEy+zqIqGDwWOpPp/sccIG62kKqr8qL6dA501E5lSkqTxsIgiSdK4ZSFVweSJwFOA7Yd58uWsH8rwHSd/1cySHYBDWb/S1CHA/GE2uIeqqPLD+vY7l+GWJPUDiyiSJI1J9gKeBDyBakLYOUM88Tqqnib1rXP15OSTpoJsyvqiypHAUcBmw2xwA+t7bv0QOtdMeERJkiRJ0lhlDuQRkPdBLoZkiNsKyPchx0EeUDq1NLVkDuRIyNshP4GsHOb/Wur/i/8FeUY9/5AkSZIkqYzMhzwZ8gXI7cNcyN0E+Wx9ITfcp+iSxiSbQh4H+Q/IbyFrh/l/uAryI8jrIHuWTi5JkiRJM0BmQx4K+TDklmEu2C6rn3N0PZmspAmXzer/cydBzh2hl8plkJMhT4LMK51ckiRJkqaJzIY8qr7gunWIC7LlkO9CXgPZrXRiSQDZFfISyP9B7himoLIE8r+QF0C2KZ1akiRJkqaYzKp7nHwUcsMw85t8C/I8yKLSiSUNJ3PrYuj7IZcMU1BZAzkL8mbIg0qnliRJkqQ+lgPqi6xrhplX4TuQF0O2LJ1W0nhlL8g/Qs6s/18PN+znJMjBpRNLkiRJUh/IFpBX1p8+d7uIWlt/73jIdqXTSuq1LKznRjkZcv0wBZWr6rmOHgrplE4tSZIkSZMknbpr/xcgy4YonPy0nuNk+9JpJU2WzIb8FeTfIBcMU1C5AvJeyGEWVCRJkiRNU9kF8va6i363C6M/Q06A7FQ6qaR+kH0h74D8cZiCypWQ90EeYkFFkiRJ0hSXeZBnUa2es6bLBdBSyGeqLvqSNJTsA/lnyPkjDPl5P+Tw0mklSZIkaQyyG+TfIbcMcbHzC8jLIJuVTippqsnekLdBzhumoHJJ3fNt99JpJUmSJGkIORhyCmR1l4ua2+rJI/cvnVLSdJE96omnh5qcOpBz6+dsXTqtJEmSpBkvCyAvhfyhy8XLGsi3IU+BzC2dVNJ0lr0gb4VcNEQxZRnkS5C/hswpnVaSJEnSjJIdICcOMWTnDqrlSHcvnVLSTJT9ICdBbhqioDLQM875mCRJkiRNpHuH7KzqcmFycd1tfmHplJJEtWzy0XWbdc8QBZUL6oLwbqXTSpIkSZoWMgvy7HpugfYFyFrINyGPKp1SkoaWxZBX1RNbdyumDAw/fGJVfJEkSZKkMclcyIuGmGPgzro7/N6lU0rS2GRXyAmQS4coqFxXDweyd4okSZKkkWQTyN9BruxycXEh5NUO2ZE09aUDeRjkk5C7urR3q+uedsdUPfIkSZIk6V5ZVM9pcl2Xi4nf171S7OYuaRrKAsizGHq55Gvr3im7lk4qSZIkqahsXndtv63LhcNZkCeVTihJkycH18MVlw4xd8oZdcHForIkSZI0c2QHyHu7XCisg5wGObJ0QkkqJ5vXk9H+fojeKZdB3gzZrnRSSZIkSRMmi6mW9GzPAbAWcirkkNIJJam/DNs7ZSXky5DDSqeUJEmS1DPZtB62c3vrAmAV5BRX2pGkkWRzyCuH6Z1ylkN9JEmSpCkt8+rVdm7oUjz5hBMlStJ45AjI5+ueKO1iyqWQ4yCblU4pSZIkaVQyq/5E9NIuc558GbJn6YSSNPVl+3qI5C1diil3QT4M2b10SkmSJElDytFDdDc/A3JQ6XSSNP1kPtVS8Od3aXsH5pw6unRKSZIkSffKwyG/7HIC/3PIQ0unk6TpLx3IYyCn1z3/2u3xOZDnV0MtJUmSJBWQPSBf73Ky/nvIX5dOJ0kzUx4A+S/I3V3a52shr4MsKp1SkiRJmiGyaT0Wf3nr5PyKegUJV4iQpOKyOeR4yJVdiim31u34VqVTSpIkSdNUOpDnQq7pcjL+asjc0gklSW2ZAzkWcvYQk9C+F7Jj6ZSSJEnSNJIDIT9rnXyvhpwM2aZ0OknSaOSh9WSz7WLKSsgpkL1KJ5QkSZKmsGxVL5W5pnXCfSZk/9LpJEnjkQfXRZN22z6woo8rqkmSJEmjlzn1/Ca3tE6wr6mW05QkTX25f92jcHWrrV9XF1OOKJ1QkiRJ6nN5GOSC1gn13ZC3QRaUTidJ6rXsUa/o054wPJAfQR5dOqEkSZLUZ7JFPXRnbesE+lTIbqXTSZImWrarV+1Z0qWY8kuLKZIkSRIAeRrkutYJ8++rSQglSTNLFkPe3mVIZyBnOMxHkiRJM1TuA/la6wR5GeRN1bwokqSZKwsh/wi5qUsx5TQnoJUkSdIMkU41QWxua50U/xTygNLpJEn9JJtCjofcOETPlANLJ5QkSZImSO5fTxTYPAleUq/G0ymdTpLUr7IIckKXOVPWQb5sEV6SJEnTSObUJ7/t1RdOhexUOp0kaarIZvXx5I7W8WRtXUzZs3RCSZIkaSNkf8h5rZPdqyBPKJ1MkjRVZVvI++u5tJrHl1WQkyG7lE4oSZIkjUFmQV4PWdH6pPDD1SeJkiRtrNwH8tHWsSb1/fdXq/1IkiRJfS27QM5sndBeBDm8dDJJ0nSUXSGfrHuiNI89t9Wr/MwvnVCSJEnqIs9tTfy3DvIxyCalk0mSprvcD3JK3fOxWUy5HHKsk5hLkiSpT2SL+sS1edJ6I+SJpZNJkmaa7As5rcuyyL+BPLx0OkmSJM1oORJyWetE9WuQbUonkyTNZDka8ocuxZQzIA8snU6SJEkzSuZCTmx1m74L8srSySRJqmQW5FmQK1uFlNX1Sj47lE4oSZKkaS/7dlm6+CzIHqWTSZK0oSyE/DNkaevYdSfkn5y7S5IkSRMkz2+dhK6qT0Bnl04mSdLwsgPkE3VPlGYx5RrIi6ueK5IkSdJGy3zIx7ssXXxw6WSSJI1N9oWc2mW+lHMgDymdTpIkSVNadoX8unWi+fVqVR5JkqaqHAH5Rev4tq5ecW770ukkSZI05eTJkCWNk8sVkNeUTiVJUm+kA3kB5NpWMWUJ5HjInNIJJUmS1Pcyp8vqO1dBDi+dTJKk3sum9XFvRauY8mfIY0unkyRJUt/KdpAftk4iT4VsVTqZJEkTK3tCTu8yX8qpkN1Kp5MkSVJfySMg1zdOGtfUn8y5YoEkaQbJkyCXtwop99THxPml00mSJKm4vKkumgycLF4POap0KkmSysgmddFkWauYcgnkmNLpJEmSVETmQz7bOkH8CWTH0skkSSovO9cr9nQb4nPf0ukkSZI0abId5KzWSeG/QWaXTiZJUn/J4+qJZpvHzOWQt0PmlU4nSZKkCZUHQa5oLV/84tKpJEnqX5lbL318Z5chPo8snU6SJEkTIk+GLG2c/N0IOaJ0KkmSpobsBPl/rULKWsh/QbYonU6SJEk9k+PrE72Bk77zIbuXTiVJ0tSTR0AubhVTboC8qHQySZIkbZSuE8ieDtm8dDJJkqauLine truncated
}
},
"cell_type": "markdown",
"id": "74d5b606",
"metadata": {},
"source": [
"<div>\n",
"<img src=\"attachment:g13389.png\" align=\"left\" width=\"350\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 44,
"id": "5ea413fd",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[rank 0] I am sending: [20, 30, 40, 50]\n",
"[rank 0] I have received: 20\n",
"[rank 1] I have received: 30\n",
"[rank 2] I have received: 40\n",
"[rank 3] I have received: 50\n"
]
}
],
"source": [
"code = quote\n",
" using MPI\n",
" MPI.Init()\n",
" comm = MPI.COMM_WORLD\n",
" nranks = MPI.Comm_size(comm)\n",
" rank = MPI.Comm_rank(comm)\n",
" root = 0\n",
" rcv = Ref(0) \n",
" if rank == root\n",
" snd = [10*(i+1) for i in 1:nranks]\n",
" println(\"[rank $rank] I am sending: $snd\")\n",
" else\n",
" # Only the root rank sends data\n",
" # so this can be anything\n",
" snd = nothing\n",
" end \n",
" MPI.Scatter!(snd,rcv,comm;root)\n",
" println(\"[rank $rank] I have received: $(rcv[])\")\n",
"end\n",
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
]
},
{
"cell_type": "markdown",
"id": "ed8da7f9",
"metadata": {},
"source": [
"## MPI_Bcast\n",
"\n",
"Similar to `MPI_Scatter`, but we send the same message to all processes.\n",
"\n",
"In Julia:\n",
"```julia\n",
"MPI.Bcast!(buf, comm; root)\n",
"```\n",
"\n",
"In C:\n",
"```c\n",
"int MPI_Bcast(void *buffer, int count, MPI_Datatype datatype,\n",
" int root, MPI_Comm comm)\n",
"```\n"
]
},
{
"cell_type": "markdown",
"id": "36c05ce8",
"metadata": {},
"source": [
"### Example\n",
"\n",
"Rank 0 sends the same integer to all processes."
]
},
{
"attachments": {
"g1657.png": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA9MAAANWCAYAAAD9cN8uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13nC1lYf/xz7n9ApdeLyACgghIUbEhVowVOxqN0RhLjJpgDfYYuybGoPGn2BLRWIgVCyq2SBGlKCBNFOm9t9vv9/fHzMLs7Nm9Z8+W5+zu5/16zWvvPbsz892zu8/MM0/rIElSUekAWwJbNT5uBSwBNgcWAJvW/18KbAIsrD+3EFjW+Nyy+uvHYwtgBbASWAvcXr9+c/3xDmBN/fkVwDrgtvpzt9bbLaNvnTvHmUeSJM0AndIBJEmzTeYB2wM7ATsCOzCystysNG/J7L4erWFkhft64DrgGuAq4Nr643XQubZQTkmSNA6z+eZFkjTpsoiqcrwT91SWd64/7gjci6oiPd7W4cmynqriOh6hapFeCCwGNprsUOO0hqqifRVVZfvqersWuBK4BPgzdG4f7QCSJGnqWZmWJLVkCbBHl+0+VBXpybx23AncBNxYbzc0/n9H/fnVVBXkNVTdq1cBd3FP9+tbqo+TWbnMfKqu5QCbAfOoKtmLgUXAxlTvw+b15zfvYdt48vIB1Xv1Z+6uXA/bLoXOqkk+nyRJarAyLUlzUhYCuwJ71luz0rwz/V8f1lO1pl5B1bJ6GVWL6vXcU2G+6Z6PnZX9fw8zTYbGeW8ObANsR9Wav239cTuqhxXb1/+eP4GTrad6/4cq1xcDFwLnAxdY0ZYkaeKsTEvSrJcdgf3qbf/6430Zf1fstVTdjK9geGX5ynq7HLgaOmsnJ/dclnlUlex2hXs51UOQoW1xHwdfR1XBPpeqcn0+cB5VJfuOCUeXJGmOsDItSbNGFgP7cE+FeajyvNU4DrKeqoJ8UWP7Q/3xEuismcTAmpB0qFqyd2N4BXto25HxtW4HuBS4gKqiPfTx947PliRpJCvTkjQjZTHwAOChwEGMv7X5aqrK0kWt7U92AZ4tsohqQrhdgd2BvYH71duO4zjQeqrfjTMb22+hc/OYe0mSNMtZmZakGSHLgQcCBwOPqP+9pIcd11BVhM6gamU8DzgNOtdMUVDNCNmMakK5fagq2bvV/96LarK1XlxN9Xs1tPl7JUmaU6xMS9LAyUZUleWHAg+rP+7Qw45XAOcAZ9XbOcCFjmFW77IxVYX6ftxT0T6QalK6XlzOPa3XpwO/sgVbkjRbWZmWpOKyDHgk8FjgUVTjnDfUXfsG4NR6+zVwJnRumsqUmsuyDdWwgua2Ww87rqea4OzkejsFOn+cqpSSJE0nK9OSNO2ygKrCfGi9PZJq7eLRrKNa1ugM4CSqSsl50MkUB5XGkE2pxuo/sLH10k38WuA07vl9PmluLZEmSZotrExL0pTLAuBBVC3PjwUeDiwdY4eruafV+VTgdOjcNdUppYnLMqpu4Q+gGqJwMBue7GwFVZfwodbrX0HnxqlMKUnSZLAyLUlTIvcBngI8jqrr9qZjfPG1wM+Bn1Vb509Tn0+aLtmFqlL9cKrJ8/Zl7CW71gNnAz8BfgqcCJ07pzqlJEnjZWVakiZF5gMHAIcBT6Xq8jqaO6hanH9Sb2faZVtzRzah+lsZmpn+YGCLMXZYSzWh3tDfyy+hs3qqU0qStCFWpiWpb9kceAJVBfpJwJajfOFdVGND65ZnzoTOummJKA28zKcaez3Uev1oxp69/nbg/6harX8K/N6HUZIkSdLAy30gr4P8FLIaklG2cyAfgDwSMtbkYpJGyN6Qf4B8G3LLGH9ngVwD+R/I30J2Kp1ckiRJ0t2yP+T9kPPHuKFfBflxXQHYtXRiafbIfMhDIG+tH2KtGOPvcD3kTMh7IA+GbGhmcUmSJEmTK3tD3rWBCvQNkGMhL4JsVjqxNDdkKeTQ+u/zhA30ELnev1FJkiRpymX3uvXrrDFuzs+uW6kfZquXNAiyBeRZkE9BLh/jb3cl5Ed175HdSqeWJEmSZrjcC/JGyGlj3IT/DvIWb8ClmSAHQt4OORWyboy/63MhH4IcUnUllyRJkrQB2RLyGshJ9RjL0W603wm5b+m0kvqVbSEvgXwdctsGhmx8BvIXkAWlU0uSJEkDJPMgj4AcDblzlBvqSyBHVV8naXbJgroM+OAG5kK4CXIM5DCciV+SJElzV+5dT1R06RgV6A9DHlA4qKRplb0h/wQ5cYzu4DdCPgd5EmRh6cSSJEnSFMsSyPOpZvrtdpN8K+TTkIdDOqXTSiotO9RDP/5vjIr1TZDPQ55si7UkSZJmmRwI+Xh909tt/dlfQl4M2bh0UkmDKjtAXg35xQYq1v8NeYot1pIkSZqhsqSeYOiMUW56r4R8ALJn6aSSZppsD3kV5OeQtaOUMdfVcy08qHRaSZIkqQfZvh4LfV2Xm9u1dRfvw201kjQ5shXkRZDvQlaPUrE+ry6Xdi2dVpIkSWrJwyBfGeVm9jzIGyDblk4paTbLdpC/rycv67a83rr6gd6LIJuUTitJkqQ5K4vqVuaTx7hpPczJxCRNv+wMORLyh1Faq++CHFuXUa5hLUmSpOmQ7eqb1Mu73KDeRrVm9P1Kp5SkSh5Yj5++fow5HI6CHFA6qSRJkmal7A35AmRVl5vRC6hm2rXrpKQBlSV1b5rjxhhffSrV5IkblU4rSZKkGS/7Q47pMmvuertyS5qZsgXkFZCTRqlU31r3srG1WpIkSeOVQyDHj9KV+2O4rJWkWSH7QD5Yd/fuVrE+uZ60bGnppJIkSRpoeUTd4ty+obyBanmZLUsnlKTJl3mQQ+uJybqtX31L3Vq9X+mkkiRJGhjp1N21f93lBvLauhK9WemUkjQ9smM90eJlo7RWn153E3dstSRJ0tyUBZAXQs7tcrN4MeSVkMWlU0pSGVkIeQ7kx3Rfu/oGyEcc9iJJkjRn3N0SfXaXm8M/1i0urr0qSXfL7vXY6mu7lJtOyChJkjT75VDIaV1uBn9XT7Izv3RCSRpcWUS1xNYJo7RWXwg5wi7gkiRJs0YeCvlZlxu/30CeWDqdJM082bPu5n1Ll7L1esh7ITuUTilJkqS+5H717LTtFpTz69YVuyRK0oRkk3p4zHldKtWr6jL4oaVTSpIkqSfZuV7Gpb3Ey2WOiZakqXD38lrfHaUL+On1cBrLX0mSpMGTbSEfq1tDmjdx19Xj+JydW5KmXA6A/Bdk5SirJbwOsmnplJIkSaomxXkj5NbWTdutkHdClpVOKElzT7aDvGuUWcBvhXwIsn3plJIkSXNUDoX8vss4vaOrGzlJUll3zwL+61HGVR8D2aN0SkmSpDkie0C+17opW1/flO1UOp0kqZs8dpRx1Wvq8nvv0gklSZJmqWxcdxtsj8U7A3Jw6XSSpF5k37ryvKbLQ9HvOgO4JEnSpEmn7iZ4WevG64Z6crH5pRNKksYr94YcBbmrSxfwkyCHlU4oSZI0g+WBkJNbN1mr6xuwzUqnkyRNVLatex3d0qVSfWb9MLVTOqUkSdIMkS0hn4Osa91Y/QBy39LpJEmTLVvWleobu1Sqz4a8wJ5IkiRJY8phkCtaN1IXVa0TkqTZLRvXQ3gu71KpvhjyCivVkiRJw2TnevKZ5o3THZB/qpZXkSTNHVkM+TvIn0ZpqX6G3b8lSdIcl3mQf4Dc1qVL9y6l00mSSsoCyF9BLuhSqT4N8qTSCSVJkgrIvpBTWjdHN1Xd+CRJGpJ59WRkF3WpVP/K2b8lSdIckYWQIxm5ZvSxkG1Kp5MkDaoshLxolO7fJ0EeUzqhJEnSFMnBkHO7TCrzhNLJJEkzRRbVk5Fd2aVSfQLkoNIJJUmSJkmWQj4GWd+44VkL+TfIRqXTSZJmomwEeRPk+laFej3k25D9SieUJEmagDwAcl7rRue3kAeWTiZJmg2yDPJ2yM2ta806yP9A7l06oSRJ0jhkPuTNkFWNG5uV9WsLSqeTJM022Rzyni4rRKyEfLj6vCRJ0kDLvSG/bN3M/B6yf+lkkqTZLltD/hVyV+s6dAPktdWYa0mSpIGTw+slrppj1452bLQkaXplp/r6s7ZVqb60nhW8UzqhJEkSkM0gX2rdsFwNeXLpZJKkuSz7QL7fZebv30AeWTqdJEma0/I4yOWtm5T/hWxVOpkkSZUcCjmrS6X6u5A9SqeTJElzShZBPtpa8uoWyAtLJ5MkaaTMh7wcclWrQr0aclQ13lqSJGlK5V6QU1s3I7+A7FI6mSRJY8smkHdB7mhdx26GHAlZUjqhJEmalfKkelbUoZuPVZB/gswrnUySpN5lB8hnukxSdjHk6aXTSZKkWSPz6yf56xo3HJdDHl46mSRJ/ctekGO7jKf+CWTv0ukkSdKMlu0gP23dZBzv+DJJ0uyRQyHntK51qyAfqLqGS5IkjUsOgVzZWjv6g3brliTNPlkAOaKeULNZqb7S9aklSVKP0qlvKFY3biauh/xF6WSSJE2tbFXP8L2OkZNt7lc6nSRJGljZFPL11g3ELyHLSyeTJGn65EGQX7Wuh2sgR1cVbkmSpLtlX8ifWt26P1x1fZMkaa7JPMjftVayCOQayEvs+i1JkoA8sTVO7FbIc0qnkiSpvGxRd/1uL6V1OuShpdNJkqRi8obWDcJZkN1Kp5IkabDkQMjJrQr1Osj/g2xWOp0kSZo2WQT5XOum4AfeEEiSNJp0IIdDLmtdP6+uZv2WJEmzXLauJxZr3gh82GWvJEnqRTaF/Hs9KVnzWvptyI6l00mSpCmRPSEXNi78qyB/UzqVJEkzT/aH/LpVob4DciRkful0kiRp0uQJrYnGboA8unQqSZJmrsyDvAJyW6tSfQbkAaXTSZKkCcsrWt3RzoHsWjqVJEmzQ3aEfKtVoV4N+SBkSel0kiRp3LIA8unWxf27kGWlk0mSNPvk+ZBrW9fdC+0JJmmyuMi9NC2yLine truncated
}
},
"cell_type": "markdown",
"id": "26b56c81",
"metadata": {},
"source": [
"<div>\n",
"<img src=\"attachment:g1657.png\" align=\"left\" width=\"350\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 46,
"id": "4de15781",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[rank 0] I am sending: 20\n",
"[rank 0] I have received: 20\n",
"[rank 1] I have received: 20\n",
"[rank 2] I have received: 20\n",
"[rank 3] I have received: 20\n"
]
}
],
"source": [
"code = quote\n",
" using MPI\n",
" MPI.Init()\n",
" comm = MPI.COMM_WORLD\n",
" nranks = MPI.Comm_size(comm)\n",
" rank = MPI.Comm_rank(comm)\n",
" root = 0\n",
" buffer = Ref(0)\n",
" if rank == root\n",
" buffer[] = 20\n",
" println(\"[rank $rank] I am sending: $(buffer[])\")\n",
" end \n",
" MPI.Bcast!(buffer,comm;root)\n",
" println(\"[rank $rank] I have received: $(buffer[])\")\n",
"end\n",
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
]
},
{
"cell_type": "markdown",
"id": "5e8f6e6a",
"metadata": {},
"source": [
"# License\n",
"\n",
"This notebook is part of the course [Programming Large Scale Parallel Systems](https://www.francescverdugo.com/XM_40017) at Vrije Universiteit Amsterdam and may be used under a [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) license."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.9.0",
"language": "julia",
"name": "julia-1.9"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.9.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}