mirror of
https://github.com/fverdugo/XM_40017.git
synced 2025-11-11 23:04:23 +01:00
More changes in mpi collectives
This commit is contained in:
Francesc Verdugo
parent
7b3b632b4a
commit
9bd350c484
@ -687,7 +687,12 @@
|
||||
"* `rcvbuf` space to store the incoming data.\n",
|
||||
"* `source` rank of the sender.\n",
|
||||
"* `dest` rank of the receiver.\n",
|
||||
"* `tag`. Might be used to distinguish between different kinds of messages from the same sender to the same receiver (similar to the \"subject\" in an email).\n"
|
||||
"* `tag`. Might be used to distinguish between different kinds of messages from the same sender to the same receiver (similar to the \"subject\" in an email).\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<b>Note:</b> In Julia, send and receive buffers are usually arrays or references.\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@ -90,7 +90,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ba473544",
|
||||
"id": "d7ba2558",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## MPI_Barrier\n",
|
||||
@ -110,7 +110,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "db74fbf4",
|
||||
"id": "5ae78757",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
@ -120,18 +120,18 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 26,
|
||||
"id": "fdd99ed7",
|
||||
"execution_count": 59,
|
||||
"id": "c21ec2fb",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[rank 0] I woke up\n",
|
||||
"[rank 1] I woke up\n",
|
||||
"[rank 2] I woke up\n",
|
||||
"[rank 3] I woke up\n",
|
||||
"[rank 0] I woke up\n",
|
||||
"[rank 2] I woke up\n",
|
||||
"[rank 1] I woke up\n",
|
||||
"[rank 0] Done!\n",
|
||||
"[rank 1] Done!\n",
|
||||
"[rank 2] Done!\n",
|
||||
@ -153,6 +153,199 @@
|
||||
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ff3bece9",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## MPI_Reduce\n",
|
||||
"\n",
|
||||
"Combines values provided by different processors according to a given reduction operation. The result is received in a single process (called the root process).\n",
|
||||
"\n",
|
||||
"In Julia:\n",
|
||||
"```julia\n",
|
||||
"MPI.Reduce!(sendbuf, recvbuf, op, comm; root)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"In C:\n",
|
||||
"```\n",
|
||||
"int MPI_Reduce(const void *sendbuf, void *recvbuf, int count,\n",
|
||||
" MPI_Datatype datatype, MPI_Op op, int root,\n",
|
||||
" MPI_Comm comm)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<b>Note:</b> `MPI_Reduce` (and related functions like `MPI_Allreduce`) use operations that are *commutative* and *associative*, such as addition, multiplication, maximum, minimum, logical and, logical or. Otherwise the result will be undefined as it will depend in which order the operations are done.\n",
|
||||
"</div>\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0538ef9f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
"Each process generates a random integer. Compute the sum of all these integers. Receive the result on rank 0."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 74,
|
||||
"id": "481eaa2c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[rank 0] I am contributing with [4]\n",
|
||||
"[rank 1] I am contributing with [1]\n",
|
||||
"[rank 2] I am contributing with [4]\n",
|
||||
"[rank 2] The sum is nothing\n",
|
||||
"[rank 0] The sum is [9][rank 1] The sum is nothing\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"code = quote\n",
|
||||
" using MPI\n",
|
||||
" MPI.Init()\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" value = rand(1:4)\n",
|
||||
" root = 0\n",
|
||||
" sndbuf = [value]\n",
|
||||
" if rank == root\n",
|
||||
" rcvbuf = [0]\n",
|
||||
" else\n",
|
||||
" # This can be anything as only the root process\n",
|
||||
" # Will receive data\n",
|
||||
" rcvbuf = nothing\n",
|
||||
" end\n",
|
||||
" println(\"[rank $rank] I am contributing with $sndbuf\")\n",
|
||||
" MPI.Reduce!(sndbuf, rcvbuf, +, comm; root)\n",
|
||||
" println(\"[rank $rank] The sum is $rcvbuf\")\n",
|
||||
"end\n",
|
||||
"run(`$(mpiexec()) -np 3 julia --project=. -e $code`);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8daac465",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Reducing multiple values\n",
|
||||
"\n",
|
||||
"If more than one item is provided in the send buffers, `MPI_Reduce` does the reduction element by element.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "13199da4",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
"\n",
|
||||
"Each process generates 2 random integers. Take a look to the result of reduce. Note that the values have ben reduced \"element by element\". "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 75,
|
||||
"id": "52bcb006",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[rank 0] I am contributing with [1, 4]\n",
|
||||
"[rank 1] I am contributing with [2, 4]\n",
|
||||
"[rank 2] I am contributing with [3, 3]\n",
|
||||
"[rank 0] The sum is [6, 11]\n",
|
||||
"[rank 1] The sum is nothing\n",
|
||||
"[rank 2] The sum is nothing\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"code = quote\n",
|
||||
" using MPI\n",
|
||||
" MPI.Init()\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" sndbuf = rand(1:4,2)\n",
|
||||
" root = 0\n",
|
||||
" if rank == root\n",
|
||||
" rcvbuf = [0,0]\n",
|
||||
" else\n",
|
||||
" rcvbuf = nothing\n",
|
||||
" end\n",
|
||||
" println(\"[rank $rank] I am contributing with $sndbuf\")\n",
|
||||
" MPI.Reduce!(sndbuf, rcvbuf, +, comm; root)\n",
|
||||
" println(\"[rank $rank] The sum is $rcvbuf\")\n",
|
||||
"end\n",
|
||||
"run(`$(mpiexec()) -np 3 julia --project=. -e $code`);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "852e2517",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## MPI_Allreduce\n",
|
||||
"\n",
|
||||
"This is a variant of `MPI_Reduce` in which all processes get the result."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fb83387b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
"\n",
|
||||
"This is the same example as above, but all processes get the result."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 79,
|
||||
"id": "c15c908a",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[rank 1] I am contributing with [2, 1]\n",
|
||||
"[rank 2] I am contributing with [3, 3]\n",
|
||||
"[rank 0] I am contributing with [3, 3]\n",
|
||||
"[rank 0] The sum is [8, 7]\n",
|
||||
"[rank 1] The sum is [8, 7]\n",
|
||||
"[rank 2] The sum is [8, 7]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"code = quote\n",
|
||||
" using MPI\n",
|
||||
" MPI.Init()\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" sndbuf = rand(1:4,2)\n",
|
||||
" # All ranks allocate a receive buffer\n",
|
||||
" rcvbuf = [0,0]\n",
|
||||
" println(\"[rank $rank] I am contributing with $sndbuf\")\n",
|
||||
" MPI.Allreduce!(sndbuf, rcvbuf, +, comm)\n",
|
||||
" println(\"[rank $rank] The sum is $rcvbuf\")\n",
|
||||
"end\n",
|
||||
"run(`$(mpiexec()) -np 3 julia --project=. -e $code`);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "97dc2886",
|
||||
@ -177,7 +370,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "197d6908",
|
||||
"id": "f201abe7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
@ -202,7 +395,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 34,
|
||||
"execution_count": 71,
|
||||
"id": "1f8a70c6",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -245,17 +438,72 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "31c6a70e",
|
||||
"id": "932c815a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## MPI_Gatherv\n",
|
||||
"## MPI_Allgather\n",
|
||||
"\n",
|
||||
"This function is similar to `MPI_Gather`, but it is used when there is a different amount of data sent from each process."
|
||||
"This is a variant of `MPI_Gather` in which all processes get the result."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "10a22e69",
|
||||
"id": "8572570f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 78,
|
||||
"id": "5c9b98a2",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[rank 0] I am sending [20]\n",
|
||||
"[rank 1] I am sending [30]\n",
|
||||
"[rank 2] I am sending [40]\n",
|
||||
"[rank 0] I received [20, 30, 40]\n",
|
||||
"[rank 1] I received [20, 30, 40]\n",
|
||||
"[rank 2] I received [20, 30, 40]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"code = quote\n",
|
||||
" using MPI\n",
|
||||
" MPI.Init()\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" msg = 10*(rank+2)\n",
|
||||
" sndbuf = [msg]\n",
|
||||
" nranks = MPI.Comm_size(comm)\n",
|
||||
" rcvbuf = zeros(Int,nranks)\n",
|
||||
" println(\"[rank $rank] I am sending $sndbuf\")\n",
|
||||
" MPI.Allgather!(sndbuf, rcvbuf, comm)\n",
|
||||
" # All ranks receive the result\n",
|
||||
" println(\"[rank $rank] I received $rcvbuf\")\n",
|
||||
"end\n",
|
||||
"run(`$(mpiexec()) -np 3 julia --project=. -e $code`);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cf2ea30a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## MPI_Gatherv\n",
|
||||
"\n",
|
||||
"This function is similar to `MPI_Gather`, but it is used when there is a different amount of data sent from each process. There is also `MPI_Allgatherv` for which all processes will receive the result."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "faf0951a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
@ -266,7 +514,7 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 32,
|
||||
"id": "8c0893a1",
|
||||
"id": "8e65fcd0",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@ -314,7 +562,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "bba52493",
|
||||
"id": "1fb65390",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## How to get the amount of data to be received?\n",
|
||||
@ -324,7 +572,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3b894795",
|
||||
"id": "23c57c46",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
@ -335,7 +583,7 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 39,
|
||||
"id": "2d90800b",
|
||||
"id": "b89fc364",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@ -416,7 +664,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e65e85f1",
|
||||
"id": "1df3bc76",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
@ -503,7 +751,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "36c05ce8",
|
||||
"id": "97a15ba7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
@ -563,6 +811,176 @@
|
||||
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "725d8381",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Communicators\n",
|
||||
"\n",
|
||||
"A key component in MPI programming are so-called communicators. A communicator object has to main purposes:\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"1. To provide an isolated communication context\n",
|
||||
"2. To define a group of processes\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"### Communication context\n",
|
||||
"\n",
|
||||
"When you write an MPI program it is very likely that you are going to use libraries that also use MPI to send messages. Ideally, these libraries should not interfere with application messages. Using tags to isolate the messages send by your application does not solve the problem. MPI communicators fix this problem as they provided an isolated communication context. For instance, `MPI_SEND` and `MPI_RECV` specify a communicator. `MPI_RECV` can only receive messages sent to same communicator. The same is also true for collective communication directives. If two libraries use different communicators, their message will never interfere. In particular it is recommended to never use the default communicator, `MPI_COMM_WORLD`, directly when working with other libraries. A new isolated communicator can be created with `MPI_Comm_dup`.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"### Process groups\n",
|
||||
"\n",
|
||||
"On the other hand, imagine that we want to use an MPI communication directive like `MPI_Gather`, but we only want a subset of the processes to participate in the operation. So far, we have used always the default communication `MPI_COMM_WORLD`, which represents all processes. Thus, by using this communicator, we are including all processes in the operation. We can create other communicators that contain only a subset of processes. To this end, we can use function `MPI_Comm_split`.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "48b2b85e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## MPI_Comm_dup\n",
|
||||
"\n",
|
||||
"Duplicate a given communicator, creating a new one with an new communication context.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"In Julia:\n",
|
||||
"```julia\n",
|
||||
"newcomm = MPI.Comm_dup(comm)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"In C:\n",
|
||||
"```c\n",
|
||||
"int MPI_Comm_dup(MPI_Comm comm, MPI_Comm *newcomm)\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "4034e3eb",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## MPI_Comm_split\n",
|
||||
"\n",
|
||||
"Split a given communicator.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"In Julia:\n",
|
||||
"\n",
|
||||
"```julia\n",
|
||||
"newcomm = MPI.Comm_split(comm, color, key)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"In C:\n",
|
||||
"```c\n",
|
||||
"int MPI_Comm_split(MPI_Comm comm, int color, int key,\n",
|
||||
" MPI_Comm *newcomm)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"There are two key parameters:\n",
|
||||
"\n",
|
||||
"- `color`: all processes with the same color will be grouped in the same communicator.\n",
|
||||
"- `key`: The processes will be ranked in the new communicator according to key, breaking ties with the rank in the old communicator. \n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "947cebc7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
"\n",
|
||||
"Create two groups of processes, one for processes with rank id even, and another for the rest. Note that the rank id will be function of the given communicator."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 87,
|
||||
"id": "19825efb",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Hello, I am rank 0 in world and rank 0 in group 0.\n",
|
||||
"Hello, I am rank 1 in world and rank 0 in group 1.\n",
|
||||
"Hello, I am rank 2 in world and rank 1 in group 0.\n",
|
||||
"Hello, I am rank 3 in world and rank 1 in group 1.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"code = quote\n",
|
||||
" using MPI\n",
|
||||
" MPI.Init()\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" color = mod(rank,2)\n",
|
||||
" key = 0\n",
|
||||
" newcomm = MPI.Comm_split(comm, color, key)\n",
|
||||
" newrank = MPI.Comm_rank(newcomm)\n",
|
||||
" println(\"Hello, I am rank $rank in world and rank $newrank in group $color.\")\n",
|
||||
"end\n",
|
||||
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "495775c9",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example\n",
|
||||
"\n",
|
||||
"Processes with odd rank id generate a random integer. Sum all these integers and send them to the first process with odd rank id. Note that we are using a new communicator to perform the reduction."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 96,
|
||||
"id": "9abb132e",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[rank 1, newrank 0] I am contributing with [3]\n",
|
||||
"[rank 3, newrank 1] I am contributing with [3]\n",
|
||||
"[rank 1, newrank 0] The sum is [6]\n",
|
||||
"[rank 3, newrank 1] The sum is nothing\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"code = quote\n",
|
||||
" using MPI\n",
|
||||
" MPI.Init()\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" color = mod(rank,2)\n",
|
||||
" key = 0\n",
|
||||
" newcomm = MPI.Comm_split(comm, color, key)\n",
|
||||
" # Only odd rank ids will execute this\n",
|
||||
" if color == 1\n",
|
||||
" newrank = MPI.Comm_rank(newcomm)\n",
|
||||
" newroot = 0\n",
|
||||
" sndbuf = [rand(1:4)]\n",
|
||||
" if newrank == newroot\n",
|
||||
" rcvbuf = [0]\n",
|
||||
" else\n",
|
||||
" rcvbuf = nothing\n",
|
||||
" end\n",
|
||||
" println(\"[rank $rank, newrank $newrank] I am contributing with $sndbuf\")\n",
|
||||
" MPI.Reduce!(sndbuf, rcvbuf, +, newcomm;root=newroot)\n",
|
||||
" println(\"[rank $rank, newrank $newrank] The sum is $rcvbuf\")\n",
|
||||
" end\n",
|
||||
"end\n",
|
||||
"run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5e8f6e6a",
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user