mirror of
https://github.com/fverdugo/XM_40017.git
synced 2026-11-15 06:07:17 +01:00
More on MPI
This commit is contained in:
Francesc Verdugo committed
1 parent
b91a64c5a1
commit
03a3d629e2
3 files changed
+124
-441
No files matched your search
@@ -121,6 +121,7 @@ makedocs(;
|
||||
"Julia Basics" => "notebooks/julia_basics.md",
|
||||
"Tasks and channels" => "notebooks/julia_async.md",
|
||||
"Remote calls and remote channels" => "notebooks/julia_distributed.md",
|
||||
"MPI" => "notebooks/mpi_tutorial.md",
|
||||
"Matrix Multiplication"=>"notebooks/matrix_matrix.md",
|
||||
"Jacobi" => "notebooks/jacobi_method.md",
|
||||
"Solutions" => "notebooks/solutions.md"
|
||||
|
||||
@@ -1915,6 +1915,21 @@
|
||||
d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
|
||||
transform="scale(-0.6)" />
|
||||
</marker>
|
||||
<marker
|
||||
style="overflow:visible"
|
||||
id="marker8529-36"
|
||||
refX="0"
|
||||
refY="0"
|
||||
orient="auto"
|
||||
inkscape:stockid="Arrow2Mend"
|
||||
inkscape:isstock="true">
|
||||
<path
|
||||
transform="scale(-0.6)"
|
||||
d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
|
||||
style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
|
||||
id="path8527-7"
|
||||
inkscape:connector-curvature="0" />
|
||||
</marker>
|
||||
</defs>
|
||||
<sodipodi:namedview
|
||||
id="base"
|
||||
@@ -1923,18 +1938,18 @@
|
||||
borderopacity="1.0"
|
||||
inkscape:pageopacity="0.0"
|
||||
inkscape:pageshadow="2"
|
||||
inkscape:zoom="1.979899"
|
||||
inkscape:cx="-176.2183"
|
||||
inkscape:cy="1109.0146"
|
||||
inkscape:zoom="3.959798"
|
||||
inkscape:cx="-202.58436"
|
||||
inkscape:cy="-681.81991"
|
||||
inkscape:document-units="mm"
|
||||
inkscape:current-layer="layer1"
|
||||
inkscape:document-rotation="0"
|
||||
showgrid="false"
|
||||
inkscape:snap-global="true"
|
||||
inkscape:window-width="2496"
|
||||
inkscape:window-height="1376"
|
||||
inkscape:window-x="1984"
|
||||
inkscape:window-y="27"
|
||||
inkscape:window-width="1920"
|
||||
inkscape:window-height="1163"
|
||||
inkscape:window-x="1920"
|
||||
inkscape:window-y="0"
|
||||
inkscape:window-maximized="1"
|
||||
fit-margin-top="0"
|
||||
fit-margin-left="0"
|
||||
|
||||
|
Before Width: | Height: | Size: 2.0 MiB After Width: | Height: | Size: 2.0 MiB |
+101
-434
@@ -43,12 +43,12 @@
|
||||
"## What is MPI ?\n",
|
||||
"\n",
|
||||
"- MPI stands for the \"Message Passing Interface\"\n",
|
||||
"- It is the gold-standard for distributed computing in HPC systems since the 90s\n",
|
||||
"- It is a standardized library specification for communication between parallel processes in distributed memory systems.\n",
|
||||
"- It is the gold-standard for distributed computing in HPC systems since the 90s\n",
|
||||
"- It is huge: the MPI standard has above 1k pages (see https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf)\n",
|
||||
"- There are several implementations of this standard (OpenMPI, MPICH, IntelMPI)\n",
|
||||
"- The interface is in C and FORTRAN (C++ was deprecated)\n",
|
||||
"- There are Julia bindings via the package MPI.jl"
|
||||
"- There are Julia bindings via the package MPI.jl https://github.com/JuliaParallel/MPI.jl"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -92,17 +92,6 @@
|
||||
"## Running MPI code"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "044aaeec",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Execution model\n",
|
||||
"\n",
|
||||
"- MPI programs are typically run with a Single Program Multiple Data (SPMD) model\n",
|
||||
"- But the standard supports Multiple Program Multiple Data (MPMD)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8376135d",
|
||||
@@ -131,6 +120,17 @@
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "044aaeec",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Execution model\n",
|
||||
"\n",
|
||||
"- MPI programs are typically run with a Single Program Multiple Data (SPMD) model\n",
|
||||
"- But the standard supports Multiple Program Multiple Data (MPMD)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5f76fb65",
|
||||
@@ -138,7 +138,7 @@
|
||||
"source": [
|
||||
"### Hello world\n",
|
||||
"\n",
|
||||
"Julia code needs typically needs to be in a file to run in with MPI. Let's us write our hello world in a file:"
|
||||
"Julia code typically needs to be in a file to run in with MPI. Let's us write our hello world in a file:"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -194,8 +194,8 @@
|
||||
"source": [
|
||||
"### MPIClusterManagers \n",
|
||||
"\n",
|
||||
"- This package allows you to create Julia workers that can call MPI code\n",
|
||||
"- Very useful to combine Distributed.jl and MPI.jl\n",
|
||||
"- This package allows you to create Julia workers that can call MPI functions\n",
|
||||
"- This is useful to combine Distributed.jl and MPI.jl\n",
|
||||
"- E.g., we can run MPI code interactively (from a notebook)\n",
|
||||
"- Link: https://github.com/JuliaParallel/MPIClusterManagers.jl\n"
|
||||
]
|
||||
@@ -227,24 +227,10 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"execution_count": null,
|
||||
"id": "84adeab3",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"3-element Vector{Int64}:\n",
|
||||
" 2\n",
|
||||
" 3\n",
|
||||
" 4"
|
||||
]
|
||||
},
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"using MPIClusterManagers\n",
|
||||
"using Distributed\n",
|
||||
@@ -257,20 +243,10 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"execution_count": null,
|
||||
"id": "612a7587",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" From worker 4:\tHello, I am process 2 of 3 processes!\n",
|
||||
" From worker 2:\tHello, I am process 0 of 3 processes!\n",
|
||||
" From worker 3:\tHello, I am process 1 of 3 processes!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"@everywhere workers() begin\n",
|
||||
" using MPI\n",
|
||||
@@ -282,6 +258,20 @@
|
||||
"end"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6caa8d74",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## MPI Communicators\n",
|
||||
"\n",
|
||||
"In MPI, a **communicator** represents a group of processes that can communicate with each other. `MPI_COMM_WORLD` (`MPI.COMM_WORLD` from Julia) is a built-in communicator that represents all processes available in the MPI program. Custom communicators can also be created to group processes based on specific requirements or logical divisions. The **rank** of a processor is a unique identifier assigned to each process within a communicator. It allows processes to distinguish and address each other in communication operations.\n",
|
||||
"\n",
|
||||
"### Duplicating a communicator\n",
|
||||
"\n",
|
||||
"It is a good practice to not using the built-in communicators directly, and use a copy instead with `MPI.Comm_dup`. Different libraries using the same communicator can lead to unexpected interferences."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c87b3c82",
|
||||
@@ -304,7 +294,24 @@
|
||||
"id": "8b4254d1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Scatter"
|
||||
"### Scatter\n",
|
||||
"\n",
|
||||
"TODO explain what this code does"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g13389.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABFEAAANWCAYAAAA2uxORAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13nGV1ff/x190Ou5SlI10FKQpSBaLYsBB7xW6sUSMhxoJGjRiNIbGXRIk1qD8TuwI2FBsWBAsoICC91wUWtu++f3+cM+yZs3fq3pnvnZnX8/G4j9l7555z33Nn9nvO+dxv6SBJkjTpMg/YGtiq9XU+sBmwANik/vccYDEwF1hUP76g/t5cYMsxvvjAfu4C1gLLgRXAauDu+jlL6q9314+vqJ+3FrgTuKN+zh317c7Gv++AzsoxZpIkSVNAp3QASZI0HWQBsBNwH2BXYBvWF0YGiiTbNP69WZmck2Y5g4oq3AHcCtwIXA/cBFwH3Fx97SwtlFOSJI2BRRRJkjSCLKQqjOxU33apv+5c3+4DbFcsHqwElo1xm1XAvPrfi6h6p5S0DLiBqsjSLrTcAFwBXAmdVaUCSpIkiyiSJAmAbA3sWd/2avz7fsAWPX6xJcBt9e321teVVL02VgNLWT+MZmBYzRJgTf295dBZ0btYGRhCNBvYvH5wC2AWsCnVUKN59fe2bNwWt+43b/PonXVUxZUrhrhdB511PXw9SZLUYhFFkqQZI5uzvjjSLpZstRE7XglcS3WBf3X99Ua6F0lum1kX+tmU9QWVbal68GxH1XtnB2DH+rY91XCnjbEKuIr1RZXLgYuAC6l6sazdyP1LkjTjWUSRJGnayRyqAsn+wAH11/2pht6M1TKqC/Pr6ts19ddrubdw0rm5B6FF5lMVWHZqfN0e2A3Yo77tRNVTZqxWAH+ubxewvrjyF+is3ujokiTNEBZRJEma0rIV8GDWF0r2B/ajWr1mtFYAl9a3vzT+fSl0ru9pXG2kzKWan2aPIW5jnZtmNdXv+kI2KLBYXJEkqc0iiiRJU0a2AQ6vbwdRFUx2GuXGa6kKJJcwqEjCpcA10EnP46qALGR9QWUvYB9g3/rrWJaCXgX8Efhd43Z+b+egkSRp6rGIIklSX8psYG/gYOCvgIdSXQiP5th9J/Anql4FFwK/BX4HnbGuYKNpJYupeint2/h63/o2GmuoinC/rW8XVF87S3qfVZKk/mQRRZKkvpAdqHqYHFF/PYRqRZjhrAEupuoxcF59+yN0rp3AoJp2sg3re6vsCzyQqqfTaHqurKUqrAz0VvkNcK49ViRJ05VFFEmSishuwKOBRwIPo5o8dNgNqC5WzwZ+XX+9ADorJzKlZrLcl6qY0rxtO4oNVwHnAr8EflF9dfJhSdL0YBFFkqRJkW2BRwBHUw3N2XeEDe6m6llyFtWF6K+hc8tEJpRGlvtQDTEbuA0MCRrJDaz/Wz4L+P3MWupakjRdWESRJGlCZAvg4VS9TR5FNQfFUMfddVRzl5wN/Iqqp8lFXmRqasgOVL1UDqEajnYEsMUIG91O9bf+i/p2DnSWT2RKSZJ6wSKKJEk9kTlUE8A+nqpocjAwe6gnU03KeSbwI+Bn0LljMlJKEy+zqIqGDwWOpPp/sccIG62kKqr8qL6dA501E5lSkqTxsIgiSdK4ZSFVweSJwFOA7Yd58uWsH8rwHSd/1cySHYBDWb/S1CHA/GE2uIeqqPLD+vY7l+GWJPUDiyiSJI1J9gKeBDyBakLYOUM88Tqqnib1rXP15OSTpoJsyvqiypHAUcBmw2xwA+t7bv0QOtdMeERJkiRJ0lhlDuQRkPdBLoZkiNsKyPchx0EeUDq1NLVkDuRIyNshP4GsHOb/Wur/i/8FeUY9/5AkSZIkqYzMhzwZ8gXI7cNcyN0E+Wx9ITfcp+iSxiSbQh4H+Q/IbyFrh/l/uAryI8jrIHuWTi5JkiRJM0BmQx4K+TDklmEu2C6rn3N0PZmspAmXzer/cydBzh2hl8plkJMhT4LMK51ckiRJkqaJzIY8qr7gunWIC7LlkO9CXgPZrXRiSQDZFfISyP9B7himoLIE8r+QF0C2KZ1akiRJkqaYzKp7nHwUcsMw85t8C/I8yKLSiSUNJ3PrYuj7IZcMU1BZAzkL8mbIg0qnliRJkqQ+lgPqi6xrhplX4TuQF0O2LJ1W0nhlL8g/Qs6s/18PN+znJMjBpRNLkiRJUh/IFpBX1p8+d7uIWlt/73jIdqXTSuq1LKznRjkZcv0wBZWr6rmOHgrplE4tSZIkSZMknbpr/xcgy4YonPy0nuNk+9JpJU2WzIb8FeTfIBcMU1C5AvJeyGEWVCRJkiRNU9kF8va6i363C6M/Q06A7FQ6qaR+kH0h74D8cZiCypWQ90EeYkFFkiRJ0hSXeZBnUa2es6bLBdBSyGeqLvqSNJTsA/lnyPkjDPl5P+Tw0mklSZIkaQyyG+TfIbcMcbHzC8jLIJuVTippqsnekLdBzhumoHJJ3fNt99JpJUmSJGkIORhyCmR1l4ua2+rJI/cvnVLSdJE96omnh5qcOpBz6+dsXTqtJEmSpBkvCyAvhfyhy8XLGsi3IU+BzC2dVNJ0lr0gb4VcNEQxZRnkS5C/hswpnVaSJEnSjJIdICcOMWTnDqrlSHcvnVLSTJT9ICdBbhqioDLQM875mCRJkiRNpHuH7KzqcmFycd1tfmHplJJEtWzy0XWbdc8QBZUL6oLwbqXTSpIkSZoWMgvy7HpugfYFyFrINyGPKp1SkoaWxZBX1RNbdyumDAw/fGJVfJEkSZKkMclcyIuGmGPgzro7/N6lU0rS2GRXyAmQS4coqFxXDweyd4okSZKkkWQTyN9BruxycXEh5NUO2ZE09aUDeRjkk5C7urR3q+uedsdUPfIkSZIk6V5ZVM9pcl2Xi4nf171S7OYuaRrKAsizGHq55Gvr3im7lk4qSZIkqahsXndtv63LhcNZkCeVTihJkycH18MVlw4xd8oZdcHForIkSZI0c2QHyHu7XCisg5wGObJ0QkkqJ5vXk9H+fojeKZdB3gzZrnRSSZIkSRMmi6mW9GzPAbAWcirkkNIJJam/DNs7ZSXky5DDSqeUJEmS1DPZtB62c3vrAmAV5BRX2pGkkWRzyCuH6Z1ylkN9JEmSpCkt8+rVdm7oUjz5hBMlStJ45AjI5+ueKO1iyqWQ4yCblU4pSZIkaVQyq/5E9NIuc558GbJn6YSSNPVl+3qI5C1diil3QT4M2b10SkmSJElDytFDdDc/A3JQ6XSSNP1kPtVS8Od3aXsH5pw6unRKSZIkSffKwyG/7HIC/3PIQ0unk6TpLx3IYyCn1z3/2u3xOZDnV0MtJUmSJBWQPSBf73Ky/nvIX5dOJ0kzUx4A+S/I3V3a52shr4MsKp1SkiRJmiGyaT0Wf3nr5PyKegUJV4iQpOKyOeR4yJVdiim31u34VqVTSpIkSdNUOpDnQq7pcjL+asjc0gklSW2ZAzkWcvYQk9C+F7Jj6ZSSJEnSNJIDIT9rnXyvhpwM2aZ0OknSaOSh9WSz7WLKSsgpkL1KJ5QkSZKmsGxVL5W5pnXCfSZk/9LpJEnjkQfXRZN22z6woo8rqkmSJEmjlzn1/Ca3tE6wr6mW05QkTX25f92jcHWrrV9XF1OOKJ1QkiRJ6nN5GOSC1gn13ZC3QRaUTidJ6rXsUa/o054wPJAfQR5dOqEkSZLUZ7JFPXRnbesE+lTIbqXTSZImWrarV+1Z0qWY8kuLKZIkSRIAeRrkutYJ8++rSQglSTNLFkPe3mVIZyBnOMxHkiRJM1TuA/la6wR5GeRN1bwokqSZKwsh/wi5qUsx5TQnoJUkSdIMkU41QWxua50U/xTygNLpJEn9JJtCjofcOETPlANLJ5QkSZImSO5fTxTYPAleUq/G0ymdTpLUr7IIckKXOVPWQb5sEV6SJEnTSObUJ7/t1RdOhexUOp0kaarIZvXx5I7W8WRtXUzZs3RCSZIkaSNkf8h5rZPdqyBPKJ1MkjRVZVvI++u5tJrHl1WQkyG7lE4oSZIkjUFmQV4PWdH6pPDD1SeJkiRtrNwH8tHWsSb1/fdXq/1IkiRJfS27QM5sndBeBDm8dDJJ0nSUXSGfrHuiNI89t9Wr/MwvnVCSJEnqIs9tTfy3DvIxyCalk0mSprvcD3JK3fOxWUy5HHKsk5hLkiSpT2SL+sS1edJ6I+SJpZNJkmaa7As5rcuyyL+BPLx0OkmSJM1oORJyWetE9WuQbUonkyTNZDka8ocuxZQzIA8snU6SJEkzSuZCTmx1m74L8srSySRJqmQW5FmQK1uFlNX1Sj47lE4oSZKkaS/7dlm6+CzIHqWTSZK0oSyE/DNkaevYdSfkn5y7S5IkSRMkz2+dhK6qT0Bnl04mSdLwsgPkE3VPlGYx5RrIi6ueK5IkSdJGy3zIx7ssXXxw6WSSJI1N9oWc2mW+lHMgDymdTpIkSVNadoX8unWi+fVqVR5JkqaqHAH5Rev4tq5ecW770ukkSZI05eTJkCWNk8sVkNeUTiVJUm+kA3kB5NpWMWUJ5HjInNIJJUmS1Pcyp8vqO1dBDi+dTJKk3sum9XFvRauY8mfIY0unkyRJUt/KdpAftk4iT4VsVTqZJEkTK3tCTu8yX8qpkN1Kp5MkSVJfySMg1zdOGtfUn8y5YoEkaQbJkyCXtwop99THxPml00mSJKm4vKkumgycLF4POap0KkmSysgmddFkWauYcgnkmNLpJEmSVETmQz7bOkH8CWTH0skkSSovO9cr9nQb4nPf0ukkSZI0abId5KzWSeG/QWaXTiZJUn/J4+qJZpvHzOWQt0PmlU4nSZKkCZUHQa5oLV/84tKpJEnqX5lbL318Z5chPo8snU6SJEkTIk+GLG2c/N0IOaJ0KkmSpobsBPl/rULKWsh/QbYonU6SJEk9k+PrE72Bk77zIbuXTiVJ0tSTR0AubhVTboC8qHQySZIkbZSuE8ieDtm8dDJJkqauLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "74d5b606",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g13389.png\" align=\"left\" width=\"350\"/>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -315,7 +322,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"@everywhere workers() begin\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
|
||||
" nranks = MPI.Comm_size(comm)\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" root = 0\n",
|
||||
@@ -336,7 +343,24 @@
|
||||
"id": "97dc2886",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Gather"
|
||||
"### Gather\n",
|
||||
"\n",
|
||||
"TODO explain what this code does"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g13884.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABFEAAANWCAYAAAA2uxORAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13mGRlnf7/d01iYMg5BxUlKCiggiImDKiYV9c16xpXZY3o6q7u18SuusZVMcvq+jMHRFcRTBgQDICIiGSQnAcmz/3745xmTp+uTtXV/VR3v1/XVVdPV9c5dVd1z3PO+dQTOkiSJM24LAG2AbZufd0I2AxYCmxc/3sRsBWwGNi0vn9p/bPFwJaTfPKh/dwKrANWACuBNcDy+jE31V+X1/evrB+3DrgFuLl+zM317ZbGv2+GzqpJZpIkSbNAp3QASZI0F2QpsAuwM7A7sC0bCiNDRZJtG//erEzOGbOCYUUVbgauB64G/gZcA1wJXFt97dxWKKckSZoEiyiSJGkcWUZVGNmlvu1Wf921vu0MbF8sHqwC7pjkNquBJfW/N6XqnVLSHcBVVEWWdqHlKuBi4BLorC4VUJIkWUSRJEkAZBtg7/p298a/7wps0ecnuwm4ob7d2Pq6iqrXxhrgNjYMoxkaVnMTsLb+2QrorOxfrAwNIVoIbF7fuQWwANiEaqjRkvpnWzZuW7W+b96W0D/rqYorF49yuxI66/v4fJIkqcUiiiRJ80Y2Z0NxpF0s2XoKO14FXEF1gX9Z/fVquhdJbphfF/rZhA0Fle2oevBsT9V7Z0dgp/q2A9Vwp6lYDVzKhqLKRcB5wJ+oerGsm+L+JUma9yyiSJI052QRVYHkAODA+usBVENvJusOqgvzK+vb5fXXK7izcNK5tg+hRTaiKrDs0vi6A7AHsFd924Wqp8xkrQT+XN/OZUNx5a/QWTPl6JIkzRMWUSRJmtWyNXBvNhRKDgD2p1q9ZqJWAhfUt782/n0BdP7W17iaoiymmp9mr1Fuk52bZg3V7/pPjCiwWFyRJKnNIookSbNGtgUOrW8HURVMdpngxuuoCiR/YViRhAuAy6GTvsdVAVnGhoLK3YF9gf3qr5NZCno1cA7wu8bt7P7OQSNJ0uxjEUWSpIGUhcA+wMHAA4HDqS6EJ3LsvgX4I1Wvgj8BvwV+B53JrmCjOSVbUfVS2q/x9S71bSLWUhXhflvfzq2+dm7qf1ZJkgaTRRRJkgZCdqTqYXJY/fUQqhVhxrIWOJ+qx8BZ9e0c6FwxjUE152RbNvRW2Q+4J1VPp4n0XFlHVVgZ6q3yG+BMe6xIkuYqiyiSJBWRPYCHAw8FHkQ1eeiYG1BdrJ4O/Lr+ei50Vk1nSs1nuQtVMaV5224CG64GzgR+Cfyi+urkw5KkucEiiiRJMyLbAQ8BjqQamrPfOBssp+pZchrVheivoXPddCaUxpedqYaYDd2GhgSN5yo2/C2fBvx+fi11LUmaKyyiSJI0LbIF8GCq3iYPo5qDYrTj7nqquUtOB35F1dPkPC8yNTtkR6peKodQDUc7DNhinI1upPpb/0V9OwM6K6YzpSRJ/WARRZKkvsgiqglgH01VNDkYWDjag6km5TwVOAX4GXRunomU0vTLAqqi4eHAA6j+X+w1zkarqIoqp9S3M6CzdjpTSpLUC4sokiT1LMuoCiaPA54A7DDGgy9iw1CG7zn5q+aX7Ajclw0rTR0CbDTGBrdTFVV+VN9+5zLckqRBYBFFkqRJyd2Bo4HHUk0Iu2iUB15J1dOkvnUum5l80myQTdhQVHkAcASw2RgbXMWGnls/gs7l0x5RkiRJkjRZWQR5COS9kPMhGeW2EvIDyCsh9yidWppdsgjyAMi/Qn4CWTXG/7XU/xc/CnlKPf+QJEmSJKmMbAR5POQLkBvHuJC7BvLZ+kJurE/RJU1KNoE8CvKfkN9C1o3x/3A15BTIqyF7l04uSZIkSfNAFkIOh3wQct0YF2wX1o85sp5MVtK0y2b1/7njIGeO00vlQsjxkKMhS0onlyRJkqQ5IgshD6svuK4f5YJsBeT7kJdD9iidWBJAdoc8H/JlyM1jFFRuqh/zbMi2pVNLkiRJ0iyTBXWPkw9DrhpjfpNvQ/4BsmnpxJLGksV1MfR9kL+MUVBZCzkN8kbIvUqnliRJkqQBlgPri6zLx5hX4XuQ50K2LJ1WUq9yd8hrIKfW/6/HGvZzHOSQ0oklSZIkaQBkC8iL60+fu11Erat/dgxk+9JpJfVbltVzoxwP+dsYBZVL67mODod0SqeWJEmSpBmSTt21/wuQO0YpnPy0nuNkh9JpJc2ULIQ8EPJuyLljFFQuhrwHcj8LKpIkSZLmqOwG+de6i363C6M/Q46F7FI6qaRBkP0gb4WcM0ZB5RLIeyGHWlCRJEmSNMtlCeTvqFbPWdvlAug2yGeqLvqSNJrsC/k3yNnjDPn5r6qgIkmSJEmzRvaA/AfkulEudn4BeSFks9JJJc022QfyFshZYxRU/lIXXfYqnVaSJEmSRpGDISdA1nS5qLmhnjzygNIpJc0V2aueeHq0yakDObN+zDal00qSJEma97IU8gLIH7pcvKyFfAfyBMji0kklzWW5O+TNkPNGKabcAfkS5DGQRaXTSpIkSZpXsiPkbaMM2bmZajnSPUunlDQfZX/IcZBrRimoDPWMcz4mSZIkSdPpziE7q7tcmJxfd5tfVjqlJFEtm3xk3WbdPkpB5dy6ILxH6bSSJEmS5oQsgDytnlugfQGyDvItyMNKp5Sk0WUryEvria27FVPWQk6EPK4qvkiSJEnSpGQx5DmjzDFwS90dfp/SKSVpcrI75FjIBaMUVK6shwPZO0WSJEnSeLIx5J8gl3S5uPgT5GUO2ZE0+6UDeRDkk5Bbu7R3a+qedkdVPfIkSZIk6U7ZtJ7T5MouFxO/r3ul2M1d0hyUpZC/Y/Tlkq+oe6fsXjqpJEmSpKKyed21/YYuFw6nQY4unVCSZk4Orocr3jbK3Ckn1wUXi8qSJEnS/JEdIe/pcqGwHvJdyANKJ5SkcrJ5PRnt70fpnXIh5E2QHUonlSRJkjRtshXVkp7tOQDW1atTHFI6oSQNljF7p6yCfAVy/9IpJUmSJPVNNqmH7dzYugBYDTnBlXYkaTzZHPLiMXqnnOZQH0mSJGlWy5J6tZ2ruhRPPu5EiZLUixwG+Z+6J0q7mHIB5FWQzUqnlCRJkjQhWVB/InpBlzlPvgLZu3RCSZr9skM9RPK6LsWUWyEfhOxZOqUkSZKkUeXIUbqbnww5qHQ6SZp7shHVUvBnd2l7h+acOrJ0SkmSJEl3yoMhv+xyAv9zyOGl00nS3JcO5BGQk+qef+32+AzIM6uhlpIkSZIKyF6Qb3Q5Wf895DGl00nS/JR7QD4KWd6lfb4C8hrIpqVTSpIkSfNENqnH4q9onZxfXK8g4QoRklRcNoccA7mkSzHlFshxkK1Lp5QkSZLmqHQgz4Bc3joZvx7yMsji0gklSW1ZBHk65PRRJqF9D2Sn0iklSZKkOST3gfysdfK9BnI8ZNvS6SRJE5HD68lm28WUVZATIHcvnVCSJEmaxbJ1vVTm2tYJ96mQA0qnkyT1Iveuiybttn1oRZ+DSyeUJEmSZpEsquc3ua51gn15tZymJGn2y93qHoVrWm39+rqYcljphJIkSdKAy4Mg57ZOqJdD3gJZWjqdJKnfsle9ok97wvBAToEcWTqhJEmSNGCyRT10Z13rBPpEyB6l00mSplu2r1dfu6lLMeWXkIeXTihJkiQNgDwJcmXrhPn31SSEkqT5JVtB/rXLkM5AfuQwH0mSJM1T2Rny9dYJ8h2QN1TzokiS5q8sg7wGck2XYspJTkArSZKkeSKdaoLY3NA6Kf4p5B6l00mSBkk2gRwDubpLMeVkyH1KJ5QkSZKmSe5WTxTYPAm+qV6Np1M6nSRpUGVTyLFd5kxZD/mKRXhJkiTNIVlUn/y2V184EbJL6XSSpNkim9XHk5tbx5N1dTFl79IJJUmSpCnIAZCzWie7l0IeWzqZJGm2ynaQ99VzaTWPL6shx0N2K51QkiRJmoQsgLwWsrL1SeEHq08SJUmaquwM+XDrWJP6+/dBti6dUJIkSRpHdoOc2jqhPQ9yaOlkkqS5KLtDPln3RGkee26sC/oblU4oSZIkdZFntCb+Ww/5CGTj0skkSXNd7go5oe752CymXAT5eycxlyRJ0oDIFvWJa/Ok9WrI40onkyTNN9kP8t0uyyKfAXlw6XSSJEma1/IAyIWtE9WvQ7YtnUySNJ/lSMgfuhRTTobcs3Q6SZIkzStZDHlbq9v0rZAXl04mSVIlCyB/B7mkVUhZU6/ks2PphJIkSZrzsl+XpYtPg+xVOpkkSSNlGeTfILe1jl23QP4FsknphJIkSZqT8szWSejq+gR0YelkkiSNLTtCPl73RGkWUy6HPK/quSJJkiRNWTaCfKzL0sUHl04mSdLkZD/IiV3mSzkTcmjpdJIkSZrVsjvk160TzW9Uq/JIkjRb5TDIL1rHt/X1inM7lE4nSZKkWSePh9zUOLlcCXl56VSSJPVHOpBnQa5oFVNuhhwDWVQ6oSRJkgZeFnVZfedSuzlLkuambFIf91a2iil/hjyqdDpJkiQNrGwP+VHrJPJEyNalk0mSNL2yN+SkLvOlnAjZo3Q6SZIkDZQ8BPK3xknj2vqTOVcskCTNIzkaclGrkHJ7fUxcWjqdJEmSissb6qLJ0Mni3yBHlE4lSVIZ2bgumtzRKqb8BfKY0ukkSZJURDaCfLZ1gvgTyE6lk0mSVF52rVfs6TbE5y6l00mSJGnGZHvIaa2TwndDFpZOJknSYMmj6olmm8fMFZB/gywpnU6SJEnTKveCXNxavvi5pVNJkjS4srhe+viWLkN8Hlo6nSRJkqZFHg+5rXHydzXksNKpJEmaHbIL5H9bhZR1kI9BtiidTpIkSX2TY+oTvaGTvrMhe5ZOJUnS7JOHQM5vFVOugjyndDJJkiRNSdcJLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "4bdf9c02",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g13884.png\" align=\"left\" width=\"350\"/>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -347,7 +371,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"@everywhere workers() begin\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
|
||||
" nranks = MPI.Comm_size(comm)\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" root = 0\n",
|
||||
@@ -372,7 +396,9 @@
|
||||
"\n",
|
||||
"- `MPI.Isend`, and `MPI.Irecv!`\n",
|
||||
"\n",
|
||||
"There are *Non-blocking directives*. MPI also offers blocking directives with different blocking behaviors (communication modes). Blocking communication will be discussed later in the course.\n",
|
||||
"which are *non-blocking directives*.\n",
|
||||
"\n",
|
||||
"MPI also offers *blocking* directives with different blocking behaviors (communication modes). Blocking communication will be discussed later in the course.\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
@@ -381,29 +407,35 @@
|
||||
"id": "750fdacb",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Pingpong example"
|
||||
"### Example\n",
|
||||
"\n",
|
||||
"The first rank generates a message and sends it to the last rank. The last rank receives the message and multiplies it by a coefficient. The last rank sends the result back to the first rank."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g5369.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABCoAAAMNCAYAAABeW5XKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13nB11ufjxzySBJCQkBBI6hF5CBxGkKcUuYMOCAnpVrPdix3LvT67Xgli5eq8i9mvFLoKCAtJEpEPovSYktBBCCCnP74/vbHJydua03T2zZ/fzfr3mtXvOTnnmnLMzc575fp9vhiRJGobiWGBCzRPnQHZvzd/3B94M7A5sBmwAPAncCvwJOB2yuQ3WPwk4Gjg8X8cMIICHgX8CPwd+B1l0GP904HXAQcDOwMbAZGBMHueTwL3AbcAVwIWQ3drBdjYEjgKOBLYGNgQWAXOA24GzgJ9CtjifPwPeUbeS30D2SPvbliRJkiRp1Ij5EFEzHZE/vwnE2XV/K5oWQBxTsu43QsxtYR0Xpu21FfdkiC9DPNvC+uun2RDTWtzOOIgPQTzVwnrnQLw6X25Mwd/3aG8fJUmSJEkadYoSFbEdxP1tfvl/d916T2lz+VshNmgx5vUhru0gQVE7bdzCdtaE+EOb612RJzZMVEiSNMyNqzoASZLUknVIXTo2rXnuHuBm4GlgI2BvYI265f4b4lLIrof4GPCRmr8tBa4C5pKuCXYkdZ+otR3wP8BrG4cXGam7yG51f1gBXA/cnW/nSWAisC6wEzALGN943f228zNSl5V6DwEX5NtZE9gGOJjUhSYDvkh6zSRJkiRJUnv6tai4t+b3c4pbAcR6ED8oaDHwJ4jnQSzLHz8J8VGIqQXrOBTivoJ1HNAk3lcULPNziJlNlpuYL/sjiCXNW1TE2wq2Mwfidam1RL/5p0J8FWJ5Pu88W1RIkiRJktS2fomKvumreauCsuWyPEFQ3+3hzvz3hyF2b7Lt7SAW1a3ju02W+WHd/H/oYJ83gpjQ4O/rQTxet52HIOpbgRQte3z+OhS9piYqJEmSJElqrDBRcW7jJMXKZTeraT1Rn7B4cYvb/1Ldsk1GxYgr6+Z/aWvbaUd8sGCfDm1j+e+ZqJAkSZIkqSP9EhUrIGa1sfxFBV/I/9jG8s8pWH7zBvPfXTfvnq1vq+WYbq7bxlltLr8BqXuJiQpJkoaxgr6ckiRpGLoUspvamP/KgueadN9YzXWkYpu1dmww/5N1j/dtY1stiE2BHeqe/EF768geBtpMbkiSpG4zUSFJUm+4uM357617HMAlrS+eLQUerHtynQYLXF/3+DMQh7W+vaaeW/c4gHM7WM+fByEWSZI0hExUSJLUG+5oc/6FdY+fhGx+m+t4qu7xlAbz/qLu8TTgLxCXQrwTYss2t11vl7rH90C2oIP1XDfAOCRJ0hAbV3UAkiSpJU+0Of+yAS4P/bt+rNFg3rOAc4D6Yp375RMQt5NadVwC/A2yu9qIZVrd4/vbWHYwlpMkSV1iiwpJknpDfeKhXfVJh0GWBfB6Gnet2BZ4K6lWxp0Qt0F8CmKTFjZQ3+2kvsVIqzpphSFJkrrIRIUkSRok2QLgZcAbaa2LxbbAScAdEJ+DaNRiY2zd4+Udhdj5cpIkqUtMVEiSpEGUBWQ/h2x3YDfgROBP9B8VpNYE4OPAnyHGl8xT33WlUb2MRqZ2uJwkSeoSExWSJGmIZNdDdgpkLwPWJY3c8QngAuDZggUOAT5TsrL6RMWGHQbV6XKSJKlLTFRIkqQuyJZDdgVkn4fsEGAz4HP0r73xPoii1hK31j3eFmJiB4Hs1sEykiSpi0xUSJKkCmTzIPsk8P66P0wADixY4PK6x2NL5mvm4A6WkSRJXWSiQpIkVel0YFHdczML5rsDeKjuuWPa21RMBl7T3jKSJKnbTFRIkqQKZc8C8+uerB/hg3z40+/VPfkGiF3b2NjHgbXbiU6SJHWfiQpJkjRAkaWpo2XXATaue/L+kplPZ/UinOOA/4OY1sJ2Xgh8uIMAJUlSl5mokCRJA7UecAPEWyDWbHPZzwK1yywBLiyeNbsvn7/WrsAFELsXLxMZxPHAb/PtLG4zPkmSJEmSBDEfImqmI9pc/ti65W/vIIar69bxnpL5ptfM8yjEdyFeATGjZP4MYh+I39etPyC+0ySmNSH+WbDcMojzID4BcVyKNb6Y9nu1+T5QsOwe7b82kiRJkiSNKj2bqKif7of4B8SfIP4CMRvi8ZJ5b8+7gjSLa12Iaxpss2w6HWJswfM7tv/aSJKkoTKu6gAkSdKItmk+NXMV8ArInmg+a/YYxMHAl4G3As3qYzwLfBr4HMXFNBe0EJ8kSeoSa1RIkqSBWgC8E/gdsLDNZe8C3gvsA9nc1hfLnoDsbcDzgG/Sf+jSAG4DvgTsCNln85FDigpvtpAckSRJ3dJhhW5JkjS0YjdWb/l4Z2utDVYuvx6wRc0TSyCb3WYMOwJr1TxxH2T1Q4nWLzMWmAXsA2wPbA1Mr1nP48DDwHXApcBleQJhEMREYAPSjZiHIVtUMM/LgT/WPDEPsg0GZ/uSJEmSJEltiZPr6lOcVXVEkiRpdXb9kCRJo0SMB95S9+Q/KghEkiRJkiQp/rOuNcVyiK2rjkqSJEmSJPW0OCCvwdHOMm+DWFaXqDh7aOKTJEmSJEmjSHwD4imIb0G8IC/gWTbv3hC/rktQBMTTELO6FrIkSWrZuOazSJIkDTuTSEOivhN4GuJ64D7SUKMTSSON7AWsX7L8ByG7qRuBSpIkSZKkES2+UdBCotVpCcRxVe+BJEmSJEkaMeJYiHvbTFAsh/ip3T0kSRr+sqoDkCRJ6kzsDOwD7A5sAWwETCUNv74AeAy4E7gEOB+yB6uJU5IkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZKkAlnVAUiSpCrFOqy6HpgMrJH/PgGYmP++Rv63Pk8A0eIGngSWA08DS4ClkD01kIglSdLIZqJCkqSeEWsD6+bT9PznVFISYSKwdj5NzJ+bAqyVT+vkz08EpnU78hKP5z8XAsuAZ4DFwCJSgmNhPj2e/6x9bkE+9T3/OGSLuxm8JEkaGiYqJEmqRIwDNgA2BjbMpxmsSkSsC6xX9/sahatSn0XAI8C8/Gff9Gj+3Pz895q/Za22DJEkSV1iokKSpEEVawKbsCoBUftzg/xvGwDr43m4akuBucD9wEPAg/n0UP7cHOABW2pIktRdXiBJktSWGA9sDmwBzMynLWqmjYExlYSmofIYq5IXDwB3AXevmrL5FcYmSdKIY6JCkqR+YjqwfT5tw+rJiI0YGefPvnoQkOpDLKz52+M1v/fVj2jV2sC4uuf6inSOJ9XLGEOqrTFSLGS1xMVqiYy7bJEhSVJ7RsKFliRJHYiJwHb5tC2rEhPbMXyKTdZ7lHR3v296lNULTD5NqtOwgJSEeJqUdFicT0+kv2fPdj3yUjEFGEsq8jkBWJOU7JiaT1NYVSR0Cqko6JSax2vnz62X/z4cPQjcAtwK3JT/vBWy+yuNSpKkYcpEhSRphIupwC75tDMpKbEdqftGlefBZ0kFHh/Ifz7IqmKP9cmI/HcLPzYW40mjoUwnFSadUfN4vfzx+jWPN6DabjoLSUmLW4Cba36/fXglkyRJ6i4TFZKkESLGkVpE7ALsyqrkxMwuB7KUlHx4gFSMsXaaS6p18LB1DYaDGEcqdLoZqUvPpqwqhLppzc+1uhzYMuB24Hrg2vzn9ZA90OU4JEmqhIkKSVIPivWBPVg9ITGL1G1gqD0L3AfcC9xT87Pv9wchW96FONQ1sQ4pgbEZqU7JljXTVqThY7vhUeC6fMqTF8y29YUkaaQxUSFJGuZiGrAXsDfwnHzafIg3ugC4LZ9uIRVH7EtIzIFsxRBvXz0lprIqabFlwe/jh3DjS0ldRq4DrgL+CVwD2dNDuE1JkoaUiQpJ0jASk4HdSYmJvmkHhqaOwFLScJN3kQoc3pj/no/YYD0IDZbYmNTiZytgp/z3nUjdTYbCclLy4qqa6Z+2vJAk9QoTFZKkisQY0pe1A4DnkVpKbM/gJyWWkBIRNwCzSQmJ24B7IGtn2E1pkMV0YEdSMm57UgJje1L3ksH+P1gEXE1qcXEFcDlk9wzyNiRJGhQmKiRJXRITSd03DgD2B/YjDSs5mO4hJSRuIPXfvwG4zYSEektMIo1Qs1vNtAtpONbBNA+4FLgYuITUZcT/FUlS5UxUSJKGSEwnJST6EhN7MXjFLheT+uRfw6qExA2QPTlI65eGmchI9S76Ehe75j+3ZPCu554C/kFKWlxManWxaJDWLUlSy0xUSJIGSWwIHAq8gJSc2GGQVryUlIi4Argyn2Z751cCiCmkpMVewHNJrZa2HaSVLyV1F+lLXFwK2SODtG5JkkqZqJAkdSimAs8nJScOJdWbGKjlpHoSV9ZM10G2ZBDWLY0SsS6rkhZ9PzcYjBWTarz8NZ8uhOypQVivJEmrMVEhSWpRTCB14TiElJh4DjB2gCt9BPg76Y7tZaQ+8jY1lwZdzCQlLfqmPYHJA1zpUuByViUuLrelkyRpMJiokCSViDGkZERfi4n9gQkDXOkdpOJ9l+Q/b3EYUKkKMQ7Yg/R/fVD+c/0BrnQhcCEpaXEeZLMHuD5J0ihlokKSVCPWAV4EvBx4CQP74rKMVOyyJjGRzR1wiJKGSOxASlgcSKozs/UAVzgX+AtwNnAOZI8PcH2SpFHCRIUkjXoxi5SYeBnpy8m4Dle0nFTw8vx8+ofdOKReFhuTkhZ9rS52AcZ0uLLlpG5eZ6XJ1haSpHImKiRp1IkJpITEYcCRDGx0jrtY1T/9r94xlUayWI9Uo+Yw0jFk1gBWdi9wDunY8SeLckqSJEmjTqwPcTzEmRCLIKLD6R6I70IcnQ9HKmnUiq3y48ovLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "b479bbd4",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g5369.png\" align=\"left\" width=\"350\"/>\n",
|
||||
"</div>\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"execution_count": null,
|
||||
"id": "dccc3f6a",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" From worker 2:\tI am sending: 20\n",
|
||||
" From worker 4:\tI have received: 20\n",
|
||||
" From worker 4:\tI am sending: 80\n",
|
||||
" From worker 2:\tI have received: 80\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"@everywhere workers() begin\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" nranks = MPI.Comm_size(comm)\n",
|
||||
" snder = 0\n",
|
||||
@@ -425,7 +457,8 @@
|
||||
" MPI.Wait(req)\n",
|
||||
" msg = buffer[]\n",
|
||||
" println(\"I have received: $msg\")\n",
|
||||
" msg = msg*(rank+2)\n",
|
||||
" coef = (rank+2)\n",
|
||||
" msg = msg*coef\n",
|
||||
" println(\"I am sending: $msg\")\n",
|
||||
" buffer[] = msg\n",
|
||||
" req = MPI.Isend(buffer,comm;dest=snder,tag=0)\n",
|
||||
@@ -436,378 +469,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8d800917",
|
||||
"id": "162682c1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Message Passing Interface (MPI) is a standardized and portable library specification for communication between parallel processes in distributed memory systems.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Julia offers a convenient way to work with MPI for creating efficient parallel and distributed applications. In this tutorial, you will learn how to use MPI from Julia to perform parallel computing tasks.\n",
|
||||
"\n",
|
||||
"## MPI launches separate Julia instances\n",
|
||||
"When you run an MPI-enabled Julia script, MPI takes care of spawning multiple instances of the Julia executable, each acting as a separate process. These workers can communicate with each other using MPI communication functions. This enables parallel processing and distributed computation. Here's a summary of how it works:\n",
|
||||
"\n",
|
||||
"-- TODO: insert picture here --\n",
|
||||
"\n",
|
||||
"- **MPI Spawns Processes**: The `mpiexec` command launches multiple instances of the Julia executable, creating separate worker processes. In this example, 4 Julia workers are spawned.\n",
|
||||
"\n",
|
||||
"- **Worker Communication**: These workers can communicate with each other using MPI communication functions, allowing them to exchange data and coordinate actions.\n",
|
||||
"\n",
|
||||
"- **Parallel Tasks**: The workers execute parallel tasks simultaneously, working on different parts of the computation to potentially speed up the process.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## Installing MPI.jl and MPIClusterManagers Packages\n",
|
||||
"To use MPI in Julia, you'll need the MPI.jl package, and if you intend to run MPI programs in a Jupyter Notebook, you'll also need the MPIClusterManagers package. These packages provide the necessary bindings to the MPI library and cluster management capabilities. To install the packages, open a terminal and run the following commands:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "3cb5f151",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"using Pkg\n",
|
||||
"Pkg.add(\"MPI\")\n",
|
||||
"Pkg.add(\"MPIClusterManagers\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ed45a4b2",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
" <b>Tip:</b>\n",
|
||||
"The package <code>MPI.jl</code> is the Julia interface to MPI. Note that it is not a MPI library by itself. It is just a thin wrapper between MPI and Julia. To use this interface, you need an actual MPI library installed in your system such as OpenMPI or MPICH. Julia downloads and installs a MPI library for you, but it is also possible to use a MPI library already available in your system. This is useful, e.g., when running on HPC clusters. See the documentation of <code>MPI.jl</code> for further details.\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7a36916e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Writing a HelloWorld MPI Program in Julia\n",
|
||||
"Let's start by creating a simple MPI program that prints a message along with the rank of each worker. \n",
|
||||
"\n",
|
||||
"Create a new Julia script, for example, `mpi_hello_world.jl`:\n",
|
||||
"\n",
|
||||
"```julia\n",
|
||||
"using MPI\n",
|
||||
"\n",
|
||||
"# Initialize MPI\n",
|
||||
"MPI.Init()\n",
|
||||
"\n",
|
||||
"# Get the default communicator (MPI_COMM_WORLD) for all processes\n",
|
||||
"comm = MPI.COMM_WORLD\n",
|
||||
"\n",
|
||||
"# Get the number of processes in this communicator\n",
|
||||
"nranks = MPI.Comm_size(comm)\n",
|
||||
"\n",
|
||||
"# Get the rank of the current process within the communicator\n",
|
||||
"rank = MPI.Comm_rank(comm)\n",
|
||||
"\n",
|
||||
"# Print a message with the rank of the current process\n",
|
||||
"println(\"Hello, I am process $rank of $nranks processes!\")\n",
|
||||
"\n",
|
||||
"# Finalize MPI\n",
|
||||
"MPI.Finalize()\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6caa8d74",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### MPI Communicators\n",
|
||||
"In MPI, a **communicator** is a context in which a group of processes can communicate with each other. `MPI_COMM_WORLD` is one of the MPI standard communicators, it represents all processes in the MPI program. Custom communicators can also be created to group processes based on specific requirements or logical divisions. \n",
|
||||
"\n",
|
||||
"The **rank** of a processor is a unique identifier assigned to each process within a communicator. It allows processes to distinguish and address each other in communication operations. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "19f41e38",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Running the HelloWorld MPI Program\n",
|
||||
"\n",
|
||||
"To run MPI applications in parallel, you need a launcher like `mpiexec`. MPI codes written in Julia are not an exception to this rule. From the system terminal, you can run\n",
|
||||
"```\n",
|
||||
"$ mpiexec -np 4 mpi_hello_world.jl\n",
|
||||
"```\n",
|
||||
"In this command, `-np 4` specifies the desired number of processes. \n",
|
||||
"But it will probably not work since the version of `mpiexec` needs to match with the MPI version we are using from Julia. You can find the path to the `mpiexec` binary you need to use with these commands\n",
|
||||
"\n",
|
||||
"```julia\n",
|
||||
"julia> using MPI\n",
|
||||
"julia> MPI.mpiexec_path\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"and then try again\n",
|
||||
"```\n",
|
||||
"$ /path/to/my/mpiexec -np 4 julia mpi_hello_world.jl\n",
|
||||
"```\n",
|
||||
"with your particular path.\n",
|
||||
"\n",
|
||||
"However, this is not very convenient. Don't worry if you could not make it work! A more elegant way to run MPI code is from the Julia REPL directly, by using these commands:\n",
|
||||
"```julia\n",
|
||||
"julia> using MPI\n",
|
||||
"julia> mpiexec(cmd->run(`$cmd -np 4 julia mpi_hello_world.jl`))\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Now, you should see output from 4 ranks.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0592e58c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Running MPI Programs in Jupyter Notebook with MPIClusterManagers\n",
|
||||
"If you want to run your MPI code from a Jupyter Notebook, you can do so using the `MPIClusterManagers` package.\n",
|
||||
"\n",
|
||||
"1. Load the packages and start an MPI cluster with the desired number of workers:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "cf66dd39",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"using MPIClusterManagers\n",
|
||||
"# Distributed package is needed for addprocs()\n",
|
||||
"using Distributed\n",
|
||||
"\n",
|
||||
"manager = MPIWorkerManager(4)\n",
|
||||
"addprocs(manager)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d40fe3ee",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"2. Run your MPI code inside a `@mpi_do` block to execute it on the cluster workers:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0a51d1f2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"@mpi_do manager begin\n",
|
||||
" using MPI\n",
|
||||
" comm = MPI.COMM_WORLD\n",
|
||||
" rank = MPI.Comm_rank(comm)\n",
|
||||
" println(\"Hello from process $rank\")\n",
|
||||
"end\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "38ed88c1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"MPI is automatically initialized and finalized within the `@mpi_do` block.\n",
|
||||
"\n",
|
||||
"3. Remove processes when done:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "e0b53cc1",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"rmprocs(manager)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5466a650",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Point-to-Point Communication with MPI\n",
|
||||
"MPI provides point-to-point communication using blocking send and receiving functions `MPI.send`, `MPI.recv`; or their non-blocking versions `MPI.Isend`, and `MPI.Irecv!`. These functions allow individual processes to send and receive data between each other.\n",
|
||||
"\n",
|
||||
"### Blocking communication\n",
|
||||
"\n",
|
||||
"Let's demonstrate how to send and receive with an example:\n",
|
||||
"\n",
|
||||
"```julia\n",
|
||||
"using MPI\n",
|
||||
"\n",
|
||||
"MPI.Init()\n",
|
||||
"\n",
|
||||
"comm = MPI.COMM_WORLD\n",
|
||||
"rank = MPI.Comm_rank(comm)\n",
|
||||
"\n",
|
||||
"# Send and receive messages using blocking MPI.send and MPI.recv\n",
|
||||
"if rank == 0\n",
|
||||
" data = \"Hello from process $rank !\"\n",
|
||||
" MPI.send(data, comm, dest=1)\n",
|
||||
"elseif rank == 1\n",
|
||||
" received_data = MPI.recv(comm, source=0)\n",
|
||||
" println(\"Process $rank received: $received_data\")\n",
|
||||
"end\n",
|
||||
"\n",
|
||||
"MPI.Finalize()\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d4dfe654",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In this example, process 0 sends a message using `MPI.send`, and process 1 receives it using `MPI.recv`.\n",
|
||||
"\n",
|
||||
"### Non-blocking communication\n",
|
||||
"\n",
|
||||
"To demonstrate asynchronous communication, let's modify the example using `MPI.Isend` and `MPI.Irecv!`:\n",
|
||||
"\n",
|
||||
"```julia\n",
|
||||
"using MPI\n",
|
||||
"\n",
|
||||
"MPI.Init()\n",
|
||||
"\n",
|
||||
"comm = MPI.COMM_WORLD\n",
|
||||
"rank = MPI.Comm_rank(comm)\n",
|
||||
"\n",
|
||||
"# Asynchronous communication using MPI.Isend and MPI.Irecv!\n",
|
||||
"if rank == 0\n",
|
||||
" data = \"Hello from process $rank !\"\n",
|
||||
" request = MPI.Isend(data, comm, dest=1)\n",
|
||||
" # Other computation can happen here\n",
|
||||
" MPI.Wait(request)\n",
|
||||
"elseif rank == 1\n",
|
||||
" received_data = Array{UInt8}(undef, 50) # Preallocate buffer\n",
|
||||
" request = MPI.Irecv!(received_data, comm, source=0)\n",
|
||||
" # Other computation can happen here\n",
|
||||
" MPI.Wait(request)\n",
|
||||
" println(\"Process $rank received: $(String(received_data))\")\n",
|
||||
"end\n",
|
||||
"\n",
|
||||
"MPI.Finalize()\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "024db538",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In this example, process 0 uses `MPI.Isend` to send the message asynchronously. This function returns immediately, allowing the sender process to continue its execution. However, the actual sending of data is done asynchronously in the background. Similar to `MPI.Isend`, `MPI.Irecv!` returns immediately, allowing the receiver process to continue executing. \n",
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-warning\">\n",
|
||||
"<b>Important:</b> In asynchronous communication, always use <code>MPI.Wait()</code> to ensure the communication is finished before accessing the send or receive buffer.\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## Collective Communication with MPI\n",
|
||||
"MPI provides collective communication functions for communication involving multiple processes. Let's explore some of these functions:\n",
|
||||
"\n",
|
||||
"- MPI.Gather: Gathers data from all processes to a single process.\n",
|
||||
"- MPI.Scatter: Distributes data from one process to all processes.\n",
|
||||
"- MPI.Bcast: Broadcasts data from one process to all processes.\n",
|
||||
"- MPI.Barrier: Synchronizes all processes.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Let's illustrate the use of `MPI.Gather` and `MPI.Scatter` with an example:\n",
|
||||
"\n",
|
||||
"```julia\n",
|
||||
"# TODO: check if this runs correctly\n",
|
||||
"using MPI\n",
|
||||
"using Random\n",
|
||||
"\n",
|
||||
"MPI.Init()\n",
|
||||
"\n",
|
||||
"comm = MPI.COMM_WORLD\n",
|
||||
"rank = MPI.Comm_rank(comm)\n",
|
||||
"size = MPI.Comm_size(comm)\n",
|
||||
"\n",
|
||||
"# Root processor generates random data\n",
|
||||
"data = rand(rank == 0 ? size * 2 : 0)\n",
|
||||
"\n",
|
||||
"# Scatter data to all processes\n",
|
||||
"local_data = Vector{Float64}(undef, 2)\n",
|
||||
"MPI.Scatter!(data, local_data, comm, root=0)\n",
|
||||
"\n",
|
||||
"# Compute local average\n",
|
||||
"local_average = sum(local_data) / length(local_data)\n",
|
||||
"\n",
|
||||
"# Gather local averages at the root processor\n",
|
||||
"gathered_averages = Vector{Float64}(undef, size)\n",
|
||||
"MPI.Gather!(local_average, gathered_averages, comm, root=0)\n",
|
||||
"\n",
|
||||
"if rank == 0\n",
|
||||
" # Compute global average of sub-averages\n",
|
||||
" global_average = sum(gathered_averages) / size\n",
|
||||
" println(\"Global average: $global_average\")\n",
|
||||
"end\n",
|
||||
"\n",
|
||||
"MPI.Finalize()\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "e65cb53f",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"using MPI\n",
|
||||
"using Random\n",
|
||||
"\n",
|
||||
"# TODO: check if this runs correctly\n",
|
||||
"\n",
|
||||
"MPI.Init()\n",
|
||||
"\n",
|
||||
"comm = MPI.COMM_WORLD\n",
|
||||
"rank = MPI.Comm_rank(comm)\n",
|
||||
"size = MPI.Comm_size(comm)\n",
|
||||
"\n",
|
||||
"# Root processor generates random data\n",
|
||||
"data = rand(rank == 0 ? size * 2 : 0)\n",
|
||||
"\n",
|
||||
"# Scatter data to all processes\n",
|
||||
"local_data = Vector{Float64}(undef, 2)\n",
|
||||
"MPI.Scatter!(data, local_data, comm, root=0)\n",
|
||||
"\n",
|
||||
"# Compute local average\n",
|
||||
"local_average = sum(local_data) / length(local_data)\n",
|
||||
"\n",
|
||||
"# Gather local averages at the root processor\n",
|
||||
"gathered_averages = Vector{Float64}(undef, size)\n",
|
||||
"MPI.Gather!(local_average, gathered_averages, comm, root=0)\n",
|
||||
"\n",
|
||||
"if rank == 0\n",
|
||||
" # Compute global average of sub-averages\n",
|
||||
" global_average = sum(gathered_averages) / size\n",
|
||||
" println(\"Global average: $global_average\")\n",
|
||||
"end\n",
|
||||
"\n",
|
||||
"MPI.Finalize()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "dfd5da9e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In this example, the root processor generates random data and then scatters it to all processes using MPI.Scatter. Each process calculates the average of its local data, and then the local averages are gathered using MPI.Gather. The root processor computes the global average of all sub-averages and prints it."
|
||||
"<b>Important:</b> In non-blocking communication, use <code>MPI.Wait()</code> before modifying the send buffer or using the receive buffer.\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
Reference in new issue
Block a user