Adding asp notebook

notebooks/asp.ipynb

@@ -0,0 +1,726 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "2133c064",
+   "metadata": {},
+   "source": [
+    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/39/VU_logo.png/800px-VU_logo.png?20161029201021\" width=\"350\">\n",
+    "\n",
+    "### Programming large-scale parallel systems\n",
+    "\n",
+    "\n",
+    "# All pairs of shortest paths\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a7b64d5a",
+   "metadata": {},
+   "source": [
+    "## Contents\n",
+    "\n",
+    "In this notebook, we will learn\n",
+    "\n",
+    "- How to parallelize the Floyd's algorithm\n",
+    "- How communication can affect the correctness of a distributed algorithm\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ade31d26",
+   "metadata": {},
+   "source": [
+    "## The all pairs of shortest paths (ASP) problem\n",
+    "\n",
+    "Let us start by presenting the all pairs of shortest path (ASP) problem and its solution with the [Floyd–Warshall algorithm](https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm).\n",
+    "\n",
+    "### Problem statement\n",
+    "\n",
+    "- Given a graph $G$ with a distance table $C$\n",
+    "- Compute the length of the shortest path between any two nodes in $G$\n",
+    "\n",
+    "We represent the distance table as a matrix, where $C_{ij}$ is the distance from node $i$ to node $j$. Next figure shows the input and solution (output) of the ASP problem for a simple 4-node directed graph. For example, the solution of the problem contains the minimum distance from node 2 to node 3, which is $C_{23}=8$ as highlighted in the figure.\n"
+   ]
+  },
+  {
+   "attachments": {
+    "g37385.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAABGgAAAGgCAYAAAD2CVFOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N15fBT1/cfx1+zm4iYcAipCCIfggZUqihYFxIpnrYr1bNVqb6+qJEF/jRZyeFXb2npWrUetZ+utXFqtWhUVxQOBBBTkkpsQcuzO749vwu7ObpLdZHdnd/N+Ph77MDPz3ZnPhDg7+5nv9/O1EBEREZFY9QMuClpeAfzTnVBERCQOxgAnAscAewF7AL2BDU2vL4CXgReBdS7FKBnOcjsAERERkTS0H7A4aHkOcKxLsWSa8cDYoOX7AJ9LsYhI5hsP3AwcGWV7H3A/8H/AmkQF1U4jgaODlp8GvnUnFFedCOzZ9PMm4EkXYxERERGRBNsPsINer7obTkapIPR3m+duOCKSobKAvwB+Qq850b62A2cnPerW/YTQGA92NRr3zCPwO/jE5VhikuV2ACIiIiIiIiJJ1AX4B3CKY/1W4FnMUKavgR1AX2A4cCowmcB36O7AQ5ihULclPmQREREREYlEPWgSRz1oRCTRHif0OuMH7gH6tPG+A4C3CO9N8+OERRqbn6AeNJDGPWg8bgcgIiIiIiIikiQXA2cELdvApU3rN7Xx3k+Ao4B/O9bfAYyKV4DSeSlBIyIiIiIiIp3BHsAfHOvKgT/HsI8GYDrwXtC6bph6NiIdoho0IiIiIol3KNCz6ectwPtB2wqB84HRmKldGzC1DxZgZp7YFuUxxgH5TT/vAN4J2vYd4EfACGBg0/aVwHOYWgv1UR7je0Bu08/riL7r+ADM0IBm7xJ6Xl2BCU0/D3W8d3Ir8S0G1kYZg4jILzDJlGafAze0Yz/1mB437xP4Tj0ZM6TogxbeczCBIVTOa3RrugOHBS1/SvjsUVMwMzSPcawfT8vDtqqB5Y51kwBv08/fAJ81/expOsYpwBBMomsDsBR4CvgvpidSW3IxnyPNlmI+i6IxounYzeY6tg8E9m/6Oficu2OmTm/JW8DOKGMQERERkRQUaw2a94Pavtm0rgemwGRrM4isA06PMqYFQe9b1LRuEOYmtrWZSJYSOi1ra1YFve8fUb4HTHIo+JiHOLYPbyPGll5nxRCDiHRuuZiEbjyvIY869vf3VtrOCWr3UQzHGOs4xvkR2tQT+/Xz+gj72RG0/d6mdQcDS9rY1+uYBEpb9nS877dRvKfZzY73Op3VRowtvYbHEEPCaYiTiIiISPL1x/QiORfz1LMlewD/xCQ4YjUU8yR3ShvthmOSOO05hohIuhiP6c3XbBPwTAf3eZ9j+UQy6zv2VEzvmJFttJsI/A/TW1M6QEOcRERERJIrG3gC2Bfz9O4V4EXgK0yy5gDgIgJduT3AX4HXiH44Tw4msTOw6RhPAY8BVZhZkQ4EfkbgZtqLefK7GnijvSfWARuBoqafp2GKcDa7Fmhs4X2xPIUWkc7tSMfyHGBXB/e5ANiO6REJZpjpfiR/5qCZmM+KcYQWQP4T5roeyX/b2OdewMOYz4xdmGTU85jhVT0xv89fAIOb2udjPs8OxJ2hpx8R+By5GDN8GExPVGfdoWAbExmUiIiIiCReR4Y4Nb/WEKi74tQFUxsmuP3v2jhG8BCn5tcuzBPdSLKAWx3tv6D1aa0TNcQpmKbZFpFEeI7Qa8tVcdrva479/qyFdokc4tTsJ462sU6zHTzEqfm1GvOZF0kv4CVH+yda2X8ihzgF0zTbIiIiIhK1WuBYTHHClrafj3ky2+zMdhznl5gnnpE0Ym6Og7v4jwLOa8dxRERS3d6O5cVx2q9zP87jpLNGTJL/0xa2b8XUSVsStO504KAEx5WxlKARERERSb7baPup3nrCkyc9W2gbybvA/W20sYHLCJ0l6ZIYjiEiki6csxltjtN+N7VxnHR2N/BhG21qgKsd6/Q50k5K0IiIiIgkn7OwZEveDvrZQ2BMfTT+RtvdwMFM6f1K0PJ3CS2kKSKSCfIdy1vjtF/nfjIpQRPtZ9ULhNa6OSEBsXQKStCIiIiIJNcaYHmUbVc5lnvHcJw5MbR11tBprUaMiEg68juWW5tBLxZex7IvTvt120bMTIDR8GNmA2y2DzAo7hF1AkrQiIiIiCRXLLNb1DiWu0X5vh1AdQzHcQ63GhXDe0VE0oFzSFMsCe/WOIeexmvolNtiLa7rbD8iXoF0JkrQiIiIiCTXzhjaOocoRfvE99sI723NeseycyiAiEi6c9aK6Run/fZv4zjpakOM7fU5EgdK0IiIiIhknliSQGB63ATrEa9ARERSxArH8tg47de5n1h6L6YyfY64QAkaERERkczTJcb2zqFTzhttEZF091/HcjxqbeUQnqBxHidddY2xvT5H4kAJGhEREZHM05fYCmA6u+hHU0Mhlv3HeqMvIhJvbzqWpwK9OrjPE4C8oOW1wNIo3pcO189Yh4C153MkFp3ic0QJGhEREZHM0xMYEkP7/R3LX7bQLrjLeyw3ywNjaCsikggLCR1+1BU4u4P7vNix/FQrbdt7/XRrNqQDOtg+UqLKOWwqHX4PSaUEjYiIiEhmmhJD22Mcy++20G5r0M97xbD/CTG0bXQs58bwXhGRlviAPzrWXUf7Z3M6BpgWtOwHbm+lffD1cxDRfxc/PIaYGhzLHbl+9gcOjLKtRehnzirgmwjtthE63Xm0nyMWcFiUbSH0cyStPkOUoBERERHJTBdE2W4Qppt+s49oeSrw4Cei+xFdEcjBwLFRxgLhdQv6xfBeEZHW3Efo7ESDMEmVWIYcgZmh6K+OdU/R+vCm4G3diK6HSi5wbgxxxfv6Ge3nyFRgn6Dll1po5weqgpajTT6dSGw9MbcH/ZxWnyFK0IiIiIhkpiOAM6NodxOhTxjvaqXte0E/5wLntbFvC7gDyI4ijmZfO5YLYniviEhrtgMXAXbQuvOB24g+SdMbeBUYHrRuHfCbNt73nmP5p1Ec6/fElphwXj+HxvDeSH4BjG6jTS7mcyTY3a20D/49HEjbxZp7ALe20cZpVdDPvdGU3yIiIiIZbT/MDX7z69U22r8f1NZZqLI1RzuOc1IrbRc42tqYp6mTW2jvAW5wtF9G6zUBBmO6jje33wQc1ELb7sBDEWKyaf2GfJSj7SuEzw4iItIRtxN+XXodGNPG+04BvnK8rxE4Lopj5gDrg97XgOkZEkk25vrsjxDn+a0cIxdT56W57WfEluDZEeF4VUBhC+27Ak872v+7jWOc4mi/GNijhbZ7A+9EiMluoX2zcxxtbwG8bbxHRERERNJUqidolmCGKtmYugsPYmol7IvpVv9j4C3HvhuJrm6NM+lSC/wZ+CEwqem/NwJrmrbXA3c63tPWE9P/Odpvx9TF+TfweNDriCjiFRFx8mDq0Ti/9Psx159STE+bM4BfY3pwVEdoX4u55kWrxPF+H/AIcBbm+nkS8DtMofbmNn9yvKe1BA2Y631w+13Ah8DzhF4/p0d4b3CCZg4mCd+c7L8JOArTc+i7wGWYpH7wsTZjkiqtyQI+cbxvPSYhdULT7+Ec4B6gpmn72qaYo03Q9MDUu3Ee401MQin496Ai9iIiIiJpLtUTNIuAEcBGx/tbevmIvtZAP8wT1Wj3ezHwI8f6thI044j8JNf5OivKmEVEIrmCQBIg1tcq4MgYj5eN+QyI9hg3A2Md69pK0AwCVkex7+sjvDf4unsv5jOnPspYtxF9Id/vYhLv0ex3OyYZf7NjfVvOx3wGtbX/4S3twA2qQSMiIiKSmZZiEh3OugdOX2O62d8f5X6/BSbSdqJpPabXzj1R7jfYQuBg4GHCi16KiMTLH4CRmOtfXZTv+RbT22MMsSXcwQxrOg54oo12OzE9eK6Kcf9gei9+B9PzZkMbbdvyHGaY7Ko22r2Hma3vnSj3+z6msPCyNtp9gkno/zfK/Qb7O6ZX6EuEz26VsmKtVi0iIiIipiZK8OwTm4APWml/KNCz6ectmJvTaORjkizNPsJ8OYhkAabHDcDHmKeuYO73JmIKBhcCe2KedK7A3Hw/i/ky0B7HYuoJ7A/0xXRvXwm8iOlGvqup3QBCZyx5tymGaHiAYU376OLYtpiWZ5wSEYlFN8zU2VMwMxLtgblub8RcZ77EfNn/H6ZnRkcdCpyG6U2yByYZvQqYD/wD81kBpp5XcM+UTzFJmGgNxgw7ctbyqgaWO9btCGp3H4FCxrmY4UcnA0MwPSk3AZ8DzwBzad/vJAfzOzgec53vhSm4XIUZfjSPwLTcI5qO3WxuDMfJxiTi+hFetP4t2v8ZKCIiIiISkXOIk4iISCycQ5wkyTTESURERERERETEZRriJCIiItKyHsAbbgcRpUJMV3gwQ4uWuBiLG2ZgpuQWEWlLLmaokoQ6gEAnjk2YGmWdwVQ6Xq8nLrLcDkBEREQkhXkJ1HJJJ3mkZ9wdke92ACKSNjx0vmtkrPo0vToDZ10a1yhBIyIiItKyHZgna+mgeTpWMAUWf+ZiLG74xO0ARCRt1JE+1/Zkeg6T4AdTEPlWF2NJppaK74tIgnXBVO8eBhyEqQa+J4GZNURERNKVigSLiEhHqEiwy9SDRjLZYGASZhrUkcAoYK9W2m/DTJ/3JfAh8FrTf+MxjZ6IiIiIiIhIi1QkWDLN/sD5wKnA8Djsbwum4ODDwMtAYxz2KSIikgiXYHqIAqwFbnMxFhERST83ADlNP78HPOViLCKSprIxSZmFBLrkJeK1DigHBiTntEREREREREREUl8WpgBiNYlNzDhfO4E/AgMTf4oiIiIiIiLSGWiIk6SrCcBfaHt6PBszq8N7mNoySzBz3NcA24FcoBumcPBQTK2ascBhQNc29r0V+D/gDlSnRkREREREREQ6kS6YxIyflnu41AFPA9OB/u08Tg4wETO13JpWjmUDLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "7605a53f",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g37385.png\"  align=\"left\" width=\"450\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d9979e39",
+   "metadata": {},
+   "source": [
+    "### Floyd's sequential algoritm\n",
+    "\n",
+    "The ASP problem can be solved with the  [Floyd–Warshall algorithm](https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm). A sequential implementation of this algorithm is given in this function."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "4fe447c5",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "floyd! (generic function with 1 method)"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "function floyd!(C)\n",
+    "  n = size(C,1)\n",
+    "  @assert size(C,2) == n\n",
+    "  for k in 1:n\n",
+    "    for i in 1:n\n",
+    "      for j in 1:n\n",
+    "        @inbounds C[i,j] = min(C[i,j],C[i,k]+C[k,j])\n",
+    "      end\n",
+    "    end\n",
+    "  end\n",
+    "  C\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b27707b9",
+   "metadata": {},
+   "source": [
+    "You can check that this function computes the solution of the small ASP problem in the figure above by executing next cell. Note that we used a large number to represent that there is no direct connection between two nodes."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "860e537c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "inf = 1000\n",
+    "C = [\n",
+    "      0 inf inf   1\n",
+    "      2   0 inf   9\n",
+    "    inf   3   0 inf\n",
+    "    inf inf   5   0   \n",
+    "]\n",
+    "floyd!(C)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b9eeb64d",
+   "metadata": {},
+   "source": [
+    "### The algorithm explained\n",
+    "\n",
+    "The main idea of the algorithm is to perform as many iterations as nodes in the graph. At iteration $k$, we update the distance matrix $C$ by finding the shortest paths between each pair of nodes, allowing indirect paths via nodes from 1 to $k$. At the last iteration, it is allowed to visit all nodes and, thus, the distance table will contain the solution of the ASP problem.\n",
+    "\n",
+    "This process is cleverly done with three nested loops:\n",
+    "\n",
+    "```julia\n",
+    "n = size(C,1)\n",
+    "for k in 1:n\n",
+    "    for i in 1:n\n",
+    "        for j in 1:n\n",
+    "            C[i,j] = min(C[i,j],C[i,k]+C[k,j])\n",
+    "        end\n",
+    "    end\n",
+    "end\n",
+    "```\n",
+    "\n",
+    "At each outer iteration $k$, we do a loop over the distance matrix $C$. For each pair of nodes  $i$ and $j$ we compare the current distance $C_{ij}$ against the distance via node $k$, namely $C_{ik}+C_{kj}$, and update  $C_{ij}$ with the minimum."
+   ]
+  },
+  {
+   "attachments": {
+    "g42544.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAABAIAAAD1CAYAAADK+FcHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N11uBzV/cfx99wbJYEkkADBSQnBCxQpUpwCBYprixYtFC3Swg9rcdfiUlwKheIUh9JCgbZIgOAWLCRAlCT3/P44u9nZsZ3ZHVn5vJ5nnnt3duac797dnTvznSMOIu1tKWA1YFRpGQkMLC2DgSnAxNLyCfAm8BbwH+BZYHL+IYuIiIiIiIhIXH2BrYEbgbGAaWCZBjwF/A5YKM8XISIiIiIiIiLRlgAuBsbR2MV/2DITeBzYCejO6TWJiIiIiIiIiMfywG3YC/UsEgBBy9vAXkDvHF6fiIiIiIhIqpyiAxCp05zAacCvgK4a284A3qDS//8LYBIwAegPDCiVtwi2ZcHSwJAYMYwGfg08kTR4EREREREREYlvJ+zFfNRd+0+A84DNgDkSlt+FbWlwKPAk0BNRTw/wZ2BQIy9IRERERERERPxmA64kuh//ncBGpNuPfxHgeKKTD+8AK6dYp4iIiIiIiEhHWwj4H+F35W/GNunP0gDgEODzkDimYccOEBEREREREZEGLAl8SPDF95vABjnHMxg4Hzv2QFBMp+Ucj4iIiIiIiEjbWIHgKQF7gNMpduT+NYGPCE4GnF1gXCIiIiIiIiItaTHgM/wX2V8DPyswLrehwAMEJwOOKTAuERERERERkZYyD3YAPu/F9cfAMgXGFaQXcA3BrRZ+VWBcIiIiIiIiIi2hC3gU/4X1u9gR/JuRA5yLP+apwEoFxiUiIiIiIiLS9I7Hf0H9BTCqyKBicAie3vBtYFCBcYmIiIiIiIg0rVXxj8Y/GfhRzP2HY7sUuJe+6YcZqhfwIP5kwLU5xiAiIiIiIiLSErqAf+G/iN47QRkLBuzfL90wa5oTeN8TQw+wTs5xiIiIiIiIiDS1A/FfxN+SsIxmSAQA/ASY6YnjFWyLAREREREREZGONxvwOdUXzhOwTf2TaJZEAMClAbHsXlAsIiIiIiIiIk3lEPwXzfvXUU4zJQLmxA5y6I5lDNBdUDwiIiIiIiIiTaE38BHpXDA3UyIA4PCAeLYvMB4RERERERGRwm1BYwMEujVbImAA/lYBjxQYj4iIiIiIiEjh7qD6Qvkz6p/yr9kSAQAnUh3PTGCBQiMSERERERERKchgYCrVF8rnNFBeMyYCRmCnD3THdEShEYmIiIiIiIgUZGv8F+7LN1BeGomAgcAQzzJbAzEBPO2J6dEGyxMRERERERFpSRdSfYH8QYPlNZoI2AqY7Nn/bWCxBuP6rafMKUD/BssUERERERERaTmvUn2BfE2D5TWSCDgY23/fve9zwLAGYwJYISCu9VIoV0RERERERKRl9AVmUH1xvFuDZdaTCHCAEwL2u4v07tp3AV97yj8spbJFREREREREWsLS+C++V2iwzKSJgL7AzQH7nI+9eE/TM546Lk25fBEREREREZGm5h0osAc7UF8jkiQChgBPBsRwQoMxhLnKU9fjGdUjIiIiIiISqlfRAUhHm8/z+HNgYk51LwLcDyzpWvc9sDu2hUAW3vE8nj+jekREREREREIpESBFmt3z+Juc6l0OmwRwX4hPALbEthDIygTPY+/rFxERERERyVzafaBFkvB2A/guhzo3BJ6mOgnwKbAO2SYBwP/6lAgQEREREZHcKREgRerrefx9xvXtDtwHzOFa9yrwY+C/GdcNMNXz2Pv6RUREREREMqdEgBRpkufxbBnWdRRwDdDbte4xYE3gowzrdfO2AMhrPAQREREREZFZNEaAFCmvpvJXATt71l0P7EX2rRDcvK8vj64QIiIiIiIiVdQiQIrkHRxwWEb1eJMAVwG7kW8SAPyvL6/BEUVERERERGZRIkCK9J7n8RzAvBnUM93zeAtg+QzqqWVxz+N3C4hBREREREQ6nBIBUqS3AtaNyqCe7YEprsdDseMDrJZBXVG8r+3NnOsXERERERERKZSDHTDPuJbfNljmgp7yDNAPWAv41rN+IrBeg/XFNRDbFcFd/1451S0iIiIiIiLSNB6j+uL4vgbLC0sEAKwMjPM8NwnYsME649gkIK6lc6hXREREREREpKkcS/XF8bdAnwbKi0oEAKwIfOl5fip23IAsneWp8zNsiwgRERERERGRjrI6/gv3rRoor1YiAGAp4FPPNtOAbRqoN0oX8IGnvpszqktERERERESkqXUBH1N9kXxnA+XFSQSAHbjvI892M4BdG6g7zHoBMW2fQT0iIiIiIiIiLeFM/Hfn56+zrLiJAIBFgHfwJwP2rLPuMDd76pgA9E+5DhEREREREZGWsRz+i/dz6ywrSSIAYCHsNIbu7XuAg+qs32txbHLBXf4VKZUtIiIiIiIi0rIexz+139x1lJM0EQAwL/AK/mTAYXXU73VNQLnLpVCuiIiIiIiISEvbEP8F/NV1lFNPIgBs0uE/AfseX0cMZT8CZnrKu6uB8kRERERERETayj/x3z1fM2EZ9SYCAAYDzwXsf1rCGMAOgvgC/tezUh1liYiIiIiIiLSllfHfQR8DDEpQRiOJAICBwGMBZVwEOAnK+b+AMq5NsL+IiIiIiIhIR7gc/wX0bQn27wKGeJYkF/AAvQPKGFIqO4618A8Q+A0wPGEcIiIiIiIiIm1vLuAT/MmAo4oMKoERwFj88e9dZFAiIiIiIiIizSzojnoP8Ksig4phKPAGjbVoEBEREREREelIx+K/oJ4O7FpkUBHmA/6LP+YxwBwFxiUiIiIiIiLSEhzgevwX1j3AEQXGFWQU8B7+WL8AFi8wLhEREREREZGW0hu4H/8FtsEmCQYWF9osWwBf449vErBagXGJiIiIiIiItKQBwIMEJwNGY6ccLMJA4EJsCwVvXN8A63i275VncCIiIiIiIiKtrA9wI8HJgJnAJdjp/fKyHfBRSDxjgeU92y8IvA78PMcYRURERERERFpG/4B1XcCJ2Av/oAvw8cBJ2OkHs+BgL+T/FVK/Af4NLOrZbwHg7dLz01AyQERERERERGSW1YG7sFMHrhuyzQbAZ4RfjE8Erint35VCTAsAR2Hv6IfVabDdBPoG7Pu2Z7upwGYpxCUiIiIiIiLSkrqwA+49Q+Vi+Xtg/oh95sEOFhjUP9/bTP8GYE9gGfwX6kEWADYGTgOeJ7wFQnl5G9gkpKxBwD8D9plWes0iIiIiIiIiHaMvsBfwBpW+/uUL5atilrEO8D+iL9TdywzgHezF+SPAbcC9wBPAf4HvEpQ1EdtVIagbg1tUMmDLmK9TREREREREpGXNARwMfIo/AWCwd/mXSlCeA2xOdN/9NJfvgPOB4QliHAQ8F1DW9ygZICIiIiIiIm1qXuAE7NR65Qt+74VxD3BnA3WsBVwJTAgou5GlB9t1YV9gcJ2xRSUDtqqzTBEREREREZGmsxzwZ2A64QkA97JaCnX2x7YSOBfb5L9WnUHLl8DtwP74ZwKo1xy0dTLAzAVmZTA/A/MLMFuBWQdM1HgP3jJOBWMiliUi9n3btd1xDb+cupiDXTF8GrFdV43X+Zf8YhYRERGRTtOr6ACkba2JHW1/09Jjx/MzyJPYC+VGTQH+VloABgKjgMWBkcDs2IvyQcBkbH//icDHwFvAm8BHKcTh9S121oP7gLVd63tjkw67ADdnUG+GzELYZMlPgeUJnanBfAQ8DFwLzjM5BSciIiIiIgGUCJA0dWEv/I8FVsHe7Y668Pc6PYugsBf5L5aWok3C/o28yYBu7EwI0BLJADME29VjX+LNxrAg8Cu7mJeAQ8F5qsY+nwPXetaNSxZn0zL4P+9bYhNWIiIiIiIiTa8vsCswhuABAGstM7Gj/ydJGrS6AcDj+P8WM4CdC4wrBjMCzOiQJu3vgXkczO1g7gbzHJgJIduuEVC2u2tAwsRNK3UNCNz3DnUNEBEREZE8qEWANGIYcABwEDAE2xcfQpuHh+oCTsNeCHeKScBm2OkL13Gt78aOqeAAN+YfVi1mFPA09r0v+w44G7gRnLcD9ukFrAHsBOwB9Ck9UWvqxVZ0M3ZgSbDjYoiIiIiIiLSFEdgp9CYTbwDAWq0BPsb2k+9EA4DHCG4Z8IsC4wpg+oP5n+eu/rNg5k5Qxggwfy3tu0HA8y3eIqARahEgIiIiIvlIeudWOtsK2LvVb2FbAfQrrW+kSX+5NUCn3j0ttwx43LO+G7gO+GXuEYU7HVjW9fgZYENwvohfhPMudoaEo7DJDhERERERyVmTdQ0wA7FNjMM8A871wU+ZZYDfuFYcDM7UFGJaFjjQteIQcKakUO522BHkwxwDzleN15OKH2Hfl7WpHgCw0T79BhgPXN1gOa1uMpVuAuu61ndTGSzvhpxj8jDzAHu7VnwL7ALO5ORlOQY4A0wTjAlhhmFnkSibmCyx4StvEDBX6cFMcD6ovywRERERkaZleoFZE8xJYB60I4KbD8GMBfM6mCfAnAlmCzD9apQ1NGRQsfJyecS+m3i2nT1820SvbzNPuYNSKvecGq81rbnr03AhlSbrjXQDCFqOz/F1NLvZgEcJ7iawS4FxAeaPns/nSRnVk2PXALMpmImufd4GM7K+uGeVqcECRURERKSdmT5gfg3mkxoXtO7lazAXgZk/pEwlApozEdANrAkcA/ydyqwAjSQFerB3wudC3Jo0GWBec302Z4BZIKN6ckoEmH3ATHdt/y8SjXUQWq4SASIiIiLSrswoMG8mSAB4l4lg9g0o15sIWNe/TWhMLZYI8NWzehMnArx+gO0qMInGkgHn5R14i5gNm3AJSgbsmn84ZhiYHtdn84UM68o4EWAcMCd4vmsPpXi8UCJARERERJpeHYMFmp8A/wAWd63sAR4EDsZOEzYKmA/4IbA5cA7gnlZsALBlLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "b132bccb",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g42544.png\"  align=\"left\" width=\"400\"/>\n",
+    "</div>\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "722e330c",
+   "metadata": {},
+   "source": [
+    "The update of the distance matrix at each iteration is illustrated in the next figure for a small ASP presented above. We highlight in green the distances that are updated in each iteration. Note that adding values to the table can be interpreted as adding new connections in the graph, as illustrated in the figure."
+   ]
+  },
+  {
+   "attachments": {
+    "g38106.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAABGgAAAGhCAYAAAA9VYLrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13nBT1/cfx114vcPQOAkoURFEpKgooaGLvoljQkKixYie/2DUaRU0sMTE2ogY1auyKIHbRaBARaSKKICC9Hndc3fn98d29nZ3bu9u9292Z3X0/H495sDM3u/eZO2525r3fAiIiIiIiIiLSHPcDlm3xuVuOpLIctwsQERERSUG7Ai/Y1r8ALnGpFhERkWRpCwwBhgb+3dX2tRrgQDeKShcKaERERERiV4i5MA3a7FYhaeg84HTb+jFAlUu1iIgI/Ab4FeZ9r18j+1Unp5z0pYBGRERERLykH3C4bT3LrUJERASAS4H93C4iE+gNT0RERERERERitQq1cIwrtaARERERERERkcasAebYltnAWmA50Nu9stKLAhoRERERERERachoYJvbRWQCdXESERER8Y4OQKsEf4+OQEGCv4eIiKQPhTNJohY0IiIiIol3PbBL4PEPwN2Bx22AcwPLnoSCk1LgXeBhYGaU3+NqYPfA49XAbYHH3TEzI40DdgPyAtvXAW8AfwfmRvk9JmOmWAXTvP3xKJ83LFBD0J2YZvFBnYE/Bh4f4HjuQ0BtA6/7eKAOEZFU4gMmYs77Qcsx51i/GwWJiIiIiKSqgYBlW95pYv8vbfvOCmw7EHNBbjWx/JPoPlT7wPaceYFtJ2CmAG/s9WswNwXRtKxeZXvec1HsHzTO8T2HOb7er4kaG1rOiKEGEZFEuJ/w85Kvif1zgEccz5kH9ExgjYm0nNBxaMDgFlILGhEREZHkOwh4H8iPYt9fAzuAy2L8HscArxC6WagB1mNa0HS07ZcNTApsOw9zkS0iIvHXCngeONq27V3gFGC7KxWJp2gMGhEREZHk6gy8hAlnVgCXArsCuYFtg4EpjudcDAyN4Xt0BJ7ChDNrMSFPW6AH0AnohelmVGN7zm8CtbhhJeb4hmLqtjvI9jXnMiOJNYqItEQ34CPCw5knA+sKZ0REREREmqklXZyCy0waHxD4Jsf+TY33Yu/iFFx+ALo28pwjMU3Sg/tvx4Q4DUlUFye7uxz7akBjEfGyaLo4DcQE8sF9/MDNUbx2B0w31UQuT8R2uPUsR12cRERERMRFLQ1olmEGCG5MDvCj7Tlrm9jfGdDUAkOaeA7AjY7n/bGRfRXQiIiEayqgGQ1ssX29Grggytfu6HjtRCz/ju1w61luey0FNC2kLk4iIiIiyfcnmp62tAbTFSqoC2ZGpmi9CMyJYr97gE229d/Q9CCXIiLStFOBaYRmv9uBGbz9UdcqEk/TIMEiIiIiyeUnPHhpzNeO9V7Az1E+99ko96sI1BP8RLc78AvguyifLyIi9f0eM9ZXMPBegxm8fW4Mr1EKnBbnupxWJvj1JQYKaERERESS60dMc/dobHSsN9Utyu6zGPb9lPAm9/ujgEZEpDmyMd2eLrFtW4QZDHhFjK9ViWkNKRlCAY2IiIhIcm2OYd8Kx3o003KDCXac4U5jFjvWe8bwXBERCXkZOM62/jFwItEH85LBFNCIiIiIJFcyBlHc2sL928WrEBGRDGMPZ9ZgWs6UuVSLpBgFNCIiIiLppzrG/Z2hUbQtdUREJNwSYI/A427Ak8BZNC+c9xEaYDhRqlCA5BkKaERERETST6sY92/tWI+1BU5TNCuUiGSKQ4B3gEGB9VOBYuAUYGeMr9UB2BC/0iJ6HhiX4O8hUdI02yIiIiLppzOQG8P+zjFnNkXcy8xAFZQdw+uXxLCviEgqWw8cCnxh23YU8Daxh+eSYRTQiIiIiKSffGDPGPbfz7H+TQP77bA9juVGo28M+4qIpLotwBGEz6Z3CCakUWAtDVIXJxEREZH0dDwwL8p9T7A9rgHmNLCffQaqfjHUcngM+1Y61ltRfzYrERGv24YJad7AtKgBGIHp/nQU0c3qtBUYmojibDS7lIcooBERERFJT+cB99L0mAcHAvvb1qcR3lLGbj5wcOBxP6A3sKKJ1x8FDGliH7vtjvVOxDZluIiIV+wAjgRewITmAAcA7wO/ounxZRoLzCUNqYuTiIiISHraBbi9iX2Kgb8TPojvPxrZ/1PbYx9wdROv3wF4ool9nL5zrO8b4/NFRLykEhgLvGzbti/wMdDdlYrEs9SCRkRERCT9VGMGCb4q8Pgm6k/x2hOYSvj4M9MxYyQ05BVMC5fgGAqXAD8CDxA+gDDASOBxTEubcqAoyto/t9UPcCemCf5XmME3RURSTRVmpqSnCc2Y1B/4ANMFdKVLdUWrHQ23hCywPfbRcJfWBcDaeBYlIiIiIgIwELBsyztN7P+lbd9ZMXyfQx3f57hG9v3Att88zI1AcH0l8FfgcuD3wHNAmeO112K6LDXlWsfzLGAp8BhwV+Df+bavrQKudOw/rInv8UyE72FhxqLZbFtOiaJeEZFEup/w85SvkX2zMa0K7fsvJ7YxvdxwKJHPybEsZyS76FSkFjQiIiIi6ekiTOAyCtNa5tJG9l2P+dSzqfFkAP4CDAdOsm3rR+QbjNWYcRd2j+J17SYCe1D/E9v8wBKUF+Prioi4qRYzPlgpJjAHc57+BHMOXuhSXeIRGoNGREREJD2VAYcBd2BmAomkBtPNaS9M8/No1AKnATdgbjIiqcYMijkY0zUpVpswIdBZwEvAD418LxGRVGJhWhX+2batK2bg4EGuVCSe0VjzKxERERFJHR8Qmsr1G2Af29cKMDOG7IoZlHIHZuyYt2nZDEnFwBhgb8wYBVsxrXDeQePFiIiIiIiIiEgGco5BIyIiIilEY9CIiIiINMyHaXqeCuzjseQC3dwqxCVbMIMIi4i4pSvqpbIV2Ol2ESIiIiKSftrS8pkrtCRnCU5dKyLillrcPxe6vUxo8U8xg6kFjYiIiEjDLMDvdhFR8hH+yW2q1B0vmXa8IuI9tW4X4AE6F4uIiIhIxvuA0CeYGoNGREQkxagFjWSKfGB3zOwVrTGzTrQFyjHTkJYCq4ElmKk9RURERERERJJGAY2kq92A0YFlONAbyIryuZuAucCHwPvAbKAm/iWKiIiIiIiIiKSfrsBVwNfEd6CrDcCDwLDkHYqIiEjMdgeGBJY9Xa5FRERERDLQQGAqUE3iRyX/CjiV6FvjiIiIiIiIiIiktd2AlzAjhSd7+rhFwAmJP0QREREREREREW8qAG7CDPCb7GDGubwB9E3s4YqIiIiIiIiIeMuewALcD2bsSxkwIZEHLSIiIiIiIiLiFb8GduB+INPQ8hRm+m4RERERERERkbTjA/6E+wFMNMsXQMfE/BhERERERERERNyRDTyG+8FLLMu3QO9E/DBERERERERERJLNBzyO+4FLc5afgF7x/5GIiIiIiIiIiCTXZNwPWlqyzAPaxv2nIiIiIiIiIiKSJL/D/YAlHst7mG5aIiIiIiIiIg3SjaN40d7Af4BctwuJg76YoOZDl+sQEREREREREYlaK2AJ7rd8iedSC4yO5w9JRERERERERCSR7sH9QCURy/dAQRx/TiIiIiIiIpJGfG4XIGIzEJhLnLs2dezYkaFDhzJkyBCGDBnC4MGDKSkpCdvn2GOP5bPPPovnt43kFuDWRH8TERERERERST05bhcgYvMgcQhnunbtyoQJExgyZAhDhw6ld+/eTT4nJycpfwr/B0wBVibjm4mIiIiIiIiIxOpA4tSd6IgjjrBiNWrUqGR1dXowMT8+ERERERERSWVZbhcgEnCj2wUkyXlAV7eLEBEREREREW9RFyfxggHAUYl6cb/fz9KlS/nyyy+ZM2cOAH/5y18S9e2aUghcBNzsVgEiIiIiIiLiPQpoxAvOIY4DVm/dupVnn32WOXPmMGfOHObOncv27dvrvj5y5Mh4favmGo8ZMNhyuQ4REREREREREcB0s/uJJE55PXLkSDfHoAkuo+L3IxQREREREZFUpzFoxG0HAr3cLsIFp7pdgIiIiIiIiHiHAhpx2xi3C3DJYW4XICIiIiIiIt6hgEbclqkBzQCgm9tFiIiIiIiIiDcooBE3ZWG6OGUiH3CQ20WIiIiIiIiINyigETf1xkw7nan6u12AiIiIiIiIeIMCGnHTHm4X4LLd3S5AREREREREvEEBjbhpN7cLcFk/twsQERERERERb1BAI25q63YBLmvjdgEiIiIiIiLiDQpoxE2t3C7AZSVuFyAiIiIiIiLekON2AZLRMj2gae12ASIikjBZwC+AAUBHzOx9W4DvgQVAjXultUhbYDhmFsZBQKfA0hEoxxzjPOAj4AVguztliogkVBtgH6AXUAzsBDYCc4D1LtbVElmY96wDA0svoCvm/F4AbAOWA18CLwNfuFKliCTMfYCV7GXkyJGW06hRo5JeB7AuXj9IEZEU0g04DrgFeBNYBWy2LUe6VlnLtQV+i7lw3UbD5/9S4DHMhXAqeRjwE/373A7g96TXB4KdgB8J/z8729WKRLwjFxgMXAA8ggkr7H8rn7hXWlwMAyYDX9P4ufBr4HeYn0eq6AtsJbZ7mc8wQX06uY7w/7ObgXGuViSSRLeR2QHN9/H6QYqIpIC7gdU0fW48zq0CW+hCzCeosbwPVABXu1FsM71E897vXgfyXag3EZ6n/vEtcbUiEfftAfwPc05r7Fwwz60CWygXWETs5745pM6kIHvSvPN7OXCEC/UmwgAiv4+fm8wi0ukTDUk9pW4X4LJMP34RySzDgO5uF5FAu2GagNuVA3Mxzd2rgF2AIUBe4Ov5wL1AEfDH5JQZF0uBdzHN25dijs+PaeY/CBgLHI/p1gUmdLsduDbplcbX8cBpbhch4kEdMef4dJVN5BaPy4DvMC0mCzDn9562rw8GPgBGACsSXGO87MS0dPoA0x13Line truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "50b782df",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g38106.png\"  align=\"left\" width=\"450\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "attachments": {
+    "g38354.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAABGgAAAGhCAYAAAA9VYLrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13nBT1/cfx1+xVDjh6BxFFBRUsiIoBFTQRS+zYwRBLLBFbxK6oMYAaNcaYGJWIsUSNvaHYRaM/RURRFJWiICC9Xb+d3x/fXXZ2du9u9253Zsv7+Xjsg5vvze5+hrub3X3Pt1iIiIiIiIiISHPcCVzo2A4Atk+1eKUN0B/ojDneDcBSYJmfReWCQr8LEBEREclC2wFPOLY/As73qRYREZF0KgV+BYwGRgE7YIIZtx+B54F7gK88q05ERERE8toumCuk4dtr/paTU84EZjpuxf6WIyLSqDuJfj2w/C0npYqBe4F1RB9jU7d64HaglfclZzf1oBERERGRTNIfONixHe8qrYiIpF8ZcHYz7hcALgYGA0cAVaksKpfpBU9EREREREREmrIM02PoGGAbzNAnC+gNjAU+c+1/EHCHlwWKiIiISP7REKf0mUL0/22pv+WIiDQql4c4tccc01vAoUBBE/sXAf8kdrjTLmmsMaeoB42IiIiIiIiIuFVjJgUeCbyCCVsaUwucA7zvaAsAJ6WluhykgEZEREQkc3TCLF+aTp1RrxQREWlaJab3TDKCwG2utl+kppzcp0mCRURERNLvasx4fYDvgVtCX7cDTg/ddiYSnGwCXgf+jlnJKBGXAjuGvl4G3Bj6uidmZaSTgO2JrIq0EngBsxzqnASfYyqmyzvAx8D9Cd5vaKiGsMnAYsd2V+Cm0Nf7uO57Nw1ftb0/VIeISDaxgAmY837YYsw5NuhHQSn2kWu7hy9ViIiIiEheSHYOmk8c+84Kte2LeUPe1HKl/yKxi2pvOe4zN9R2FLC2icevw3woSKRn9VLH/R5LYP+wk1zPOdT1/f5N1NjQ7eQkahARSYdk56ApAqa57jOb3AoxuhB9fJ/7W072UA8aEREREe/tB7wJlCSw72+AzcAFST7H4cAzRD4s1AE/Y3rQdHbsVwBMDLWdiXkzLSIiqdcW+C/wK0fbDGAM5jyfK3Zybf/oSxVZSHPQiIiIiHirK/AUJpxZAvwe2A5zVbUE2BNzddXpPGCvJJ6jMzAdE86swIQ87YFemCubfTDDjOoc9/ltqBY//Ig5vr0wdTvt5/ie+/aqhzWKiLRED+BtosOZB4Ejya1wBmInBX7dlypEREREJC+0ZIhT+DaTxicEvs61f1PzvTiHOIVv3wPdG7nPaKDGsf9GTIjTkHQNcXLSMtsikk0SGeK0CyaQD+8TBK5P4LE7YYappvP2QHKH26RtMYFT+Fgrgd4pfg4RERERka1aGtAsxEwQ3JhCYJHjPiua2N8d0NQDQ5q4D8C1rvvd1Mi+CmhERKI1FdAcCKxzfL8GGJ/gY3d2PXY6bv9J6mgbF8D0bHQ+vntFJ2mEhjiJiIiIeO9PwIYm9qnDDIUK64ZZkSlRT2ImnmzKrcAax/ZvaXqSSxERadrxwCtEVr/bDByNmfw9F00iegjX90RWFJQEaJJgEREREW8FiQ5eGvOZa7sP8FOC9300wf2qQvWcHdruCewALEjw/iIiEutyzFxf4cB7OWby9jlJPMYm4IQU1+WWqgl8TwCucWzXYHpPbkzR4+cFBTQiIiIi3lqE6e6eiNWu7aaGRTl9kMS+7xMJaAD2RgGNiEhzFGCGPZ3vaPsKOAwzD00yqjG9ITPdIcBDRMIoGzgXM7xXkqCARkRERMRba5PYt8q1nciy3GCCHXe405j5rm1N6Cgi0jxPA792bL+LGdaUaDCfbX4JPEvk9SkczrhXI5QEKKARERER8VaNB8+xvoX7d0hVISIiecYZzizH9JzZ4lMt6bY/JpxxTuY+EbjXn3KynwIaERERkdxTm+T+7tAo0Z46IiIS7Rtgp9DXPYDpwCk0L5y3iEwwnC41NC9A2hd4EShztN2AVm1qEQU0IiIiIrmnTZL7t3VtJ9sDpylaFUpE8sWBwGvAoND2cZiJ2McQO2y1KZ2AVSmrLL7HMZP5JmMIZnUq52vHZMwqTtICWmZbREREJPd0BYqS2N8958yauHuZFajCCpJ4/PIk9hURyWYrgQOAjxxtRwAzSD48z0S7Aa8S3bPnTuAqf8rJLQpoRERERHJPCbBzEvvv4dr+vIH9Nju+TuaDRr8k9hURyXbrMCsbOVfTOwDT6ySbA+tBwOuYnj1hdwEX+1NO7tEQJxEREZHcdCQwN8F9j3J8XQfMbmA/5wpU/ZOo5eAk9q12bbch+WEBIiJ+24AJaZ4HRobahmOGPx1KYqs6rQf2Skt1EYmuLrUzJpzp7Gj7B3BRyisSEREREUnCLpilRMO315rY/xPHvrOSeJ4DXc/z60b2fcu17xKgVQLPsS9m6FL4fs81su/fHfsFgb4JPP7+rrpsYGgj+1/q2ndgAs8hIuKXO4k+Z7nn3CrBnFed+8wBunhYY0vtCPxE9DFMQyNyUk7/oSIiIiK5aRvgj03s0xq4h+gPFP9oZP/3HV9bmDClMZ2BB5rYx22Ba3v3JO8vIpJJqjETBD/taNsdeBfo5UtFydkBcwGgh6PtIeBMouclkxTQECcRERGR3FOLmST4ktDX1xG7xGtv4GGi55+ZgZkjoSHPABuJzKHwe2AR8Bdi36jvD9yPGQpVQfRSrI350FE/mJVB1gGfAj8n+BgiIpmkBjgR+DeRFZMGYIKPg4AffaqrKT2BN0P/hn0FPAqMSvKx3gLqU1SXiIiIiMhWmT7EaS7mCmd4+0fgr8CFwOXAY8AW12OvILEhS5cRO2TpW+A+YEro33mO7y3FTCCZ6BAngEfiPIeNmYtmreN2XAL1ioikU1NDnJwKML0K3cNRk5nTy0ujiH8ubs4t0ZBeRERERCQp2RDQtAbeIbE3ziuBXROsqQDTVT+Rx10K7Im5YpxMQNOJ6P+zhm4nJ1iziEi6JBPQEPq++z7LMa8rmUYBjcc0B42IiIhIbtqC6Tp/M2YlkHjqMMOcdsX0eklEPXACcA2wqYF9aoEnMOHMpwk+rtMaYBhwKvBf4DvMiigiItnOxqx85JwjrDtmKNFgXyqSjNFUuiciIiIi2eEtTI8bgM+B3RzfKwV+BWyHmUtgM2bumFeA1S14ztaYK6yDgA6YIGgJpkeR5osRERERERERkbzjHuIkIiIiWUSrOImIiIg0zMJ0Pc8GxY6vi4heEjUfrMNMIiwi4pfuaJTKeqDS7yJEREREJPe0J3UTJOqW3lt46VoREb/U4/+50O/b+Bb/L+Yx9aARERERaZgNBP0uIkEW0Vdus6XuVMm34xWRzFPvdwEZQOdiEREREcl7bxG5gqk5aERERLKMetBIvigBdsSsXtEWs+pEe6ACswzpJmAZ8A1maU8RERERERERzyigkVy1PTAydBsG9AUCCd53DTAHeBt4E/gYqEt9iSIiIiIiIiIiuac7cAnwGamd6GoVcBcw1LtDERERSdqOwJDQbWefaxERERGRPLQL8DBQS/pnJf8UOJ7Ee+OIiIiIiIiIiOS07YGnMDOFJxOybASWA98DS4C1JL8k3lfAUek/RBERERERERGRzFQKXIeZ4LexEKUemA3cCpwA7A6UNfCYBZgJhEcDE4GXMRMHNxXUvAD0S/HxiYiIiIiIiIhktJ2BeTQemnwAnAt0auFzFQGHAo/SeBi0BRjfwucSEREREREREckKvwE2Ez8kqQMeAwan6bk7YHrtrG7g+W1gOmb5bhERERERERGRnGMBf6LhYGQGsJNHtbQB/ghUNVDLR0Bnj2oREREREREREfFEAXAf8cOQlcAYn+raCXingbq+Bvr6VJeIiIiIiIiISEpZwP3ED0HeAXr5VxpgwqNJmOFV7vp+APr4VpmIiIiIiIiISIpMJX4481dMOJIpDiP+3DhzgfY+1iUiIiIiIiIi0iK/I344c42fRTViH2AVsfW+QWaFSSIiIiIiIiIiCRlE/GWtr/SzqATsCqwltu5JPtYkIiIiIiIiIpK0NsACYkOOO/wsKgkHAJVE114PjPSzKBERERERERGRZNxG9g8TOoPYY/gOKPWzKBERERERERGRROwC1BAdbKwAurfwcTsDo4GrgaeBxZihSM7bfi18DrfpxIY016f4OUREREREREREUu5NYkON0c14nO6Y+Wr+iwlj4k027L7t37LSY7QFfnQ9RyVaeltEREREREREMtgwYkOTJ5v5WIfEeSyvAxqAE+I8z11peB4RERERERERkZR4mdT1NsmUgAZiewVV0PIhWyIiIiIiIpJjAn4XIALsDBzqavsXZohQKgSBb4BHgEtCN6/c4NpuBZzr4fOLiIiIiIiIiCRkCtG9TGqBfi14vH2IhDEHAOWu74/Aux40AO+5nmshYKXx+UREREREREREkhIAfiA6wHguzc/pdUBzisfPJyIiIiIiIllGQ5zEb8OInWvm334UkkbPAhtdbcf7UYiIiIiIiIhkJgU04rdRru1NwAt+FJJGFZiQxukgPwoRERERERGRzKSARvzmDmjeBar9KCTNXndtDwR6+FGIiIiIiIiIZB4FNOKnAmBfV9tbfhTigTdd2xawnx+FiIiIiIiISOZRQCN+6guUuto+9KMQDywjdtnwAX4UIiIiIiIiIplHAY34aac4bV97XoV3vnFt7+hLFSIiIiIiIpJxFNCIn7Z3ba8J3XKVO6Dp70sVIiIiIiIiknEU0Iif2ru2V/hShXdWurbb+VKFiIiIiIiIZBwFNOKnNq7tzb5U4Z2Nru1yX6oQERERERGRjFPodwGS1/ItoNnk2m7rSxUiIuKFALADMBDojFm9bx3wHTAPqPOvtBZpDwzDrMI4GOgSunUGKjDHOBd4B3iC2IsTIiK5oB2wG9AHaA1UAquB2cDPPtbVEgHMLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "eb5e46a2",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g38354.png\"  align=\"left\" width=\"450\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "attachments": {
+    "g38620.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAABGgAAAGhCAYAAAA9VYLrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13mFTV/cfx991lC73YQUQWCzZU1EQSLKhR7C0omlhjN7Ggwi5qJEbZJRpbmj1qNCqJvaEo9vKzERVFUFhQUBBEkLrsMuf3x5lh7tyZ3Z1675TP63nmce/dO3e+s7J3Zz7zPec4iIiIiIiIiEg6bgIudG2XASagWvzSBdg4fOsGrAWWADOApgDrKngdgi5AREREpADVABNd2/8HnB9QLSIiIrlSBfwUGBK+7Qls0sqxzcAHwAPAfcCPfhQoIiIiIqVtB+wnpJHbC8GWU1TOACa7bpXBliMi0qabiP174ARbTtYdSOzzS/Y2Hzg8gHoLmjpoRERERCSfbAUc4NouC6oQERFJW2/gMWzofk+wpRQO/cETERERERERkbYsBh4GzsIOedoY2y3kYOehGQI0AMtd9ykHbgW287VSERERESkpGuKUOw3E/myrgy1HRKRNxT7EqTcwmOSbO/phJwt2/0zuz01pxUcdNCIiIiIiIiKSyDfAh0AoyePnAid79h2Bsoek6IckIiIikj82wC5fmksboq4UERHJnf8DvnRtdwU2CqiWgqJJgkVERERy73Jgi/DXs4A/hb/uDpyK/bRxe6LByXLgReAf2JWMknEJsE346/nA1eGve2MnaRwJDCC6KtJC4Cng78DUJB9jAtAj/PV7wJ1J3m+PcA0R9cAc1/bGwB/DX//Uc9+/AutaOe+d4TpERAqJA1yAve5HzMFeY5PtVMl387GTvkd0CqoQERERESluqc5B877r2DfC+4ZgW8HbW6r0nyT3odrLrvt8FN53JLCknfO3YN8UJNNZPc91vweTOD5ipOcx9/B8f6t2amztdkIKNYiI5EKqc9BUAHd77vMBsFkOawzCZ0Sf31qgY7DlFAZ10IiIiIj47+fAS0BVEseeCqwAfpfiYxyKXeI08mahBfgO20Gzoeu4cmB0eN8Z2BfTIiKSfV2B/wIHuvZNAkZgr/PFYntgoGv7NWB1QLWIiIiISJHLpINmJvBt+Os5wPlAf+wHZ5XArsBdnvOvA3Zv5zHcHTTzsUuimvBjnQJ0dh27OTAeaPY8TnshUK46aKqA3cK3ezzHDnF9z3vrlUINIiK5kGwHTW/scFL3sXdRfE0TvbAdQZHnGAKGBVqRiIiIiBS1TAKayG0ybU8I/HvP8e3N9+IOaCK3WcCmbdxnOLb1PHL8j0CfNo7PVUDjpmW2RaSQJBPQ7EDskNYQcFUS594AO0w1l7e7Unu6CXXDfohwBXZ+M/fP44osnF9EREREpFWZBjSzsRMEt6UD0Oi6z4J2jvcGNOuwXSbtudJzvz+2cawCGhGRWO0FNPsCP7i+vxY4Lclzb+g5dy5uD6X0bGG7JM/bCByb4rlLnpbZFhEREfHfeGBZO8e0AI+4tjfBtsgn6z/YNvP2XAd879o+nfYnuRQRkfadiJ1jJrL63XLgcOzk78XKAPdiO2oeaedY8Si28W4iIiIi+S5E8i9a/+fZ7gt8k+R9/53kcWvC9ZwV3u4NbI2dK0dERNIzBqgnGnh/i528fWoK51gOHJflury+zvL5HOy8Z0cDf8b+DJqz/BhFSwGNiIiIiL8ase3uyVjs2W5vWJTbWykc+ybRgAbgJyigERFJRzlwC3Cua99nwCHYeWhS0YTthswnjcROWt8J+7dpR2wANTS8vxvwB2Bv4AhglY81FiwFNCIiIiL+WpLCsWs828ksyw022PGGO22Z7tnePIX7iohI1KPYYUwRrwFHkXwwn+/WkHj47NPYOcT2Bu7HdnwC7I8NrM7wpboCpzloRERERPy11ofHWJrh8T2zVYiISIlxhzPfYjtniiWcScZrwD7E/l05HdthI+1QB42IiIhI8Ul1vL83NEq2U0dERGLNALYNf70ZdsLcE0kvnHeITjCcK2uBlVk+ZyNwPXBNeNsBfgXUZflxio4CGhEREZHi0yXF47t6tlPtwGmPVoUSkVKxL/ACsFN4+1jsROwjiB+22p4NgEVZqyyxh4GROTjvU0QDGoidt0ZaoSFOIiIiIsVnY6AiheO9c858n/AouwJVRHkK5++WwrEiIoVsITCM2HlaDsMGFp0CqSgY8z3bGwRSRYFRQCMiIiJSfKqA7VM4flfP9setHLfC9XUqXTr9UzhWRKTQfY+dHNe9mt4BwCTiOxaLlXfVwWWBVFFgNMRJREREpDgdAXyU5LFHur5uIfEKHRC7AtVWKdRyQArHNnm2u5D6sAARkaAtAw4CnsR21ADsBUwGDia5iYOXkvuhQbmawHiIZ9vbUSMiIiIikhU7AMZ1e6Gd4993HftGCo+zr+dxDm/j2Jc9x84FOibxGHtihy5F7vdEG8f+w3VcCOiXxPn39tRlgD3aOP4Sz7HbJfEYIiJBuYnYa5Z3zq2OwHOeYz4ENvKxRr+VA+8R+5xPDbKgQqEhTiIiIiLFaQtiJ2hMpDPwd2LfUNzaxvFvur52sGFKWzYE7mrnGK+Znu1dUry/iEg+WQ0cBTzu2rcrNlTfLJCKkrcR9oOBVCZ67wDcRmznz4/YOXhERERERLIu3zto1rq+bgAqExy/OfCK5/zPtVNPZ2zbvruL5mISf+i3NzZsMdglXJPtoNnIU/8cYDh24mMRkXzTXgdNRAfgQc+xM4G+PtSYrq2wdX4CXA7sTOsT0HcDTgKmEd81eXHOKxURERGRkpXvAc1HwH2u7a+BvwAXAmOwbxK8ockCkhuydBnxL76/AO7AhkF3EPsCfR72xXmyAQ3AAwkew2Dnolniuh2bRL0iIrmUbEADdujP3Z7jG4GaHNeYrkhA4741YSeSfx07n87rwJfAugTHGuzz1cgdEREREcmZQghoOgOvkvgFs/e2ENgxyZrKgUeTPO88YDAw0rO/vYBmA2J/Zq3dTkiyZhGRXEkloCH8/b8Rf63cNoc1pitRQJPsbTW260bhTAr0wxIREREpTiuxy7xei10JJJEW4H5sODMtyfOuA44DrgCWt3JMMzARG858mOR53b7HrgDyK+C/2E9ntUSriBQDA/wWuN61rw82UN8pkIpa9xU2YH8Q22WZjIXAn7EfZFyLHQorSUplsh8RERERyV8vYztuwLaf7+z6XjVwILaNvjewAttW/xywOIPH7Azsh31T0RMbBM3FdhR9l8F5RUQk//THXu9rsPOCVWGHnv6IvfZPxQ55VSgjIiIiIiXNO8RJRERECkiHoAsQERERyWMOsGnQRSTJvVJTBfm/fGu2/YD9JFdEJCibolEqS7Hzz4iIiIiIZFUP0p8gUTd/byNb+X8oIuKX1lYyKqXbaRn/FEuYOmhEREREWmconLH0DrGf3BZK3dlSas9XRPLPuqALyAO6FouIiIhIyXuZ6CeYmoNGRESkwKiDRkpFNbANdsbxbkAnoDt2fOQq7NKd3wAzyGw1CxEREREREZGUKaCRYrUVMCx8GwJsAZQled8l2CXiXgGmAO8CLdkvUURERERERESk+GwGXIJt687mRFeLgb8AP/HvqYiIiKRsG2C38G37gGsRERERkRK0I/AA0EzuZyWfCowg+W4cEREREREREZGiNgB4FDtTeCohyzJgPjALaMQOaWpJ8RzTgaNy/xRFRERERERERPJTNXAVdoLftkKUFuB94E/YrpedgY6tnLMM6A8cBFwKPAMsb+f8BngaO/GwiIiIiIiIiEjJ2B6YRtuhyZvAOUCvDB+rAzAcO3xqVRuPtxI4LcPHEhEREREREREpCKdhw5DWumX+DQzK0WP3AK4EFrXy+Aa4D+ico8cXEREREREREQmUA9TTejDyHHblCj90Bq4G1rRSy7vARj7VIiIiIiIiIiLii3LgThKHIQuBXwZU1zbAK63UNQPYMqC6RERERERERESyyqH1cOYVoHdglVllwBgSrwD1FdA3uNJERERERERERLLjTyQOZ27Bdtbki4OBFcTX+TF27hoRERERERERkYJ0DvGBRwi4PMii2vATEk8g/BL5FSaJiIiIiIiIiCRlJxIva10bZFFJ2BFYQnzd4wKsSUREREREREQkZV2AmcSHHDcEWVQK9gZWE1v7OmC/IIsSEREREREREUnFn0m8jHZZkEWl6HTin8OXQHWQRYmIiIiIiIiIJGNHYC2xwca3wKYZnndDYDh2/ppHgTnYoUju288yfAyve4kPaa7K8mOIiIiIiIiIiGTdy8SHGgelcZ5NgTrgv9gwJtFKUN7b3pmVHqcr8LXnMVYDW2T5cUREREREREREsmYI8aHJxDTPdVCCc/kd0ACMSPA4t+TgcUREREREREREsuI54rtNNk/zXPkS0IBdZtv9OKvIfMiWiIiIiIiIFJlCmnhVitf22Dli3P4JzMvS+UPADOABYFT45pc/eLY7Auf5+PgiIiIiIiIiIklpILbLpBnYMoPz/ZRoGLMP0M3z/b3wr4MG4DXPYzUCTg4fT0REREREREQkJWXAV8QGGE/k+DH9DmhO8PnxREREREREpMBoiJMEbQjQ17PvX0EUkkOPAz969o0IohARERERERHJTwpoJGj7ebaXA08FUUgOrcaGNG7e5y0iIiIiIiIlTAGNBM0bVLwGNAVRSI696NneDtgsiEJEREREREQk/yigkSCVA3t69r0cRCE+mOLZdoCfBVGIiIiIiIiI5B8FNBKkfkC1Z987QRTig/nA1559A4MoRERERERERPKPAhoJ0rYJ9n3uexX+meHZ3iaQKkRERERERCTvKKCRIA3wbH8fvhUrb0CzVSBViIiIiIiISN5RQCNB6uHZXhBIFf5Z6NnuHkgVIiIiIiIikncU0EiQuni2VwRShX9+9Gx3C6QKERERERERyTsdgi5ALine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "d02606e0",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g38620.png\"  align=\"left\" width=\"450\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "attachments": {
+    "g38913.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAABGgAAAGhCAYAAAA9VYLrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13fBR1/sfx12xICL1KRzooKiLq2UDBCvaGevZer6onB+rp3SlFvVPvflctZ69nOQuiIKjYFREbSgtFlN4TCEl2fn98s9nZ2U2ym+zMbHk/H499kJ3M7nw2JLO77/1+P18QERERERERkYa4B7AdFyvYciSbNQm6ABEREZEs1Bd4xnH9I+CagGoREREJ2qXAla5t04HxAdSStRTQiIiIiKSuGbCv4/qGoArJQZcCZzquHwfsDKgWERGpX2/gbqCla/ti/0vJbgpoRERERCST9AeOdFwPBVWIiIjUywL+RXw4Iw2gJzwRERERERERaYhLgaOrv94aZCG5QAGNiIiIiIiIiKSqG3BH9ddh4NbgSskNCmhEREREREREJFX/ANpWf/034L0Aa8kJCmhEREREMkcHvJ/H3xEo9vgYIiKS2y4ATqz+ejlwY4C15Aw1CRYRERHx3o3ArtVfLyY6JLwN5kXuBcBgosHJVmAG5tPJ6Uke4zpgYPXXK4E/VH/dDdMj4CygH1BUvX018DLwd2BukseYQvTT0k+A+5O83f7VNURMApY6rncC/lj99QGu2/4fUFXL/d5fXYeISDaxgF9gzvsRSzHn2HAQBaWoC/Bnx/WrUf8ZEREREQnIHoDtuLxRz/6fOvZ9t3rbgZgX5HY9l/+Q3Idqsxy3mVe97STMEuB13X8l5k1BMiOrv3fc7skk9o84y3XM/V3f719PjbVdfppCDSIiXriH2POSVc/+TTCrHjlvMw/o4WGN6fYc0dqfcGw/gNjH9bT/pWU3jaARERER8d/BwEygaRL7XghsA36e4jGOA14g+mahEliDGUHT0bFfAXBD9bZLMS+qRUQk/VpiQotjHdtmAKcBWwKpKHVnAqdWf70B+HWAteQc9aARERER8VcnzKePTYFlwM+AvkBh9bZhwIOu21wN7JfCMToCD2PCmVWYkKct0B3YBeiJmWZU6bjNxdW1BGEF5vHth6nb6WDH99yX132sUUSkMboCbxMbzjxUfT1bwpkOwL2O69dipsuKiIiIiASmMVOcIpfp1N0Q+Heu/evr9+Kc4hS5LMb0CqjNaGCnY/8tmBCnNl5NcXKa7NpXDY1FJJMlM8VpD0wgH9knDNySxH13wIxS8fLyQAqP9Qlin8Pcj1VTnERERETEd40NaJZgGgTXpQlQ4rjNqnr2dwc0VcC+9dwG4GbX7f5Yx74KaEREYtUX0IwCNjq+XwFcnuR9d3TdtxeXp5Ks5XjHbUoxTefdFNA0kqY4iYiIiPhvIrC5nn0qMVOhIjpjVmRK1rPAnCT2uxNY77h+MfU3uRQRkfqdDkwluvrdNkzz9n8HVlHDtAH+6bh+C2aEpqSZmgSLiIiI+CtMbPBSl89d13sCPyR52yfq3wWAHdX1RD7R7QYMABYkeXsREYk3DtPrKxJ4/4hp3j43hfvYCpyR5rrcViSxz91Ep7/OwYwaEg8ooBERERHxVwlmuHsy1rmu1zctyun9FPZ9j9gh9z9BAY2ISEMUYAKMaxzbvsE0A16W4n2VY0ZDBukITKN5MCM7Lye2wbykkQIaEREREX9tSGHfHa7rySzLDSbYcYc7dZnvut4jhduKiEjU88AJjuvvACeTfDCfSVphVhWMjAL6E/BZcOXkPgU0IiIiIv7a6cMxNjVy/3bpKkREJM84w5kfMSNnSgOqpbHuAHat/noR8PsAa8kLCmhEREREck9Fivu7Q6NkR+qIiEis74BB1V93BR4GzqZh4bxFtMGwV3aSOEDaDbii+mu7+uvtHteS9xTQiIiIiOSelinu38p1PdUROPXRqlAiki8OA94AhlRfPw14sfrfVAOODsDa9JWW0NPAWQm2tyN67raANxtw32cQ2+T4feCQBtxP3tAy2yIiIiK5pxNQmML+7p4z6xPuZVagiihI4f5bp7CviEg2WwOMBD5ybBsDvEbq4bnkGQU0IiIiIrmnKTA4hf33cV3/opb9tjm+TuWNRp8U9hURyXYbgWOIXU3vMExIo8BaaqUpTiIiIiK56URgXpL7nuT4uhKYU8t+zhWo+qdQy5Ep7Fvuut6S+NWsREQy3WZMSPMSMKp623DM9KcxJLeq0yZgP0+qi6qtji8acOw9gYcc16cD4x3XtyF1UkAjIiIikpsuBe6i/p4HBwI/cVyfSu0vor8k2j+gP9ALWFbP/R8K7FvPPk5bXNd3IbUlw0VEMsU2TBjzDCY0BzgAmAkcTf39ZeoKzL1W2oBju/OFjQ24j7ymKU4iIiIiuWlX4LZ69mkB/J3YJr7/rGP/9xxfW8B19dx/R+CBevZxW+C6PjTF24uIZJJyYCzwvGPbUOAdoHsgFUnGUkAjIiIiknsiy2xfC0wGihLs0wN4ldj+M9MwPRJq8wKxI1yuAX5N4teUh2L6L/QHypKq2viQ2GXCJwGjMY2PRUSy0U7MakaPOLbtBsxGPbpERERERBplD8B2XN6oZ/9PHfu+m8JxRrqOc0Id+85y7DcP80Ygcn0F8Ffgl8A44EnM8HXnfa/CTFmqz29ct7OBhcB9mDDoPuArx/e+x4Q4zv33r+cYjyc4ho3pRbPBcTktiXpFRLx0D7HnKauOfQswowqd+y8jtZ5emewAYh/b08GWIyIiIiL5IBsCmhbA2yQOOtyX1ZjmjskowAzVT+Z+vweGAWe5ttcX0HQg9mdW2+WnSdYsIuKVVAIaqr/vvs2PmOeVbKeAppE0xUlEREQkN5UCRwC3Y1YCSaQSeAwTznyV5P1WYYbq3wRsrWWfCkxTzGHAZ0ner9N64CDgHOC/wCLMiigiItnOBn5FbI+wLpjGwUMCqUgyRn3pnoiIiIhkh1mYETdglkfd2/G9YsyKIX2BbpiVRUow/WYas0JSC+BwYC+gHSYIWoYZUbSmEfcrIiIiIiIiIpKV3FOcREREJIu41ykXERERkSgLM/Q8GzhXaioEugZVSEA2YpoIi4gEpQuapbIJ2B50ESIiIiKSe9qSXDNcXYK/nFXL/6GIiF+qCP5cGPTlokb/FPOYRtCIiIiI1M4GwkEXkSSL2E9us6XudMm3xysimacq6AIygM7FIiIiIpL3ZhH9BFM9aERERLKMRtBIvmgKDMSsXtEKs+pEW6AMswzpVmAl8B1maU8RERERERER3yigkVzVDxhVfTkI6AWEkrztemAu8BYwE/gEqEx/iSIiIiIiIiIiuacLcC3wOeltdLUW+Auwv38PRUREJGUDgX2rL4MDrkVERERE8tAewGNABd53Jf8MOJ3kR+OIiIiIiIiIiOS0fsBzmE7hqYQsW4AfgcXAMmADqS+J9w1wkvcPUUREREREREQkMxUDv8M0+K0rRKkC5gB3AmcAQ4HmtdxnAaaB8GjgBmAqpnFwfUHNy0CfND8+EREREREREZGMNhj4irpDk/eBq4AOjTxWITAGeIK6w6BS4KJGHktEREREREREJCtcCGwjcUhSCTwJDPHo2O0wo3bW1XJ8G3gYs3y3iIiIiIiIiEjOsYCJ1B6MTAMG+VRLS+A2YEcttXwEdPSpFhERERERERERXxQA95E4DFkNjA2orkHA27XU9S3QK6C6RERERERERETSygLuJ3EI8jbQPbjSABMe3YqZXuWubznQM7DKRERERERERETSZAqJw5m/YsKRTHEsiXvjzAPaBliXiIiIiIiIiEijXEHicOamIIuqwwHAWuLrfZPMCpNERERERERERJKyF4mXtR4fZFFJ2BPYQHzdtwZYk4iIiIiIiIhIyloCC4gPOe4OsqgUHAZsJ7b2KmBUkEWJiIiIiIiIiKTiLrJ/mtAlxD+GRUBxkEWJiIiIiIiIiCRjD2AnscHGKqBLI++3IzAauBF4HliKmYrkvBzcyGO4PUx8SHNLmo8hIiIiIiIiIpJ2M4kPNUY34H66YPrV/BcTxiRqNuy+HNq40uO0Ala4jrEdLb0tIiIiIiIiIhnsIOJDk2cbeF/HJLgvvwMagDMSHOcvHhxHRERERERERCQtppK+0SaZEtBA/KigMho/ZUtERERERERyTCjoAkSAwcAY17b/YKYIpUMY+A54HLi2+uKX37uuNwOu8vH4IiIiIiIiIiJJmUzsKJMKoE8j7u8AomHMYUBr1/dH4N8IGoDZrmMtASwPjyciIiIiIiIikpIQsJzYAON/Hh/T74DmbJ+PJyIiIiIiIllGU5wkaAcR32vm0SAK8dCLwBbXttODKEREREREREQykwIaCdrhrutbgZeDKMRDZZiQxumIIAoRERERERGRzKSARoLmDmjeAcqDKMRjM1zXdwe6BlGIiIiIiIiIZB4FNBKkAuBA17ZZQRTig5mu6xZwcBCFiIiIiIiISOZRQCNB6gUUu7Z9GEQhPlhJ/LLhuwVRiIiIiIiIiGQeBTQSpEEJtn3rexX++c51fWAgVYiIiIiIiEjGUUAjQernur6++pKr3AFN/0CqEBERERERkYyjgEaC1NZ1fVUgVfhntet6m0CqEBERERERkYyjgEaC1NJ1fVsgVfhni+t660CqEBERERERkYzTJOgCJK/lW0Cz1XW9VSBViIiIH0LAAGB3oCNm9b6NwCLgK6AyuNIapS1wEGYVxiHALtWXjkAZ5jHOA94GniH+wwkRkVzQBtgb6Am0ALYD64A5wJoA62qMEOY568DqS0+gC+b8XgxsBpYCnwLPAx8FUqWIeOYewHZc3vbpuCNcx7WBQ3047pWuY7qnPImI5IOuwAnArcArwPfABsdldGCVNV5b4BLMC9fNxD/XRC5bgfswL4SzyT+AMLU/LvdlGzCO3PpAcBeghNjf2U8CrUgkcxQCw4DLgX9hwgrn38rs4EpLLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "0aca29d3",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g38913.png\"  align=\"left\" width=\"450\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0c95ea88",
+   "metadata": {},
+   "source": [
+    "## Parallelization\n",
+    "\n",
+    "Now, let us move to the parallelization of the method.\n",
+    "\n",
+    "### Where can we exploit parallelism?\n",
+    "\n",
+    "Recall:\n",
+    "\n",
+    "```julia\n",
+    "n = size(C,1)\n",
+    "for k in 1:n\n",
+    "    for i in 1:n\n",
+    "        for j in 1:n\n",
+    "            C[i,j] = min(C[i,j],C[i,k]+C[k,j])\n",
+    "        end\n",
+    "    end\n",
+    "end\n",
+    "```\n",
+    "\n",
+    "- The outer loop cannot be parallelized since the operations at iteration $k$ require the result of iteration $k-1$.\n",
+    "- The inner loops over $i$ and $j$ can be parallelized at a given iteration $k$."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9a9e8c44",
+   "metadata": {},
+   "source": [
+    "### Parallelization strategy\n",
+    "\n",
+    "As for the matrix-matrix product and Jacobi, any of the iterations over $i$ and $j$ are independent and could be computed on a different processor. However, we need a larger grain size for performance reason. Here, we adopt the same strategy as for algorithm 3 in the matrix-matrix product:\n",
+    "\n",
+    "- Each process will update a subset of consecutive rows of the distance table $C$ at each iteration $k$.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9eab6b5d",
+   "metadata": {},
+   "source": [
+    "### Data dependencies\n",
+    "\n",
+    "Recall: we perform this update at iteration $k$:\n",
+    "\n",
+    "`C[i,j] = min(C[i,j],C[i,k]+C[k,j])`\n",
+    "\n",
+    "If each process updates a block of rows of matrix $C$, which data we need for this operation?\n"
+   ]
+  },
+  {
+   "attachments": {
+    "g14170.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAycAAAGYCAYAAABYqp0gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d13nCRlnfjxT29OLCwLLEFgSUpakoKIeAqyZhFEFARE9E4FBRQRRThdUDEAnqee+hOEUxEDKmAmLEFQ1ANFQJAgGQQ2sSwLbKzfH0+3Xd1b1d0z09X1zPTn/XrVa3qmq5/6TnWlb9UTQFIMZgNJdfpeybHEai3q6+gfJcciSZ2YRP24dX8B5X+tWvZ8YGrG+zNTy/9TAcvvd8dQX79zyg2lJ66g/v++cAjlrAc8VS3nM81vjhlCwWrtPsJBIcvTwJLqdA/wZ+D/gMuB53oRXIaxwKbV18uAR0qKQ70zrToBLAAWlxiL1CsLgHVz3luSmu6ifmy+Aljek+jWNB7YpPr6WeCfJcWh+GwP/Ef19ZmEi73B8PwfuK/1znxCYv1R4EPAN4CHSo2oT9xHPbvsdFoA/DewYQnxbpOK4w8lLL/flfHk5BOpZX6oR8scCp+cqBsWMPBj8+OEi7/pJcT7wlQcV5SwfA1NkU9OfkV9+5ycM8/M1PLznpw8H8//ALtSXw9zO/yMT04GL/305LvpN3xy0hs3AA83/W0K4US3EzCh+rd1geOAtwNHAz/uVYCS1Id+S7iwS1uLcGzeGRhX/dsGwInA4cC/A7/sVYBSjtnAa6uvzwSW5sy3FLio+tqbOt13D/X1e3uZgQxD84GvAicDhwFfAm4qNaI+kH5y8pYW840FXgZcAKxIfWY1cGjBMab55KRcPjlpzycn6ob0k5NXtZhvPLAv8CNgZeozK4E3Fhxjmk9Ohreinpz8plrmcsId6KHwyUkwmCcn/aabT04AtiJc7yaE62AARnWhYA3NCuA6wh25l1N/wlIBzifsLJKk3loGXAW8FXg18ET176OB7wMvKCkuaUvCDS0IScr8EmORhuIfhNpFEG7krwdW64rN7wmPaf9AqD86Hjid9nfpxgC7ATsC6xOSzqcIDTpvIDTAzzOWUMUs3cvHGOoNpdNWkd/gbjrwIsIJe3J13nmER3S3ETLjbpkATKy+fpb2nQik53+GcNHRbDL1KhxLCHdHIbT/2RfYmPA/3UfouOCZAcY8mXCXdibh+3mUcGfmiRafaWUasDthfU+pxjaf0ID3Flqv79r/OiH1t4lkf+ft1u844CXAtoRqic8SGrVdBSzq4P9IGw28gtDIcxLwGGFfuHOA5UjdNpdwHP4t4bg8mfDk8bA2nxtLOC7uQDhGjiJ0PPF3wrbd6jgyrrqctZrKy9pPVxKOW1nWp35snliddx6hof/thDuW3TKR+nEl71ibN/9SWnc6MBZ4MeE8tw71HqrmE9bnQI4TY4G96N5x6+WE77h23PpjNaaivIf6zeULWs1YnW/t6uvm7WQg5/9W21jNtoTvaL1qOU8AvyNci7STt+2MB/Yh3GGfSviesv7n2r62fXX5g9nX0uuh031tPOF7h3CufLbFcmomE7aZLQn7d61zpGsH8Pna9crThJvcVMuaDWxWjesh4GrKa9if3vYgfKdZ38MFhP1xPHAUoZqiCtJpta4sH6OxeteOOfO9iXAHb2Fq/ubpGeArNG4gaa9t8dnm6f+aPrsWoaeF39JY7aF5ugd4R2f/ekc+nCr7Yx3Mn66+dGzOPD9OzbMPYX19l8aqdrXpqWoMoztY9phqjEsyylkOnEf9gNKuWtdk4CPANTlx1aZ7gXe1iOnbLT7bPH04p4y1gTMIB/+sz60AvgXMaBFH2kGEp4ZZZV0FbIHVutQdnVbrynJG6rMrCdtllrcS6qHn7R8J4ZhwFvkNmd/W4rPNU3MVlHWBUwg3p1a1+NwdwMGd/esdmZMq+5gO5j8zNf+ROfOMBk4iJFSt1sH9hHZBraxN6La01XHrPDrvlObNhAvArLKuJlx8drtaV4VwsZkQLvYmtp69ZYP41+XEnjXlNaavAIcQkoBWn/23NnHOSc1/HOHC9qOseX3zYNPnDiZUvXyyxfKfBs4mJGJZDh7Aeri66bMDaRA/HfgyIQHJKvsZwjFhnTblfCf1mTcQrjNOJ/s6YyXwdcKFfzd0Wq1rMvCL1Lx/J/94OSM139+6FKdy3Ed9ZQ80OZlGuItU+3zeAffPdLYz1U5CG2SUMZTkZNYAPpsQGjt1Q9HJyf6EHaT2+2rCQaP5//k2ratG1qp/ZB2AVqd+/ytwQOr3vOTkBRlltZq+llPOUJOTrYC7O/z8/YT6zK2ckPG55TQmYAtprHtvcqLBGkpyshGN2+V7cubrdP9IgJvJvhgZSnLy0gF8NiFc2HTDnFSZ3UpOLiQ75qyL0Z+1WNaWhKcrnayPBwhPAVr5YMbnllen2u+LCLUa0sfDodo+VV7zOTnLzNT83U5OxtL5+WQFrW+azUnNexzwg5xymjsX6vQ7TQjn2qynIb1ITrag8+PC7dS7ds6STk7eROigo12ZPyckkkPVSXIyg7Bt1ua7nvY9Hf4jNX8ZPdb2jfuor+iBJicQ7ha3O+D+GbgVOBXYm8a++7cgXIg/lirnFxllrE3YwNIXx7dW/9Y8bdf02VmEA84lwDsJB83aY9lxhMe7/4/GO3fduEtXdHJyV/XnXwgXL7XqjzOB/6Lx/2l1p+7U1HwrgM9RP+CMJZwYbmtaZkLr5GQlYXt4F6EaQe2u2VhgD0JCkn6KlVXtZCbh+/x6ar6zyf7Om598TCfcuap97rbqMjaqvj8JeA3hMXptnjvIvzu8H42J2kWEXpIgHEh3p37gTa8jkxMN1lCSE4AbU5/Pq1JzN+H4/FFCtcd08rE1ISGfnyrnwowyphH2wcNT8/2R7P20+QbASwl31S8CjiAcO2rH5gnV979Nfd9bTdhvh2pOKtZuJCfp89IiwhOUmdSPyaMI6/NIwnnokpzlTCMkBrWy/kZYr+nj1qsJT5pq89xJ/p32V9J43PoxsEv1vQqhelHtrnH6uHV/TnkDcXSqvK92MP/M1PzNCUbt/H9gap68839Wsvat1OeeAj5JuA4YQ7g5tyPwP9TX1UpCB0BZ5qTKuqX6cz6hJ6cXEq5pdickLml3Es7VHyNUDUonH1n72g8yll3b195O47rqZF/rJDmZSOMNz/mE77F2ft0QOJ7GhPsv1KtuNUsnJ7Wb1LdU49+s+v/MIqz7dIJySE55A9EuOXkBjYnGT2j/dA8ab0K8rQtxKsdQk5PPpT6fV19zN9pnwhs3xbJbznyD6a1rGvmP6dIOTZV9S4dlt1J0clK7CJiUM2/65LCUsI6bbU6of1qb74icsqYSDkLpZeclJ2sTnlq0cxD1k8Ed5G8jg+mtK/0k6BfkH3TGAb9OzfufGfOMpvFO0v/klFUhVLFLryOTEw3WUJOT2ojcCSFRyZJ3nE2bSWh7lhBueGyTM99geuvaAHheB/O9J1X29R2W3cqcVHndSE6+l3p//w7KWyvn7+lyfknr41b6LvScjHlG0ZhwfCOnrAprPlW4v038nUg/Ucg7r6TNTM3fzXFO0onjE6x58zItvZ3dSvY5aQ6N6+peWj89qOlkX9uc+r62mvzOLIoa5+S01DzzyK9NMIvGBCXv+iadnCSEc+2EnHnPTs13df6/0bFWycleNCaC/03nHW8dn/pcXq0PdcFQk5P0I+Oh9sRxWKqsT+fMU3RXwpelym9XzaedopOTlYSnEq3MTc1/asb7p6fev7RNWbvReBeuG10J/zxVXl6bpYEmJ9tQf2r0BI2NB7NsRL1u7T9Z8yCVrlJ4P63vrkwljEdhcqKhGmpykt5v7h1iLOkbHR/NmaforoTTTwuGWp1iTqqsbiQn16feb1cPP8/W1J8mzyO//WXNDOrVeB9nzbaFr07F9CD5N7EgJEvp2gv3Dyz0TDenymvXjgOKS07S1XYO6GD+WtfHCaFdZ7M5qffz5hmK96bKPjlnniKSk/E0nrvatb/9QGrehwm1Ipqlk5PFtK4ytR71qobP5ZQ3EHnJyZup7zeryG+vmied7F5lV8LxejL1ehpD6/Y5vZPtPoRyhuKqCGLo1LW0b5SVfpz+poz303/LeyJQ82dCT23dVMT6fjv17fBc8ntuq/kn4YQE4aKnOeFLn9DOo3UvJU/RNIKsVJL0sXnd3Lk647G5tXTPPnsNsoxDqScY5xEu5lp5nHAnGsITqOabO+lj+/m07gVqCeHpSTetn3q9sMtld2pbQtU1CAl6XnW6tPR6eGWbeW+nO3f508ra1/ak3t73CbKrcKZ9i/q5dRPq6znPDwk3XPLMp54Uj6ez2i4DdTyhCulEQnXStxOe2AxE+n/YwK6E45XOblfQvivebQh1Xjcn1JNNP+JL3/kpqqHROEIWXesyc20aE6qdU683Im5XtZ/lX22CKoT/bQL1LncnU78QX0nozayduYR64J0aR3jisgPhzshUGr/nWanX3Vrf6brCnZ44bqGehOxEeKRfkz5BdLLO5zLwuzFStzUfm9vZjrDtb0Y4Nqd7zUnfdS/q2DyBeher0whPINLVavZIvY7t2Hw99fE8LgA+S7hJ8dgAyhjMceuvhDvBEL67v6beS6+vTo9bJ3W43HYqNA64WFZykl6n13T4mfQ63Dl3ruC6AUVTt2217M0I5+Hm7vJretnges/U66upD1OQ51nCNcMbqr+/mPo4IFn+2EEMD1GvNjrYJ5BZRhGSkBOqvz9JON9fO4iyTE6GiXSjrrwDUIWQoZ5M+2pINd3cMCHE+QnCI/msXjCytHusXrZO+stfTDhBbkS4WNmQ+t2JjagnCg/SfhyWTpcJYd39J6Ev8E7v2nZrfW+den062Sfc5gufdIP65lGM0/WJO+kH3/FOFIP0cS7vjuUoQqcVH6HzaqzdPjavT6jrfhjtq2DWxHZs/hrwH4T2M9OALine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "5cf07baf",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g14170.png\"  align=\"left\" width=\"450\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "db9a3294",
+   "metadata": {},
+   "source": [
+    "TODO explain this part"
+   ]
+  },
+  {
+   "attachments": {
+    "g12830.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAGgCAYAAAC0SSBAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAABqISURBVHic7d15nF11ff/x1wyTIQnEBBGJbCFIJeyEXcpmlJCQIFo2W6kslqq1NZCCta611BXJT0RFS6nW0ooboAJSIKBIILKEStghbEIIaxIIhCwz9/fHd6ZzZrz3M5PJzP2ee/N6Ph73kXu/c2fOOzNn7nvOOfd8DzSHZUCl63Zd5iyS1FTacgeQpMzGAFsUHi8BXsuUpZRacweQpMxOABYVbkfkjVM+FoUkKdQsu56uAEZ33V+YM4gkNZtmKYpTcgeQpGblridJUsiikCSFmmXX0+H0/F9eAhbkiyJJzaVZiuIKYGzX/evx7W2SNGSapSgkqd7agb8GNimMPQ78KEuaYWRRSNK62xT4CTCtMLYQOCpPnOHlwWxJWjdbATfRuySuBw4GnsqSaJhZFJI0cLsCtwKTC2P/QdqSeDlLojqwKCRpYKYA84DtCmNfAU4F1mRJVCceo5Ck/h0P/AAY2fW4A/go8N1sierILQpJis0CLqWnJF4FjmEDKQlwi0KSatkIuAD4SGFsCTATuDNLokwsCkn6Y5uQtiJmFsYeAaZ3/btBsSgkqbfNgWuBgwpj84GjgReyJNKQ8JrZkgbrg/S8flSAVX0e/4Se4xMbJA9mS1Jv7X0evwiszhGkLCwKSertXHqfPPch0ltjN9hd9RaFJPU2j3Ry3YuFsfcDlwAjsiTKzKKQpD92J3Ao8Exh7ETgMjbA4xUWhSRVdx9py+LpwthMUlmMypIoE4tCkmp7gDQr7KOFsenAr4AxWRJlYFFIUuxx4B3Aw4Wxw4CrgTfkCFRvFoUk9e9J0jGLewpjBwM3kE7Qa2oWhSQNzBLgncDvC2P7kC5atEWWRHViUUjSwD0HHE6a0qPbXqQr3m2dI1A9NEtRbFS435kthaQNwTLgCODGwtgk4LfAxCyJhlkzFEU7aabHbityBZHUkFYBSwu3gUzXsYI0SeA1hc8bB/yMdE1tlcx4ek/gdVHeOJLUXJphi2L3Po/vy5JCklRKo4Bf03uLYnLOQJKk/KYBc0in0T9D75K4I2MuSVJJfIne5dB9ew3YL2MuSWpKzXCMAmAh6USY23MHkaRm05I7wCC8l7T76SXSmZLzSQXh+ROSJEmSJEmSJEmSJEmSJEmSJEmSJEmShkemM7MrZwAfzbNsSdIALICWEwHa8ix/2bYwbsc8y5Yk9W/NUlJHrK3zFkXl48CucO9JsGuzTEgoSU3ohQqsvAa2fb7eRTEXmFLfZUqS1kemXU//50nSxclL5nsfgL3nwZ6Lciep7rJp0NoJ77k2d5LqFvwJ3H0gnPKfuZPUdtHpcHqJr6/+/ZNg99tgn4dyJ6nuiqnQ2Qp/VsLfX4C7d4A7D4ZTf5A7SW2lXQf3AA7MuPzKXKhUCrerMoaJPAecmDtE4ErgitwhAieRpoAvs47cAfrxDHBy7hCBy4Gy/v4CHA88nztEP0q6DlZm93mdrnicQJIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSaGW+i6uMheY0vP4mg6Yvrq+GQZkFLAGWJs7SA0bd/27KmuK2tqAEcDK3EECoyh/PtfBwXMdHLSz2uDcEcWRtlxRkgNuAf4mb4aqfgt8Cbg6d5AaLgQ6gL/NHaSGo4GzgMNyBwn8Htg/d4jAr4GvA1dkzlHLN0kvxB/KHaSGacCngENyBwmUdB08+QPA2cWRzEWx2SvAPXkzVLUGeIJyZgNYTvpLs6z59iJ9D8uar1uZ860B/kB5My4F2ilvvp0p9+9ItxLm221J3xGPUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQi31XVxlLjCl5/FvlsPhD9U3w4DsA/wBeC53kBp2ASrA/bmD1LAlsDWwIHeQwH7A7blDBPYGngaezR2khkmkPzTvyx2khi2AbSn3OrgvcEfuEH/s8+Phs9sWR9pyRUnGLKeU3yj2AB4FHsgdpIatgU7K+b2DVGRbUt58UNpf0v+zB/AEcE/uIDW8hVQUZf0e7gRsRXnzQWnXwc33JJVsLpW5UKkUbldlDBN5Djgxd4jAlcAVuUMETgKW5A7Rj47cAfrxDHBy7hCBy4Gy/v4CHA88nztEP0q6DlZm93mdrniMQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUasu//MobMmeoYlwLvHkUPFTCbABvaoMW4PmS5ttxFLzQAstKmg9gLLC8xPnGtcDmo2BRSTNu0QaVNnihpPl2GgXPug4Ozsi+Ay31XX5lLjClvsuUJK0Pdz1JkkIWhSQpZFFIkkK5D2YvAn6cOUMVs2bBjOtg6n25k1R3zgkwuhP+/qe5k1R3xW5w8xT42jdyJ6ntI5+AC7+cO0VtZ34Mpt4I0xfmTlLdF46F9jY4+0e5k1R39c5w/VSYc37uJLWVdh3cH3hncSD3wexbgDPrm2EgRl0Dx54Hl1yXO0l1b54Dm3fA/WfnTlLd1Gnwu1mwfHruJLW13Qpr3547RW2jr4aZ34IfX5U7SXXjvwrj2uGBM3Inqe7od8JvPg4vH5k7SW2lXQf/AphVHHDXkyQp1L3raTfg8ML45cDTdU8jSSqd7qI4FLigMH4fFoUkCXc9SZL6YVFIkkIWhSQp1H2M4jLg94XxezJkkSSVUHdRLOm6SZLUi7ueJEkhi0KSFOouihnArwq3PbMlkiSVSvcxignAtML4uRmySJJKqMF3Pa1shXMnwFk7w5PtudNIUjPKPc34IK1pgXccC/M/DB1j0tictTDhF3DZN2Dyyrz5JDWPxSNgzo6wcHuY9CTMehh2WJ07VT01aFHsfCYsOrH3WKUNHv8zOHQ3ePA02GpNnmySmsdJB8BPPwWrtkyPrwUuXArTvgK/uDFrtDpqwF1PH94LFp1Q++Mr3gZHnlK3OJKa1Cn7wX+d31MS3dZsBr/8Msx8V55c9deARXHDYfR7waXHD69HEknN6qWN4NJPE77W/M/ZsHBU3SJl1IBF8dKO/T/n1YnpBy1Jg/H1ibBqfPyctePgm5PqkyevBiyKTRf3/5yNn4c3dgx/FknN6a4JA3ve/QN8XmNrwKKYdFf/z3nTHcOfQ1LzmvzEwJ436cnhzVEODVgUl18L4xbU/njbcvh/365fHknN54zHYONn4+e0LYWP3F+fPHk1YFGM6oRLPglbVnlr2ujH4MyPwXEv1j+XpObxxg543zlApfZzpn91Qzlnq0HPo5ixFJZ8AmbvCvN3gZWjYY/74Zz/he02qBNhJA2X798Oa2f1Po8C0pbE9K/CL27Il62+GrQous25F7g3dwpJzeqS38EXj4Ov/wksnAA7PwlnPbSh/UHa4EUhScNtu9Ub+h+lDXiMQpJUTxaFJClkUUiSQhaFJClkUUiSQhaFJClkUUiSQt3nUTwE/LAwviRDFklSCXUXxfVdN0mSenHXkyQplHkKj0oLLC3pNCKVFnippNm6L89Y1nyVrj9AypqvW9nzVVrLm7HSAp2UOJ/r4KCNbYXeFwjt59rTQ60yF5jS8/iaCswMpvHNppU0vXAZswGdLdDSAi2duZNUVyH9oraWNB9AZ8nzVVqhxXVw0FwHB+/MFji3VzdkbrNpt8LaM/NmqGbUNXDseXDJdbmTVPfmObB5B9x/du4k1U2dBr+bBcun505SW9utsPbtuVPUNvpqmPkt+PFVuZNUN/6rMK4dHjgjd5Lqjn4n/Obj8PKRuZPUVtp18C+AWcUBj1FIkkIWhSQpZFFIkkIWhSQpZFFIkkIWhSQpZFFIkkIWhSQpZFFIkkIlnGdEGrQW4G3APsCWwFjgdeB54AFgAbCyxufuQ98JbnosBx4HVgXL3q9r+S8Bjwwg60Rgi677dwIdA/icvtqBScDOwOZdN4BlwH3ALdT+/0oDZlGoGbQARwCnA9sFz3sN+BXwb1U+9jVgdPC5HaQX3ouAB6t8/BukLfRfA//Qb+I0TcK7u+6/C3hlAJ9TdDjwL8CI4DkrgAuBn67j15Z6sSjU6NqBTwLFeaUWAwuBpaQX0rcAk0lFcCxwWPD1XgdeLDx+IzCKtLVxCPB24NPAjUMTf9A2Jf3fOoHHSBcbe6lrfHvSFsumwNmkLauLs6RUU7Ao1Og+S9qagLTL5zzSLqa+2oGppK2OTYOvtwDoO1HleOBE0lZAW9cy/5dURLk8DPwjcAfwcpWPHwh8HhgHfBC4Cq9cqUHyYLYa2TH0lMTdpBKoVhIAq4ErgT9n3bcGlgDnA1d0PR4N5J6V9EHgBqqXBMB84Ftd9zcCDqpHKDUni0KNqg04rev+a8Bnuv7tz2ukffuDcXXh/q6D/Br1tLBwLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "e17c5271",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g12830.png\"  align=\"left\" width=\"225\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "attachments": {
+    "g12957.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAGgCAYAAAC0SSBAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAABprSURBVHic7d15lFx1nffxdyWdkAQCYY/syaiAERBBNkEwQkhMGFS2eZw8sszAHMcZIxF4HB11HMcVwhlkEBnGQZERRAQRAyhEVEiIyPLIEiLKpiwxEEIIELJ1zR+/bvt2U/Xt6qXqdzt5v86p01W3b/f9pHOrP3233wU1U7Xw+H7mLJLUL225A0hSZmOBbQuvlwCvZspSSsNyB5CkzE4AHi08jsobp3wsCklSyF1PzVU8LrEwWwpJGgCLorlOzB1AkgbKXU+SpJBFIUkKueupuY4sPF8K3J8riCT1l0XRXLcUnl9DOg1PkoYUi0KS+mckcAawaWHaE8D3sqRpIotCkvpuM9Lp71ML0x4A3psnTnN5MFuS+mYH4Jd0L4lbgUOBp7IkajKLQpIaNwm4E9i3MO3bpC2Jl7IkagGLQpIaMxmYD+xSmPYV4FRgbZZELeIxCknq3QnA5cCojtfrgY8Al2RL1EJuUUhSbBZwFV0l8QpwLBtJSYBbFJJUz3DgQuDDhWlLgBnAPVkSZWJRSNLrbUraiphRmPZ7YFrHx42KRSFJ3W0N/BQ4pDBtIXAM8HyWRNqgec9sqfxOo/t7dTWvf++OqvvVGwEPZktSdyN7vF4GrMkRpCwsCknq7ly6Xzz3d6RTYzfaXfUWhSR1N590cd2ywrS/Bq4ARmRJlJlFIUmvdw/wLuDZwrSTgGvZCI9XWBSSVNsi0pbF04VpM0hlMTpLokwsCkmqbzFpVNjHCtOmATcBY7MkysCikKTYE8C7gd8Vph0O3AhsniNQq1kUktS7P5COWTxYmHYo8DPSBXobNItCkhqzBHgP8JvCtP1INy3aNkuiFrEoJKlxS4EjSEN6dHob6Y53O+YI1AoWRfMM7/G6PUsKSYPtReAo4LbCtD2A24EJWRI1mUXRPFv1eP1ylhSSerMaWF54NDJcx8ukQQJvLnzdOOAHpHtqSw3Zg+4Di52bN44k9Y9bFM2zV4/Xi7KkkCSV0ljg13RtTbQDE7MmkqR+2mhHQ2yC95EuwpkAvBPYpvC52+h+ZackaSP0dbofk+h8rADekjGXJA2Ixyia627SVobHJyQNWe56Gjx3kIr3BdLQxAuAe0lbFZIkSZIkSZIkSZIkSZIkSZIkSZIkSZKGskqexVY/Bnwkz7IlSQ24DyonQrYhPF7cGca9Mc+yJUm9W/siqSPWtXiLonoOMAkemgmTHJBQkkrr+Sqsuhl2fq7VRTEPmNzaZUqSBiL36LF/IN2cvGQu+xC8fT7s82juJLVdOxUqVXj/T3Inqe3eN8H9B8Ep38mdpL5LT4fTL82dor5vzYS97oL9HsmdpLYfToH2YfCBEr5/Ae6fCPccCqdenjtJfaVdB/cGDsq4/Oo8qFYLj7kZw0SWAiflDhG4Abg+d4jATGBJ7hC9WJ87QC+eBU7OHSJwHVDW9y/ACcBzuUP0oqTrYHV2j9/TVY8TSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKWRRSJJCFoUkKVRp7eKq84DJXa9vXg/T1rQ2Q0NGA2uBdbmD1LFJx8fVWVPU1waMAFblDhIYTfnzuQ72n+tgv53VBueOKE5pyxUlOXAB8Pd5M9R0O/Al4MbcQer4OlAFPpI7SB3HAGcBh+cOEvgNcEDuEIGfA/8O/DBzjnr+g/SL+O9yB6ljKvAp4LDcQQIlXQdP/hBwdnFK5qLYciXwYN4MNa0FnqSc2QBWAO2UN9/bSD/DsubrVOZ8a4E/Ut6My4GRlDffnqStsbLm61TCfG9d0nOKxygkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIUsigkSaFKaxdXnQdM7nr9ixVwxCOtzdCQ/YA/AktzB6njLUAVeDh3kDq2B3YE7s0dJPAO4Ne5QwTeDjwN/Cl3kDr2IP2huSh3kDq2BXam3Ovg/sDduUO83ufGw2d2Lk5pyxUlGbuCUv6g2Bt4DFicO0gdO3R8LOPPDlKRbU9580Fp36R/tjfwJPBg7iB1vIFUFGX9Ge5Oep+UNR+Udh3ceh9SyeZSnQfVauExN2OYyFLgpNwhAjcA1+cOEZgJLMkdohfrcwfoxbPAyblDBK4Dyvr+BTgBeC53iF6UdB2szu7xe7rqMQpJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUsiikCSFLApJUqgt//Krm2fOUMO4Cmw3Gh4pYTaAbduAdniupPneOBqer8CLJc0HsAWwosT5xlVg69HwaEkzbtsG1TZ4vqT5dh8Nf3Id7J9RPSdUWrv86jxgcmuXKUkaCHc9SZJCFoUkKWRRSJJCuQ9mPwpcnTlDDbNmwfRbYMqi3Elq+/wJMKYKH78md5LafvhWuGMynPe13Enq+/An4OIv505R35kfhSm3wbQHciep7QvHwcg2OPt7uZPUduOecOsUOP+C3EnqK+06eADwnuKE3AezFwBntjZDI0bfDMfNgStuyZ2ktu3mwFZVWHxW7iS1TZkKv5oFK6blTlJf252w7uDcKeobcyPMuAiunps7SW3jvwrjRsLij+VOUtsx74FfnAMvHZ07SX2lXQc/CMwqTnDXkyQp1Lnr6a3AEYXp1wFPtzyNJKl0OoviXcCFhemLsCgkSbjrSZLUC4tCkhSyKCRJoc5jFNcCvylMfzBDFklSCXUWxZKOhyRJ3bjrSZIUsigkSaHOopgO3FR47JMtkSSpVDqLYldgauGxdbZEG4xVw+DS8bA098CLkjQg7noadNdvCTv+G4ydB2dcD+Nvg20uhHN3zZ1MkvrDohhU39oeTrgSnjkK1o9J06ojYdkB8InvwD/snTefpL57ZgSctSccPQ1mTYLHRuZO1GruFhlUZ30K1m5Z+3Ptm8B/fwZmfxAmrmltLkn9M/NAuOZTsHr79PqnwMXLYepX4Ee3ZY3WQm5RDJq5W8KyA+N5Vu0MX31La/JIGphT94f/uaCrJDqt3RJu+DLMODJPrtazKAbNVW9qbL77dm9uDkkD98JwuPLThDd3+8nZ8MDolkXKyKIYNBOXNTbf1i80N4ekgbtgN1g9Pp5n3Tj4jz1aEiczi2LQzHoC2lbG81Ta4f0lvQeypC73NXiW4sMbxdmMFsWg2Wo9HNrLjdx3vwJOd0wtqfT2fbKx+fb4Q3NzlINFMahuuwHe9nWorO3xiSrsdh385NIssST10ccfh016+aOubTl8+OHW5MnL02MH3X3fhnN/DtccDEsmwjZPwRF3w5xFuZNJatTm7XD8F9NZT/UOaE/7Cuy7qqWxMrEomuLsJ9ND0tB1xa+g8lH4/j93P0W2bTlM++rGdB2FRSFJdX3nLvjC8fDvb4IHdoU9/wBnPQK7bFQXzVoUkhTaZQ2c/xDwUO4kuXgwW5IUsigkSSGLQpIUsigkSSGLQpIUsigkSSGLQpIU6ryO4hHgysJ0B66TJAFdRXFrx0OSpG7c9SRJCmUewqNageUlHUakWoEXSpqNClAtb75qxx8gZc3Xqez5qsPKm7FagXZKnM91sN+2GAbDu00J7gfbDNV5wOSu1zdXYUa1tRkaMgyodjxKqL0ClUq6Y14ZVUlv1GElzQfQXvJ81WFQcR3sN9fB/juzAud264bMbTb1Tlh3Zt4MtYy+GY6bA1fckjtJbdvNga2qsPis3ElqmzIVfjULVkzLnaS+tjth3cG5U9Q35kaYcRFcPTd3ktrGfxXGjYTFH8udpLZj3gO/OAdeOjp3kvpKuw5+EJhVnOAxCklSyKKQJIUsCklSyKKQJIUsCklSyKKQJIUsCklSyKKQJIUsCklSqITjjEj9VgHeDOwHbA9sAbwGPAcsBu4FVtX52v3oOcBNlxXAE8DqYNnv6Fj+C8DvG8i6G7Bdx/N7gPUNfE1PI4E9gD2BrTseAC8Ci4AF1P/3Sg2zKLQhqABHAacDuwTzvQrcBPxXjc+dB4wJvnY96RfvpcBva3z+a6Qt9J8D/6/XxGmYhGM7nh8JrGzga4qOAP4NGBHM8zJwMXBNH7+31I1FoaFuJPBJoDiu1LPAA8By0i/S8cC+pCI4Djg8+H6vAcsKr7cCRpO2Ng4DDgb+GbhtcOL322akf1s78DjpZmMvdEzfDZjQ8fxs0pbVN7Ok1AbBotBQ9xnS1gTAo6Qtg3trzDcSmELa6tgs+H73Aj0HqhwPnETaCmjrWOb/JxVRLr8D/gm4G3ipxucPAj4HjAP+BpiLd65UP3kwW0PZsXSVxP2kEqhVEgBrgB8D/4e+bw0sAS4AftjxegyQe1TS3wI/o3ZJACwELup4Phw4pBWhtGGyKDRUtQGndTx/Ffg08EoDX/cqad9+f9xYeD6pn9+jlR4oPN8iWwoNLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "3261af17",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g12957.png\"  align=\"left\" width=\"225\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9d1de618",
+   "metadata": {},
+   "source": [
+    "In summary, at iteration $k$ a given process will need row number $k$ which might be stored remotely in another process. The owner of row $k$ needs to *broadcast* this row to the other processes at iteration $k$."
+   ]
+  },
+  {
+   "attachments": {
+    "g12703.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAycAAAGYCAYAAABYqp0gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d13nCRlnfjxz2wOsLBLXOKSRHJQUAFPQdCfCUFERYKAioIBM6Koq6cYEM/DdGeA00PvMGI8JSMgBvSAJXpKVoFddmFhWTb2749vt/10T1V3z0zP1LPL5/169W5PT3X1d6qrnqpvPQkk5eAQoFZ/fKviWHK1Ls1t9JeKY5GkXkyjWW7dNQrr/1J93QuAGQW/n5N8/u9G4fOf7E6huX3nVhvKmLiY5t/7tBGsZ0NgcX09H2//5YQRrFid3UkUCkUeAx6tP/4M/BH4PXAR8MRYBFdgIrBl/fky4K8VxaGxM7P+AHgIeKTCWKSx8hAwq+R3jyaPP9Esmy8Glo9JdINNBjavP18K/L2iOJSfnYE31J+fRVzsDYfn/+CxNnYWEIn1acA7gH8D7q00oieJO2lml70+HgL+Fdi0gnh3SOL4TQWf/2RXRc3Jh5LPfMcYfeZIWHOifniIoZfNDxAXfxtUEO/TkjguruDzNTKjWXPyc5r75/SSZeYkn19Wc/IUPP8D7EVzO1za43usORm+tPbkP9NfWHMyNq4F7mt7bR3iRLc7MKX+2izgbcBrgJOB741VgJL0JPQr4sIutS5RNu8BTKq/tjHwbuAY4PXAz8YqQKnEIcAL68/PApaULLcE+G79uTd1+u/PNLfvLVUGsgZaAHwBOB04Gvgc8IdKI3oSSGtOXtFhuYnAs4HzgRXJe1YDR41yjClrTqplzUl31pyoH9Kak+d3WG4ycBDwHWBl8p6VwEtHOcaUNSdrttGqOflFfZ3LiTvQI2HNSRhOzcmTTT9rTgC2I653a8R1MADj+rBijcwK4CrijtxzaNawDADnEQeLJGlsLQMuA14JvAB4sP76eOC/gB0rikvalrihBZGkLKgwFmkk/kK0LoK4kb8h2KwrN78mqml/Q7QfnQx8lO536SYAewO7AhsRSediokPntUQH/DITiSZm6SgfE2h2lE6torzD3QbA04kT9vT6svOJKrqbiMy4X6YAU+vPl9J9EIF0+ceJi4526xDbAqIz7Mr6802Ju6ebEX/TncTABY8PMebpxF3aOcT38zfizsyDHd7TyUxgH+CpNLf3AqID74103t7TieYqU5LXplL8nXfbvpOAZ9XjmFVf/l7iom5RD39HajzwXKKT5zTgfuJYuH2I65H67VKiHP4VUS5PJ2oej+7yvolEubgLUUaOIwaeuI3YtzuVI5Pqn7Nu2/qKjtOVRLlVZCOaZfPU+rLziY7+txB3LPtlKs1ypaysLVt+CZ0HHZgIPIM4z61Pc4SqBcT2HEo5MRHYj/6VW88hvuNGufXbekyj5SSaN5fP77Rgfbn16s/b95OhnP877WMNTyW+ow3r63kQuIa4FummbN+ZDBxI3GGfQXxPRX9z41jbuf75wznW0u3Q67E2mfjeIc6VSzt8TsN0Yp/Zlji+G4MjXTmE9zeanD5G3OSmvq5DgK3qcd0LXE51HfvTfQ/iOy36Hs4njsfJwAlEM0WNkl6bdRV5H63Nu3YtWe5lxB28hcny7Y/Hgc/TuoOkXtjhve2P37e9d11ipIVf0drsof3xZ+C43v70nrwrWff7elg+bb701pJlvp8scyCxvf6T1qZ2jcfiegzje/jsCfUYHy1Yz3LgXJoFSrdmXdOB9wBXlMTVeNwBnNghpm90eG/7410l61gPOJMo/IvetwL4OrBJhzhSRxC1hkXrugzYBpt1qT96bdZV5MzkvSuJ/bLIK4l26GXHR40oEz5DeUfmV3V4b/ujvQnKLOADxM2pVR3edytwZG9/ek/mJus+pYflz0qWf23JMuOB9xIJVadtcBfRL6iT9YhhSzuVW+fS+6A0LycuAIvWdTlx8dnvZl0DxMVmjbjYm9p58Y4d4l9UEnvRo6wz/QDwaiIJ6PTef+oS59xk+bcRF7anMfj65p629x1JNL18uMPnPwacTSRiRY4cwna4vO29Q+kQvwFwDpGAFK37caJMWL/Ler6ZvOclxHXGRyi+zlgJfJm48O+HXpt1TQd+mix7G+Xl5SbJcjf3KU6VuJPmxh5qcjKTuIvUeH9ZgftHejuYGiehjQvWMZLkZLchvLdGdHbqh9FOTg4lDpDGz6uJQqP97/kGnZtGNpp/FBVAq5OfbwAOS34uS052LFhXp8eXStYz0uRkO+D/enz/XUR75k7eWfC+5bQmYAtpbXtvcqLhGklyMpvW/fKkkuV6PT5qwPUUX4yMJDnZfwjvrRE19P0wN1lnv5KTb1Mcc9HF6I87fNa2RO1KL9vjbqIWoJO3F7xvef3R+HkR0aohLQ9Haudkfe3n5CJzkuX7nZxMpPfzyQo63zSbmyz7NuC/S9bTPrhQr99pjTjXFtWGjEVysg29lwu30BzauUianLyMGKCj2zp/QiSSI9VLcrIJsW82lrua7iMd/iVZvooRa5807qS5oYeanEDcLe5W4P4RmAecARxA69j92xAX4vcn6/lpwTrWI3aw9OJ4Xv219sdObe/djShwLgSOJwrNRrXsJKJ6999pvXPXj7t0o52c/Kn+//8SFy+N5o9zgH+h9e/pdKfujGS5FcAnaRY4E4kTw01tn1mjc3KyktgfTiSaETTumk0E9iUSkrQWq6jZyRzi+/xystzZFH/n7TUfGxB3rhrvu6n+GbPrv58G/D+iGr2xzK2U3x0+mNZE7bvEKEkQBek+NAvedBuZnGi4RpKcAFyXvL+sSc3/EeXzaUSzxzT52J5IyBck6/l2wTpmEsfgMclyv6X4OG2/AbA/cVf9u8CxRNnRKJun1H//DZrH3mqaIz+NxNwk1n4kJ+l5aRFRgzKHZpk8jtieryXOQxeWfM5MIjForOtmYrum5dYLiJqmxjK3U36n/Xm0llvfA/as/26AaF7UuGucllt3laxvKE5O1veFHpafkyzfnmA0zv+HJ8uUnf+LkrWvJe9bDHyYuA6YQNyc2xX4Is1ttZIYAKjI3GRdN9b/X0CM5PQ04ppmHyJxSd1OnKvfRzQNSpOPomPtvws+u3GsvYbWbdXLsdZLcjKV1hueC4jvsXF+3RQ4ldaE+39pNt1qlyYnjZvUN9bj36r+9+xGbPs0QXl1yfqGoltysiOticb36V67B603IV7VhzhVYqTJySeT95e119yb7pnwZm2x7F2y3HBG65pJeTVd6qhk3Tf2uO5ORjs5aVwETCtZNj05LCG2cbutifanjeWOLVnXDKIQSj+7LDlZj6i16OYImieDWynfR4YzWldaE/RTygudScD/JMt+sGCZ8bTeSfpiyboGiCZ26TYyOdFwjTQ5aczIXSMSlSJl5WxqDtH3rEbc8NihZLnhjNa1MbBFD8udlKz7mh7X3cncZH39SE6+lfz+0B7Wt27J6+l6fkbnciu9Cz23YJlxtCYc/1ayrgEG1yrc1SX+XqQ1CmXnldScZPl+znPysuQ9DzL45mUq3c/mUXxOmkvrtrqDzrUHDb0ca1vTPNZWUz6YxWjNc/KRZJn5lLcm2I3WBKXs+iZNTmrEuXZKybJnJ8tdXv5n9KxTcrIfrYngv9L7wFunJu8ra/WhPhhpcpJWGY90JI6jk3V9rGSZ0R5K+JfJ+rs18+lmtJOTlUStRCeXJsufUfD7jya//1GXde1N6124fgwl/JNkfWV9loaanOxAs9boQVo7DxaZTbNt7d8ZXEilTQrvovPdlRnEfBQmJxqpkSYn6XFzxwhjSW90nFayzGgPJZzWFszusmw3c5N19SM5uTr5fbd2+GW2p1mbPJ/y/pcNm9BsxvsAg/sWviCJ6R7Kb2JBJEtp64W7hhZ6oeuT9XXrxwGjl5z8LnnP4T0s3xj6uEb062w3N/l92TIj8cZk3aeXLDMayclkWs9d3frfviVZ9j6aA/Wk0uTkETo3mdqQZlPDJ0rWNxRlycnLaR43qyjvr1omTXYvcyjhfD2cPJ/JyIZ9Tg+yfUawnpG4LIMYenUl3TtlpdXpLyv4ffpaWY1Awx+Jkdr6aTS292to7odfo3zktoa/EyckiGrr9oTvsOT5uXQepWQxbTPIShVJy+ZZpUv1Jrey+ekVxVAmHdlnv2Gu4yiaCca5xMVcJw8Qd6IhaqDab+6kZft5dB4F6lGi9qSfNkqeL+zzunu1I8399Q7ghz28J90Oz+uy7C305y5/qqpj7Zk0+/s+SHETztTXaZ5bN6f7MXkBccOlzAKaSfFkemvtMlSnEk1IpxLNSV9D1NgMRfo3bOxQwvlKs9sVdB+KdweizevWRDvZtIovvfMzWh2NJhFZdGPIzPVoTaj2SJ6P9O7caLus+yL/6BM0QPxtU2gOuTud5oX4SmI0s24uJdqB92oSUeOyC3FnZAat3/NuyfN+be+0rXCvJ44baSYhuxNV+g3pCaKXbX4pQ78bI/Vbe9nczU7Evr8VUTano+akd91Hq2yeQnOI1ZlEDUTarGbf5HluZfPVNOfzOB/4BHGT4v4hrGM45dYNxJ1giO/uhuR36fbqtdx6b4+f280ArRMuVpWcpNv0ih7fk27DPUqXClcNKZqmp9bXvRVxHm4fLr9hLDtcPzN5fjnNaQrKLCWuGV5S//kZNOcBKfLbHmK4l2az0eHWQBYZRyQh76z//DBxvr9yGOsyOVlDpJ26ygqgASJDPZ3uzZAa+rljQsT5IaJKvmgUjCLdqtWr1st4+Y8QJ8jZxMXKpjTvTsymmSjcQ/d5WHr9TIht90FiLPBe79r2a3tvnzz/KMUn3PYLn7RDffssxml74l7GwXe+E+UgLefK7liOIwateA+9N2Ptd9m8EdHW/Wi6N8FsyK1s/hLwBqL/zEzg08CniAvdXxFNTC6lc61rWm7NpfgGLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "8412a275",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g12703.png\"  align=\"left\" width=\"450\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eba45ea4",
+   "metadata": {},
+   "source": [
+    "### Efficiency"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e96eda2d",
+   "metadata": {},
+   "source": [
+    "- Each process updates $N^2/P$ entries per iteration\n",
+    "- 1 process broadcasts a message of length $N$ to $P-1$ processes per iteration\n",
+    "- $P-1$ processes receive one message of length $N$ per iteration\n",
+    "- The receive/computation ration is $O(P/N)$ which would be small if $P<<N$"
+   ]
+  },
+  {
+   "attachments": {
+    "g20431.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbwAAAGqCAYAAABwP/FZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAB6mSURBVHic7d15nBxlgf/xT89MJichgQRIIoIgyLEgRpBFQORmkUPkjIIg6OLiEcQTEPhxqFwrBlZBF6IigqJo+IkvxEUOdSE/5FoQ5CbcEAI5IOSe/v3x1Ox0hurpyaS7n+p5Pu/Xq19dU93p/qanp79dVU9VgdSadgfKFZfv9+PfLMzuO72BuSQVVFvsAFKdfAbYOHYIScVl4WmwGAKcGTuEpOKy8DQYPJxdTwG2jhlEUnFZeBoMLgReJ7yfvxU5i6SCsvA0GMwHLsim9wN2iZhFUkFZeBospgHPZ9PnxAwiqZgsPA0Wi4BvZ9M7AftGzCKpgCw8DSaXA09k0+fi+1tSBT8QNJgsA87IprcCjoiYRVLBWHgabK4B7sumzwE6I2aRVCAWngabMnBaNv0u4LiIWSQViIWnwej3wG3Z9GnAiHhRJBWFhafBqnspbwLwxZhBJBWDhafB6q/ADdn014G1ImaRVAAWngazU4AuYAzwlchZJEVm4Wkwe5AwahNgKo7YlJJm4Wmw+yawlDBwpSNyFkkRWXga7GYRjsAiKXF+41WregX4VTb9fF93JOyAPr7i57sbkkiSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmDzwRg3dghJKnFrAtsFDuEahsOHAX8DVgO7BU3jiS1nJ2BFcBDwAmEU1epIErAh4AZwMvAm0AZuCNmKElqYbcTPkcXET5XbwH2w2MbR7MJcBHhoLyvEX453Ze5wD7xoklSS9sdeJ2VP1fnAS8B04Ft4kVLx1jgi8DDwGzCYnc55/ISMCxSRklqdUMJS3Z5n69lwufvU4RzOU6IlHFQKgH7Al8FNgPWofY58lYAcxqcS5IGs3FAe437lAlr2J4DLgCuaXSowa4D+ARhEXoI/TshbDuO0JSkRisRlgbfSRgkaOGtplJ2PRw4BPgaMBEYTfUNqHOAPYGuhqeTpMGnBNxMWMrLswx4A3iGcPLiG4ClzYmWngnAaYR1yLPJX788OVo6SWptW/P2z9aubN6jwJepXoZqoG2Ay4EXCKMzu38xP40ZSpJa2HTC52iZMFrzJeBSYPOYodSjA/gI8EfCL2cpsHHURJLUejYmfH7OBm4E9qB/YycUyRjCEQL+OXYQSWoxk4FjgVGxg0iSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmS9L9KPZPldmCdaEkkSf0xB0rLYodoRR0V0+sDT8cKIknql+2Bu2KHaEVtPZNf2jxeDElS/1y0ZewEraoE5f2Bi2HuOBg7KnYgSVJf5i6Csa8AZ0LpJ7HTtJIOYBSwIYyNHEWSVNvY4cCGwJqRg7ScjirzPw/Ma2aQ2s4+HJ5/H/zwG7GT5HtkLHzrIvjZMbGTVPfJ6XDMWbDbrNhJ8p1wJqz1DJwzPXaSfOcfAI/vAv/55dhJ8r0xBE64An72ydhJqjvmR7D/xXDw32MnyfeFU2HE63DepbGT9NIGXBk7xCBQngLlcq/LhNipcpwLPBg7RB8mAUtih6hhITA5dog+zAQuix2iDycDT8QO0YdhwPLYIWqYB+wZO0QfbgWuih3i7crtOZ/TU2OnajVtte8iSVLrs/AkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUnoyJ/9yzWAhU1NUlsn0A6Mjh2kilHZdVHzdRtJcTN2EH7PRc03jPAlsaj5hmbXRc0HUAJGUNyM7RTyPbh1OzwQO0TLK0F5CnD1yrMnLoeXogTqQzvhj2V57CB96KD4+VYA5dhBqmjPrldETVFdW3Yp+u+46Pl8D66ydmB57wWUE6E0LUaaFlaeAuVyr8uE2KlynAs8GDtEHyYBS2KHqGEhMDl2iD7MBC6LHaIPJwNPxA7Rh2EUu+wA5gF7xg7Rh1uBq2KHeLtye87n9NTYqVqN2/AkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSejIn/2Oh4GupiapbRQh72uxg1TRDnRS3HwAI4DbgGWRc1QzBpgMHBo7SBUjKfbvuER4HxY1H8CawG+ApbGDVLEmUAb+JXaQlXUAy2OHaHlVCm/zqfDC3OZGqelYYFvghNhBqhgH/Ag4JnKOvvwaOAN4InaQKi4CngYujh2kiiOAPYBPxw5SRSdwLcV+D/4SuBC4N3aQKs4DXs+uC2RoGzAjdopBoDwFyuVelwmxU+U4F3gwdog+TAKWxA5Rw0LCElRRzQQuix2iDydT3C8LAMMo/mLAPGDP2CH6cCtwVewQb1duz/mcnho7VatxG54kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCRaeJCkJFp4kKQkWniQpCR1V5k+G8pymJqnpiInwynC4dfvYSfL9cTycUYI7C5oP4INtcOQ/wQlDYifJt/soWHMd+E1BX8Oj14dZw+D2guZ7qRMOKsHMguYD2LEd9t4MTl8QO0m+PUfDsCXwu6K9hi6c1EEJylOAq2MHkSStkhOhNC12iFbitwZJUhIsPElSEiw8SVKecbED1Fu1QStfA+Y3M0htJx4EL24F154VO0m+h8bA1G/BzZ+LnaS6PS6BUy6A3Z6NnSTfwd+Adz0HF/48dpJ8J+8Dj+0E130zdpJ8C4bARy+GW/4tdpLq9vweHP9DOOQfsZPkO+wkWHceXDI9dpJe2oBLm/h8mwO3AhOBriY+b0NVK7yHgIKN0rxpR5i7GLg/dpJ8942Hv5QpbD6AP3fBro/Cbo/GTpLvlrdgo9co7Gv4h/fC80sobL45nS3wHlwBmz8JhxQ04y1vwsS5FO81bPbauBOAscCHgVua/NwN4ypNSVKlNuBgoBM4KXKWurLwJEmVdgXWzqY/BIyKmKWuLDxJUqUvEZbuAIYCh0TMUlcWniSp20hg24qfO4EvRMpSdxaeJKnbYbx9FeY7skvLs/AkSd2+SFjKq7QW8OkIWerOwpMkAUwCJuTM7wCObnKWhrDwJEkAx9MzOrO3kcDkJmZpCAtPkgRwFNUPRjKesLqzpVl4kqT3U3t/u72Bgp5Ls38sPEnSVMLBoucALwJzs/kvZ5f5wBrAPlHS1YmFJ0lpawMOAG4iLMVNAv6Y3fYdwi4JRxKK77MxAtZLtfW1kqQ0dAE7Ek4akGcFcANwI7BJs0I1gkt4kqRqZVdpBfBIo4M0koUnSUqChSdJSoKFJ0lKgoUnSUqChSdJSoKFJ0lKgoUnSUqChSdJSoKFJ0lKgoUnSUqChSdJSoKFJ0lKgoUnSUqChSdJSoKFJ0lKgoUnSUqChSdJSoKFJ0lKgoUnSUqChSdJSoKFJ0lKQkf+7NO3gacXNDdKLfPWh8Uj4agPxE6S7/UxsKKtuPkAyu1w89bwyJqxk+Rbuga8NqG4r+GcDWDJ8OLmWzoEyqXi5gMod8A9m8NRsYNUsWQszC/gazikDabHDtHySlCeAly98uxNFsKL5SiJqlo2LHxgdy6MnSRfuQ0Wj4Dhb8ZOUt2SUdC5CEorYifJt2QktK+AjsWxk+RbNhS6OmBoQd+DAItGtcB7cDGUlsdOkm/JCGgvQ8ei2ElW1gHMH9Vr5olQmtagJ/wFcDgwFbi4Qc/RdFWW8B4/DJjT1CQ1bf45mLsjvPzx2EnyXTUejvstLNw9dpLqOm+HU4+HUx+NnSTf2Omw0WNwz7mxk+R739Hw/AHw6sGxk+R7qhPec1ux34ND/wSfPhkuvit2knzjfgATX4UHzoidpJc24M7YIVqd2/AkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJsPAkSUmw8CRJSbDwJElJqHKkFamutgC2BzYCxhDedwuAJ4H7gHuArpx/tyWwSZXHXAK8CDySTed5L/CubPoPQK1Dlo0BPpxN/wMYyBFpRhBybw1MBMYCI4FFwCxgJnAX+f9fSQ1k4amR3gd8HvinKrd/OLt+BZgO7aVet+9K7aMMvwXMAH5EKJVKewKHZtN/pXbhrQucnE1fysAK7/eE0suzAzAle9zTCQUoqUksPDXKIcCX6Vlt/jzwN8JS2RLCks+WhFLMiubjz8N/V3u8/6FnSW4o8M7sMUYAHwe2AU7g7aXXbCVgBfAQ8BwwG1gOrAfsCKwFvAe4DDiSwh2zVhq8LDw1wl7AV7PpBcD5wM1A3hk4xgLHAIeGrqjqdODlXvM2Ac4irCrdgrA0+KOBhq6TU4AHgLwzFnQSjj5/CD3/7wublkxKnINWVG/rED70ARYCnwX+i/yyA5gLXAR8Dl5d1VPGPE5YilyW/fyRLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "96b3a09f",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g20431.png\"  align=\"left\" width=\"225\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "494aa965",
+   "metadata": {},
+   "source": [
+    "## Parallel Implementation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c624722a",
+   "metadata": {},
+   "source": [
+    "### Generating test data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "09937668",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "rand_distance_table (generic function with 1 method)"
+      ]
+     },
+     "execution_count": 1,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "function rand_distance_table(n)\n",
+    "  threshold = 0.4\n",
+    "  mincost = 3\n",
+    "  maxcost = 10\n",
+    "  infinity = 10000*maxcost\n",
+    "  C = fill(infinity,n,n)\n",
+    "  for j in 1:n\n",
+    "    for i in 1:n\n",
+    "      if rand() > threshold\n",
+    "        C[i,j] = rand(mincost:maxcost)\n",
+    "      end\n",
+    "    end\n",
+    "    C[j,j] = 0\n",
+    "  end\n",
+    "  C\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "3116096c",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "10×10 Matrix{Int64}:\n",
+       "      0       3  100000  100000  100000  …       3  100000       7  100000\n",
+       "      8       0       3      10       5          9       9      10      10\n",
+       " 100000       7       0  100000       8     100000  100000       7       7\n",
+       "      3  100000       6       0       7          9       5      10      10\n",
+       "     10       5  100000       7       0     100000  100000       8  100000\n",
+       "      5      10  100000       3       9  …       4      10       7  100000\n",
+       " 100000       9       8      10  100000          0       4  100000       4\n",
+       "      5      10       9  100000       6     100000       0       5  100000\n",
+       "      3      10       7      10       3          4  100000       0  100000\n",
+       "     10  100000       9       4       6          3  100000       7       0"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "rand_distance_table(10)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0b451628",
+   "metadata": {},
+   "source": [
+    "### Creating MPI workers"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "91a1cbd8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "] add MPI, MPIClusterManagers"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "6bc0122b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "MPI.ThreadLevel(2)"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "using MPIClusterManagers\n",
+    "using Distributed\n",
+    "if procs() == workers()\n",
+    "    nranks = 3\n",
+    "    manager = MPIWorkerManager(nranks)\n",
+    "    addprocs(manager)\n",
+    "    @everywhere using MPI; MPI.Init()\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "680e56cf",
+   "metadata": {},
+   "source": [
+    "### Code"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "22234b2f",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "floyd_dist! (generic function with 1 method)"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "function floyd_dist!(C)\n",
+    "    m = size(C,1)\n",
+    "    @assert mod(m,nworkers()) == 0\n",
+    "    nrows_w = div(m,nworkers())\n",
+    "    @sync for (i,w) in enumerate(workers())\n",
+    "        rows_w = (1:nrows_w) .+ (i-1)*nrows_w\n",
+    "        Cw = C[rows_w,:]\n",
+    "        @async C[rows_w,:] = remotecall_fetch(floyd_worker!,w,Cw,rows_w)\n",
+    "    end\n",
+    "    C\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "id": "d5a8d3a8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "@everywhere function floyd_worker!(Cw,rows_w)\n",
+    "    comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
+    "    rank = MPI.Comm_rank(comm)\n",
+    "    nranks = MPI.Comm_size(comm)\n",
+    "    m,n = size(Cw)\n",
+    "    C_k = similar(Cw,n)\n",
+    "    for k in 1:n\n",
+    "        if k in rows_w\n",
+    "            myk = (k-first(rows_w))+1\n",
+    "            C_k .= view(Cw,myk,:)\n",
+    "            for proc in 0:(nranks-1)\n",
+    "                if rank == proc\n",
+    "                    continue\n",
+    "                end\n",
+    "                MPI.Send(C_k,comm;dest=proc,tag=0)\n",
+    "            end\n",
+    "        else\n",
+    "            MPI.Recv!(C_k,comm,source=MPI.ANY_SOURCE,tag=0)\n",
+    "        end\n",
+    "        for i in 1:m\n",
+    "            for j in 1:n\n",
+    "                @inbounds Cw[i,j] = min(Cw[i,j],Cw[i,k]+C_k[j])\n",
+    "            end\n",
+    "        end\n",
+    "    end\n",
+    "    Cw\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "id": "dd77ee3d",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\u001b[32m\u001b[1mTest Passed\u001b[22m\u001b[39m"
+      ]
+     },
+     "execution_count": 24,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "using Test\n",
+    "load = 10\n",
+    "n = nworkers()*load\n",
+    "C = rand_distance_table(n)\n",
+    "C_seq = floyd!(copy(C))\n",
+    "C_par = floyd_dist!(copy(C))\n",
+    "@test C_seq == C_par"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "91a772df",
+   "metadata": {},
+   "source": [
+    "### Is this implementation correct?\n",
+    "\n",
+    "- Point-to-point messages are *non-overtaking* (i.e. FIFO order) according to section 3.5 of the MPI standard 4.0\n",
+    "\n",
+    "- Unfortunately this is not enough in this case"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1277772d",
+   "metadata": {},
+   "source": [
+    "If we are lucky all messages will arrive in order and we will process all rows in the right order in all processors."
+   ]
+  },
+  {
+   "attachments": {
+    "g33659.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUYAAAFECAYAAABbH5f7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d15mBxVvf/xd00mmUz2CdlIAiRhk0UjCZvoRWQJgsBl1x8octm9onABAVkERHABREFUEFEEIiooggthFRcWIayCbCYsySSThAwkITNZ6/fH91Sqpqa6urqnt5n+vJ6nnu6qrqo+PTP9mVOnTp2CuuDvBb7fi6ddqv0TrB5/bg38/IudflLtn171+OfVwM+/6Kmh2j8+EZFao2AUEYlprHYBqmAh8PFqFyKDJ4Fh1S5EjTke+Hu1C5HH2Vg5JTQL+HK1C5HHJKycQH0G41rwXq12IfLz11e7BDVofu3/7vz2apegBq3oBb+3LnM6lBYRiVEwiojEKBhFRGIUjCIiMQpGEZEYBaOISEw9dtcRSeEPADYDJmJ9ON4Eb251yyT5bfi9jQYGAauAV8FrK2ZvCsYu/DnAiIQX1gHvAM8DvwPuAG9Nwvb9gG2AHYHpbtoKq5mvA290WYpd9/xLgC/leHEZMBf4C3AjePMTtt8EOAQ4DPgo0C/2+tvAdcDV4K0uTZkF/M2xCxmSrAIWAU8BM8F7MMc+LgH2xr5rTQmvPw9cBdwCnt/99brWZRCJt1PWW5rxIvPZ7ssU3/6YlG3WFljm9si2GkTCpn1zrPOdjL+3FeAfnbD9vxLWXZmw7DHwh+cp7xUaRAJig0jckWOdrQoY2OE34A9M2Mf82HrrwO9I2P4WSBsbomtZ1MaY7Clgn8h0EHAO8Jp7fRpwt6shJlkPvALMBP5V3qJKzFGEv7d9gROB37jXBgM/B/+jCdstB24E9gOGgzcIuyTzU8BLbp1dgR+WreT17Va6fueOBC4DlrrXDweuTdjuWeDbbpuJQBN4zcA47Hc/z633WTcvoYJrjPfmeH0w+E9G9nV47PUtwP84+JFrnP1bVWPsiYJrjAk1eQD/jMg6sd+vfyL4I1PKMBT8l92268GfmLKuaoxAETXGS3KsMxn8JZHa4KQCyzE5UoN8KmU91RiL570PfCOyYJ/Y66+D9wh4yypZKsnk+9gAIgB7gN8/fMn7CXhLE7YJXl8OXBPMYDVHqQhvLnC9m2kA9ipi+2Dgka2ybqVgLNxzkec5aidSe7x1hM0aTcCYAnfwZuT5kJIUSbJ6PvI8pbaeU/BP8J2sGygYCzcg8rzAw2OpskgtkXUFbrtl5PlbJSiLZNeD35s/GjtjDfBw1q0UjIXbMfL8jWoVQgrlDwQ+5GaWA4sL2LYROMHNtAP/KGXJJK+dIs8z9in1m8A/EHgQq+EvAC7K+obqx1gQfzzw9ciCP1arJFII38POXLa4BbPcoXVWXwG2c8+/Cd6qUpZO0vi7Ev5TWgU8kGO9McBjbmYwMNY9XwPcAfwfePOStkyiYEy2CfgnReYHAx/AuhAEHcAfAG9Wty2l2o4iHCy2H7ApMAPrYgWwErg4++78GcClbuZR4OpSFFK6mR77zo0EPgwcSngofVXKlSz9gCkJy1uxLj0dhRRGwZhsW8IzYUnuAY6pUFmkMN9Kea0V+Bx4L2bblb8rVtvohzWbHAme2pXL41NuSrIe+C5wYcr2Swl7iYzEblWwG3Ag1pPkC+DvmXUkcQVjsmWEnbmh2yWB3hNVKZVk8QIQvWxvOTAHuyTwDvAy1hz8acCfgaFYoO6dfDmhlMgC7OcciF4SeDt4/0nf3MtxmO3PAO4CJgC/BH/HLJcGKhiTPQbeJ6tdCCnKp8BL6cSfhb89dmOkEdhJmn3yfzGlh34CXuaTI9l594F/NXAe1pwyDZidbyudlRbpwv8AVvMYhR0l7APeS+nbSI2L1iS3zbKBglFkA39LrHvHWOA9YD/wnkvfRnqB6KFzwgg83SkYRQDwNwPuB8Zjbcz7gpdrSCzpXfaIPJ+TZQMFowj+ptjJmc2w7jwH6gRbb+BvA36eyzP9nYEz3MwiwuumU+nkS8n5l9P1H84O7rEB/HhXktvAe6Ey5ZIUD2DdO8AGTt0f/P1T1n/IGvWlyo4ATgP/LuCvwMtYL4SR2D+5vYGjsX6QPnBm1oGGFYyldzbdRoAGbFSWc2LLnsK6l0h1RQcm+Lib0qwGFIy1YSRwnJtyWQmcAd6tWXeqYOzqEey6ymd6sI8Hyd5EUdT9KKSb1wnPPHYWsf1DZGyUdzK1U0leKwl/b8X8TK/HBqLdB9gF2BgIRvlehVU6/ox1BSqoC5eCsQvvkBLsI8dgqlI+3g3ADT3Y/oCSFUUK4AWhVuz2bcBNbnL84cB6N4Zm0RSMItKHeO+VYi86Ky0iEqNgFBGJUTCKiMTUYxtjo905rObpn1Z343rB7y7Pfafr0uBe8Hvrci+ZegzGcai7RW/182oXQIrySXrZd061EhGRGAWjiEhMPR5Kzwdqvb0D7IL3EXnXqi8HUPuX4n2LcNACMb8DPl3tQuSxJbDhlhf1GIw+eGuqXYj88o6+Xo/W1v7vzl9f7RLUoPW94PfW5V4+OpQWEYlRMIqIxCgYRURiFIwiIjEVDEb/SfCfBX905d5TRKRwlTwrPRUbYrx/Bd9TRKRgOpQWEYlRMIqIxCgYRURiFIwiIjH1eElgLg3Y9ZLTgU2AV4C7qloiEakKBWPoTuDg2LyCUaQOKRhDtwI3Ak+5Kc3HgS8AwR3J2t3jMmAd8D52U/YO7D7Hq92ytcBybISId90277r55e51EakyBWPozgLW3QyYgf38hpa4HC5Uxw2DwcAAYP7PsdBd5tYJgvg9YD2wAliD3cB8lZtWumUr3Dq5QjzYNghxkbqnYCzOL9wU1wwMLOIxYds9j4W1AyyvFi/HwqvFvT4ltu0IwCvh5+skDMroY9prpdgmCHaRqkoJRn8ScCrwCew+Kc+Ad4B7bRzweeBp8O4vcxl7kw43tedbMb+ZR2LVRYAvgfdEho16EMSZHwdiAR1dNgzoV/RH7S4SqmOHwRD3Nq3XYgP4ljqQo9uU4HcnJbYR8E4l3zBHMPoHAbdhf5GB1yPPlwBfxv5IdyhT2aRwJQzmopQhkKd+CvxBtmnbapKDeQilv9S0yJrxYbvCB13R2rYHTsq+LR2EzSMSegg7YvoBMBNrsy+rhGD0JwG3YycCTsfa3m4CmsJ1vLXg3wZ8BfxNwHu73AWVXqEMwXzfXGCSmzkTvFkpK5eyZpy2bTyYB7Hh+/FbNwGwq5uKUY4acZZta/Ek4IVYIF4PfA14zC0r250Hk2qMJ2G/8CPAu8MW+Unj7Af3R9gSUDBKLaiBGvNb34Ehp1oxWn8FO11K6Zs34sEMNdfOfPSusLMr4oLJwDH5t6GDsEdH1J+BNqxtfbKb9sSC8Trg15Q4zJOC8cPYf43fJrwW9ZZ7HF/KAon0Yh2wiTuz3wKMX07kBksVELT1Bk0LQXA2YbXaoBdFAzDcbRME6lD3etAVIgjeAW5ZP7d/D9jYbTvc7SvYNmKmmwCYBtxcwOdYR/ceGBvH1hnnpl2Ab2Ph+Q3CXOqRpGBsAt4HL187xxj3WPbj/QoJ/hDAftkDsL9uSD68GIU1COuuVVIr4mFSaUGTwkB45EwYe6b1Blv8MOx3uVsn+E4VGuJjc7xnP+zkzAxgMXB+KT5IUjC+BewJ/njwWt2ypC//zu7xtVIUpAacAVwWmR8PLHXP9wYejLzWD6vaN2CHAK3AAuwPsj1lfjHWiCzSF610E7D70sjypcADPdjvBOCfWM018C72nXoWOynzGCWspCQF493AscD3wD/KTrTE+TsAJwNzgedKVZgqm4n98JM8E5vvD1yK1RrHuGkUsAUwmvTBOZZgAbnETW2R+cXYmf7F0Npgu9K4vlL3zscOpd/DKhllCcOopGC8C/gLcASwGfg/B0YC/cA/GLsc7kQsvc/OcMjdW7zhpiw6gYtTXm/GfpHjsUOHYIovmwpMJOyvGDHBPQ4E1v0BeJP0Wmk7dhJsWbddifReLcCnsJ4yZQ3DqIRg9HzwD3UFmUF4yAzwO/e4But0fEe5C9hLdWBnzLJ2J2imW3Be+WNY3Gz5d+/LsNgDtsUSsyn3rugk/yF9MD+f8JptkVrUjl2CW1E5Onh77eB/EtgHG3FmO6yGuBR4ArgJvLkVKmM9CLqZtAKzbdGZ3yc8GXRW7MqXIEiTaqHR+fHAToRnIJOkBWmucBXp01IuCfR84D43SW2JBmmW7iCFBOl2dO8aERUN0iy10gVU58z9F7A+tl9HtWIpkAaRqA89CdJ8YToF60+Wq3PxKuxII2uYLqQ0l8TNAi7Hjnj+BpyHNR2I5JUnGP0BWJvWyJSVngdPXVD6lmiQZjEQ+xvJGqT7k37mPqhptsOMMXahw8bAPw7G+rNFg7QN6xAcF7TxTiO8UuIZ4FzgpYyfS+pUrkEkdgauAD5G/vvCjMf+SKV+BX05C2l/DA7d84Tpc002hsA6gFPcFBcJ0i610OhYmRPdtAvwb+AC4O8FlFfqSNIgElOx7jrN2B/Yv9nQaTORxs+TYgQBlkfbXFg7ybp4/voUOP01rNY4KjKNc4+jga3c81yH9kG/0/uBF4BvEva2EAGSa4xfxkLxGuAc8DSqs1RZI1aBPO0NOP2hDBs0YOH3d2DzhNfXE/6z3xkFo8QkBeNk7Prnr4DXV66DlvqyHmtbHBVZtshNr2NXOd2LXQMv0k1SMC7EgrHWxmQTyaoZuAE7G/48dob6V3QdbFkkp6Rg/B3w/4DdUOO09E4NwNHAo2jQDilC0hnnO7DLAW8Gf5cKl0ekFN4HHkGhKEXKda300diouI+B/xp2CPJ+jn2cbJcQioj0DUnddRqBW4DD3IKt3JTLaSUvlYhIFSW1MZ4AfAY7CXM11tdrUco+lpShXCIiVZMUjPu5x/3Biw/QKiLS5yWdfBkIvKtQLine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "f84ef616",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g33659.png\"  align=\"left\" width=\"225\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "df60e4e7",
+   "metadata": {},
+   "source": [
+    "However, FIFO ordering is not enough. In the next figure communication between process 1 and process 3 is particularly slow. Note that process 3 receives messages from process 1 after the messages received from 2 even though FIFO order is satisfied between any two processors."
+   ]
+  },
+  {
+   "attachments": {
+    "g33721.png": {
+     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUYAAAFECAYAAABbH5f7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7Z13mBvV9YZfua+NyxpjjGnGYAi9mGp6semEFtKAJISW5AcJnRAILRDAEEpCEloCpoSEHiBgsGmh2KEabCAUF4K7YXHvnt8f3wyanR1JI2mk0UjnfZ55tCPNjM7dXX06c+93z4WGwNkXHCfF205J/waTw5lUA7//Urfbkv7tJYdzQQ38/kve2iX96zMMw6g1TBgNwzACdEg6gASYAeyZdBAReB3okXQQNcaPgZeTDqIA56I4jSwjgdOTDqIAA1CcQGMK4wrIfJR0EIVxViUdQQ0ytfb/dk5L0hHUIAtS8HdrtWe30oZhGAFMGA3DMAKYMBqGYQQwYTQMwwhgwmgYhhHAhNEwDCNAI9p1DCMPTidgfWAd5OGYAplJycZkFObrv9saQFdgKfARZGaWcjUTxlY4k4CeIS+sBL4A3gUeAR6EzPKQ8zsAmwKD3W17YBDKzFdApm9Fwm54nEuB03K8OA+YBLwA3A6ZqSHnrwscARwF7Aq0D7z+P+Bm4HrILIsnZgOcDdFEhjCWArOAN4D7IDM6xzUuBfZDn7fOIa+/C1wH3A0Zp+3rDU2rIhL/y3NcS8RJ5m+6H6bg+T/Ic06IkOaN2R+LFZHQtn+OY66J+HdbAM73Qs4fH3LsopDnXgMn7IvTf63hVkQCAkUkHsxxzMZFFHZ4AJwuIdeYGjhuJTiLQ86/G/LVhmgdi/UxhvMGMNS3HQacB3zsvr4d8E9w2oefzirgv8B9wPjKhmoE+B7Zv9v+wEmA98HsBtwFzq4h580HbgcOBHpCpiuaknkw8L57zM7AHysXekNzD60/c8cAVwBfuq8fDfw+5Lx3gKvdc9YBOkOmCeiH/vafu8cd6+4bWYrOGJ/O8Xo3cF73XevowOsbgbMnOL45zs49ljGWQ9EZY0gmD+Cc6Tsm8Pd1TgKnd54YuoPzoXvuKnDWyXOsZYxACRnjpTmO2QCcOb5scECRcWzgyyDfyHOcZYylk1kI/Mb3xNDA659A5kXIzKtmVEYkbkQFRAD2Aqdj9qXMbZD5MuQc7/X5wE3eDsocjaqQmQTc4u60A/Yt4Xyv8MjGUc8yYSyecb6fc2QnRu2RWUm2W6MzUOxA2BTfz6vFEpIRlXd9P+fJ1nPifQl+EfUEE8bi6eT7eUViURil4MsSWVnkuYN8P38WQyxGdMr4uzlroBFrgOejnmXCWDzb+36enFQQRrE4XYCt3J35wOwizu0AnOjutACvxBmZUZAdfD9H9JQ6ncE5FBiNMvzpwMVR39B8jEXh9Acu8z3xZFKRGMXgtEMjl83uEyPdW+uonANs7v78W8gsjTM6Ix/OzmS/lJYCo3Ic1xd4zd3pBqzp/rwcuRLOgMznYWeGYcIYzrrgnOzb7wZ8A1kIernPjYLMyDZnGknzPbLFYtsD6wHDkMUKYCFwSfTLOcOAy92dV4Hr4wjSaMPgwGeuN7ANcCTZW+nr8sxkaQ8MDHl+GrL0LC4mGBPGcDYjOxIWxuPA8VWKxSiOq/K8Ng04FjITol3K2RllG+1Rt8kxkLF+5cpwsLuFsQr4HXBRnvO/JOsS6Y2WKhgCHIqcJD8BZ5+olcRNGMOZR9bMDW2mBGbGJhKVEYX3AP+0vfnARDQl8EHIRMwcnO2Ap4DuSFD3C59OaMTEdPR79vBPCbwfMp/mPz2T4zbbGQY8CqwN/A2c7aNMDTRhDOc1yByQdBBGSRwMmTwm/ig4WwLPoG6T2cDQwh9Mo0xug0zkwZHoZJ4B53rgAtSdsh3wZqGzbFTaMFrhbAo8C6yO7hKGQub9/OcYNY4/k9wsygkmjIbxNc4g9CFaE5gLHAiZcfnPMVKA/9Y5pAJPW0wYDQMAZ32UKfZHfcz7QyZXSSwjXezl+3lilBNMGA0DZz00OLM+sAg41AbY0oCzKTgFpmc6OwJnujuzyM6bzosNvsSOcyWtv3C2dR/bgRO0ktwLmfeqE5eRh1HI3gEqnHoQOAflOf45deobCfMt4OfgPAq8BHyIXAi90ZfcfsD3kQ/SAc6KWmjYhDF+zqVNBWhAYnle4Lk3kL3ESBZ/YYI93S0fy9CotZE8vYET3C0Xi4AzIXNP1IuaMLbmBTSv8u0yrjGa6F0UJa1HYbThE7Ijj0tKOP85InbKu0TqpzIKsojs362U3+ktqBDtUGAnYC3Aq/K9FCUdTyErUFEWLhPGVmSOiOEaOYqpGpUjcytwaxnnHxJbKEYRZDxRK/X8mcBf3M3F6QmscmtolowJo2EYdURmbhxXsVFpwzCMACaMhmEYAUwYDcMwAjRiH2MHrRxW89iXVlv6peBvV2Dd6YakWwr+bq3WkmlEYeyH2S3Syp1JB2CUxAGk7DNnWYlhGEYAE0bDMIwAjXgrPRWo9f4O0IT3XgWPaiwOofan4l1FtmiBIR4Bvp10EAUYBHy95EUjCqMDmeVJB1GYgtXXG5EVtf+3c1YlHUENsioFf7dWa/nYrbRhGEYAE0bDMIwAJoyGYRgBTBgNwzACVFEYndfBeQecNar3noZhGMVTzVHprVGJ8Y5VfE/DMIyisVtpwzCMACaMhmEYAUwYDcMwApgwGoZhBGjEKYG5aIfmSw4G1gX+CzyaaESGYSSCCWOWh4Fv+vYfwoTRMBoSu5XOcg+q3tIPrVWbj2Zg14pHZBhGIpgwZnkQeBKYGeHY3wIvAlcCnSoZlGEY1ceEsTQuAh4Hfgm8AWyVbDiGYcRJHmF0BoBzLThvgjMVnCd8r/UD5zxwhlY8wtpkNnAE8ANU9HYscB72RWMYdUGOwRfnMOBeYDXfk5/4fp4DnI6qTG9bodjSwAjgJffxKmA/4IeoSrhhGCklJMNxBgD3oxLSP0fWlWdbH5NZgYRzG3DWrWyINc9kYG/gfGAPYDzw/SQDMgyjPMJu/U4GmoATIHMTZHKN0HrrIwyqSGTpYiVwNRqpnolGuP+BRq8Nw0gZYcK4DTAf+fo8whYg+cx97B93UCnmDdS1cBNwNPAOsFeSARmGUTxhwtgZWAiZQov69HUfl8UbUmJ0QRleM/q9dPLth/XF9gEyIc8vRl0QB7jnPQfciH6vhmGkgLAP/GfAPuD0h8y0POfu6D5+HH9YiXAmcIVvvz/wpfvzfsBo32vt0S1zO2AJMA2YDrS4m7d/JepvPB0YBhwLvFmxFhiGEQthwvhPNLJ6AzjfcwdaArfSzrbAKcAkYFxlQ6wa9wH/yfHa24H9jsDlKGvs6259gI2ANQjPxL+BbrUXoi+fOe42E9l/5riPs/Q4rZ0uZXV9DaPahAnjo8ALwLeA9cG5E+gNtAfncGBPNEDTFTg3wi13WpjsblFYAlyS5/UmYC2UdXq34xsCP0Kj/GsAC1BV83UInT2ztvvYBVj5BDCF8Ky0xbf9D5gXsQ2GYeQgRBgzDjhHIsvOMLK3zACPuI/LgdMg82ClA0wpi4GJ7ubnUuA04BpgY+D/0Ah2ExJPn5he+2eY3ST9e/pDmJ0BNkOKma+/cgm5xTO4PxX4qtzGGka9kcPgnWkB5wBgKHA4sDnQDfW5jQX+AplJVYqxnliFBmKeQz7Qu4HDULfENHdz+yDPuhEJJsDZkBnru44npAExbbPfH9gB6JknpnxCmktcDaOuyVN2LOMAz7ibES/voUz8EuAcYCc0vfCFiOcvdrdpZP2k+ShGSDd3n8uFX0ijZKXTCbd7VZqfII/tZVhWbBSJ1WNMjiVotsxo4K8oi/w9cC6wNOb3KkdIC4npQFSqLcy6BGrLl0QX0xkosy6XkcgVcDjwb+ACbKqmEZECwuh0QoMD+WZwvAuZ5THG1Gg8C2wB3IxsPfsiW0+S+IU0Cl3QAF1UIT2I/AU3vEyzBYb1VZ2OtYBXDgfWpLWQzkQzj4J4fbzboQvsg9wF5wPvR2yX0aDkKiKxI3AtmuJWqGLMWuhb3iidr5Df8VHgFuA1uMJRVbNUFOzxvJzF9D96t+4FxHRcZyXTKwFOdbcgPiFtlYV29x2zjrvtBHwAXAi8XES8RgMRIozO1qivqwn9g30ALMpzjXqZ+VILPIC8lHfBhXtKEO5EDp+6wxOwAsycBCsGyOL5j1PhFx+jrLGPb+vnPq6BRvtzzUqCrO/0WdTX+1uybgvDAMIzxtORKN4InA+ZJdUNqeGZAuwD1y6AC5pgS2CnA5EboEHpgBLIn0+GXzwX4YR2SPxeRv7RIKvIftnviAmjESBMGDdAWeC5kLFsMBlWwVlLYViTuhufuRh5GE8lO03RyM0q1LfYx/fcLHf7BM1yehoVSzGMNoQJ4wwkjCuqHIvRhi1Rorjd3fDB94Gdka3n+UTDqn2agFvRl8i7aIT677QutmwYOQnr2X8EVe4eUuVYjFC6AO/fDOyP/l6jsWo9hWiHBrM2QcWDr8BE0SiCMGF8EE0HvMsdnTZqg1HI1nMf6gd+E9XONNqyEK3iaDYyoyRyzZX+PqpAPQacj9G37cIc1zgZMjazoDp8hTodH0O2njHAxcBw4jFFG4ZBuF2nA5rDe5T7xMbulovTY4/KKITP1sNVaL71cbQtWmEYRgmEDb6cCHwHGWZvQF6vWXmu8UUF4jIKMwUtwnU6Wm/mLTSd8NYkgzKMeiBMGA90Hw+CzDvVDMYoGgcNxIxG5ctuQdXGzdZjGGUQNvjSBfjKRDFVjEdWnqtRF8gEsl9whmEUSZgwjgN6gtOj2sEYZeFV6xmGPKhPogyya5JBGUYaCRPGm9Ho56+rHIsRD6ORredetATF62hJV8MwIhLWx7gMlXW5HpxN0Qj1ZHJ7wqzsWO0xF41S/xNljWNRbcLLCS/RZRiGj1yj0pe5Px/kbvmwsmO1ywNIFO9CfsehwPHALine truncated
+    }
+   },
+   "cell_type": "markdown",
+   "id": "bdae65f8",
+   "metadata": {},
+   "source": [
+    "<div>\n",
+    "<img src=\"attachment:g33721.png\"  align=\"left\" width=\"225\"/>\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "86534783",
+   "metadata": {},
+   "source": [
+    "### Possible solutions\n",
+    "\n",
+    "- Use synchronous send MPI_SSEND (less efficient). Note that the blocking send MPI_SEND used above does not guarantee that the message was received.\n",
+    "- Barrier at the end of each iteration over $k$ (simple solution, but synchronization overhead)\n",
+    "- Order incoming messages (buffering and extra user code needed)\n",
+    "- Use a specific rank id instead of `MPI.ANY_SOURCE` (one needs to know which are the rows owned by the other ranks)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "db2b586f",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Julia 1.9.0",
+   "language": "julia",
+   "name": "julia-1.9"
+  },
+  "language_info": {
+   "file_extension": ".jl",
+   "mimetype": "application/julia",
+   "name": "julia",
+   "version": "1.9.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}