XM_40017/notebooks/jacobi_method.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2e15ced8",
   "metadata": {},
   "source": [
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/39/VU_logo.png/800px-VU_logo.png?20161029201021\" width=\"350\">\n",
    "\n",
    "### Programming large-scale parallel systems"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8549215",
   "metadata": {},
   "source": [
    "# Jacobi method"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e0ef563",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "In this notebook, we will learn\n",
    "\n",
    "- How to paralleize a Jacobi method\n",
    "- How the data partition can impact the performance of a distributed algorithm\n",
    "- How to use latency hiding\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1dc78750",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "jacobi_4_check (generic function with 1 method)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Printf\n",
    "\n",
    "function answer_checker(answer,solution)\n",
    "    if answer == solution\n",
    "        \"🥳 Well done! \"\n",
    "    else\n",
    "        \"It's not correct. Keep trying! 💪\"\n",
    "    end |> println\n",
    "end\n",
    "gauss_seidel_1_check(answer) = answer_checker(answer,\"c\")\n",
    "jacobi_1_check(answer) = answer_checker(answer, \"d\")\n",
    "jacobi_2_check(answer) = answer_checker(answer, \"b\")\n",
    "jacobi_3_check(answer) = answer_checker(answer, \"c\")\n",
    "jacobi_4_check(anwswer) = answer_checker(answer, \"d\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4cb59d5",
   "metadata": {},
   "source": [
    "## The Jacobi method\n",
    "\n",
    "\n",
    "The [Jacobi method](https://en.wikipedia.org/wiki/Jacobi_method) is a numerical tool to solve systems of linear algebraic equations. One of the main applications of the method is to solve boundary value problems (BVPs). I.e., given the values at the boundary (of a grid), the Jacobi method will find the interior values that fulfill a certain equation.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a2bdbc6",
   "metadata": {},
   "source": [
    "<div>\n",
    "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAkQAAAEVCAYAAAAB2vqVAAAACXBIWXMAAB7CAAAewgFu0HU+AAAAGXRFWHRTb2Z0d2FyZQB3d3cuaW5rc2NhcGUub3Jnm+48GgAAIABJREFUeJzt3XecXHW5+PHPbnpIIxgChhJAQhWViIKIgCIqSLGBV0QvihQBCyrCVRFRQP151WsBsfdCEWFd1HtBrKA0BQxIUDqBEDpJKMnu/v54ZpizszO7OzNn5pzd83m/XvPac2bOnHl2yjnP+daunp6eASRJkgqsO+sAJEmSsmZCJEmSCm9icqW7u/s1/f39t2cUiyRJUkd0d3cv7O/v/1V5fWLV47fut99+SzsckyRJUkf19vb2J9etMpMkSYVnQiRJkgrPhEiSJBWeCZEkSSo8EyJJklR4JkSSJKnwTIgkSVLhmRBJkqTCMyGSJEmFZ0IkSZIKz4RIkiQVngmRJEkqPBMiSZJUeCZEkiSp8EyIJElS4ZkQSZKkwjMhkiRJhWdCJEmSCs+ESFInTQXWBbqyDkSSkkyIJLXT+sAxwMXAA8ATwENAH3Aj8CVg68yi64xTgZuI90FSTk3MOgBJ49aXgHcRpUKPAucC/yQSoi2BtwHHAUcBHwH+XzZhttVmwMdKy/8DfJtICttpHvC80vI/gPva/HrSuGBCJKldDiWSoRuBVwLLqh4/A+gBdgc+C6wCzuxkgB2wAlgOzAf+BTzZgdfcFbigtHwo8MMOvKY05lllJqme9wHnAO9tcT/vYGgyBPA48BbgqdL6GcDsFl8rb1YCOwL/QSQqA9mGI6keEyJJ9ewMvAl4cQv7WA5cOczjy4DLSsuzgL1beK28Wgb8FHgw60Ak1WeVmaR2+QTRfmWkUpGlwKtLy89pa0SSVIcJkTQ2HAdMr7rvXOBW4CXAscBzgUlEgvF94HyyraL54ii3m5xYXt3ka32IoSXePwLuBvYlGnBvDUwB7iLa2HyLSnXdcBYA/wm8AtiAGDJgOVGy9T3g9jrP+3CN+74P3JtY3434/JJuAX4OPAt4T+J17wMuJd7Xh2rse39gG2C7xH2vLcWf9EtgSdV9i4G3Ay8k2jv1Ee/d1cR79Res7tN419PTM1C+9fb2Lso6Hkk1LSdOSMnbPsD7gf4ajw0QvZpa8dPSfn7c4n5G0ksl5t2b3MfTDP3/X0b06hogkpZriESi/PgSYJMR9vseorH3ANE4/OvA14DrS/c9CZxY57m1Ppedqrb5ZI1tLgC2Au6s8dgAkQRvWOP1flJn++rbWxPP6QJOJxKgPuCS0v/3LSIZKj/nTmD7Ov+nNCb19vYuSuZAtiGSxoatgLnA0Yn7XgZ8FHg3cWLfgGjzUy6BeA+wSwdjbMZUKiUkNwN/bHI/84n354TEfccDexKNmRcSpSDzifelD9iWSABm1NnnR4ikcjpwElHycgQxTMAOpf1MIRqDn1Hj+euVYjp5mLhPL20zl0pp1SSi9O+3pZjnlv5eWHp8M+C0Gvs6vLRtMuE5KrH/8u2cxOOHlP63J4jvyl6l57yTKC06iHivNiZKrKRxy4RIGhseAR5mcJXS+4DXEVf0dxGlSOcBRya2OaRTATbpAGBOafk0olSlGQ+XbskxfvYB9gMuT9y3BvgylQRmS6KtU7WXJO4/D/g0Q6uMvgx8t7T8YSKZGCmmak8ktit7JfB3opru2tJj1xLJ7p2lbQ4iEqekVaVtV9W4L3l7OvH44aW/l1C78fu5RKmYNO6ZEElj1++AP9S4/zdUTnrP71g0jZsOfKq0fCHwg5T3fxExMGEtn6NSInM4Q0uJPgJMKC1/fpjXKD/WxfAlQY2YRGUwx6Q1xGcLsA6QRhOHhaW/U4fZ5nwiYXp4mG2kMc9G1dLYVa966WliQMAFxKjF9RwNbDrM4+XRjp9PlJDUcztRStWo04heZXcQI1qn7c/DPPYocAWwB9Hdf3eiLRMM7v7/OPDXYfZzA9HYeQOigfSGDG403Yy7iPeklnsSy8N9tqN1N/Ed2Jv4DL7J0JKwS0s3aVwzIZLGrnuGeexBIiGaNsw2hxDta0ayTelWzx9pPCE6nKjyewB4FZHApW249wdifrE9Sss7UkmIdqJybLyFkavx/kkkRBBjN10wzLajMVzcyd5lw322o/V14jvQVVp+H9Fz7lzgthT2L40ZJkTS2DVcEjGa+bJuZviqks2JmekfYviT482jeK2k1wJnAY+Vlht9/mgtH+Hx5ECJ69dZfmQUr5OsSpo/iu1H0urn2ogfEP/v6URV3bbAZ0q364lpP75DJK7SuGZCJI1drY4L884RHv8pcDDRbuUtLb5W2auI0ofVxGCMw1VHtapvhMfXJJan1FleO4rXqbefZjXbsLwZA0R7qp8QPejeRqVd0Q7EHHOfIMZ5+moH45I6zkbVkjrlVcAviDZOryba8LTTrBEen5lYfrjOcvVgmLUkG2TXGjBxLLgH+DhRKrgz8BUq/8u00no72nlJuWFCJKkTRpMMbQxslOJrVo/QXG1hYnlpYvnGxPJwjc5rbXNj3a3GhgGi1O44YmyrryQeOymTiKQOMSGS1G7JZOg11C8Z+h1wcYqvO9y8aF1ESUhZssfeLVSm49iI2qNCl82m0v39AWL8oKw1UpV6CvDmOo+tIgafvKG0vpB0GnJLuWRCJKmdXs3gZOjy4TdP1ZsYPE9a0l5USqP+xOASIohBFyESp+HaTyUHSDyTkdstdUKy2q66yu9rxEjVs0vr7wU+MMy+ytN2QIzbNJq536QxyUbVktrl1UQX9KnAr4lxenYbZvs5DB5luVUziN5SxzO41GQelaqgfgZP91H2VSIRWkxUFZ3P0ElcNyTa3UB0vf9sGkGn4K7E8gaJ5VlEQ/pHS7eyHYlpO2qV3M2jUpJ2EZ1t8C11lAmRNDacAGxBTDVRdiwx9cUfie7RM4keQxDzXUF0mz+7tPx52tfFvdq6VJIhgANLt5GMNHZQIz5AzEX2IqJU5H5iTrgjqFSDHU/tROApIt6LgeeWtvk48HsiKdiV6H21APgXMXxAdTL3NaKEaYfEfScR3epvLMW2CzFFB1SOx8+n8pmVp2E5unT/1ol9HVeK8SpiQMWyO4gSr0XAoUQy+hjwX6XX+EVVnN2lbT5b+v/uJpLJnYAPEnOy3VlalsYtEyJpbNgTeHFpudwLaufSbTWREE0hqomo2q583w9pLCGqNTfWaE0ixsxpdNycx5p4rXpuI96zbwJfrHrsViJhqk4Oku4m5jQ7gUhIzq56/GEiAT2N2uMVvYlIiMrbQmUgyN8SCdGmVD6f8v8+O3FfOSHalZibLbmv8uffzeCEqDyj/TlEO6rk0Aa/YnBisx/wjtK+P8VQK4kBG09m5HGdpDGtq6en55mi5O7u7q323Xff6rp0SRorjqXS/mc3on0QRIPgLYk2NbcTgw420vi4myjp2YhIcpYB1zG6cYqyMoEoJdoceJJoLH5nnW27iORsM2JW+yeJ//Ef2G5I41Rvb++i/v7+Zy4SLSGSVAS3M7QNUCP6iR5keehFNlp9xPQkN41i2wFaf4+kMc1eZpIkqfBMiCRJUuFZZSZpPPgQcYG3S+K+Q4jGyHcSc3VJUl0mRJLGg9OoDJBYdlTp7+8wIZI0AhMiSePB/GEey3NPMEk5YUIkaTx4eORNJKk+G1VLkqTCMyGSJEmFZ0IkSZIKz4RIkiQVngmRJEkqPBMiSZJUeCZEkiSp8EyIJElS4ZkQSZKkwjMhkiRJhWdCJEmSCs+ESJIkFZ4JkSRJKjwTIkmSVHgmRJIkqfBMiCRJUuGZEEmSpMIzIZIkSYU3MesAxpH5wNTE+uPAQxnFIkmSGmAJUXrOBW5P3D6bYSySJKVhHnBf1W33TCNqE0uIJElSPROIGpCkKVkE0m6WEEmSpMIzIZIkSYVnQiRJkgrPhEiSJBWeCZEkSSo8EyJJklR4JkSSJKnwTIgkSVLhmRBJkqTCMyGSJEmFZ0IkSZIKz7nM0jEVWLfqvg2IOWD6Oh/OENOAlwELgTnAcuBfwBXkI77pxGSBmwKziPhuAf4C9GcYV9k6VOKbScR3M3Al+YhvBpX41iHi+ydwFTCQYVxls4jv36bEZ70cuBG4hnzEN5tKfFOB+4F/ANdmGVTCusBuwCZEfMuB64HrsgwqYS7x/m0MTCbev78DN2QZVMKziPdvIyK+5cDfgCVZBpUwj3j/FhDn5PuBq4nfcB7Mq3HfRh2PohN6enoGyrfe3t5FWcczxmwMfAN4nDiwV99WAJ8mkpAsbAp8B1hZJ777gU8RJ4QsbAZ8H1hVJ777gFOIE34WtgR+DKyuE98y4GNEEpKFrYCfAU/Uie9u4EQiIc7CtsD5wJN14rsT+BBxks/CDsAvgKfqxHc78H6ym8jyBUAP8HSd+G4FjgUmZRTfTsDFwJo68f0LODrD+HYGfgOsrRPfzcDhxIVrFnYFLhkmvpuAwzKMbw/gMuKiuVZ8S4BDGcM1Tb29vYuSOZAJUfMOof6JslbisUeH4zuM+ifKWonHSzsc3xHUPxHVOrG/uMPxHUf9E1GtE/viDsf3AeqfiGqdOHfocHwnUf9AX327Bdiug7F1AR+n/oG+1olpqw7G1w2c1kB8NwBbdDC+CcD/I0pHRxPf34nS6U6ZCPxPA/FdRWdLPCYBZ44ytgGiJH/DDsY3GfhWA/H9EZjfwfhSY0KUjvcx+h9b+fY0sH+H4vtwg7ENEFfxr+lQfCc3Ed8TwMs7FN/pTcS3iiiW74T/biK+x+lcUvmVJuJ7FNixQ/F9o4n4HqYzSWUXUWraaHwPANt0IL5uolSy0fiWEyWunYjvgibiW0ZnkraJQG8T8d1JZ5K2ScD/NRHfbXQ2aUtFdUI0Zou6MvQq4HPEgasRk4AfEdUI7bQfcUJv1BTgJ0C7k+I3EtVgjZoKnEtUs7XTIUTpRqOmE9VDG6cbzhDvAI5v4nkziBNFuw9aRwPHNPG8WcCF1G6vkKb3E9UkjZpDxDc33XCG+DBRDdGo9YCLaH/198nAQU08b33i/Wt39fdpwIFNPG9DIr52Vy9/FtiniedtTMTX7urbLwJ7NfG8hcTxJavq0VSYEDVmEvBlmq/TnQF8Ib1whphMfKGb/VxnE8leu0wDPk/jyWTZXOAz6YUzxAyi9KVZ82guGR2tOcQBtVkbAqemFEstzwLOaOH5GxFVWe2yPvCJFp6/EPhIOqHUtIBok9as5wAnpBRLLQuJNmnN2oao6m2XRS3ufwfgPSnFUsv2Le5/R+KCo12eBxzVwvNfTHMXG7lhQtSYg2m92Hdv4EUpxFLLocDmLe5jP+KH0Q6H0XoJyhuBrVOIpZYjaL0u/C20/hnUczRREtCKw2hf0fuxtF5CkcZnUM/xRC/BVhzD0B6lafkgUdLYivfR+v9Yz4dpvYTieNpXCnMirZdQnEBcWLbDSbTeQPrEFPZRz8m0nhP8Vwr7yMyYDTwjr8vZftq13zzH15XSfmpJY7/dwAEp7KeWNOKbQL7jm0Qk5e2QRnxTgH1T2E8tacQ3HXh1Cvupltb3ehbNVcmMZALptNGcS3s6wEwmne/NfNrTAWYa6XxvNiJ6H45JXT09PQPlle7u7q323XffpVkGlHPLSKcNxmrgjhT2U20R6Vw9rCIa8aVtK9JJwlcCd6Wwn2pb03x1XtJjwD0p7Cepi/RKxh4lvstp6ia9nliPAPemtK+yCaTXPu5homdmmiaSXqPjh4hGzGmaRFTJpeFBoudtmqaQXsnsCqKRepqmkl77x/uJ9zBN00ivUfn7iaYbudfb27uov7//5vK6AzOO3kSiDUIaptOZHiHNWod8xzeDfMc3q3TLq9lkN/bUaMwhu7G7RmNd2ldtloa5tL/xdyvWo/Wq33aaR/sb97difdI7F7XDs7MOoFlWmUmSpLSkUcqeCROi0VtLFKVKkqTa0m4u0DFWmTXmCtJp+PhX4A8p7KfaG0inHv1y4M8p7KfaQcR0Iq36IzHPWdreQnR9btVlxFxEaTuUmCOvVZcQczml7T9Jp6rh17RnHqzDSaeqq5eYiy1tR5JOVetFxLQUaeoiejmmMU3Nz4F/p7CfpG7g3aTTg+1cYtqWNE0gemGm0YPtJ8To/WmaRPSgTGMcoXYcmzvChKgxF5BOQvReIilK21JiFN5WHUMMt5+2O4hRjFsxQJw4bmo9nCGW0/o4TP3Au0j/gA/R2Pi0FvfRR3S9T/uACjGa+Mkt7mMNkVil3SgY4rvT6jg9TwFvIxoup20ycWxoxWrgrcTI5GmbQWvj1EDE9Vbiu5K2ucR3uxUPEfE93Xo4Q2xAXHS14n7iwqgdk3JvAry+xX3cQ0x6PSZZZdaYnxITFrbif2lPMgQx5P+tLe6jh/YkQwDfpvUT8Xm0JxkCOJvWT8Q/pj3JEMT8R632LvkO7UmGIAYtfbTFfXyd9iRDEINutpoofJX2JEMQ84OtbnEfX6Q9yRDEoKhPtbiPz9GeZAhiIu21Le7jM7QnGYIYtLTVROb0FPZRz6nEBV0rPpXCPjJjQtSYNcSkn81+4CtpbtqF0Xqa6PI4MNKGdTxGDA7XLk/QWnwPE4PDtctKWvv/HyAGJmuXR2ithOM+Wi/BGc4DNDftSdk9tDaS9Ejup7WRsO+g9RK64dwDfLKF5/+L1kYyH8ntRNLRrH8SI9W3y1JaK+G9gUjq2+UfwJdaeP7fgK+lFEst1xEXhc26kpgUdswyIWrcr4EPNfG8tURR55J0wxniIpqbXuBp4D+Ig0o7nUdz00c8RbRBui3dcIb4Ic1ND/IE0YarHeMjJX2b5sb4WEUUh6c9vk+1s0q3Rj1OzEHV7o4LXyDew0Y9SgxM2K7SobLPEHMeNuohIr5WS+hGcirxG27UCmLgxJXphjPER4ljYKPuI96/dpVelZ1AnEMadQ8RX6sldCN5L/DbJp53B9GcZE264XSWCVFzPg+8ndH/eB4gJoX9RdsiGuwMogHpaH889wOvBC5uW0SDnUI0gBxt0fS9wJ5EY+BOOJEoyRrtj/tuYHfa01C+lvcTB9bRVg/cToxue0W7AqpyDHFiGm3R/r+Bl9Cehui1HE6UxIy2pHcpsAtxBd1uA0QbpU8z+pLUG4GdaU9D72r9wJuJxHK08V1PzHN1S7uCSugjLkwaaat4LRFfuy+2IH6z+9NYScyVRHztvtiCOObtA3y3gedcTnz/0h7stfPK09739PQM9Pb2tnum8/FmE6KIcCVxcKi+PUgUYWc1iNtC4HtE6UCt+FYQyVNWg/RtQZTIrK4T3/3EFWm75mYaySKi3dgTdeK7j6iCafcM3vVsQ/SIebJOfMuIKrxW58dq1vZER4Sn6sR3N1EF2u4Zxut5HlGa8HSd+O4kJgtt9wzj9SwmerStqRPf7cQVfbvm3hrJi4jSjrV14ruV9HouNeMlxEVUvfhuIebOy6pz0W5Ej9S+OvHdDLyT9s1dNpI9gd8PE99NRMHAmC1Y6e3tXZTMgZy6Ix3TiCx++8R9vUQVQKuN/NIwHXgZ0SV/DtFo9Raia327Gug1Yh2ihGVzotvxfcTB4C/kI74ZRHybEfHdSyW+PDQgnEXEt5BIHpcRB6uryEd8s4n4NiXey2R8zbYnS9McKvGtQ8S3BLiGfMQ3l4hvY+K3vIxoj3JtlkElrEclvmlEfNfTvs4ZjZpHxLeAmEJjGRFbO4Z2aMZ84vi8gEi+lxGfbbubV4zWDgwtHT2cMd5eCJy6o12eIBr8Jt1HPpIhiBKYZuqtO2UVnauua8ZKIsHNq8eI3oF59SjNtevolEeAC7MOYhgPESVtefUgMbZQXq2guXZPnbKcKOnNq1rzznWi+q7jxmxRlyRJUlpMiCRJUuGZEEmSpMIzIZIkSYVnQiRJkgrPhEiSJBWeCZEkSSo8EyJJklR4JkSSJKnwTIgkSVLhmRBJkqTCcy6z9PQxeCLNPEyqKUmSRsGEKD17Zh2AJEkpewDYqeq+pVkE0m4mRJIkqZ61wNVZB9EJtiGSJEmFZ0IkSZIKz4RIkiQVngmRJEkqPBMiSZJUeCZEkiSp8Ox2n56pwIHALsDGwDTgXmK8hp8Bt2UXmiRJGo4JUeu6gOOAU4B162xzOvBL4EgiSZIkKc+OAGaMcttrgN+3MZaOMCFqzRTgXGC/0vpq4GzgEuAxYHsiWdq2tM3OwAHAFR2PVJKk0TsF2HCU234RE6LC+zKVZGgF8ArghsTjfwK+AXwfeAswD+gBXgz8u3NhSpLUkAuBuaXlWcCrE489Cvwmsf63TgXVTiZEzTsMeFdi/e0MTobK+krbvhBYBKwH/JgoLRpoc4ySJDXj6MTytgxOiO4CDu5sOO1nL7PmTANOTaxfAvxqmO2fBk5KrL8IeGMb4pIkSU0wIWrOO4GNEuvfHcVzeoAHE+sfTzMgSZLUPBOi5iSLCtcSda0jWUP0NCvbjmh0LUmSMmZC1Lj5xFhDZUuAlaN87l+r1t+QSkSSJKklJkSN2x2YkFi/toHnXlO1vkfL0UiSpJaZEDVuu6r1Rkagvr1qfdvWQpEkSWkwIWpcdUK0rIHnriB6nJWtDzyr5YgkSVJLTIgat3HV+v0NPHeASIqSNmktHEmS1CoTosbNrFp/osHnP1m1Ptq5YiRJUpuYEDWuOoF5qsHnVydE1QmWJEnqMBOixk2rWl/T4POfrlqf3kIskiQpBSZEjauuIpvU4PMnV62vbiEWSZKUAhOixlUPwjilwedPrVp/vIVYJElSCkyIGledwFRXoY2kevvRjnItSZLaxISocXdWrc9v4LndwLwR9idJkjrMhKhxS6rWFzTw3PUZ3OZoOfBAyxFJkqSWmBA1rjohWtjAczetWr+xtVAkSVIaTIgadxmwNrG+uIHnvrBq/dLWw5EkSa0yIWrcA8CfE+vbMvrBFV9UtX5BKhFJkpR/+wNHAHtmHUgtJkTN+WlieQJw4CieMxl4bWL9OqwykyQVx4nA2cDhWQdSiwlRc74D3JFYP2wUzzkAmJtY/0SqEUmSpKaZEDXnKeBjifU9gf2G2X4qcEZi/QrgF22IS5IkNcGEqHk/AL6RWP8u8IIa200mqti2KK3fCxwMDLQzOEmSNHoTsw5gjDsGmA0cRFSHXQF8E/gt8BiwHXAs8JzS9suIqrO7Oh6pJEmjt5BKjrBJ1WPzgP9qYp+NjNvXcSZErVlDlPZcCnySGHjxmNItqQ84n0iOVnQyQEmSmnA5sGGdx+YDp3Uwlo4wIUrH14mG1vsALyWy4JlESdBS4ByidEiSpLHgcWBGm/a9uk37bYkJUXrWABeWbpIkjWVbZR1Ap9moWpIkFZ4JkSRJKjwTIkmSVHgmRJIkqfBMiCRJUuGZEEmSpMIzIZIkSYVnQiRJkgrPhEiSJBWeCZEkSSo8EyJJklR4JkSSJKnwTIgkSVLhmRBJkqTCMyGSJEmFZ0IkSZIKz4RIkiQVngmRJEkqPBMiSZJUeCZEkiSp8EyIJElS4ZkQSZKkwjMhkiRJhWdCJEmSCs+ESJIkFZ4JkSRJKjwTIkmSVHgmRJIkqfBMiCRJUuGZEEmSpMIzIZIkSYU3MesA8uCss87atbu7e5us46hnyZIl86666qqtAfr6+tZceeWVf8k6piqTgJ0T69cAqzOKpZ4dgXVKy/8GlmUYSy1bAM8uLT8A3JRhLLU8Cyj/RtYAef8OXgusyiiWevwOtmY9YNvS8lrgigxjqWUisEti/W/AyoxiqWnx4sU7Tp48eR2A7bbb7taddtrpnqxjqmdgYOCOo4466v86+ZomREB3d/ehwJFZx1HP6tWrueKKQb/9d2YVyyi9LesA1HZ+B5W1d2QdwAhy9x285pprnlneaKONMoxkZF1dXRcBHU2IrDKTJEmFZwnRUKuBFVkHkdTX1zcXmAnQ1dXFwMDAHRmHVG0SlaJ2iPcvb1VmmwBdpeXcfcbE+zeptNwH3J1hLLWsC8xKrOftOzgRWJBY9zvYOL+Dran+Dj5Azqptu7u7N+nv7+8C6O/vfwK4P+OQqj1zrsuCCdFQvzryyCPfmHUQSbvtttuXgOMApk+fPrBq1aqF2UY0xPbADYn1o4CfZxRLPU8Bk0vLFwNvyjCWWv4BbFdaXg4szC6Umr4AvK+0PED+4tuawW1ejgHOzSiWep4EppSWfwO8PsNYarkeeG5peQX5+4w/B3ygtJzH7+CWwNLE+nuAn2QUS03z5s17Yvny5VMB7r777v898sgjD8w6pqSzzz77y8CxWb2+VWaSJKnwTIgkSVLhmRBJkqTCMyGSJEmFZ0IkSZIKz4RIkiQVngmRJEkqPMch6pxJwAuAnYAZpfu+ADydWUSSJDVuDjF34PbABOA24JxMI0qBCVH7HQB8iJjYcVrVY2diQiRJGhs+A+xHDITalbj/N5gQaRReCOxaWl5LDOe+QXbhMAnYG9gd2IgY8fVfwAXA3zOMK2kK8GpgN2Io/D7gFuA8YEmGcZWtRyS6i4F5xPD81wI/Ix9D4c8E9gdeRHzXniRGwv4ZcGeGcZUtIN6/5xHTMTwKXEkcUB/NMK6yzYj377nEVBEPETOrn0c+pmJ4IfH72BKYCtwDXAr8CujPMK6JwF7AHsDGpftuBS4Ers4opmrTgNcSs9I/G1hDjHB+LnGMyYNJwKuAl5G/Y/T+RDIE8Fjp76w624457U6IPkIc9AB+yvDTOUwGfphY/xD5m6umGdcT78PlwFXAfwDfyCiWQ4kMf8Maj51MlFgdS/wAs3IE8Cki0aj2ceC/ie9GFiYCXyJmep9c9dh/AqeX/p7f0agG+yTwfmCdGo99iph+46yORlQxm/iOHUwUsycdDpwGvBH4Q4fjKlsfOJtI1rqqHjuSeP8OIJLfLCwGvkYkRNXeD/yVONk/0MmgSg4ifpu1plD/KPBt4rfd18mgqnygFMucGo+dShwDT+9oREO9jThG17poPhn4CjElSFbH6POIOe4uJy5Oe4nkfFxod0K0G5HpAlw3wrYTGDy/1GmMj4QoT/MpvZ1Ihq4grtruIg5gRwBbAO8mvuRnZhUgkWzMI06KPcAyYFPihLQp8EFi3rTvZxCU5HF3AAAL/0lEQVTbNOBooprzHOAyYCVRHfouom3Yj4C/EVfGWTiOiLMHuIQ4OW5LzC+3HvBV4irzigxiWwC8hXjPzgP+TFyh7wIcRnzuPweeAzySQXxbAQeWXvsc4C9EYvQy4K3Eb+WC0nZPZhDfK4lk6HbiAnMJcXX+euAVwIuBbxL/Q6eV35+riPfoDqIE5nDi/XoHURLzuQxiKzuKSMr/jyhNW058144ijounERewv8wqQOKCagMi4biQSD42Jo4vWxAXrEuIxDgLH8vodTvCKrNi+QdxJVR9BX4WcRLfgvjhZZkQXUtcAf216v6ziYPVAiLGLBIigB8DH2bwTOA/JE7wfyKq+94GnNLxyMKviSvJpVX3f59IhKYRSWcWCVE/cYV7CvBg4v7vEcnlT4mk7Q3AtzodHJGcfRo4g0p1AETpxpVEMrkJcZF3Yceji6q7dwHfJarfy84kvoOHEFUa84mTfSf9E/gfouou6UyiumwbIvYsE6I/Ed+t66vu/1bpvrlEjFkmRDcQJeF/rLr/TOL3uzlxAZtVQjSu2e2+WN5H7eqIx4GLSsubdy6cmo5maDIEcTK4uLScVYyPEyedu2s8djnR0wKyfQ/fzNBkiNJ9V5WWs4rvn0QJ1oM1HjsfeKK0nFV8fwFOYnAyVPajxHJW8X2dKAFaW+OxcnODLqINVKedwNBkCGA1lSrkzRlaFdlJhzE0GYJog3VZaTnr4997GZoMQb6O0eOWCZHK5pf+Lss0iuHlOcbJRANhyGd8XeT7/VuXSrusPMY3P7Gcx/iSbU7yFl/5vbuXbNsnDifPv42ysRDjmGZCJIA9iUaRkF2D75HsSzQYhXzGeDpxUn+a7KrzhvNeoi3HAFHKkCddRFXaBOBh8tXuDqJpwZdLy8uolFTmxXwqVbS/IR89CcteQrRdhCjhyqNDgZeWlvMa48vJ/zF6zDMh0kuAXxAH/YuJdgB583KikWs30VYnbweEjxA9WCCqJW/MMJZaDiN6AEG0j/lddqEM0UW0jziI6IF0GPkYuqBsIlEdtTfwFNEo/PFMIxpsHvC/RIeDu4j2YXmxmGiPMxn4LdF7Km8OpNJe7etk20O0nl2JY/QEolfXl7INZ/yyUfXIJhFdmUfSR5wYx5LFRCPcmcRJ8k1k2y22lpcSB9VpRM+Qt5KvYvcPEd2xIT7/rLq013MIUSLUTSQeH802nCG+RPTy6Sd6JGXRWLmeLqIB88FEyd/BwO+zDKjKbKIn4Q5EddTeRHuYPHgu0ZtrXaIB/+uIRut5sg9xoTWJGKPr3dmGU9NOxHFvJtHOqXzhoDYwIRrZJKJX0UjWMLYSotlEI72ZRGPSfYkGkHkyj+jCO424wnwdcZWeF6+kctV7KtmPYVJtB6KHVDdxFZz1GFPV3kHEBDGswnezC6WmDxIJZT9RMpSnZA3i/dqBqGZ8OdFoPQ/WIY4t6xK9V19FvkrVABYSSdAkovTlEPKXaMwhvnMziU4bryV/x+hxxYRoZGuAE0exXZYjxDbjXcQ4IQ8R45jk8Yd2DPAs4ur3IPKVDEF0b+8iqhpPyTaUmk4iqiuuIa5+85QMdRHvH1R6T+XJZOL9A/gs+atKeR6V8YbeRn6SIYg2QwuJJOhA8pcMQVRxzyBGgD6U/CVDEN3rNyR6Zb6BfB6jxxUTopGtIZ91363asfT390TCkUeLS38voXZX7ayV4zuHfCUbZeXP+ALyN2feekS7F4jxh/JmMyq9Bn+SZSB1lL97j5O/Rt7l793l5KuBd1I5xl8SA4XmUfIYfV+WgRRFnhpV5ymWIihPNFtrzJW8yHuM5fjyeAUM+Y4vOdGx8TWuHN9K8lc6nfffLRijamh3CVHyqrl67qJqs9sZiIb4INEYOI8lL2XvJurPV2QdSB3lOaX+nWkU9e1LVP3clXUgNSyn8v7dlGUgdSylEl8ex30pTy2St4bKEA33P0+0bcqrQ4Dp5Ld0HOB4om1ino/R40q7E6LkzNDz624VntPOQDI0m+gpULZNYnkPKqPzrqKz0ynk9SSeVGvE5Ty5JusARnBD1gEM42ny/f6tJt/xrSC/Fwq3URm1Pa/ymIRXy+MxeksqVd0QVd/lv3sl7r+NfMY/rHYnRMnMttYsyEmvaGcgGdqW6H5ay0WJ5RuB7dofzjOmAIuAW8hmosrRmEr8AJeSvwbVEAeB9cnvwXUmcfC6kfxVq0A0GJ1Ofg+cGxNV+XmdZHprIinKWwnCJCK2f5PfhsDrEFNg3ETtqVDyII/H6COpjLmW9EIGn+dOJeZkG1PanRAlB6h7GfElXFVju1lEi/rxaCWju9Ls5BXVNGKyyu2JA8JO1P5csjSLeN+eQ3Td3YV8JUUvJoYCmE6MpfPebMMZYgviM55LNKp+fbbhDHEAMSL1JGIspywn/azl7cSQBRDHpiwmmx3O14iT00qipDkvpVmTiZLuHYmEaDHwaKYRDTWfmET62USD5ZeTvwuG6cTcg9sS59GdyEdyeTej+67luSqyrnYnRJcklucQo+W+m8FfvtnEeBDJuXjGkxuotEXIix2JZAiiCm8xtSd9zdIuVKpRX0CUnl2bXThDvIE4aEF0281bQrQPkQxBjN9U72IkK28mkiGI9y9vCdEhVDp6HEL+EqJDS39nEJ9vXhKi7aj0jtqC+B3/OrtwanoFkQwB7E6UBOatFHAxkQxR+rsj8KfswnnGF0u3candPbuWEF0vy44krlpPIYrdzgJuJgbu+ir5KgEYz5ZS6bmwkvgM8uZGKu2rHgZuzTCWWq5OLOflZJR0LZVODf8kX8kQDH7Prq67VXbyHt/VdZazdhsxthnE7zdv09gAXEelmuwe8tmlfSmV3o2Pk//2lONCJ8YhOoYofZhZWl9MZQyNsj8BJwCvIep11V4riCu3vYl5kJZnG05NdxEx7knM3/NItuEMcQ6RZGxKzHWVN38mqlKeT5TA5s1/E5/xTOAHGcdSy8eotA37cZaB1HEAMY3NLcSErnnxCPG73YeoUs7jOERLiPnBXgL8nHxeiC8HdqZyjM7T/H7jVicSor8TdbRfY2gi9BAxSu3HGdrrJG8DyY03N5LPq7ek60q3vOrNOoAR/IH8VYWWDZDPRK1sLfD9rIMYxiPAV7IOoo6l5L9E48rSLc/GwjF6XOnUSNVXE+1otirdJhGNs65jcOv5gzoUjyRJ0jM6PXXHzeSzvYokSSowp8uQJEmFZ0IkSZIKz4RIkiQVXqfbEI0Fk775zW/OHXmzzvnRj340pbw8MDAAlQH38mJW1foM8hdj0mTyF19y8uNu8hfflKr1vMVXPTl0Hr+DXYnlSeQvvuR3sIv8xTe1aj1v8VUfB9chZzH29/c/8x2cMGFC7s51a9eundLV1TXyhm1iQjTU/n19fbmaG2iLLbbgsssuA2D16tVd5G/uomrfyzqAEexPvt/DDch3fGPhO/jtkTfJ1GvJ93s4n3zHNxa+g98o3XJjxYrKfMALFizYJ2/nuiyTIbDKTJIkyYRIkiTJKjNg7dq1p02cOPHrWcdRz4wZM2bttddezwYYGBjou/TSS2/JOqYqE4AtE+u3kr+Rxjej0g5mOTE/Wp7MB9YtLT9OzLGUJzOBBaXlPmLKiDyp/g7eRv6mZPA72Jq8fwe7gUWJ9dx9B3fbbbfNpkyZMgVgzpw591OZdy53+vr6Oj5dU1dPT095Aki6u7u32nffffM+5LokSVJLent7F/X39z8zWLRVZpIkqfBMiCRJUuGZEEmSpMIzIZIkSYVnQiRJkgrPhEiSJBWeCZEkSSo8EyJJklR4JkSSJKnwTIgkSVLhmRBJkqTCMyGSJEmFZ0IkSZIKz4RIkiQVngmRJEkqPBMiSZJUeCZEkiSp8EyIJElS4ZkQSZKkwjMhkiRJhWdCJEmSCs+ESJIkFZ4JkSRJKjwTIkmSVHgmRJIkqfBMiCRJUuGZEEmSpMIzIZIkSYVnQiRJkgrPhEiSJBWeCZEkSSo8EyJJklR4JkSSJKnwTIgkSVLhmRBJkqTCMyGSJEmFZ0IkSZIKb2LV+uY9PT0mSZIkabxbmFwZlBD19/f/qqOhSJIkZaC/v3/QuqVBkiSp8EyIJElS4f1/csf0sRRtfuMAAAAASUVORK5CYII=\" align=\"left\" width=\"300\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e63a5792",
   "metadata": {},
   "source": [
    "### Serial implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14a58308",
   "metadata": {},
   "outputs": [],
   "source": [
    "function jacobi(n,niters)\n",
    "    u = zeros(n+2)\n",
    "    u[1] = -1\n",
    "    u[end] = 1\n",
    "    u_new = copy(u)\n",
    "    for t in 1:niters\n",
    "        for i in 2:(n+1)\n",
    "            u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
    "        end\n",
    "        u, u_new = u_new, u\n",
    "    end\n",
    "    u\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76e1eba1",
   "metadata": {},
   "outputs": [],
   "source": [
    "jacobi(5,1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e085701",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Note:</b> The values computed by the Jacobi method are linearly increasing from -1 to 1. It is possible to show mathematically that the method we implemented in the function above approximates a 1D Laplace equation via a finite difference method and the solution of this equation is a linear function.\n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Note:</b> In our version of the jacobi method, we return after a given number of iterations. Other stopping criteria are possible. For instance, iterate until the difference between u and u_new is below a tolerance. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6918c31",
   "metadata": {},
   "source": [
    "\n",
    "### Where can we exploit parallelism?\n",
    "\n",
    "Look at the two nested loops in the sequential implementation:\n",
    "\n",
    "```julia\n",
    "for t in 1:nsteps\n",
    "    for i in 2:(n+1)\n",
    "        u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
    "    end\n",
    "    u, u_new = u_new, u\n",
    "end\n",
    "```\n",
    "\n",
    "- The outer loop cannot be parallelized. The value of `u` at step `t+1` depends on the value at the previous step `t`.\n",
    "- The inner loop can be parallelized.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "798968b1",
   "metadata": {},
   "source": [
    "### The Gauss-Seidel method\n",
    "\n",
    "The usage of `u_new` seems a bit unnecessary at first sight, right?. If we remove it, we get another method called Gauss-Seidel.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f77b547",
   "metadata": {},
   "outputs": [],
   "source": [
    "function gauss_seidel(n,niters)\n",
    "    u = zeros(n+2)\n",
    "    u[1] = -1\n",
    "    u[end] = 1\n",
    "    for t in 1:niters\n",
    "        for i in 2:(n+1)\n",
    "            u[i] = 0.5*(u[i-1]+u[i+1])\n",
    "        end\n",
    "    end\n",
    "    u\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dbc5358",
   "metadata": {},
   "source": [
    "Note that the final solution is the same (up to machine precision)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca31518b",
   "metadata": {},
   "outputs": [],
   "source": [
    "gauss_seidel(5,1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c92e9c73",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "<b>Question:</b>  Which of the two loops in the Gauss-Seidel method are trivially parallelizable?\n",
    "</div>\n",
    "\n",
    "```julia\n",
    "for t in 1:niters\n",
    "    for i in 2:(n+1)\n",
    "        u[i] = 0.5*(u[i-1]+u[i+1])\n",
    "    end\n",
    "end\n",
    "```\n",
    "\n",
    "    a) Both of them\n",
    "    b) The outer, but not the inner\n",
    "    c) None of them\n",
    "    d) The inner, but not the outer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4edad93f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "answer = \"x\" # replace x with a, b, c or d\n",
    "gauss_seidel_1_check(answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9df06442",
   "metadata": {},
   "source": [
    "## Parallelization of the Jacobi method\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97a2d7d5",
   "metadata": {},
   "source": [
    "### Parallelization strategy\n",
    "\n",
    "- Each worker updates a consecutive section of the array `u_new` \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4e2d000",
   "metadata": {},
   "source": [
    "### Data dependencies\n",
    "\n",
    "Recall:\n",
    "\n",
    "`u_new[i] = 0.5*(u[i-1]+u[i+1])`\n",
    "\n",
    "\n",
    "Thus, each process will need values from the neighboring processes to perform the update of its boundary values."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "428bce86",
   "metadata": {},
   "source": [
    "<div>\n",
    "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA6gAAADyCAYAAABAvOgkAAAACXBIWXMAAB7CAAAewgFu0HU+AAAAGXRFWHRTb2Z0d2FyZQB3d3cuaW5rc2NhcGUub3Jnm+48GgAAGbJJREFUeJzt3X2YXXVhJ/DvTBISQgKIEAgghIggr8KiBaXYarG+1drFrVsfFVdLgZSK1XVd2+5afJRHfWrVRRGTVpfFvtiX3e3aqgUeLSpWfImmRhARYxAFSUI0kkBCyNz945xhzlzuzNyEmTm/mfl8nuc+93funLnnm+HO5X7n/M45A6tXr+4EAAAAWjbYdgAAAABIFFQAAAAKMb9r+fkDAwMb2wgCAADA3NLpdFYkuX54eVRBHRgY2HjxxRffMd2hAAAAmHvWrFmTTmfktEim+AIAAFAEBRUAAIAiKKgAAAAUQUEFAACgCAoqAAAARVBQAQAAKIKCCgAAQBEUVAAAAIqgoAIAAFAEBRUAAIAiKKgAAAAUQUEFAACgCAoqAAAARZjfdgAAYEY4LMnSJFuS/LzlLADMUvagAgD9+GyS7ye5Zhq3OZDkqUkuSPKbSc6fxm0POynJ1+vb01rYPmV4UZJ3J7m87SAw29mDCgD0Y7DrfipdkuSlSc5J8oTG42uTPH0att90SJKz6vGh07xtyvHvk1yU5K4kV03D9g5N9fo/J8mZSfarH39hkkemYfvQGgUVAOjHTUnuTfK1adjWq5OcW487SXYlWTQN2+1lV2O8s6UMzC1vS/L2Mb5m9iOznoIKAPTj96ZxW19NVYhvSfLlJB9I8qpp3H7T11NNNWZu+0CSG1Idgz3Vltb3P031+l+Q5HnTsF0ogoIKAJTmTW0HgC631rfp8PEkH0tye6oZBJdFQWUOUVABgLG8JcnLux67IckftpBlul2ZZF7XY3+e5M4WstCOQ5Ic1/XYw0nW7+XzPDHJ9UkeSnJeH+t/ay+fH2YVBRUAGMuhSVbW48VJFib5XntxptWbM3JimmE3RkGdS16Y5C+6HrsryYq9fJ4FqU60tWMSMsGs50BrAGAsb0m1F+mQVFMO55IXpppW+bwke1rOQjtuTfKe+vaplrPAnGEPKgDAY32uMe60loI2ratvSXJhkhe3mAXmDHtQAQAAKII9qADAZHtykv/Qx3o/SPK3U5wFptMzkpzZ9diB9f38JBf3+J4vpDpjLxAFFQCYfCcleXcf690QBZXZ5deT/LcxvrYwyeoej/9OFFR4lIIKAEy2O1OdWGYic+WMwMwdtyb5u67HFiV5SZJHkvzfHt+zYapDwUyioAIAk+32JG9tOwS04BP1remIJPcm2ZXHXlcY6OIkSQBAPx6u7w8cd63Z6ZH6vvu6qMwdw2dy9tkZpphfMgCgHz+s70/J3Pv8sKW+P7bVFLTp5/X9YUkG2gwCs91c+x8MALBvPp1qL9KxSf57qhO+TJUjkpzVuD2xfnxx1+PHTGGGpi/W9xcnOWCatklZvlXfL0ryvDaDwGznGFQAoB+3J/lQktcnuSLJHyXZXn/tnUneN4nbel2SK3s8flKSrzeWP5zksknc7ljem+qyOWcmuS3J2oxM+70oI3vXmL1+kOSzSX4lyV8l+ZOMnNzo3iQ3T+K2TkzypsbyKY3x1UmG6vHdqX73YFZRUAGAfv1+qg/qr01y9BRu56fp78ymWyZeZVJ8I1VB/UiqvbbNPbfTUZApw+tSldTjM/oySv+U8QvqpiSHZOQ41okcmd7XS02qP4gM+2YUVGYhBRUA6NdQkvfXt6l0TX0rySeTfCrJyUmWZeQ4xG2tJWK6/TDJ6UkuSLU3f/hz9ETXMB1K9UeXfn0//Z0F+769eE6YMRRUAID+7Emyvu0QtOqhJH85xdv4Yfq7jjDMSk6SBAAAQBEUVAAAAIqgoAIAAFAEBRUAAIAiKKgAAAAUQUEFAACgCAoqAAAARVBQAQAAKIKCCgAAQBEUVAAAAIqgoAIAAFAEBRUAAIAizG87APRyzTXXHDU4OPjytnOM59prrz1vaGhoMEk2bNiwfvPmzVvbztTltCSH1OO7k2xoMUsvRyd5cj3enmRti1n6cU2SnW2HoC8vT3JU2yHGcWySFfV4W5J17UXp6cAkZzaWv5hkqKUsYzkvI39kX5+ktPffpm8k+XzbIejLsUkuaDvEOAaSPLuxvC7Ve0hJnpbk4Hp8V5KN7UV5rBUrVhx7+OGHr0iSgw8+eNsFF1xQ2vvvKPfee+8Hr7jiikfazjHdFFSKNG/evOM7nc772s4xnq985SsZGnr0M9tL28wyS7yy7QATuC4K6kxxeZJz2w6xFy5sO8AESn9/Kz3fB6KgzhQnJSn6s0eX0l/7xdm4cWM2btyYJFm5cmVS+Pvv8uXLVyeZcwXVFF8AAACKYA8qM8W32g7Qw+nDg4ULF/5k165dm9oM08PJGfkdfyjJ91rM0suKVFMJk6STappeSRYlOaHtEDxu99W3kjw5yQH1eE+SW1vM0ssTM2qK9JINKW6K7/bjR8aLtibzC5vi+9CRyZ7Fbafg8VpyZ9sJugwk25/cWL4nyZa2wozhlCTz6vGOJN9vMctjLFy4cOWuXbuWJEmn0xlK8u2WI3VbnOT4Cdea5RRUZoqnX3LJJbvbDtE0ODjYGR6fc845H/785z//jjbz9LA5yaH1+NYkz2gxSy//mOTX6vHOVMetlOTMVMeOMbN9JMkVbYfoclOSX6rHO1Lea/+yJB8aWbz5ouRpO1pL09PAV0bGZ/+f5KY/ay9LL8ven2x+VtspeDwGhpIHCjv05DuLkpOb08Xfk+SqttKM4WdJDqrH30x1vHgxTj311M+tXbv2OUmye/fuHZdccklR779r1qx5VqfT+VLbOdpmii8AAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoI6e7wlyY317bI+1v+HxvrPmMJcAAAAfZnfdgAmzSlJzq/H3+lj/V9OclA9/tOpCAQAALA37EEFAACgCAoqAAAARVBQAQAAKIKCCgAAQBEUVAAAAIqgoAIAAFAEBRUAAIAiKKgAAAAUQUEFAACgCArq3DW/7QAAAABNCursMdQY7zfBuguSLJ7CLAAAAHtNQZ09djTGyydY94QkA1OYBQAAYK8pqLPHfY3xMROs+6KpDAIAALAvFNTZ49uN8elJjh9jvQOTvGHq4wAAAOwdBXX2+FySXfV4MMm1SQ7uWmd5kk8mOTLJnmlLBgAA0Adncp09tiX5eJKL6uVzk9yZ5IYkP0vypCTPTXVypKuTvDDJyumPuW8GBgaesXr16qJK9apVqx4db968+egkZ7eXpqfm7/cBKS/fExrjwZSX74S2AzApjkp5r62ljfG8lJdvxejFD56UHPBQK0n6suWw5A2ntJ1itIeXtJ2AyVDa62pn90kwj0157x/zGuMlKSzfAw888Oj778DAwLzVq1cXla/T6ZzcdoYSKKizyx8kOS/JifXyE5O8omudf07yliSnZQYV1E6n86W2M4zntttuuzjJxW3nGMdJSW5pO8Q4FqbsfMxcF2XkD3clOiDFv/Y/enXbCcZ360urG0ymzmBy1cfaTjGBN9W3Up2Rwt7f7rjjjkfHCxYsWJzC8lFRUGeXLakK6ruS/FaqDz7D7khyTZIPppree2OSh+uv3T+NGQEAAHpSUGefzan2FvxekuNSXRP1nvrxpnfWNwAAgCIoqLPXziTfaTvEvrrnnnu+uHLlyqKP4XnZy162+JFHHhlIkvXr1++88847izpGNtW02eHf8d0Z2WNeivmpMibVXv2dLWbpx46JV6EQ52f0cVClWZDqj4dJma/9wST7N5ZLfO0vzsj1vHem7BP/7W47AH27MdVxkyVrzo57KMlQW0HGsCgj778Pp7DX/xlnnLHguOOO2y9Jli5dumf//fcv7f13lAsvvPDBtjO0QUGlSFdcccVQyvxQ1CQflKnoDxwzxANtB5iA9zemwp6U/9qS73FYt25d1q1b9+jydddd12IaxuIyMwAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAI89sOAADQomclOS/JT5L8r5azAMx5CioAMJe9KsmqJPdEQQVonSm+AAAAFMEeVABgLvufSdYl2dp2EAAUVABgbvtafQOgAAoqADDXHJDk8K7H9iS5q4UsADQoqADAXPPiJH/T9dg9SY5qIQsADQoqADDX3J3k7+rxMUnObjELAA0KKgAw13y5viXJK5L8VYtZAGhwmRkAAACKoKACAABQBAUVAACAIiioAAAAFEFBBQAAoAgKKgAwl3Xq+4FWUwCQREEFAOa27fX9E+NzEUDrvBEDAHPZbfX9fkme3WYQABRUAGBu25Dk5nr8F0lel+T8+nZmW6EA5qr5bQcAAGjZbyf5lyRHJflo4/Hrk7yglUQAc5SCCgDMdXckOTXJa5KclmRp/fi/tZYIYI5SUAEAkp8m+UDbIQDmOsegAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUIT5bQeAXtasWXNGp9P5cNs5xnPllVeeMjQ0NJAkW7duvevBBx98oO1MXVYkWVKPtyT5SXtRejo0yRH1eGeSO1vM0o8XJPl52yHoyzVJntZ2iHEsq29J8mCSDS1m6eWAJMc1lm9N0mkpy1hOSTJQj+9KUtr7b9MnklzVdgj6cnaS97cdYhwDqV77w36QZEdLWcZyXKr3kCTZnOS+FrM8xkEHHbRs6dKly5Jk+fLlD1500UWlvf+OsmvXrudcfvnlu9rOMd0UVEp1UJJnth1iPD/60Y8yNDQ0vHhqm1n6sDSjP3CWZmmSw9oOMYEFbQegb6el8PePhqVJDm87xATOaTvABEp///1K2wHo2xMyc947kuT0tgNMYGmSlW2HaNq2bVu2bduWJNlvv/2Kf/9duHDhvLYztMEUXwAAAIpgDyozwsDAwB8PNXZXFuIdw4Nly5Z9dtOmTTe1mKWXP0iyuB7/OMlHWszSy6uSnFiPH0ny9haz9LI8ye+2HYLH7V+SfK7tEF1+O9UU/CTZleSd7UXp6ewkv9ZYfmeqnCV5x8hw+deTZWvbi9LLHS9MHjqm7RQ8LkNJ/rjtEF0WJHlbY/lTSW5pKctY/ijJonp8V5I/bzHLYxx++OGvve+++1YmyZ49ex7udDrvmOh7ptPAwMCTklzcdo62KajMCJ1O512XXnrp7rZzNA0ODj76pnbSSSd9ftOmTaV9yHxDRgrqvSnzQ/BwQd2d8vKdGQV1NvhCynttnZ+yC+plGV1Q/yTlHX/d+FB5wrrkpo+1F6WXZacpqDNeJ+X9bi7O6IJ6Q8o7vvnNGSmod6ewn+HRRx/93EZB3XXppZcWlW/NmjXP6nQ6c76gmuILAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAijC/7QBdnp3kqfX4O0m+OMH6FyZZVI+vT3LXFOXq5TeSnFiP/zLJj1L9PH89ySuTPCXJQUk2J7kpyVVJfriX21hcb+fFqX4uhybZkeTHqf69H09yX4/vW5LkssbyVUke2sttD7s01b8jSf4+yff38XkAAADGVVpBfXWSi+rx6kxcUN+b5LB6/BuZ3oL6qiQvq8ffTbIrVYF7dtd6xyQ5K1XRe1mqYtmPVyd5V5KjenztpCTnJ3lbkj9IcnXX13ckeXOqQpskNyf5Up/bbVqS5ENJ5tXLn9iH5wAAAOiLKb6T49gk/5iRcvqzJPcm2dNY54Akf5vkiD6e791Jrsvocro5ybeS3N14bGmqAnlF1/d3MrqQnt3HNnt5ekbK6d2Z3j8AAAAAc4yCOjnemqoEXp/k9CRPSHJkqjL6ocZ6Bya5fILnujTJf20s31Q/97IkT0u1R/a4VNN7h70tyXO7nmcyCmrz+27ex+cAAADoi4I6OY5I8rkkL0myvvH4liSvT/LJxmMXjPM8RyV5X2P575M8L8lXu9bbmOr42z+rlwdSTQduahbKySio+zJFGAAAoG8K6uS5PMnuMb62pjF+SqpjO8d6jv3r8X1JXpvkkXG2+daMnPzoF1LtYR22tvG1Y9Pf1OJu9qACAADTRkGdHN9Ocus4X7+9MR5MNU23l1c1xtck2T7Bdrem2nM77DmN8cMZvee1117Uv05yY5IbUh0j2/SkVNOUk2RbRu8ZBgAAmHQK6uT45gRfv6dreWmPdU7ISCFMkk/3ue11jfG/6/raeNN8Vyb5rVRnA35eRpfb7vX/NclQn3kAAAD2SWmXmZmptkzw9YdSTdUd/nnv12OdU7qWz01yZh/bfmpjfGjX18Y7UdLzu5ZfkOSfxljf8acAAMCUU1Anx85JeI5Dupbfvw/PcVDX8vCez8FUl4wZzMie0F+t73+Yaspxd2F1/CkAADCtTPEtR/cxoPui+79n89jRA5OcXI8XZOSyNG9NVVqPr29J9YeLs+px97GsAAAAU2Km70EdaDvAJPpZY3x/qrP97q1eZ/y9OSNn9z071QmdnpmqsD6Q5H8n+f1UZwF+fpI7k5yWZHH9Pc2zAQMAAEyZkveg9pNtrMu1zESbG+MDk/w8yU/38vZAj+ftdRzq8PTeG1PtIR0+IdMLutZLTO8FAACmSWkFtbmnrvuEP90OS7JoCrNMt68l6dTjBUlOn6Tn/UJjPFw8h4vop7vun5NkYZJzGt+joAIAANOitILanOZ65JhrVX5xKoO0YEtGX2v0FZP0vD9Oclc9PiXJcanODtxJ8pn68bVJfpLqONhfTDXdN/U6/zpJOQAAAMZVWkH9bmN8enpfL3TY705xljZc3RivyuhLyExkXsY+JvfmxjpvTfXf/d8ycn3WoYyU1f+Y5MR6fHsmvoQOAADApCitoH4pI9Nc90/y9jHWe2OS86cl0fS6LiMlfUmq0jjRVN8Dk1xaf99YU56bx6H+p/r+U13rDE/zfU1GXhem9wIAANOmtLP4bkx1zOQv1ctvTHJ0kr9ONf33qFRTX1+U5JtJltWPzRY7k7w8yZdTnUV3Rarpt3+TqlBuSHWc7qFJTk1yXpIXpyrz42kWzf3q+890rXNjkt2Nryejiy0AAMCUKq2gJsl/TlWohvcG/mZ9a/pZqj1912R2FdQk+VaqkxX9vyRHpPpv9Mr6tq9uTXWW3yfUy1uT3NK1zrZUP/fnNB6zBxUAAJg2pU3xTao9hi/NyPGR3b6a6my061OdAGg2+mqqa5H+aZLtE6y7Ocm1SX4l1R7YXoYy+mRH/5xkT4/1Pt0Y35vk+31kBQAAmBQl7kFNkhuSHJ/k+amK2qJUhfXLSb7RWO+1qY6/TCYucpPtNUl+px6PVQybljXGva5X2m1Lkjcn+cNUl305PdXU3kWp/q13pdrbuj5VAZ3Iy1NdQiYZfTmfpv+R5KP1+JE+nhMAAGDSlFpQk6pE/UN9G8uD9a0NO+pbv366j9t5ONVxuV+YaMUJ9POz2p19zwkAAPC4lDjFFwAAgDmo5D2ok2F1Rk4M9Hi8K9VZgwEAAJgis72gviTJ8kl4nmujoAIAAEyp2V5Qr0yyZBKe5/ZJeA4en/esXr26n5NBTZtVq1Y9Ol6/fv3zkxzUXpqeDmiMn5TkvW0FGcPJjfGClJdv2cSrMAP8aibn/wOT6cmN8aKU99o/o2v5nanOh1Co234hOWGi64FPswdWtJ2Ax20w5f1udn9uf2mSY9oIMo5FjfFxKexnuHHjxuOHx/Pnz1+4evXqovIlObLtACWY7QX16rYDMGne2HaA8WzduvXcJOe2nWMch6e6xnCp5qfsfMxcz6xvpdov5b/2X992gPFtPr26waQaSPm/m8+tb6U6KoX9DO+///5Hx4ODg8W9/3Y6nbYjFMFJkgAAACjCbN+Dygy1Z8+e+wcHBz/Tdo7xrFy58qyhoaGBJNm0adP3tm/fvq3tTF2ekpFpxz9J8qMWs/RyeKqpx0l1CaTbWszSj4KnONLly0l+3naIcRyZkWlc21PeYSRLkjy1sbw2SWl/1j8r1R6uJPlektLef5tKf29jxOYkJX/2GEj12h/23SQPtJRlLCcmWVqP703y4xazPMYhhxyy/OCDDz4qSZYtW1bi++8ou3bt2tN2hjYoqBRp1apV307yorZzADPSf2k7ADAjrY3PHrPa1q1bs3Xr1iTJhg0bcsstt7SciF5M8QUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCIoqAAAABRBQQUAAKAICioAAABFUFABAAAogoIKAABAERRUAAAAiqCgAgAAUAQFFQAAgCLMby50Op0Va9asaSsLAAAAc0in01nRXJ7f9fXrO53O9KUBAACAmim+AAAAFEFBBQAAoAj/H6+X6/wiJ+vXAAAAAElFTkSuQmCC\" align=\"left\" width=\"450\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b3c8c05",
   "metadata": {},
   "source": [
    "### Ghost (aka halo) cells\n",
    "\n",
    "A usual way of handling this type of data dependencies is using so-called ghost cells. Ghost cells represent the missing data dependencies in the data owned by each process. After importing the appropriate values from the neighbor processes one can perform the usual sequential Jacobi update locally in the processes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd1e3760",
   "metadata": {},
   "source": [
    "<div>\n",
    "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA6gAAADyCAYAAABAvOgkAAAACXBIWXMAAB7CAAAewgFu0HU+AAAAGXRFWHRTb2Z0d2FyZQB3d3cuaW5rc2NhcGUub3Jnm+48GgAAIABJREFUeJzt3XmcXWVh//HPmUwyISEbgSQkEBIJW9gMCUJSaotIAz8TFgURl4ioUYtLi7SC/hRcWm2RtmqtJNRSqv6gQEUyIYKhCkoIi4ACAgIhCSQBErKQfZmZ+/vjuZM5czNLlrn3ee7M5/163dc8594z93xzZ3JnvnPOeU42a9asApIkSZIkRVYTO4AkSZIkSWBBlSRJkiQlorZkeWqWZUtiBJEkSZIk9SyFQmEMcE/zcquCmmXZkpkzZz5f6VCSJEmSpJ5n9uzZFAot0yJ5iK8kSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmojR1AkiQl7zDgbUADcEfkLJXWD6gDNgHbI2eRpG7PgipJkjpzAfDt4rg/sLkC2xwOnAKcCgwt3vd1YFkFtp33KDAe+C/gwxXetiT1OBZUSZKUkjOB64G3tPHYv1H5gtqr5GMlXAmMBR4EbqrgdiUpOguqJEnqzL2E0tQAbC3ztg6lpZy+ASwFJpZ5mx15BNgIPF3Bbb4fOB4YTPkLah1wEmFP9anAiUBv4Blgepm3LUm7sKBKkqTO/L54q9S2LgEWAs8D7wD+t0LbbsuMCNsslHwsp18Ab2/j/jcrsG1J2oUFVZIkpeSx4q0new8wCHi9AtvKgB3AE4Q/CpxKOPdXkqKwoEqSpLacRNtFsVKTJMX0OWBayX2/BL5Zoe2/WKHtAHwCWAJsKS7fhAVVUkQWVEmS1JYNhHNPAQYSLjPTUxwNvLPkvnLvzXwrcETJfa8AD+3h85wN/AT4DXDubqz/7B4+vySVlQVVkiS15QXCjLoAJxMmC+opPlW8AcwCZlZgmx8BPlty363ARXv4PH2AIcD+XRFKkirNgipJkhRfPbCiOD4XmBwxiyRFY0GVJEmK715aDqk+FAuqpB7KgipJkrraDYRZaDvzKWB1mbNIkqqIBVWSJHW1c4Bhu7He5eUO0s19lF0nVmpePhz4Vhuf8y/Aa+UMJUn7woIqSZK62seBut1Yz72n++YiWiayKnUY8IU27v8JFlRJCbOgSpKkzhRy42w31p9TriBq5b+Bx0vuOwJ4N7AUuKWNz1lV7lCStC8sqJIkqTObc+MDgE2xgkSwvfgxxcu2/LCN+84lFNRFwJWVjSNJ+64mdgBJkpS8xbQUtbfHDBLBsuLH8eze3uOu0LzH2h0Jknoc3/gkSVJntgB3AhcC1wGNwAvFx7YBT3fhtvoDn84tH54bfwR4tThuKGYpt7sJkw0dAfwV8B2gqczbXFf8OKLM25Gk5FhQJUnS7rgceBth8p2bc/cvAsZ14Xb2p+3ZZwE+lxtvpTIF9ffATcCHgX8CvkrLeZz/CMwqwzYfK358G/AW4KUybEOSkmRBlSRJu2MZcBLh2qUnE/Z0Aqzo4u1sAWbvxno7uni7HfkYsAT4EDAWGFC8f3CZtjePcFj1WOA+4EfA+uJjvwEe7MJtvY9wuZpmxxY/jgPm5+6fS9h7LEllZUGVJEm7aw3wd2XexnrgE2Xexp5qAK4p3iphO2Gio7nAocAXc499hY4L6p3s2bmyY4B3tnH/gJL7X9yD55SkvWZBlSRJSs/vgCOBvyDsSe1TvP83XbydnxEO0+7M7qwjSfvMgipJkpSmzYQCWU7PFW+SlAQvMyNJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJ8DIzSl6hUMjq6+sfiZ2jI9ddd91hixcv7gewYcOGN9auXbsqdqYSBwIHFcdbgCXxorRpP8LF4pv9EWiKE6Vto0ePPoriH/UuueSSZZMmTdoQOVK7siy7dfr06ddWYlv19fX/WSgUjq3EtvbGbbfdNuz+++8fCrBt27ZNr7/++suxM5XoD4zOLT8bK0gHjsmNXwY2xQrSjtGE1xFgNbAyYpZdDB48eOjAgQOHAYwaNWrrVVddtTh2po7U1dX9+dSpU8v+Nb7rrrve3tjYeF25t7O3tm7dWnPFFVcc1bz8+uuvL9m2bduWmJnaMIbw8xNgFfBGvChtOojw+weESyYtjZhlF3379u03bNiww5qXv/e97z2bZVnMSJ35zDnnnPNQ7BCVYEFV8r761a9mEydOnBQ7R0fWrFnDyy/v/L23P3BYB6vH1p+WHxipOil2gFK5ry+NjY1HJf5D7OFKbahQKBybZVmy/z+3bNlS+n9zWMQ4uyPZ17LomM5Xiaq08Ee3bt061q1bB0CfPn36Z1k2NHKkDtXU1FTkd8PGxsYhKb93ZFnW6n0fSPYPcUX9af2H3tT0p+UP5UnYunVr6dd4Uso/27MsGxQ7Q6V4iK8kSZIkKQnuQVU1uiHLskWxQ+Tt2LHjaoqH2YwaNWrp8uXLfxA5UqmPAkcUx9uBr0TM0pZJwAW55W8AGyNlaVNtbe23GhoaAFizZs2vsiy7J3KkUjMKhcL4mAEKhcLdNTU198XMUGrDhg1/SXGP2tChQ7esXr36q5Ejlfoz4Ozc8pWxgnTgWy3DUQ/C8CfiRWnL7z4GTXVhPOg1OPx/4uZpbdiwTdNXrvzjaIDGxkLDSy8NnhU7U16/ftuHjRix+cLIMXZkWfblyBla2bZt2/7A/21eHjFixO2vvfbabyNGasvXgD7F8YvAv0fM0pZP0XJE2RYgqfffkSNHnrZixYppzcsNDQ1fqqura4yZqVShUPgmkO5u3TKxoKrqFAqFW6ZPn/7L2Dnyxo0bdyXFgjp69Ojly5cv/4fIkUq9k5aCugNILd/HaF1Q/xV4PVKWNvXq1WtnQV28ePHD06ZNS+o1nDt37mQgakHNsuzXqb0uU6ZMOY9iQR0yZMjW1atXJ5UPKNC6oKaWD1oV1PGPwy9+FC9KW3p/KFdQ34DH/ituntaGD3/PxJaC2tjwuc+dllS+Sy99Zvz5578UvaCm9t5x8cUXDydXUE8++eR76uvrUyuA/5eWgrqU9N4/zqGloG4jsXwTJ07cni+ot9xyy7dvu+227TEzlaqvr/97emBB9RBfSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEmpjB1BZjQLqiuM3gPUdrFsLjM4tvww0lCmXJEmSJO3CPajd2xxgUfF2cSfrHpxbdxEworzRJEmSJKk1C6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKhqtl/sAJIkSZJ6Ngtq97Y9Nx7YybrDyhlEkiRJkjpjQe3eNuTGIztZd2I5g0iSJElSZyyo3dtLufFxHayXAR8ocxZJkiRJ6pAFtXv7bW58OnBiO+t9Eji5/HEkSZIkqX0W1O7tTlrOQ+0FzAXOAeoIe02PBK4Dvge8ECOgJEmSJDWrjR1AZbUK+C5wRXH5EEJpBWii5Q8UW4D3AI9THd8Tl9XX158bO0Te3/7t3+6cBfnFF18cC3wnYpy2HJkb9yG9fMeWLH8D2BwjSHsaGhp2jseNG/fO+vr6fhHj7KJQKBwfOwMwrb6+fkTsEHk/+MEPxjSPV61atR/pfe+Xnv+fWr4Sv/tTGHdQ7BStNfVtGa85GMZdHi/LrpYtaxjTPO7Vq6b3DTf8Kql8vXs3HBA7A9Cnvr4+qe/99evX97v55pt3Li9YsOB9QArvs3l1ufGRpPf+MTY3Tu79d+HChRPyy+9///uvmzFjRlOsPO3IYgeIoRrKiPbNFwlvEO8pub+5nL4KvBt4CniNUGKTlmXZu2NnKFVb2/JfadWqVQcDn42XplO9STsfwMdiByjV2Ni4czxo0KBJwKR4aZI1pXhLRr9+LX9HePPNN/uS/vd+4vlWnRhuqdo4FDZeFDtF3tq1LeNevWp6jRixKal8iaglse/9Pn36tFpes2bNGcAZcdLslkNJ7DUsUUdi+d54441WyzU1NZ+OFEUlLKjd3w7gAuBdwHuBwwl/jXkFmA/cAmwqrnspMLg4Xl3ZmJIkSZJ6Ogtqz3FX8daR+ZUIsqeuvvrqwty5c+fEztGRkSNHHt3Y2DgQYO3atSteffXVZbEzlRhJy97xjcAzEbO0ZX9gfG75t4TD0JNx9NFHT6ypqekF0Lt37xeAtZ18SjRZlj1ZwW3dD6yo1Pb21MCBAw8ZP378SIAtW7asX7x48XOxM5UYCBydW34kVpAOvC03fg5YHytIO44CBhXHrxL+AJuM4cOHHzx06NBDi+NNwB8iR+rQ1q1bd1RiO1mWvQqk/LO9Zvz48TuPlFm6dOkzmzZt2hgzUBvGE35+AiwjvffiQ2i5zOF6wvtHMvbff/8Bo0ePPqZ5uVAoPAoUIkbqUGNj48rYGSolmzVr1s4vRJZlR82cOfP5mIEkSZIkST3D7NmzjywUCn9sXnYWX0mSJElSEiyokiRJ6evT+SqSVP0sqJIkSen7MvAAcGDsIJJUThZUSZKk9D1AuG7uk8A7ImeRpLKxoEqSJKVvIfAmcDBhVv4fEq5rLUndigVVkiQpfeuBhuK4L/B+4AngLdESaW/1jR1ASpkFVZIkqTr8MjfuS7gO5kLgg3HiaC/0JRyuPQsnvpLaZEGVJEmqDnOATbnlDBgG/GvxsX4xQmmPXE84l/g84KDIWaQkWVAlSZKqw/3AxjbuHwRMBZ4FTqhoIu2JvwQ+DDQS9novjxtHSpMFVZIkqTqsAra381gfYDThMOArK5ZIu2sy8M/F8ReA+RGzSEmzoEqSJFWPxzp5/ADg68AdFcii3TMcuJXwR4Q7gH+KG0dKW23sAJIkSdptdwJnA3Ul928GXgRuBx4H1lQ4l9pWC/w3cAjwR+ASoBAzkJQ6C6okSVL1uA9YR9grB+GQ3z6E2WGvwENHU/OPwJ8BG4DzCZcLktQBD/GVJEmqHksIk+xAmNH358APCb/T/Qg4OE4steEi4K8Je0w/QpjESlInLKiSJEnV5RlC6XkWeA/wGeApwl7Vm/D3uxRMIvzhAOCbwP9EzCJVFd/AJEmSqsscwl7UC4oftxD21m0CzgS+Ei+agDFAPdAfmIdfD2mPWFAlSZKqyzzgu8DS3H3PAp8sjr8CnFfpUAJgCOHrM4IwWdVFtBySLWk3WFAlSZKqyyLg823c/2PCtTYzwvmox1UylOgD3AYcAywHzgU2Rk0kVSELqiRJUvfxN8DdwP7AT4HBceP0GBnw78AZhJl6/w+wLGoiqUpZUCVJkrqPRuCDwGLgCOD/4e97lfAN4EOEy/6cDzwZN45UvXzDkiRJ6l5WE85B3QScDXw/bpxu76PAFwkzK88Efhk3jlTdLKiSJEndz5PAJUATYfKky6Om6b7eBVxfHF9NuMyPpH1gQZUkSeqebieckwpwLXBhxCzd0TsIr3Et8B/A1+PGkboHC6okSVL39U/Adwi/8/0XcFrcON3GZOBOoC/wM+ATceNI3YcFVZIkqXu7nDCjb3OZ8vIz++Zk4OeEmZLnEa512hA1kdSNWFAlSZK6tybCzL4PAkOB+cBRURNVr1OAXwCDCJMhXUCYuVdSF7GgSpIkdX9bCBP6PA6MAP4XODxqoupzGqGcDgZ+DZxLeF0ldSELqiRJUs+wDjgDeAIYBfwKGBMzUBV5O+Fw3oHA/YSyvzFqIqmbsqBKkiT1HOsI10Z9DjgUuBdLamfOAe4GBhQ/no3lVCobC6okSVLP8jrwTuBFwmG+vwGOiZooXR8nTDC1H2HW3vPwsF6prCyokiRJPc9y4E+B3wOHAAuAU6MmSs8XgNlAL+A/CRMibYsZSOoJLKiSJEk902vAO4CHgSGECYBOj5ooDXXAjcC3istfAz6Cl5KRKsKCKkmS1HOtIRzuey8t51heGjVRXMMJl4+5hFBIPwlcHTOQ1NNYUCVJknq2jcA04L+BPsAPgesIh7b2JBOAR4AphOJ+NjAraiKpB7KgSpIkaRtwMXAl0ARcDtxFuOZnTzADeAAYDbwA/Alhr7KkCrOgSpIkCaAA/APwXmATMBV4FJgUM1SZDQZuA24C+gFzgJMJl+GRFIEFVZIkSXn/Q5jhdwkwjjDD7+eBLGKmcvhz4HHC7Lzbgb8mXEbmzYiZpB7PgipJktS+WcB8wiGvPckThHMybyWcl/ptYB5wcMxQXWR/4F8JkyGNBRYRDun9F8JeZEkRWVAlSZLa92eEWW5Pih0kgnXARcDHgc3AWcCzwF9Svb9DngU8CVxWXL6eUMR/Gy2RpFaq9c1FkiSpEhpLPvZE/044D/UhYBDwfcKEQsfHDLWHjiCcX/pzwl7TxYQ/PHwK2BAxl6QSFlRJkqT2nQwcAMyMHSSyZwmHwV5GOEdzMuH8zRuAQyLm6sxBhMOTnwamE841/TZwAuEQX0mJsaBKkiS1bzOwlnAZlp6uCfg3YDxh5tta4GOEy7J8GzgwXrRdDAOuJewp/TzhPNq7CHt9/4Zw7VdJCaqNHUCSJCkxU4G3ltz3DFAfIUuKVhAuRXMq8PfA6YQS+CngJ8D3gKciZZtQzPEBwmVjAB4GvgL8IlImSXvAgipJktTaecAnS+77CRbUUg8B7wDOBL4OnEKYUOnjwH3AjYTzPteVOcdBhK/ZR4sZ8vm+RjjvVFKVsKBKkiS1dgMt5yfOJEymo/bNL96mAJ8F3k24xuifE875nA/8FLifcEmXfVUDHEvYc3s+4ZqtvYqPbSNcx/V64DddsC1JFWZBlSRJau3x4g0sp3viweJtJGFv5gWEyYjeVbwBvAYsILy+i4q3pcBqwjmueX0Jswa/BTiSMBPvBEIRHpxbrwA8Srhm603Aqq79Z0mqJAuqJEmSutIKwiG/XweOIhTVswiXqhkBvKd4K7WDlsmLBtDx76kbgIWEw3d/CrzcFcElxWdBlSRJUrn8Efi74q2OUFKnAMcAhxP2jo4CMqA3MKTk85sI5fNFwmzBzxD2wD5Jz742rdRtWVAlSZJUCdsI5XJByf29gIHAfoTDejPC5X02AesrGVBSfBZUSZIkxdRIuNbs2thBJMVXEzuAJEmSJElgQZUkSepI83mOfaKmkKQewoIqSZLUvtXFj4dGTSFJPYQFVZIkqX0Lix/fBkyMGUSSegInSZIkSWrfLwiXNDkB+CXhuptvFh/7KXBPpFyS1C1ZUCVJktrXAJwD3E64hudFucdewoIqSV3KgipJktSxpYRDfI8DxhCu1QnwVKxAktRdWVAlSZI6VyAUUkupJJWRkyRJkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSErzMjKrCHXfcMSZ2ho7U19cfuHbt2r4AK1asWP/www+vj52pxABgUHG8HVgZMUtb+gDDcsvLCZd0SMa55547sqampgbgpJNOWn3cccdtiZ2pPfvtt9/6s846a00ltnX33XcfvGXLlrpKbGtvLFiwYOCiRYsGAmzcuHHr/Pnz34idqUQdcFBueVmsIB04JDdeBWyLFaQdB9JyXdINwJsRs+xi8uTJA0aMGDEIYMCAAdvPP//81N5/WznvvPOWZllW9vffW2+9db/evXsPL/d29taOHTuym2++eVTz8iOPPLJy+fLl22NmasMwws9PCN/3GyJmacvA4g1gK5DU++/YsWPr3vrWt+58/50xY0aK7787DR48+LXTTz99a+wclWBBVfKuueaamtra2sWxc3Tk0Ucf5amnvDRed3bnnXfuHI8fP57a2nTfPnfs2PF94NOV2Nb27dvn1NbWTqrEtvbGkiVLuOOOO2LHUA+2cOHCneNx48Zx4YUXRkzTuXvvvXcwFSj5ffv2/Yssy35W7u3srYaGBt87urnFixezeHHLr5eXXHIJxb9DJ2nTpk1nAffEzlEJ6X4VJEmSJEk9igVVkiRJkpSEdI9Rk9r3wSzLFna+WuVs3br1CYrnWUyYMOGxJ5544r2RI5W6CTitON4MHB8xS1veC3wzt3wKiZ2rUldXt2jbtnDq3XPPPXf95MmTr40cqdT1hULhzMgZrs2y7PrIGVpZuXLl7cAEgDFjxry5ZMmSkyJHKvVx4Mrc8uGxgnRgUW78LeCGWEHa8Tg7z7Ef+Tx88coO166wI4+888vPPz9/AsCOHTu2ZFl2XOxMeU1NTccncKhtcq/LK6+8ciDwcPPyKaecctXDDz98a8RIbXkK6FccLwBmRMzSlluBicXxeorvxamYMmXKpQ8++OCXmpfXr19/9JAhQ3bEzFSqUCi8QA/coWhBVdUpFAqvTp8+/aXYOfLGjRvX1Dzu27fvNiCpfITJCZoVSC9faRldCrweI8ju2LRp07pp06Yl9RrOnTt3c+wMwNrUXpcpU6bsnNCntra2ifS+99eWLKeWr9Ra0svY2DKs3Q6XLY8XZVd1db/c+T1YKBSaUvs/MmfOnKGxMwCF1F6Xiy++eFN+ediwYW+Q3vd+U268lfTy5SdUS+79d+jQoavzy3PmzFl82223JTURVn19fewIUfS4Ri5JkiRJSpMFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKQm3sAD3IYcBooD+wDPgDUNiH5xsOvAUYDKwHFgMr9jGjJEmSJEVTDXtQbwR+W7xd1Mm6B+TW/S0wqrzRdjEfWFS8DS3eNw14BFgC/Br4OfAUsBT44B4+fx/g08DjwKvAg8A84AFC6X0MeB+QtfG5E3LZftfOOrvjwNzzLAKG7OXzSJIkSVIr1bAH9WhgYnE8rJN1e+fWBagrS6L2HUrYq9k8/hjwTdoug4cCPwIOAv55N577aGAOcEQ7j2fAScDNhCJ/MbA19/izhMLe/JqMA17Yje2WOoWWf+MLwNq9eA5JkiRJ2kU1FNRq9QHgcqAJ+AVh72aBUPDOpKW0/j3wU8Ie1fYcD9xH2EPc7FfF+9YQSu5ZwNuKj51HKL8X5tbfStir/CfF5VPY+4La7IG9+HxJkiRJapMFtXw+T9i7+C7goZLHzgduJxxi3Rf4MPC1dp5nP+AWWsrpSsIe0vtK1rsa+BDwQ8Ke5AuKt9tz6zxA64L64z349zTLF9QFe/H5kiRJktSmajgHtVplwEfYtZwC3FG8NTuzg+f5BDC+ON5OOKf1vnbW/RHhkOJmV5Q8ni+Up7DnMuDk3LJ7UCVJkiR1GQtq+TxDOGe0PXNz42PbWScDPpdbng082sl2vwvsKI5PAcbmHltAy8zBJxL23pYaSzjHtK0Jpo6iZVKklcDznWSRJEmSpN1mQS2f+Z08viw3HkLbh1sfC4zJLf/Hbmx3NWGW32ZTcuM1hOIMYUbgCSWfOw54iTA77x/ayFR6eO++XCZHkiRJklqxoJZPZxMQvVay3L+Ndf40N95IuDzM7ngpNy6d9Td/WG7pYb5/kRsPAiaXPO75p5IkSZLKxkmSymd9J483lCy39ceC/GG2ven88F4Ie2MPLFnOW0A4rxV2LahTS5bPAn6TW3YGX0mSJEllY0Etn6YueI78ZWXqaH2N193Vr2S5vT2ofYDTi+OfES5VMxX4Uu55ji+ON9P6MGJJkiRJ2mcW1LTlS+7rwK/34jkeK1leDCwn7J0dCwwjTHg0GRgAbACuIhTUCbnHTyLsxQV4mJaJmCRJkiSpS3S3gtrd/j1rc+NFwHu76HkfIFxLFcJe1HpaDu+9F3iueDuacF7qj/HwXkmSJEllVg2TJDXmxn06Wbf0fMtqtyQ3PqQLn7et66E2T5A0r/jxruLHs0rWK/18SZIkSeoS1VBQN+XGIztZ95hyBokgP0HRaML1SbtC6XmowwiH8xaAnxfvby6qZxK+T5oLaiOwsItySJIkSdJO1VBQ85djGdPJutPKmCOGF4Bnc8uf6aLnfZKWWYZPJhzeWwP8nnB+KoQSu55QXt9FKMilnytJkiRJXaYaCuqTufE7aT2zbd4xwPvKH6eiCsC3c8ufofW1SjtSy66XjWmW3ws6CPhscTwvt852YH5x/JXc/Z5/KkmSJKksqqGg3pUbDwRuIsw2mzeJUK4KlQpVQTcB9xfHvYA5wBVA33bWH0O4NMwLwPc7eN78eaSTih/nlawzr+Tx0s+TJEmSpC5TDbPePgfcTctkPdMIkwctIJyfehTwViADPkkoZb0qnrJ8GoH3A78CjiRcD/Va4MuE1+AVoAEYQXgd8uepLurgeUv3hK4BHiq57+eE0p918HmSJEmS1CWqoaACfIowYVDzTLYHANNL1rkWmEU4HLWzyZSqzQrCdUpvBM4p3jcQOLuTz3uug8ear2XafG3Te2g9YzLAq8DjwMTicvM1VCVJkiSpy1VLQV1CmEX2G8C7CedNQthz+CvgO7QcCvxjYGxxnJ8BuBLupuWc2aWdrLsBuC23vL2T9dcA5wKnAZcSZtctvfTMWuB3hGuZ/gx4poPn2wx8l5bJj25sZ73vESZJglBqJUmSJKksqqWgQtiLeCnwCeAgwvmzr7LrXr8vVDhX3l/twbrLgffuxTYeoOUw237AUMJe0LXF2564YjfWual4kyRJkqSyqqZKzFM7AAADyElEQVSC2mwHoawq7AXdHDuEJEmSJHWFaiyo+yIDBnfRc70JNHXRc0mSJElSj9fTCuoBwBtd9FyHAS930XNJkiRJUo9XDddBlSRJkiT1AD1tD+oawl7UrvBmFz2P9lCWZbfW19d3NutxRV111VUDm8ePPvroJNI7Tzr/fd+P9PLtV7L8exI7hH779pZvuQkTJnz6sssu+3DEOLsoFApDYmcArqqvr/9M7BB5N95449Dm8eLFiweR3vd+/5Ll1PKV+hJ7NiFgJeRO3Vl+NPS5q/1VK++ZZ7LmKw/Qp0+ffvX19al9jXt3vkrZ7Zfa67J58+aam2++eefyvHnzrgW+Fi9Rm/LvH6eR3vvH0Nx4IInlmzdvXqv33w984ANLZsyYEStOe3rkzsSeVlAL7PlMt0rP0M5Xqawsy3aOGxoa+gAHx0vTqYy08wEMjx2gVKFQ2DmuqanZH9g/XppkDSjeklFT0/KzvbGxsYb0v/dTzzeweEtUYy00Hhg7RV5j62sNVMP7bwzJvS759w6AxsbGwXTdPCblUEdir2GJ5N5/G0v/c2ZZUvl6sh7ZyiVJkiRJ6elpe1BVha655pqmuXPnfih2jo6ccMIJx44cOXIwwPLly195+umnU5tA6xDCxF4AG4AnI2ZpywDghNzyQ+x6jeOozjjjjFNra2t7AfTv3//ZLMvWxM7Unqampj9Walu9evX6cqFQSGqPVd7IkSMPmzp16iEA69evX7dw4cI/xM5UYhBwXG55QawgHfiT3Php0jvF5Vha9mwtA5ZGzLKLY445ZtTo0aPHAAwePHhDlmWpvf+2MmTIkIpcvq62tvaxpqamZH+2Z1nWa+rUqac2Lz/xxBNPrly5ckPMTG04npYjGpYSvv9TMho4tDheByT1/jtixIhBJ5544s733yzLHsyyrNDR58RUW1ub9HtHV8pmzZq18wuRZdlRM2fOfD5mIEmSJElSzzB79uwjC4XCzj+ue4ivJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSUJtfKBQKY2bPnh0riyRJkiSpBykUCmPyy7Ulj99TKBQql0aSJEmSpCIP8ZUkSZIkJcGCKkmSJElKwv8Hrm+iffGnhpYAAAAASUVORK5CYII=\" align=\"left\" width=\"450\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c397005",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "<b>Question:</b>  Which is the communication and computation complexity in each process? N is the length of the vector and P the number of processes.    \n",
    "</div>\n",
    "\n",
    "    a) Communication: O(P), computation: O(N/P)\n",
    "    b) Communication: O(1), computation: O(N)\n",
    "    c) Communication: O(P), computation: O(N)\n",
    "    d) Communication: O(1), computation: O(N/P)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a03fc4c",
   "metadata": {},
   "outputs": [],
   "source": [
    "answer = \"x\" # replace x with a, b, c or d\n",
    "jacobi_1_check(answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ed4129c",
   "metadata": {},
   "source": [
    "## Implementation\n",
    "\n",
    "We consider the implementation using MPI. The programming model of MPI is generally better suited for data-parallel algorithms like this one than the task-based model provided by Distributed.jl. In any case, one can also implement it using Distributed, but it requires some extra effort to setup remote channels right for the communication between neighbor processes.\n",
    "\n",
    "Take a look at the implementation below and try to understand it. Note that we have used MPIClustermanagers and Distributed just to run the MPI code on the notebook. When running it on a cluster, MPIClustermanagers and Distributed are not needed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e15082fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "] add MPI MPIClusterManagers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a66cbf9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "using MPIClusterManagers \n",
    "using Distributed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0d63c6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "if procs() == workers()\n",
    "    nw = 3\n",
    "    manager = MPIWorkerManager(nw)\n",
    "    addprocs(manager)\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0923606",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test cell, remove me\n",
    "u = [-1, 0, 0, 0, 0, 1]\n",
    "view(u, 6:6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "68851107",
   "metadata": {
    "code_folding": []
   },
   "outputs": [],
   "source": [
    "@mpi_do manager begin\n",
    "    using MPI\n",
    "    comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
    "    nw = MPI.Comm_size(comm)\n",
    "    iw = MPI.Comm_rank(comm)+1\n",
    "    function jacobi_mpi(n,niters)\n",
    "        if mod(n,nw) != 0\n",
    "            println(\"n must be a multiple of nw\")\n",
    "            MPI.Abort(comm,1)\n",
    "        end\n",
    "        n_own = div(n,nw)\n",
    "        u = zeros(n_own+2)\n",
    "        u[1] = -1\n",
    "        u[end] = 1\n",
    "        u_new = copy(u)\n",
    "        for t in 1:niters\n",
    "            reqs = MPI.Request[]\n",
    "            # Exchange cell values with neighbors\n",
    "            if iw != 1\n",
    "                neig_rank = (iw-1)-1\n",
    "                req = MPI.Isend(view(u,2:2),comm,dest=neig_rank,tag=0)\n",
    "                push!(reqs,req)\n",
    "                req = MPI.Irecv!(view(u,1:1),comm,source=neig_rank,tag=0)\n",
    "                push!(reqs,req)\n",
    "            end\n",
    "            if iw != nw\n",
    "                neig_rank = (iw+1)-1\n",
    "                s = n_own+1\n",
    "                r = n_own+2\n",
    "                req = MPI.Isend(view(u,s:s),comm,dest=neig_rank,tag=0)\n",
    "                push!(reqs,req)\n",
    "                req = MPI.Irecv!(view(u,r:r),comm,source=neig_rank,tag=0)\n",
    "                push!(reqs,req)\n",
    "            end\n",
    "            MPI.Waitall(reqs)\n",
    "            for i in 2:(n_own+1)\n",
    "                u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
    "            end\n",
    "            u, u_new = u_new, u\n",
    "        end\n",
    "        u\n",
    "        @show u\n",
    "        # Gather results in root process\n",
    "        results = zeros(n+2)\n",
    "        results[1] = -1\n",
    "        results[n+2] = 1\n",
    "        MPI.Gather!(view(u,2:n_own+1), view(results, 2:n+1), root=0, comm)\n",
    "        if iw == 1\n",
    "            @show results\n",
    "        end            \n",
    "    end\n",
    "    niters = 100\n",
    "    load = 4\n",
    "    n = load*nw\n",
    "    jacobi_mpi(n,niters)\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eff25246",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "<b>Question:</b>  How many messages per iteration are sent from a process away from the boundary?\n",
    "</div>\n",
    "\n",
    "    a) 1\n",
    "    b) 2\n",
    "    c) 3\n",
    "    d) 4\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98bd9b5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "answer = \"x\" # replace x with a, b, c or d\n",
    "jacobi_2_check(answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "075dd6d8",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "<b>Question:</b>  After the end of the for-loop (line 43), ...\n",
    "</div>\n",
    "\n",
    "    a) each worker holds the complete solution.\n",
    "    b) the root process holds the solution. \n",
    "    c) the ghost cells contain redundant values. \n",
    "    d) all ghost cells contain the initial values -1 and 1. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3b58002",
   "metadata": {},
   "outputs": [],
   "source": [
    "answer = \"x\" # replace x with a, b, c or d\n",
    "jacobi_3_check(answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4537661d",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "<b>Question:</b>  In line 35 of the code, we wait for all receive and send requests. Is it possible to instead wait for just the receive requests?\n",
    "</div>\n",
    "\n",
    "  \n",
    "    a) No, because the send buffer might be overwritten if we don't wait for send requests.\n",
    "    b) No, because MPI does not allow an asynchronous send without a Wait().\n",
    "    c) Yes, because each send has a matching receive, so all requests are done when the receive requests return.  \n",
    "    d) Yes, because there are no writes to the send buffer in this iteration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e16ea5eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "answer = \"x\" # replace x with a, b, c or d.\n",
    "jacobi_4_check(answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9aa2901",
   "metadata": {},
   "source": [
    "### Latency hiding\n",
    "\n",
    "Note that we only need communications to update the values at the boundary of the portion owned by each process. The other values (the one in green in the figure below) can be updated without communications. This provides the opportunity of overlapping the computation of the interior values (green cells in the figure) with the communication of the ghost values. This technique is called latency hiding, since we are hiding communication latency by overlapping it with communications that we need to do anyway.\n",
    "\n",
    "The modification of the implementation above to include latency hiding is leaved as an exercise (see below).\n"
   ]
  },
  {
   "attachments": {
    "fig.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6gAAADyCAYAAABAvOgkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d17fFxlgf/xz0nTO71ZKKWFUrByKbctlB+2ogJVi0q5rCCLrhVd7bqC6y6yK+yq4OruqoDu/lxXWryh+/uhFERIqSDIRS2lVKwCcoe2SIG29ELb9Jpk9o9n0pxMZ5I0JPM8ST7v12tePU/mzMy3STrNN+ec58nmzp1bQJIkSZKkyGpiB5AkSZIkCSyokiRJkqRE1JaMZ2ZZtiJGEEmSJElS31IoFCYCdzaPWxXULMtWzJkz5+lqh5IkSZIk9T3z5s2jUGiZFslTfCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSkJt7ACSJCk5Q4D3FrcXAvURs1TboOJtF33r7y1JSbCgSpKkUqOBG4vbbwSe7+Tz1ABvAqYCU4BhxY9fQrrl75+AzwP3ADMiZ5GkPseCKkmSusN/Ax8Ehpe573LSLagxvBc4E3gGuDpyFkmKyoIqSZJKbQQuK26v7+RzTKalnK4qPudRrzNXNfwJeBD4YxVf83hgDvArqldQDwJOyN0OAzLgJuCzVcogSXuwoEqSpFKbga++zue4DrgKeBh4Bfgw8IPX+ZzVcF3x1ptNBZZWuG+/agaRpFIWVEmS1B3+X+wAPch3CJNRbdnLxx0GHEe4RvjhTrzuBuC3xcd+CBjfieeQpC7lMjOSJKnUQUCheDs0cpZquxC4F7imiq/5MqEkPrWXj5tFmMxqzl4+7mnC5FdvAN5FuCZ43V4+hyR1C4+gSpIktTgUOAVoquJrTiZcn7sWuK8Kr7epeJOk5HgEVZIkKa73EY6EfjF2EEmKzYIqSZIkSUqCBVWSJEmSlASvQZUkSR01AphU4b5n8LpGSdLrZEGVJEkd9Tbgtgr3nQ7cWcUsfcWhVJ6l98TinycBX6mwzy3Akq4OJUndxYIqSZI6aiOV19v06Gn3mAB8tp19jiveylmOBVVSD2JBlSRJHfVrYGrsEH3MC8BXK9x3InAa8Afgjgr7/L47QklSd7GgSpIkpet54LIK932GUFCXtLGPJPUozuIrSZIkSUqCR1AlSVJ3OI4wcVKz43Pbnwa25sbfArZUI5QkKW0WVEmS1B3amln2CyXjH2JBlSRhQZUkSXvaAswrbnd2dt4ncs/Rnq3t71I1W4C1wIbYQbrZ94EDc+NDi3/OBO7Kffxe4N+qFUqSLKiSJKnUBuCvX+dz/Lp462m+Vrz1dtOAw8t8fFzx1mxtdeJIUmBBlSRJiutm4En2vgzWEZaheb4Tr/l3wLAO7PdCJ55bkjrNgipJkhTX48Xb3nq6eOuMSuumSlJULjMjSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSXCZGSWvUChkdXV1D8XO0ZZrrrnm4OXLlw8B2Lx586sbNmxIbWHzfYH9itvbgBXxopQ1GJiYGz8FNMWJUt6ECRMOp/hLvQsvvPDFqVOnbo4cqaIsy26cNWvWVdV4rbq6uh8UCoWjqvFanTF//vwx999//2iAHTt21K9evTq1NR2HAhNy4ydiBWnDkbntF4D6WEEqmED4PAKsA9ZEzLKHkSNHjh4+fPgYgPHjx2+//PLLl8fO1JaBAweeMnPmzG7/Gt9+++1va2xsvKa7X6eztm/fXnPppZce3jxevXr1ih07dmyLmamMiYT/PyGsoftqvChl7Uf4+QNgK7AyYpY9DBo0aMiYMWMObh5/85vffCLLspiR2vOpM88888HYIarBgqrkffGLX8xOOOGEqbFztGX9+vW88MLun3uHAge3sXtsQ2n5DyNVx8cOUCr39aWxsfHwxP8TW1KtFyoUCkdlWZbsv89t27aV/tscEzFORyT7uSw6sv1doiot/NFt3LiRjRs3AjBgwIChWZaNjhypTTU1NVX52bCxsXFUyu8dWZa1et8Hkv1FXNFQWv+iNzVDaflFeRK2b99e+jWemvL/7VmWjYidoVo8xVeSJEmSlASPoKonui7Lsudih8jbtWvXFRRPsxk/fvzKVatWfTtypFJ/BbypuL0T+ELELOVMBc7Njb8MbImUpaza2tqvNDQ0ALB+/fp7syy7M3KkUrMLhcLkmAEKhcIdNTU198XMUGrz5s2fpHhEbfTo0dvWrVv3xciRSr0NeE9ufFmsIG34Ssvm+Adg/2XxopTz+49B08CwPeIVeOPNcfO0NmZM/aw1a56aANDYWGh4/vmRc2NnyhsyZOeYsWO3nhc5xq4syz4fOUMrO3bs2Af4XPN47NixN73yyiu/jRipnH8BBhS3nwW+EzFLOZ+g5ajuduDKaEnKGDdu3MkvvfTSGc3jhoaGfx44cGBjzEylCoXCvwPpHtbtJhZU9TiFQuHHs2bNuid2jrxJkyZdRrGgTpgwYdWqVau+GjlSqXfQUlB3Aanl+xitC+p/AasjZSmrX79+uwvq8uXLl5xxxhlJfQ4XLFgwDYhaULMs+1Vqn5fp06efTbGgjho1avu6deuSyke41jpfUL8GFCJlqSRXUCf/Dn7xo3hRyun/oVxBfRUe/mHcPK3tv//7TmgpqI0Nn/70yUnl++hHH598zjnPRy+oqb13XHDBBfuTK6gnnnjinXV1dakVwM/RUlBXkt7/7bNoXVCTynfCCSfszBfUH//4x1fPnz9/Z8xMperq6v6NPlhQPcVXkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUhNrYAdRtBgOTc+NlQFM7j6kBpuTGjwPbujiXJEmSJJVlQe29DgV+mxsPof2yOaDkMccAj3VxLkmSJEkqy1N8JUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBB7b0KJeOsA4/ZpzuCSJIkSVJHWFB7ry0l446Uz4O7I4gkSZIkdYQFtfdaT+ujqJM68Jh3dFMWSZIkSWqXBbX32gIsz41Pb2f/YcBF3RdHkiRJktpmQe3d7s5tXwyMr7DfQOB64KBuTyRJkiRJFVhQe7d5QFNxexTwa+BcWq5H3Rc4H3gIOAf4fbUDSpIkSVKz2tgB1K0eBr4JfLo4PgSYX9xuovUvKF4ilNdnq5au8y6qq6s7K3aIvH/8x38c3Lz97LPPHgL8Z8Q45RyW2x5AevmOKhl/GdgaI0glDQ0Nu7cnTZr0jrq6uiER4+yhUCgcEzsDcEZdXd3Y2CHyvv3tb09s3l67du1g0vveP75k/B9RUnTY798Kk/aLnaK1pkEt2+sPgEmXxMuypxdfbJjYvN2vX03/6667N6l8/fs3vCF2BmBAXV1dUv82N23aNOSGG27YPV60aNFfACm8z+YNzG0fRnrvb4fktpN7/128ePGU/PgDH/jANbNnz26qtH8kHVmFo9exoPZ+nyn++SlaF9L89m+AvwDWVSvU65Fl2Z/HzlCqtrbln9LatWsPAP42Xpp29SftfAAfix2gVGNj4+7tESNGTAWmxkuTrOnFWzKGDGn5PcJrr702iPS/9xPPt/a4cEvVltGw5fzYKfI2bGjZ7tevpt/YsfVJ5UtELYl97w8YMKDVeP369TOAGXHSdMhBJPY5LDGQxPK9+uqrrcY1NTUXR4qiEhbU3q8R+Dvgu4QSejywH2GW3+cIR1TvIxxR7Qe8P/fYP1UzqCRJkqS+zYLadzxavLWlkZZTgJNxxRVXFBYsWHBb7BxtGTdu3BGNjY3DATZs2PDSyy+//GLsTCXGAQcWt7cAj0fMUs4+wOTc+Le0XD+dhCOOOOKEmpqafgD9+/d/BtjQzkOiybLskSq+1v2ESwSSNHz48AMnT548DmDbtm2bli9f/mTsTCWGA0fkxg/FCtKG/5PbfhLYFCtIBYcDI4rbL5PYL1f333//A0aPHn1Qcbse+GPkSG3avn37rmq8TpZlLwMp/99eM3ny5N1nyqxcufLx+vr60jXmY5tMy7wiL5Lee/GBhJ8/ILxvJPX+u88++wybMGHCkc3jQqGwlNZLNCalsbFxTewM1ZLNnTt39xciy7LD58yZ83TMQJIkSZKkvmHevHmHFQqFp5rHzuIrSZIkSUqCBVWSJCl9A9rfRZJ6PguqJElS+j5PmHV/39hBJKk7WVAlSZLS9xvgBOAR4LTIWSSp21hQJUmS0rcYeA04ALidsHxc/6iJJKkbWFAlSZLStwloKG4PAj4ALAMOjZZInTUodgApZRZUSZKknuGe3PYgwjqYi4G/jBNHnTCIcLr2XJz4SirLgipJktQz3AbU58YZMAb4r+J9Q2KE0l65lnAt8dnAfpGzSEmyoEqSJPUM9wNbynx8BDATeAI4tqqJtDc+CXwYaCQc9V4VN46UJguqJElSz7AW2FnhvgHABMJpwJdVLZE6ahrwjeL2Z4G7ImaRkmZBlSRJ6jkebuf+NwBfAm6pQhZ1zP7AjYRfItwCfD1uHClttbEDSJIkqcNuBd4NDCz5+FbgWeAm4HfA+irnUnm1wE+AA4GngAuBQsxAUuosqJIkST3HfcBGwlE5CKf8DiDMDnspnjqamq8Bbwc2A+cQlguS1AZP8ZUkSeo5VhAm2YEwo+/Pge8Sfqb7EXBAnFgq43zg7wlHTD9CmMRKUjssqJIkST3L44TS8wTwPuBTwKOEo6rX4893KZhK+MUBwL8DN0fMIvUovoFJkiT1LLcRjqKeW/xzG+FoXT3wTuAL8aIJmAjUAUOBhfj1kPaKBVWSJKlnWQj8X2Bl7mNPAJ8obn8BOLvaoQTAKMLXZyxhsqrzaTklW1IHWFAlSZJ6lueAz5T5+P8Q1trMCNejHl3NUGIAMB84ElgFnAVsiZpI6oEsqJIkSb3HPwB3APsAPwVGxo3TZ2TAd4AZhJl63wO8GDWR1ENZUCVJknqPRuAvgeXAm4D/jz/vVcOXgQ8Rlv05B3gkbhyp5/INS5IkqXdZR7gGtR54N/CtuHF6vb8C/okws/Ic4J64caSezYIqSZLU+zwCXAg0ESZPuiRqmt7rvcC1xe0rCMv8SHodLKiSJEm9002Ea1IBrgLOi5ilNzqN8DmuBb4HfCluHKl3sKBKkiT1Xl8H/pPwM98PgZPjxuk1pgG3AoOAnwF/HTeO1HtYUCVJknq3Swgz+jaXKZefeX1OBH5OmCl5IWGt04aoiaRexIIqSZLUuzURZvZ9ABgN3AUcHjVRz3US8AtgBGEypHMJM/dK6iIWVEmSpN5vG2FCn98BY4FfAm+MmqjnOZlQTkcCvwLOInxeJXUhC6okSVLfsBGYASwDxgP3AhNjBupB3kY4nXc4cD+h7G+JmkjqpSyokiRJfcdGwtqoTwIHAXdjSW3PmcAdwLDin+/Gcip1GwuqJElS37IaeAfwLOE0318DR0ZNlK6PEyaYGkyYtfdsPK1X6lYWVEmSpL5nFfBW4A/AgcAi4M1RE6Xns8A8oB/wA8KESDtiBpL6AguqJElS3/QKcBqwBBhFmADo1KiJ0jAQ+D7wleL4X4CP4FIyUlVYUCVJkvqu9YTTfe+m5RrLj0ZNFNf+hOVjLiQU0k8AV8QMJPU1FlRJkqS+bQtwBvATYADwXeAawqmtfckU4CFgOqG4vxuYGzWR1AdZUCVJkrQDuAC4DGgCLgFuJ6z52RfMBn4DTACeAd5COKosqcosqJIkSQIoAF8F3g/UAzOBpcDUmKG62UhgPnA9MAS4DTiRsAyPpAgsqJIkScq7mTDD7wpgEmGG388AWcRM3eEU4HeE2Xl3An9PWEbmtYiZpD7PgipJktTirwjrgn4jdpDIlhGuybyRcF3q1cBC4ICYobrIPsB/ESZDOgR4jnBK738QjiJLisiCKkmS1OJg4GTg2NhBErAROB/4OLAVOB14AvgkPfdnyNOBR4CLiuNrCUX8t9ESSWqlp765SJIkqTq+Q7gO9UFgBPAtwoRCx8QMtZfeRLi+9OeEo6bLCcvr/A2wOWIuSSUsqJIkSS2+AuwHnBM7SGKeIJwGexHhGs1phOs3rwMOjJirPfsRTk9+DJhFuNb0asIR8nsi5pJUgQVVkiSpxVbgVWBT7CAJagL+G5hMmPm2FvgYYVmWq4F940XbwxjgKsKR0s8QrqO9nXDU9x8Ia79KSpAFVZIkqcU0whqg58YOkrCXCEvRTAPuBQYRSuBKYB5xT/2dUsywHLgUGAosISyZcwbwdLxokjrCgipJktTi3cA1hGsT1bYHgdOAdxFK4BDChEqPEIrrbMI6o91tv+LrPkg47fjjxSwPAu8B3gz8ogo5JHWB2tgBJEmS1KPdVbxNB/4W+HPCGqOnEK75vAv4KXA/YUmX16sGOAo4lXCt8FuBfsX7dhDWcb2WsFyQpB7GgipJkqSu8EDxNo6wnuy5hMmI3lu8AbwCLCIc6XyueFsJrCNc45o3iDBr8KHAYYSZeKcQinD+yGwBWEpYs/V6YG3X/rUkVZMFVZIkSV3pJeBLxdvhhKJ6OmGpmrHA+4q3UrtombxoGG3/nLoZWExYNuanwAtdEVxSfBZUSZIkdZengH8t3gYSSup04EjgjYSjo+OBDOgPjCp5fBOhfD5LmC34ccIR2EeAxu6PL6naLKiSJEmqhh2Ecrmo5OP9gOHAYMJpvRlhuZ96XO5H6nMsqJIkSYqpEdhQvEnq41xmRpIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpLgMjOSJEkt5gK3A6/FDiJJfZEFVZIkqcWq4k2SFIGn+EqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISXGZGPcItt9wyMXaGttTV1e27YcOGQQAvvfTSpiVLlmyKnanEMGBEcXsnsCZilnIGAGNy41VAIVKWss4666xxNTU1NQDHH3/8uqOPPnpb7EyVDB48eNPpp5++vhqvdccddxywbdu2gdV4rc5YtGjR8Oeee244wJYtW7bfddddr8bOVGIgsF9u/GKsIG04MLe9FtgRK0gF+wKDitubSWz90mnTpg0bO3bsCIBhw4btPOecc1J7/23l7LPPXpllWbe//954442D+/fvv393v05n7dq1K7vhhhvGN48feuihNatWrdoZM1MZYwj/f0L4vt8cMUs5w4s3CO8bayNm2cMhhxwy8M/+7M92v//Onj07xfff3UaOHPnKqaeeuj12jmqwoCp5V155ZU1tbe3y2DnasnTpUh599NHYMdSNbr311t3bkydPprY23bfPXbt2fQu4uBqvtXPnzttqa2unVuO1OmPFihXccsstsWOoD1u8ePHu7UmTJnHeeedFTNO+u+++eyRVKPmDBg16V5ZlP+vu1+mshoYG3zt6ueXLl7N8ecuPlxdeeCHF30Mnqb6+/nTgztg5qiHdr4IkSZIkqU+xoEqSJEmSkpDuOWpSZX+ZZdni9nernu3bty+jeJ3FlClTHl62bNn7I0cqdT1wcnF7K3BMxCzlvB/499z4JCCpawUHDhz43I4d4dK7J5988tpp06ZdFTlSqWsLhcI7I2e4KsuyayNnaGXNmjU3AVMAJk6c+NqKFSuOjxyp1MeBy3aPvsEn40Wp4O/5793bb+cWzuauiGn29A9cTQNDwmDc0/BPl7X9gOo67LBbP//003dNAdi1a9e2LMuOjp0pr6mp6ZgETrVN7vPypz/9aV9gSfP4pJNOunzJkiU3RoxUzqPQ/L3PImB2xCzl/ARovgRkE8X34lRMnz79ow888MA/N483bdp0xKhRo3bFzFSqUCg8Qx88oGhBVY9TKBRenjVr1vOxc+RNmjSpqXl70KBBO4Ck8gH5i+oLpJevtIyuBFbHCNIR9fX1G88444ykPocLFizYGjsDsCG1z8v06dN3T+hTW1vbRHrf+60nszqNNdSkNUFYKyPZwjsS+7eZ0dQyqN0JF62KF2ZPAwfes/t7sFAoNKX2b+S2224bHTsDUEjt83LBBRfU58djxox5lfTeP3Lf+2wnvXz5CdWSe/8dPXr0uvz4tttuWz5//vykJsKqq6uLHSGKPtfIJUmSJElpsqBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmojR2gl8uANwLjgUHAS8DjQOPrfN5JwFhgGLAReB5Y/TqfU5IkSZKiSv0I6kTgudxtWAceU1PymKO7K1zRJ3Ov9T+5DJ8CngSeAe4D7gAeIRTJLwFD9vJ1DgK+BbxYfM5fAwuBB4CXgd8BH6Htr+nluaz/2c7r/TOtP48faWf/m3L7fridfSVJkiRpD6kfQe0PHJobd7RQ5x8zsOvilDUq93orCCX6JuBdFfYfDXwOeAcwE9jUgde4CLiacBS2nAyYAnyPUCTPBtaX2e+xXNbzgL8DChWe8xxafx7PBL5fYd99gFnAgOL49xX2kyRJkqSKUi+oPU0N4Sjquwin8d4PPAxsBg4GzgL2Le77ZkLpnNPOc14JXJEb1wM/B5YBrwFjCEX3pOL9bwXuAt4CbC95rvuBBsLX/QBgMvDHMq/5BkLhzTsF6Ef505PfSks5XUs4UixJkiRJe8WC2rVOJnxO/wicz57l71LgZuC04vijwJeBFyo83yzgC7nxTwhHU9eV7HcF4YjoDwinDh8P/CvwmZL9NgFLgWnF8YwyGQFOpeVo9Q7CUeiRwAnAQ2X2n5HbvofKR2UlSZIkqaLUr0HtaWqBVYSCV674bQQ+AGwpjvsRimU5/QnXnGbF8Y3ABexZTpvNBz6WG3+ScHS11C9z2zPK3F/68Wv3cv97KuwjSZIkSW2yoHa9zxFOc61kNbAgN55aYb/zCBMjQSi0F9P+kckbCEdIIVyvekGZffIF8u2UP4reXDg3A18Fmko+nrcvcGxu/Msy+0iSJElSuyyoXWsX4Uhnex7ObU+ssM+Zue1babv05v00t/3WMvc/AGwrbo8ATiy5/yDgsOL2rwgzBP+hOH4LMLhk/9No+T5aQZjFV5IkSZL2mgW1az0FbO3Afmty2yMq7JMvl4v3IsOTue0jy9y/A/hNblx6VDQ/vrv4Z/NR0UHA9Db29+ipJEmSpE5zkqSuVW5pl3J25rYHlLl/IDAuN76E9tchhbDETX6t2DdU2O+XwDuL2zMIEzWRG+f3a/7z0tz9+SJ6Wpn9JUmSJGmvWVC7VkMXPc+okvGhZfdq39AKH89fhzqNMPNv85Hf5sL5CmHdVIBfE0r1AFoX2IOBScXtAnBvJ3NKkiRJkgU1Uf1KxvdQefbetmyr8PHfARsIRXggYXmcXxDWRW0+cptfLqYeeBB4G2GpmVHFx+fL6h8JpVaSJEmSOqU3FtRyp8z2NKVl9JvAz7rw+RuB+4BziuMZhILa1um6vyQU1H7AKcAteP2pJEmSpC6U+iRJ20vGgzrwmHJrf/Y022l9NPLYSju+DuXWQy03QVKl/TO8/lSSJElSF0q9oG4qGR/Qgcec1B1BIshfz3lmxb06L18opxCK/SnF8TPACyX7P0RYFxVCQT0KGFscNwD3d0NGSZIkSX1I6gX1NWB1bvyWDjzm492Updpuzm2fALy7i5//SWBVcbuGMEvvyOK43NHQXYR1UQGOAGbn7vste/4yQZIkSZL2SuoFFVqvAfoJ2r5u9iO0LJ/S091CmHio2feAQ/bi8fvRUjgryc/me3Fuu/T03mb54npxhY9LkiRJUqf0hIL6o9z20cD1wPCSfYYAXwCuA7ZUKVd3ayIU7ubrcMcCS4GPUnkiqH7AqcA8YCXwxnZeI18sB+det9JyMeX2L/24JEmSJHVKT5jF92fAIlpO7/0AcEbxY68RjhS+mbDmZxPh1NOfVj9mt1hK+Pv8kDBB1Gjgu8DXCUeWXyQU2JGEo6vHAfvsxfOXK5bLgPUV9n8UWEPriai20footyRJkiR1Sk8oqE3A+cBdwJHFjw1nz2sytwJ/A9xavWhVMR9YAfyAsE4pwAjg9HYe9zKwsZ19XgSeAg7PfazS6b0Q1kW9B/iL3McWsedsy5IkSZK013pCQYUwmc+JwCXAB2ldqDYQStx/AE8Qlj+ZX3J/d3oi93qPdfAxL+Qes7qtHYuWAscAfw6cSziNt3Q5nS2EiY9+BdxJODra2IHn/ibw9tz45ko7Fv2IcCpxR/eXJEmSpA7pKQUVoB74UvE2jHBq70b2PB21ALy/irl+yt6fUvxA8bY3moCbijcIpzSPJnwNy30eOupbxVtHLSzeJEmSJKlL9aSCmreZljU5+6r64k2SJEmSeoWeMIuvJEmSJKkPsKBKkiRJkpJgQZUkSZIkJaGnXoPaWR8CBnfB8zwN3NcFz6NOyLLsxrq6up2xc+Rdfvnlw5u3ly5dOhV4KWKcct6Q2x5CevlK/13+gTAxWDJ27mz5lpsyZcrFF1100YcjxtlDoVAYFTsDcHldXd2nYofI+/73vz+6eXv58uUjSO97f2ir0UlcFylHxyzkXO7kjNgxWmnIfw5XHQEDbo8XZk+PP56NaN4eMGDAEI2W2AAAAwZJREFUkLq6utS+B/vHDgAMTu3zsnXr1pobbrhh93jhwoVXAf8SL1FZ+fePk0nv/W10bju599+FCxe2ev/94Ac/uGL27Nmx4lTSJw8m9rWCehWwfxc8z/VYUGMa3f4u1ZVl2e7thoaGAcAB8dK0KyPtfNA1/067VKFQ2L1dU1OzD7BPvDTJGla8JaOmpuX/9sbGxhpS/97fTgq/aKhsF4PZ1SW/6O0mjbXQuG/sFHmNrRd86wnvvzEk93nJv3cANDY2jgRGxknTIQNJ7HNYIrmvcWPpP84sSypfX9YnW7kkSZIkKT197Qjq4XRNKU/q9NLe7sorr2xasGDBh2LnaMuxxx571Lhx40YCrFq16k+PPfbYC7EzlTgQOLi4vRl4JGKWcoYBx+bGDwKNFfaNYsaMGW+ura3tBzB06NAnsizr7NrD3a6pqempar1Wv379Pl8oFJI6YpU3bty4g2fOnHkgwKZNmzYuXrz4j7EzlRgBHJ0bL4oVpA1vyW0/BrwWK0gFR9FyZOtFYGXELHs48sgjx0+YMGEiwMiRIzdnWZba+28ro0aN2lqN16mtrX24qakp2f/bsyzrN3PmzDc3j5ctW/bImjVrUlvi8Big+RKjlYTv/5RMAA4qbr9GeP9IxtixY0ccd9xxu99/syx7IMuyQluPiam2tjbp946ulM2dO3f3FyLLssPnzJnzdMxAkiRJkqS+Yd68eYcVCoXdv1z3FF9JkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCWhNj8oFAoT582bFyuLJEmSJKkPKRQKE/Pj2pL77ywUCtVLI0mSJElSkaf4SpIkSZKSYEGVJEmSJCXhfwG8dZWmdb83nAAAAABJRU5ErkJggg=="
    }
   },
   "cell_type": "markdown",
   "id": "5ae8701f",
   "metadata": {},
   "source": [
    "<div>\n",
    "<img src=\"attachment:fig.png\" align=\"left\" width=\"450\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d4de5a9",
   "metadata": {},
   "source": [
    "## Extension to 2D"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f5d2895",
   "metadata": {},
   "source": [
    "### Serial implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4ab59b2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "function jacobi_2d(n,niters)\n",
    "    u = zeros(n+2,n+2)\n",
    "    u[1,:] = u[end,:] = u[:,1] = u[:,end] .= 1\n",
    "    heater = 1/n^2\n",
    "    u_new = copy(u)\n",
    "    for t in 1:niters\n",
    "        for j in 2:(n+1)\n",
    "            for i in 2:(n+1)\n",
    "                north = u[i,j+1]\n",
    "                south = u[i,j-1]\n",
    "                east = u[i+1,j]\n",
    "                west = u[i-1,j]\n",
    "                u_new[i,j] = 0.25*(north+south+east+west) + heater\n",
    "            end\n",
    "        end\n",
    "        u, u_new = u_new, u\n",
    "    end\n",
    "    u\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6da0aa54",
   "metadata": {},
   "outputs": [],
   "source": [
    "u = jacobi_2d(100,1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47643bf6",
   "metadata": {},
   "source": [
    "## Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0edeee84",
   "metadata": {},
   "source": [
    "### Exercise 1\n",
    "\n",
    "Transform the following parallel implementation of the 1d Jacobi method (it is copied from above) to use latency hiding (overlap between computation of interior values and communication)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66db180d",
   "metadata": {},
   "outputs": [],
   "source": [
    "@mpi_do manager begin\n",
    "    using MPI\n",
    "    comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
    "    nw = MPI.Comm_size(comm)\n",
    "    iw = MPI.Comm_rank(comm)+1\n",
    "    function jacobi_mpi(n,niters)\n",
    "        if mod(n,nw) != 0\n",
    "            println(\"n must be a multiple of nw\")\n",
    "            MPI.Abort(comm,1)\n",
    "        end\n",
    "        n_own = div(n,nw)\n",
    "        u = zeros(n_own+2)\n",
    "        u[1] = -1\n",
    "        u[end] = 1\n",
    "        u_new = copy(u)\n",
    "        for t in 1:niters\n",
    "            reqs = MPI.Request[]\n",
    "            # Exchange cell values with neighbors\n",
    "            if iw != 1\n",
    "                neig_rank = (iw-1)-1\n",
    "                req = MPI.Isend(view(u,2:2),comm,dest=neig_rank,tag=0)\n",
    "                push!(reqs,req)\n",
    "                req = MPI.Irecv!(view(u,1:1),comm,source=neig_rank,tag=0)\n",
    "                push!(reqs,req)\n",
    "            end\n",
    "            if iw != nw\n",
    "                neig_rank = (iw+1)-1\n",
    "                s = n_own+1\n",
    "                r = n_own+2\n",
    "                req = MPI.Isend(view(u,s:s),comm,dest=neig_rank,tag=0)\n",
    "                push!(reqs,req)\n",
    "                req = MPI.Irecv!(view(u,r:r),comm,source=neig_rank,tag=0)\n",
    "                push!(reqs,req)\n",
    "            end\n",
    "            MPI.Waitall(reqs)\n",
    "            for i in 2:(n_own+1)\n",
    "                u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
    "            end\n",
    "            u, u_new = u_new, u\n",
    "        end\n",
    "        u\n",
    "        @show u\n",
    "        # Gather results in root process\n",
    "        results = zeros(n+2)\n",
    "        results[1] = -1\n",
    "        results[n+2] = 1\n",
    "        MPI.Gather!(view(u,2:n_own+1), view(results, 2:n+1), root=0, comm)\n",
    "        if iw == 1\n",
    "            @show results\n",
    "        end            \n",
    "    end\n",
    "    niters = 100\n",
    "    load = 4\n",
    "    n = load*nw\n",
    "    jacobi_mpi(n,niters)\n",
    "end"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.9.1",
   "language": "julia",
   "name": "julia-1.9"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}