diff --git a/notebooks/asp.ipynb b/notebooks/asp.ipynb index 9cf1ee2..a6ddd7d 100644 --- a/notebooks/asp.ipynb +++ b/notebooks/asp.ipynb @@ -38,10 +38,21 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "id": "1dc78750", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "floyd_impl_check (generic function with 1 method)" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "using Printf\n", "\n", @@ -58,19 +69,27 @@ }, { "cell_type": "markdown", - "id": "ade31d26", + "id": "9e0f9545", "metadata": {}, "source": [ "## The All Pairs of Shortest Paths (ASP) problem\n", "\n", - "Let us start by presenting the all pairs of shortest paths (ASP) problem and its solution with the [Floyd–Warshall algorithm](https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm).\n", + "Let us start by presenting the all pairs of shortest paths (ASP) problem and its solution, the [Floyd–Warshall algorithm](https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm).\n", "\n", "### Problem statement\n", "\n", "- Given a graph $G$ with a distance table $C$\n", - "- Compute the length of the shortest path between any two nodes in $G$\n", - "\n", - "We represent the distance table as a matrix, where $C_{ij}$ is the distance from node $i$ to node $j$. Next figure shows the input and solution (output) of the ASP problem for a simple 4-node directed graph. Note that the minimum distance from node 2 to node 3, which is $C_{23}=8$ as highlighted in the figure.\n" + "- Compute the length of the shortest path between any two nodes in \n", + "$G$" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "ade31d26", + "metadata": {}, + "source": [ + "We represent the distance table as a matrix, where $C_{ij}$ is the distance from node $i$ to node $j$. The next figure shows the input and solution (output) of the ASP problem for a simple 4-node directed graph. Note that the minimum distance from node 2 to node 3, $C_{23}=8$, is highlighted in the figure." ] }, { @@ -100,10 +119,21 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "id": "4fe447c5", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "floyd! (generic function with 1 method)" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "function floyd!(C)\n", " n = size(C,1)\n", @@ -129,10 +159,25 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "id": "860e537c", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "4×4 Matrix{Int64}:\n", + " 0 9 6 1\n", + " 2 0 8 3\n", + " 5 3 0 6\n", + " 10 8 5 0" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "inf = 1000\n", "C = [\n", @@ -266,10 +311,21 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "id": "75cac17e", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "floyd2! (generic function with 1 method)" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "function floyd2!(C)\n", " n = size(C,1)\n", @@ -295,10 +351,19 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "id": "907bc8c9", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " 1.544626 seconds (10.53 k allocations: 732.570 KiB, 1.37% compilation time)\n", + " 2.646978 seconds (8.49 k allocations: 592.948 KiB, 0.47% compilation time)\n" + ] + } + ], "source": [ "n = 1000\n", "C = rand(n,n)\n", @@ -312,7 +377,7 @@ "id": "ad811b10", "metadata": {}, "source": [ - "The performance difference is significant. Matrices in Julia are stored in memory in column-major order (like in Fortran, unlike in C). It means that it is more efficient to access the data also in column-major order (like in function `floyd!`). See this section of [Julia's performance tips](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-column-major) if you are interested in further details." + "The performance difference is significant. Matrices in Julia are stored in memory in column-major order (like in Fortran, unlike in C and Python). It means that it is more efficient to access the data also in column-major order (like in function `floyd!`). See this section of [Julia's performance tips](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-column-major) if you are interested in further details." ] }, { @@ -345,14 +410,29 @@ }, { "cell_type": "markdown", - "id": "9a9e8c44", + "id": "ebb1f4d7", "metadata": {}, "source": [ "### Parallelization strategy\n", "\n", "As for the matrix-matrix product and Jacobi, any of the iterations over $i$ and $j$ are independent and could be computed on a different processor. However, we need a larger grain size for performance reason. Here, we adopt the same strategy as for algorithm 3 in the matrix-matrix product:\n", "\n", - "- Each process will update a subset of consecutive rows of the distance table $C$ at each iteration $k$.\n" + "- Each process will update a subset of consecutive rows of the distance table $C$ at each iteration $k$." + ] + }, + { + "attachments": { + "fig-asp-partition.png": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhAAAAJHCAYAAADBi2/GAAAACXBIWXMAAB7CAAAewgFu0HU+AAAAGXRFWHRTb2Z0d2FyZQB3d3cuaW5rc2NhcGUub3Jnm+48GgAAIABJREFUeJzs3XmYJWV59/HvPTsw7MMiy4AobgjiggiKu7iiqMGYxKBoYtREoxGiRhER9VUganzj8mrcADUxrogbiAugKEbAEPcFRARk3xmY5X7/eM6E09V1uvvpPt11eub7ua66oJ9Tyz1nq9+peuqpyEw0NyJiF+ApFYvcDlwP3NCbrsrMq4ZYz9OBHfuabsjMTw9r/ZI2XBFxMLB7X9OqzDypo3JmVUQsp3x3PwJ4ALAHsCWwOXAjcB1wOfBD4HvA6Zl5YzfVzp0wQMydiHgi8LUZruZqypv0G8DnM/OSGdRzNuUDsd4vMvM+MytP0sYgIj4PHNrXdG1mruiqntkQEfcG/hF4LrBpxaK3Ap8A/iUzfzobtY2CBV0XoGrbUZLwO4HfRMQ3IuKxHdckSRuMiNgkIk4E/gd4IXXhAWAz4MXAhRHxtohYNuwaR4EBYn5bADwOODMivhQRO3VdkCTNZxGxG/Bd4NXAohmubjHwOuC8DfH7eaZPjmbuOkr/hjZLuOs822SeBlwQEYdm5rnDKm6uRcQngCf3NV2RmXt1VY806iLiBOBFjebtMnNtF/XMZxGxB/BtYNeWh9cC3wdOBX4AXNWblgLbAg8EHg0cBixvLLs3cE5EPD4zfzsbtXfBANG9d2Tm8RPNEBELgBXAg4ADgIOARzH+CNL2wDd6b9KphIhTgf7zc3+cctWzZzmwdd/ft3VViDRPbMLYz8xcOZ2yA13v1g5qGJqI2Ab4Ou3h4RvAKzPzJwMWvxy4CDgpIv4eeAPwKsoRiPXuDnw2Ih6WmXcMr/LuGCDmgcxcR/mgfq03ERG7A28EXgBE3+ybAqdGxL6Z+YdJ1nvCLJQraSOQme/vuoYhOwW4Z6MtgVdk5r9OdSWZeTPwmoj4CvBFylHk9fYFjqN0zJz37AMxT2XmJZn5QuCZwKrGwyuAf5v7qiRp/omI5zH21CmU8PCimvAwZuHM71CuUlnXeOgVEbHddNY5agwQ81xmfhF4fstDT4qIJ811PZI0n0TEUuCtLQ+9NzM/OpN1Z+a3gQ80mpcCfzWT9Y4KA8QGoDf40+daHnrNXNUQEVtGxNYRsXjyuedeRCzs1bd1r0/JXG5784jYZA62s/Vsb6e3jSWzuY1hiIjFvVqrT9P2vVdi8rmnp/ee2Ho+XN4XEUsjYrOu65hFfw6sbLT9BnjtkNb/npa2Zwxp3Z2yD8SG45XA0xn7mj4qIvbMzF+1LRARL2HsSHJXZ+Y/T7ahXkh4OvAsYH/KqGzR9/htwM+B/wZOA76SmbdPsL5/Arbo/Xm/xsNbRMTbG21v7Z1nHLS+TYGnUsbL2Lu3zk0a89xA6fT0X8BngHNziqOqRcRzKecy11uTmW/oe3w74C+APwP2olwTvv55OZ/SUesDmXnNVLY3oIZdKc//k3u1rAAW9h5bBfyYcinaxzPzv6e5je0op8ieCuxHGYNkUe+xm4BfUzqXnZaZZ0/331JRzwuBe/U13ZSZb+t7fG/KtfdPB3ah/EBaExG/pLwP/29mXjZg3SspVzIcRjkPvhjIiPg1cAZlQKBfTrPuB1Neq4cC92fs6K9ExBrgl5TX7GvAFzLzpknW+TLu2ukd0DLL2yKi//383sz8fd/yD6C8P/t9eP13RUTsDLwC+FNKp8IFEXErcK/MvLw3z+GM/bzelplvbtT5MMYONrXeR6b6fEbEcygdyPutBd6WmcPouNl2NOAdQ1o3mfmLiLiM8p5cb5+IWJSZa4axjc5kptMcTcATKefV+qd/HOL6v9Sy/n+YYP6zG/P+fArbeArwu5btTDRdAfz1BOu8rHJ9dxuwniXAUZTLYmvWl8AFwKOn+Dyf0lh2Va89gL+mDD8+2fZuBo6Yxmu8HfAvwB0V/7avAveo2MZmwDG9Gqe6je8AB8zy5+erjW3+ode+DDiBslOZqMZbgb9orDOAf6AMGz/RsncCR1bWeyBw3jTeizdTevEvm2Dd51au86GN5Z/bMs8Teo89E7hpwHp27VvH5xuPXdNS5ybATwa8X2IKz+GeA16btwzpPXU3Sh+F/nXfBCwf8nv3zylHhPunLWfz8zIXU+cFbEwTsx8g/qRl/V+eYP6qAEEZEKX2y7B/et+A9c44QFB6On9zhvWtBZ4/hed5XICg/GL992ls828rXt99gN9P8992C3DwFLaxK+XIzHS2sQb4+1n8/IwLEJQxUr5X+Ro/rbe+AP618t/4d1Os9SW952Mm78czgE0GrH9WAgTlCE5zh9o/VQWI3nz7Uj4jzXX91STPYQBntiz3A2DxkN5Tz29Z/0mz9R7e0CZPYWxYvtvS1jz0Ny29w5Vva3noZsrAK5dSdlLbATsDD2f8YCovjYgf5gw7Jg1wCvCYlvZrKc/Lr4HVlA5MKyinbh7G2FM+C4CPRMT5mXlRxbaDchrk6Y32a4BfUMayuDvjLxEDODEivpmZP5twAxH7U66736Ll4V9S/o1XUY7C7AM8krHXoG8GfCkiHpWZ3x+wjXtSfhm2jZj3S0rgvIbyi//+lPFI+vtDLATeHRHbZd8pnVm0FPgKYw/hr6G81pdSxkXZh7F9vRYA74+IM4C3AH/b91hSjq79lvLL+cGM/fcBvD0ivph9pwOaIuJpQNsljmspO7+LuGvwuBWUz8wBvf/2ezzwDsqphLnwKMrlhUPt+5GZF0bEGyhHifq9IyK+lJmDxp85HGgO038L5SjS6iGVt19LW+vnQy26TjAb08QsH4HobePylm1sM2DeKR2BoIwtcU1j3lspX76bDVhmOeV+Hc1aLqFx6JJyLneP3vSNxvxX9j22flrYWP6ZLdu5HvjL5ryN5Xah7JSby352kue4eQSiOX0HeFLLv3Nfynnu5vwfmmR7W/eet+ZyPwUeM2CZnWg/IvJLYGnL/Msop3Ha5m89ctHbxkdallkHPH0WPj/NIxD902rgXcAujWX2An7VMv+3G3//J/CAxrI70P4L+NgJalxM+xG1jwE7TrLcK3r/jv7lVjX/Tb35d+auz8PJLdvbk7GfmaWN5duOQPQfeVhDGWjuzcDfUA65/yuwbd86pnQEojfvggHP5ScGzL+CcuPA5vwvHPJ7qvk+SGC/Yb93N9Sp8wI2pom5CRDNHXACew+Yd6oB4k+n+0GmfSf2kAnm/2Jj3sumsI3mjuXOqX4J9L64m+epVzHg0HFvmUEB4k5KH4yB53YpHeiafSRuBhZNsMzHW7Z1LrDFJP+2AD7Ysuxftsx74oBtbDWF5/DvW5a9dirLVr63BwWIq5ig/wqlo2/bcutf64GH0ilDFDf7A/xkgvmf0bKND1T8G1/SsvxLJ1mm7TTMwODcW6YtQKyfzgDuPoVapxwgevPv0ntfNLf3xJZ5P9Yy32eG+X7qbaetf8ZQ+z9syJOXcW54rm9pm+mgJfu0tP37FJf9cEvbvWdQyxgRsZDxpy6+mJk/nMryWQ6F/t9G81JKj/kadwLPyswTsvfNNGB7V1J+MfZbTjklME7v6oA/bzRfCzwjJ+mp36vj1YwfYvj5jW1sC7y0Mc/VlKMIg+7T0r+df6Eciei3DeV8+my7DnhkluvtW2XmDyhHUprWAM/OzIGDrmXmtZTTJP3uM8FljY9v2cYxg9bf4sOMvzfOQRXLz9SHgCdl5sXDXnGWK2D+puWh9/WunAKgd3fhwxvzXMbsvJ+aQ4CvycxbZmE7GyQDxIbnxpa2mY4N0DbO/lTHe/gOcI/GdNoM6+m3I6Uz3W/7pjMq1/HTlraakJPAn2fmVP9d32lpa16+ut7LGH+59dGZeVXbzOMKK5e7Ni+x3L8XvNZ7CeNvV/zmzLx6KtvoeR3l/HS/lze2M2x3UoLUz6cwb9tr/HeZ+eVpLLuA9v4sUMJg/3vxnBx8jn+cXqD9daN5aIF7EmcCL8tZvAlXZn6GcnSh3x6UYfnpjYvxAcb2w1hH6dx83SyUtE3j77bvTw1gJ8oNT3NYa5j6zn6Qtg/u31POj04oM++kfJHOiiz3+7jHDFfT9gVfc3OiOzPzsxXzt/26G7S95vC6twGfqNgWwJFAc3yPRZROfTC+8+cq4KSaDWTmVRHxeUq/k/V2oRy9uqBmXRWuycxzpjhvW+D6/BSXbQtSW7XNmJlHTHGdE2nWOlc3yjo+52ZcgldQjqr0f27/ISI+SRmHY8/G/P+cmd8cdhG9ESiXNprn9Q3B5poBYsPTduvvtlBRo20gomMj4iDgo8Dnc4KBouaBtuendQcxJG2DYI27uiIitmT8qY1vT3bqoinLHQRb7yLYOxT/wEbzd2q30fMFxgYIKFfjzFaAqDHtQbtoDxDNK4yGqflZms33Yr+Bp96GupHMmyPiL4GzuGsftBj4JOPDwwWUMTFmo447IuIOxoaIDXnEzaEzQGx4tmxpm2mq/jqlb0Xzl9Dje9OqiDiHcurgDODCifoBzLWI2Jq7eqOvoOysl3LXYfu2oZln89B72yHittOJD2ppH/bO+EGMP0L1095zVqttxNOHTGM9s2Eml/21/Sqf1unf3jDgu1Hei7tQwsEixn62mqezZvO92InMPDci3gK8qa95r8Zst1FODd45i6VcRxlMar22708NYIDY8LR1mLxiJivMzBsj4pWUow1tX5zLuCtMvAO4MiK+DHwZOGOuOyX1flU/s1fPYymDI81HK1ramufHZ2r7lrZX9aZh2CDuOjgTEXEgZQTXx1ECld+7xVspV6a1DcUN8Oop9m+ZiesZGyAWRcRyO1JOjZ0oNyC9m0Tt3WheRxkbYkYy8yTKTvl3U5h9R8p9BT4HXBMRn+md7phVEbFVRLyVMojQyZSrDeZreID2c9/TObVQu435tP6RFRHPjIjzKIN8vZ7xA5dt1Hr9LZ5H+ym90zKzeRfL2dB2euq+c7DdDYIBYsNyT8b3gfhNZt42jJVn5qmUHuHPB85h/H3u2ywFng2cFRGnRcTdJltgOiLioZQbVf0T43tW97uJ8VdtXDIbNQ1B23n2Yfc1mc1z+bARHhKOiOURcRIlQLeNdLjeHZS+Gb9tTBvTr9/NaT+FuEuvk+Ns+5+WtoleM/UxDW9Y2oZy/q9hbiAz76D00D8pIrYHnkbpxf8Exl8K2PRU4HsRsf9UL0Ocit6dBb/Vsv11lGv4v0q5X8Jv2zoHRsRy2n8Fda2tpmF38mo7onEaAzpdTkPNpaDzXu+y1c9RPg9Nv6aMn3I2Zcd1RVtfoYg4hXI31w1a79bzn2T8lRBQRm19K+UKotnUNl7Mw4D3zfJ2NwgGiA3L81vaasdEmLJeCPgI5f4RS4FHAAf3pgfQPqb+7pR7arTdQrdab7ufYXx4+DblDqDD7jMwl9oGBWu7F8ZMtF2i+63MfOeQt7OxeB3jw8P1wMuBT2XmVI7abSxOYPD4JwCvioivZeY3ZrGGMyhXn/R/Vz1j2P0gIuJblO/Efnv2BiqbtzyFsYGIiL0Z3xlpNaUj46zLzDsy88zMfE1mPpByOdabae9/8ZwhHp58FuMH9TmTcmvi+RweoH18insNeRtXtrTtMeRtbBR67+nmja9uAx6fmZ8wPNwlIp5KGSSt3zcZO5DTAuBjvZFSZ0VmXs74m2dtQRm+fyh6R6UeSukPtH66c76HBzBAbBAiIhg/HDOUIZ2HdqqgRmb+JjOPoVya1RzJb3PgPkPa1J81Nw28Yo4GxJltFzD+EsKh3F21sY3mOBiPHPI2NhaPZ/xVJx/MzPO7KGZURcQOlCOX/b/6b6IcQX1NY/adgf83yyW1Dbf/j71TLMPweMYfIf3xkNbdKQPEhuHFlFvx9kvab79dJSK2jojDGtOUR37s3UvhPS0PDevyvmYtv8/MtmGL553eIdTmF82jan+RRcRDW17DZb1t3MH4X2D3j4jqoxARsWfLdnauXc881va5OH3OqxhhvR87H2H85cOv690r40OMH3r92RHxwlks6xSgeXv2ewHHDWn9f9vSVjNy7cgyQMxzEfECyt34mj6emcMYdGgT4NONqfmrfzJth+oGDW7VHPBnsl8BzS+itn4Dk2kbb2FUfKnx9xKgdrjktzH29fsQ5T4S632uMX8wvc5rf8P498qw+2yMsh1a2ubi/dg2SNawfj0P299RxsTo9z3K/S/oneZ5MeUKlX7/EhGD7j8yI70QfXTLQ6+MiOfOZN0RcRhwSKP5NuA/ZrLeUWGAmKci4r4R8RnK4E7NzrAXA68c0qauYPwNZh5XuY7mnS2TwffHaB5O32qSGzI1v6DvXtO/ojfi4qlTnb8D72f8c/LG3l06JxURKyjDSff7XuN8/IcZP9TzX0XElE9l9I5oPKfRfCUw2wMBjZK2sFB1qi4i3kAZXKlG21Dss9ZvYLoiYi/g+EbzncCL+9+PvcGj3tqYbzlwSkTMVsf/kxh/tGhhb5vT6vAdEQcDH2956D2ZuUHctMsAMQ9ExIKI2DEinhoRb+kNG/0TyvgKTdcBTx3WG7R3mdkXG82PjogpdTLqfWk0O0udO8EdCpt1LwDuPsEmLmz8vQXlRl9Tqe2hlPH4m4NvjYxeH5bmOdrNga9GxI4TLdv7sv1/lJFC+4259XZvnJB3NOZZDHymN4rihHrDM3+E8YN2vX+UhjSfA833IsCrp3IuPSK2iIj3Mr3D5m2f9ZneYG6oeqH+k4x/L769d6+Wpncw/lLi/am7NfqU9d6nz2P8mDALgQ/1xrBp3qejVURsGREnUjqwN1/7XwDHzrDc0ZGZTnM0UX5ZZGO6FvjNgOkSypdDc5lB08XA/SrqObux/M8HzLc/5f4N/fOupvxK2HHAMkuAwyl3FmzW+bQJajqqZf7PAlv0zbOg7/+f0TL/2l5tW7WsPyiXm/4bpYPioOfy+AlqPKUx76rK98EeLdt7/QTzb0bpiNpc5nLKeAFLWpZ5IOVS1uYyFwKLWuZfQPnCa86/ivKrcYeWZRZSxgE5v2W53wFbD/nz89XGNv5QseybWmrcforLHtqy7CEt8y2inEtvznsu8JAB694J+IcBy62fbpykvue2LPOd/n9f/2dmgmWeUPl6fL6x/DUTzPuulu39FFg6wTIHMP57Zw3wiGG+rxrbvDdloLm212E15UqRv6d8h+xJuaJiJ8qPkOdTblV+64Dl/wjsPVu1dzF1XsDGNNEeIIYxraMcgqv6wmaKAaI373sGbHsN5W6dn6P8Uj4Z+AblcG7b/B+cpKaHD1juTkpAupEyzG3/MqcPWGYV8CPKXSJPouxQ274cbm5pG5kA0Vvm/r0voLZ/5/WUnetHKf0O2sLG+n/nwC8wygiebWEgKV/k/0U5d/vR3vauHjDvLcABs/D5GekA0Zv3TwY8J0k5bXcG5bD2l4CLGL+DbHs/ThYgdhuwnjWUIHct8OPGMnMWICjjwqxreT9NGgQoV5c167wY2HLY76/G5/PHE7yO05kup+LH3XyZOi9gY5oYfoBYQzm9sN8066kJEAspO46Z1PthYOEU6jpvkvU0A8TWlMsRp1PT+ZQBXprtIxUg+pb72TT/ndcA+09hG5tRjvhM9zX+AwN+bQ/h8zPyAaI3/+um+dzdBryw5f01YYDobXOy16yTAEG52urylm29f4rb2Jxyb5vm8ifPxnus8Tl4F+Wow3Q/C/9bK7BiNuvtarIPxPxzBeXXy8uB3TLzGZnZNhzrUGXm2sw8AjiMwR0gB7kYOCwzX5SZbbeybnoGZUc51dquBw6iDD87lfUDXEbpaHog5XTRyMvM31Lu5ng07ee926ylfIHtlZk/mMI2bqX8in42dcNZ3wqcCNw3M4c6fPp8k5n/h3LjuaneBXc15ajEvpn5kclmHuAFlKsZRs2/MfZul1ACxWunsnBm3kz7ZZDPi4jaq8GmLDNvzcxXUU5NfIz6e9CsoRyVfXRm/mVmNjspbxCil7Y0ByJiF8ZfwjSZGym/TC4HLs3Mod1bICKeTrlz5no3ZOanp7BcUE41PJUyZv29KEcBtqKcariecuj0POBrwNeychS+Xse8QylXcOxIuUTuVsov6W9m5icHLLc75cv0UZQe8NtSPszXUJ7D71HOY56emXf2llnM+EsjfzxohxsRj2Xs6JdrM7NtMJpB/7YtKL8A+/0wKy677V098iTgyZT+Dtv1ptsph6z/m3IXyFOyXF9frXd31/XDkz+S0klyO0rHsOspz+f5lFNDn+192c+aiHgyYztq3paZp0xx2QcDD240n5yZk+4Yeu+pgxvNX8/MCe9M2+s4+BzKZ34/ynt4U8p7cf1r9C3gK5n5+77lmu+vOzPzY1OocwHlM/lwynn5HSiXQ14H/CAz39837z0YfzXVlzPzD5Ntp28dB1OGpl9vVZa79q5/fAVlpNim82tDZq/TdvPGbFP6vhqGiNic8lk7iHLEcg/Kd96mlFN211H6e/2I0uflGzXP5XxlgJAkSdU8hSFJkqoZICRJUjUDhCRJqmaAkCRJ1QwQkiSpmgFCkiRVM0BIkqRqBghJklTNACFJkqoZICRJUjUDhCRJqmaAkCRJ1RZ1XcCGLCK2Ap7QdR2SpJH3x8w8q+siang3zlkUEQ+k3O5YkqSJnJGZzdvGjzRPYUiSpGoGCEmSVM1TGLMoIhYCW3RdhyRp5K3OzFu6LqKGAUKSJFXzFIYkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqLVr/PxH8A7BHh7VIkqTR9ttM3gkQmQlABN8DDuiyKkmSNNLOzeRAgEUR/AWcswusfCCs7LowSZI0si59YMQlr4VH/j6gdwhCkiRpiha1tF0H3DjXhUzukt1h90u6rmKw3+8KW94AW9zcdSXtrtoO1i6Eu13ZdSXtbl8GV+0Au/2u60oGu2R32PX3sHBt15W0u2xn2PQ22Ob6ritpd93WcNumsMsfuq6k3dqF5XM8yt8zv9sNtv8jbLKq60raXbFj+Xxsf3XXlbS7aXO4cavyOR5VI7uv2xLYpr+h7QjEUZmcOHc1TU1ErM7MxV3XMUhEXAock5kf7bqWNhFxErBjZh7cdS1tIuIg4NTM3LrrWgaJiFXArpk5kl+OEXEBcFpmHt11LW0i4jjgkMzct+ta2kTECuCyzFzWdS2DRMR1wKGZeVbXtbSJiNOBKzPz8K5raRMRRwDHZubInq8f1X1dBEcCJ/S3eRmnJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFULyBzb9NbfwRuu6KacCe0P/KDrIibwEOAPwCg+dwD3BhYD/9N1IQNsSanxvK4LmcD+wPnA6q4LGWBf4Ebg4q4LGeDulNf5wq4LGWAR8GBG+3tmP+BXwA1dFzLA/Smfj190XcgAOwK7AP/VdSETGNF93VvuBq/frb9l0fiZtr2M8iU5avYHfth1ERPYh/LFfVHXhQywHbApo/sc7grck9GtD+ChwAXA7V0XMsC9gSsZ3edwGbCE0a1vE+BBjG59UALOz4Hfd13IALsBtzG6z+E+wPaMbn0wsvu6bR9EeX3/V8sRCI7K5MS5K2pqImJ1Zi7uuo5BIuJS4JjM/GjXtbSJiJOAHTPz4K5raRMRBwGnZubWXdcySESsAnbNzKu7rqVNRFwAnJaZR3ddS5uIOA44JDP37bqWNhGxArgsM5d1XcsgEXEdcGhmntV1LW0i4nTgysw8vOta2kTEEcCxmbmy61oGGdV9XQRHAif0t9kHQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUbVHXBUgbgwi2BVYC2/aa7gRuBP6YyZWdFSZJ02SAkGZBBCuAQ4GnAg8Ddpxg3muAHwPfBb6YyflT3MZOwLvvavny7rD9syO4d8vs1wO3AX8EfgpcmMmlU9lOY5sfAZb3NZ2XyYm162lZ78eATfuavp/JO2e63mGIIICzgE16Td/N5O87LEkaCQYIaYgiuDtwNPDnwNIpLrYCeFxvemMEvwNOAt43ydGJzYHD7vrzKQBbAfedYq3/DfwH8K+Z3DTFWp8BbNP398IpLjeZQ4Eth7SuYbsH8Ii+v//YVSHSKLEPhDQEESyM4A2UX/dHMPXw0GY3Sgi5JIKPRLDzMGpssQ/wVuDXERw+S9vYELyq6wKkUeQRCGmGItgC+BzlCMIg1wK/Aq4Bbge26007U44atFlKCSNfBf5zWPW22A74eAT3B16bybpZ3Na8EcEC4DXAy7quRRpFBghpBiLYHDgTeEjLw7cC7wM+C/ywbcccwUJgP+BJwJ8Ae82som+dD495TaNxM2AJsCtwH+AJwO4tCx8F3AS8ZWY1zFcvWQ5fjwgeA+xPOQ21d8dFSSPLACFNU+8X6r/THh4+ARyVyRUTrSOTtcD3ge9H8GbgWcAbmfaO65LrMvnGZHNF8GRKB8x7NR46JoIzMvnUt13fAAAgAElEQVTB9LY/P0VwE7x/896f3+y0GGmesA+ENH0vp9dzsU8Cx2TyvMnCQ1Mm6zL5DLAv8LfAquGU2bqtr1KuDjmv8dAiNs4jEIu7LkCabwwQ0jREsD1wXMtDb83kzTNZdy9IvA94KPCTmaxrku1cDzyP8UHlsb1/nyQNZICQpue1lMso+30XOHZYG8jkIkr/iElPScxgG78CvtZoXkA5OrExORDOfjzsdyfllFT/NKVxOaSNjX0gpEoRbEq5OqJfAi/NZM0wt5XJ7ZSrNmbTNynjMPS7xyxvc6RkckHEI1cAmcmP+h+L4OaOypJGmkcgpHqHMP7SyzN7Rwzmo8tb2ppHVyRpDAOEVO+Qlrb3znkVw9P2PXDLnFchaV4xQEj1HtH4ex3wrS4KGZJdW9q8wZekCRkgpCrHbUMZarrfRZnc2EU1Q/L4lrYL57wKSfOKAUKq8qjdWxr/a66rGJbe8NVPbDT/NpOfdlGPpPnDACFV2aHtttzXzHkZQ9C7muRkxn8P/FsH5UiaZwwQUpWlm7U0Xj/nZcxQBLtRLt/ct/HQlcB75r4iSfNNyzgQpzw24i9HcVjXhRHxuq6LmMBWwDMiou0X6ijYF9h8hJ/DewCbjHB9AIvgc08df3fn9z8s4mUd1P3EFePHgLrinoOfw/22hMftCE+/P+z3AFjU+PyvWwev/gq8+xURg7Z52yawSd/fP753xL5T/bc/Crhbe32rlo69A/oF94l40Fw/p5sBi8bX95uVsEff3z+7R8T9unqfbgocHhEP72j7k9mTga/xSNgf2GqE64OR3dedfFAZuPYuLQHi1w+m/U59XQvgL7suYgKbUkbva96caFSsBBYyus/hcsr9CEa1PoAF8JsHj2++/ADg3nNeDZcvGd/2kt3geX2jYQaw5QLYdAEsGhgLYG3CkX+Edx8AHDB4vjuXjA0Ql69k6q/ZjpTXuWX+O5eMDRCX71ax3mFZRDkq29ju1TuMDRBX7jx+njmzGDiY0b3MdkdgDaP7Od6G8l09qvXByO7rfr1tS2NmYzoyMxm1CVjddQ2T1HcpcETXdUxQ30nA6V3XMUF9BwHXd13HJDWugl+/uOUz85Zu6sl7t9QynelqyMdNcZvXNpb9bMXzdxxw4YD13tBY76c7eH1XAKtaavt2o7Yvd/gevA54ZFfbn0J9pwMndV3HBPUdAVzadR2T1DiS+zrII5vfHfaBkKrc2NZhcps5L2M4bgVOBPbK5Myui5E0v3gvDKnKxX+ABzUbR/W0Vb/bKZ09rwa+B5wDfD2TayvXc2fj74VDqA3Gd+i+Y0jrlTRLDBBSlaMvhWffztiOAAdGsDRzFHZ6n/oO/NkzG423Z467Zfd03Ug5z73esiGtd9OW7UgaYZ7CkKr8bA1wQaNxE0rv7hGwanUm1zemYYUHgJsafzd3/NUi2ITxRzIMENKIM0BI9U5vaXvBXBfRkRsaf+8yhHXu3tJWe2pF0hwzQEj1PtfS9rwIVs55JXPvJ42/V/aOIMzEfVva/nuG65Q0ywwQUqVMLqJ0ROy3mHKZ4oaued+PhcATZrjO5r04EvjRDNcpaZYZIKTpOaGl7fCI4Q0AE8GSCN4VwTOGtc4hOK+l7a+mu7IItgCe3Wj+Zeb8Gx5c2tgYIKRpyOQLwNktD70/Ytwv6mq90yHfAV4JtIw42Y1MfsX4W30fEtF6S/CpeB3QHOHus9Ncl6Q5ZICQpu9FjB9SeDPgtAheHlH/+YpgqwiOBS6iDI0+it7Z0vYfEdyvZiURPA94TaP5TuD/TbcwSXPHACFNU+/X+OHA2sZDiyh3tLwwgmdGjLnJQ6sI9orgLcDFwBuBLYZd7xB9kvGnMrYBzongpZMFpwi2iODdwMco4/73e3cmlw6tUkmzxoGkpBnI5PMRvAD4CKUjZb+9KVds3BrBdyhXMFxJGUJ6e2A7YGfKGBI7z1XNM5XJ2l5fj3MZO4z31sD7gDdF8FVKR8irKGM6rIAPPwxW3h24nHKkpukHwDGzWvwAJbxdvzl8aFEEb288vEfj7/u0zLPe7zN57yyUKI0cA4Q0Q5mcEsFlwMm0j4uwGfCU3lTrf4Cfz6C8WZHJLyN4MvAF4G6Nh7cHnt+b+rxwolWeAzxryINe1Xg1bLUMjoLxp1Wa9phgnh+AAUIbB09hSEOQybeBfSj9A2a6E1wDfAl4ErBP77LRkZPJecBDKJ09p2sd8C/AYzO5eiiFSZoTBghpSHrDRr8aWEn5hfot4LYpLn4z8HXg1cCumTw9k69nkrNT7XBkcnkmjwYeD5xBCT9TcRvwYeC+mbwyk9WzVKKkWeIpDGnIer+kjweOj2Ax5Vf6vSj9BbbtTbdRruD4LeU0xY8zp7zzXb+dX9DXCTEiLgC+D0cM459RpXc78DMjWA4cSPk3b0f5N28C3ARf2gvO2xWOezZw/iiFhkw2iYgVwGWZOawbhEkbNAOENIt6O8lze9MGL5NbKPcKGXe/kIinHwccknncD+a8MElD5ykMSZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSaq2aHzTjUsjtlo+96VMLiJGsq6eADYZ4RoXA4tGuL5lMPKvMYz2a7wQWDrC9S0GFoxwfZvCyL8HA1g2wjUuAhaPcH3LgBjh+oBRfQ/esBS2HNMSkDl2pn9cCyesncOqpmoJcGfXRUxgCbC2N42ixb3/ru60isEWUL58Rv01Xg3kZDN2ZDGltjVdFzLAIsoOcFTfgzA/vmfWAOu6LmSAUf+eWdibRv01HsH6jloIxy/sb2kJEByVyYlzWNWURMTqzFw8+ZzdiIhLgWMy86Nd19ImIk4CdszMg7uupU1EHAScmplbd13LIBGxCtg1M6/uupY2EXEBcFpmHt11LW0i4jjgkMzct+ta2kTECuCyzFzWdS2DRMR1wKGZeVbXtbSJiNOBKzPz8K5raRMRRwDHZubKrmsZZFT3dREcCZzQ32YfCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqtqil7akRbDfnlUzq+AURvKPrKgZ791aw72ER3KfrStp94kGwarPRfQ7/Zxf4+iajWx/ACYvgRW+M4LauK2n3vp1gp8dFsKTrStp94SC4/G6j+xpfvwn826LRrQ/gnzeBJ7wkgqd2XUm7D+8Jy3Ya3efw23vDhVuNbn0wwvu6hzYbAjK7qESSJM1fnsKQJEnVDBCSJKlaWx+IW2AUz/FetT1sf1XXVQx27baw2a2wbFXXlbS7aQtYtwC2uqHrStqtXgw3bAXbXd11JYNdvR1sey0sWNd1Je2u2waW3lHeh6Po1s3gjqWwzXVdV9Ju3YLyOR719+BWN8Di1V1X0u6GrcrnY4ubuq6k3apl5X247bVdVzLYyO7rNgWW9ze09YE4KpMT566mqYmI1Zm5uOs6BomIS4FjMvOjXdfSJiJOAnbMzIO7rqVNRBwEnJqZW3ddyyARsQrYNTNHcgcTERcAp2Xm0V3X0iYijgMOycx9u66lTUSsAC7LzGVd1zJIRFwHHJqZZ3VdS5uIOB24MjMP77qWNhFxBHBsZq7supZBRnVfF8GRwAn9bZ7CkCRJ1QwQkiSpmgFCkiRVM0BIkqRqBghJklTNACFJkqoZICRJUjUDhCRJqmaAkCRJ1QwQkiSpmgFCkiRVM0BIkqRqBghJklTNACFJkqoZICRJUjUDhCRJqmaAkCRJ1QwQkiSpmgFCkiRVM0BIkqRqBghJklTNACFJkqoZICRJUjUDhCRJqmaAkCRJ1QwQkiSpmgFCkiRVM0BIkqRqBghJklTNACFJkqoZICRJUjUDhCRJqmaAkCRJ1QwQkiSpmgFCkiRVM0BIkqRqAZljm97xK3jtpd2UM6HHAt/suogJPBK4GPh914UM8ABgCfDDrgsZYBtgH+DbHdcxkccC5wB3dl3IAAcC1wK/6LqQAe4NbAt8r+tCBlgCPILR/p55NHAR5XUeRftRPh8/7rqQAXYFdgfO7riOiYzovu7tK+E1e/a3LBo/06Y3ApfNUUW1RrUugDXA9YxujXsC6xjd+tYy2vUBJHAFcHvXhQywGriZ0X0OdwK2YHTrW0Z5jUe1PiifkauBy7suZIC9gVWM7nO4HFjJ6Na33gjWt+mWzZaWIxAclcmJc1TRlEXE6sxc3HUdg0TEpcAxmfnRrmtpExEnATtm5sFd19ImIg4CTs3MrbuuZZCIWAXsmplXd11Lm4i4ADgtM4/uupY2EXEccEhm7tt1LW0iYgVwWWYu67qWQSLiOuDQzDyr61raRMTpwJWZeXjXtbSJiCOAYzNzZde1DDKq+7oIjgRO6G+zD4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASLMkgmURbN2b/KyNvKO2hD0igr0j2D2CrbquSBpli7ouQNoQRLA58BTgUcB+wB7ANo15bgZ+C/wYuBD4aiY/r9jGk4F9Bs/x1u1hlwMjeAWwCrgWuBj4WSa31/x7ett7APCkRvPJmVxeu67GevcFntho/ngmV85kvZU1bAM8AXgksC+wDxy/HI4H+O+++a4CfgB8DTglk5vmqkZp1BkgpBmIYC/g1cCfAcsmmX1z4AG96XDgnRH8HPgs8MFMLp1k+T8BXjj44X8C2Al4bOOBtRH8CPgccFImV0yynfUeCry90fZtmFmAAB7Wst4zYXYDRARLgWcBLwYOAhZOYbHtgUN60zsiOAF4WyZrZq1QaZ7wsKo0DREsj+C9lKMJRzB5eBjkPsDrgd9E8KkI9htWjX0WclcY+FUEb+ntTDc2TwA+CTyaqYWHpuXAscD3IlgxxLqkeckAIVWK4L7A+cDLmHhHdAvlFMLlwOpJVrsIeC7w78OocQKbUQLLN9wJjrMG/piU00x/nGC+/YCv9U5bSRstT2FIVb5xf+DT0LrzvR74BPBl4OxMbu1/MIJtgQdR+hU8GbjvDItZR++8RfG2V8Buv4C/OA/YFNizt73tW5Z9BPDtCB6ayW0zrGO+Wg2cBpwOnA2bXwW3/D4z7wH/+3o9CTiKctqp34OBN/YekzZKHoGQpmy3gMd8ivHhYQ1wInDPTF6eydea4QEgk2szOSOTV2dyP+BxwDkzKCgzecf6CV5/FTzvu5m8NpNXZPJk4G6UjoJntiy/F/DOGWx/vrqasuPfJZNnZfKBTH4Ct2T/TL3X6xOUIw4fbFnPKyLYaQ7qlUaSAUKagnIZ5smLYMEOjYduBZ6ZyVGZXFezzky+mclBlF+5vx9WrY1trMvkbMr5/7e1zPLiiImu7Nig3EoJentmcmImV01loUxWAy8Fzmo8tITSuVLaKBkgpKl5IRzU/LysBp6UyWkzWXEmX6dcSvj5maxnkm1kJq+nnF7pF8DzZ2u7oySTb/WC3o3TWHYd8NaWhx4188qk+ckAIU2id8XCm1oeOjpzRqcg/lcm12X+7yWGs+m4lrZHz/I2NxTfAbLR5ikMbbQMENLkDgN2brT9CDhh2BvKbO2rMEznAdc02u41y9vcIGRyB4w7erFlF7VIo8AAIU3uBS1t7+wd1p5XMknG97dYHsHiLuqZTyIIymWw/apPh0gbCgOENIEItqZcxdDvCuA/OyhnWO5o/L0W5l8Y6sBKGBe0/tBFIdIoMEBIE3sE43cap/d65s9Xd2v8fVUmazupZH55dEvbD+e6CGlUGCCkiT2kpe17c17FkESwK7Bbo/kXXdQyDx3e0vbVOa9CGhEGCGlie7a0nTvnVQzPS1va3AlOIoKDGH+TsnMzDV/aeBkgpInt2tI2Z7edHqYIHgS8stG8mvndn2PW9e558YGWh46d61qkUdJyL4zPHBBx2EvmvpRJLYiIUaxrvS2AgyNiVO9yuBew+Qg/h/cGlo1efVffvTFydcJmh0Xc1kGnwx/eZ+wZlYyIBf3P147Aw9qfw5fuAcf/NSzfZGz7D86Fhz0xom17H37E+LuH/82zIj74wOlUD+wPbA8ff+T4sate9OyIj7SdLppLmwMLxz5/CwIu+mu43/3GzvqzC+F+d49grt+vmwDPioj7TTpnN3YHthm9z/H/eiSwxQjXByO7r/vPA+BPxrQEZGNglKOvh7eM4qVJuwOXdFzDRFYCNwA3dV3IADtQ7hx5edeFDLCMUuPvui5krIt3ht37OlHekrB5RzV+YgX8+fK7/l4LLLqkb4ZdgNuBa8ufSwL2XwIv2RyesxksasSEy9bAQ66APw7oQPnyzeE9245te+QVcHbzKo6p2hbYBF55I7yrsd6HXwHfm+56h2Uh5Tnse33ftCUcs/XY2X6/Bh54OVzbxZUruwFXUV7nUbQT5Y050d1Mu7QFsBVwadeFTGB3RnJf94Yt4bgxn4WWAMFRmZw4h1VNSUSszsyRvVY9Ii4FjsnMj3ZdS5uIOAnYMTMP7rqWNhFxEHBqZm496cxzKIJfMGagpVyVGZsMXGB2a/kwYw8JJPCZu/78yhNgl6thnz9QDpvci3K/hjbXA4/J5McTbO+vGX8TqYdl8oP66iEijgMOgfwA8P7Gw/tl8l/TWe+wRMQK4LLMXFb+5rnAKYy9ZfudwCMyu7n6IiKuAw7NzOZ9OUZCRJwOXJmZbR1OOxcRRwDHZubKrmsZZFT3dREcSWPwPG/nLU2s8UsvlkWwSeZI/AIMyiiZPU+B8uuqreNnv/+h3ADs17NV2HwXwbOAkxkbHtYBL+oqPEijxk6U0sSub2nbtqVtPrgJeAtwgOFhsAieBnyKsT+wEnhpJqd0U5U0ejwCIU2s7VzpSuCyuS5kmq4GzgG+DZyc2RqIBlnT0tba3XIIRmRgricuoFyV0jz18+rMcadzpI2aAUKa2K9a2h7FaAwmtQ544l1/HvoB2Pn78N53UTr0Xp/JdTNYf1uH4GUzWN96bX1IRqDz8S+eCisXM3bk0aT0C3tXR0VJI8sAIU2srWPfY4D/M9eFtMhMvrH+j4gv3gxcnPneHw1p/W079WF0IF3e0tbplV+lw+i9mh071wAvzmQkO0ZLXbMPhDSx7zH+5lMHRbBDF8XMsbad+jD+3Ts1/l43YFtzIoI3UK426e8wuQo4zPAgDWaAkCaQyU3AmY3mZcCrOihnrv2Kcgi/32RXeEzFvZvb6eJmXhEsiOD/Asc1HroJeEomX5jrmqT5xAAhTe4jLW0vjWDHOa9kDvU6XP620fzoma11+QLKiJT95vyyyAg2BT4N/N3YR65J4LGZfGuua5LmGwOENLkvMn5HugVwSsTwPkMRbBMxEn0r+jV37gdEzOQoxJ9tAWw6yTZmVe/007eAZ499ZO2v4YDVmQyrD4m0QTNASJPIZA3w+paHHgecGDHzSxsj2B+4AHjOTNc1ZF9q/B3A26a3qk0XwD81j9okcNr01lcvgr2A7wMPbTz0bXjTU+DXzVM2kgYwQEhT8x/wlbZ7H7wK+FTE9K5OiGBFBMcDZ1HGlxg1n2f8/VP+JGLcXT0nVELWGYfA7s2bzZ2ROe7ozqyI4AnAdyn3Guj3MeCJ8JaaMTKkjZ4BQpqCTBKOWANrL2l5+E+Bn0XwwoipXRodwW4RvA24GDiKwfes6FRvyO5jWh765wjeEcFmk62j11fkP+DAvRsPrQXeMIQyJxXBSuArwJZ9zQm8PpMjMrlzLuqQNiSOAyFN2VUJn/tTOOzTlLsi9tsN+DDw9gjOBL5DGa3yasrARDtQbre9J3AQ8EBmb1THYfsw8AzgaX1tC4B/BJ4XweeB0ylh6BrK5ZArgPtQBrp6FqXPSNP/mcP7SmzK+O+7W4Fn9e57Ady5CC5aEjGtm3q9KpOzZ1aiNL8YIKQqz7kY8kDKcMcHtsywHfDc3jQdF0+3stmSSUbwZ8CXgUc2Ht4J+NveVONDwBuHUN5MLAcefNefi4EHMbZtyracfBZpw+IpDKlSJpdTdqRHUYaMnql1wFcpv9afMIT1DV0mtwCPB/51ZmtancDLMnlxOS0kab4yQEjTkMnaTE4E9qAEiR9TgsBUrQMupPQB2C2Tp2Ry+ijvVDNZncnLKeM4fJHxg0xN5A745vmw788zaQ4ZLWke8hSGNAO9wZZOpFzOuTXwcMoO9m6U235vQ7l/xI2U/hC/pIx7cE5m9fDNbwbe17/5mVU/PZmcBxwawU6UIzEHUYLUNr1pLeU26NcBP6HcDfQceNyrgEO6qBm4BHjIxLOcvRX8w1fgh22npibj7dG10TFASEPSCxOnMUvjGmTyO+B3s7Hu6eidyvn33jSp6LDLaCarYOIBoiIeuYJygzIHkpKmwFMYkiSpmgFCkiRVM0BIkqRqBghJklTNACFJkqoZICRJUjUDhCRJqmaAkCRJ1QwQkiSpmgFCkiRVM0BIkqRqBghJklTNACFJkqoZICRJUjUDhCRJqmaAkCRJ1QwQkiSpmgFCkiRVM0BIkqRqBghJklQtIHNs0z+ugxPWdVPOhBYBa7ouYgKLgHW9aRQt7P13badVDBaUGkf9NR71+nwPzsx8eI3XAjnZjB0Z9dd4QW8a9dd4BOs7agEcP+agQ0uA4KhMTpzDqqYkIlZn5uKu6xgkIi4FjsnMj3ZdS5uIOAnYMTMP7rqWNhFxEHBqZm7ddS2DRMQqYNfMvLrrWtpExAXAaZl5dNe1tImI44BDMnPfrmtpExErgMsyc1nXtQwSEdcBh2bmWV3X0iYiTgeuzMzDu66lTUQcARybmSu7rmWQUd3XRXAkcEJ/m6cwJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKnaopa2J0Sw6ZxXMqljFkTwxq6rGOxtW8CDnh7Brl1X0u6D+8Cq5aP7HP5oN/jystGtD+BNC+Fvj4zg1q4raXfCjrDLQaP7HH7qILhsh9Gt75pN4V8Xjm59AMctg6c8P4JHd11Ju/fsAcu2H93n8GsPhPO3GN36YIT3dQ9vNgRkdlGJJEmavzyFIUmSqhkgJElStbY+EHcCq+e6kMnduhlsNqLnngFu2xSW3AmL1nRdSbs7lkIGLFvVdSXt1i4sNW56W9eVDHbrZqW+GNHTfrdvAgvXlvfhKLpzSXmdN7m960raZZTP8ah/zyy9o7zOo2jVsvL5WHpH15W0W7OovA9H/XtmJN+Di4El/Q1tfSCOyuTEuatpaiJidWYu7rqOQSLiUuCYzPxo17W0iYiTgB0z8+Cua2kTEQcBp2bm1l3XMkhErAJ2zcyru66lTURcAJyWmUd3XUubiDgOOCQz9+26ljYRsQK4LDOXdV3LIBFxHXBoZp7VdS1tIuJ04MrMPLzrWtpExBHAsZm5sutaBhnVfV0ERwIn9Ld5CkOSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqhkgJElSNQOEJEmqZoCQJEnVDBCSJKmaAUKSJFUzQEiSpGoGCEmSVM0AIUmSqi0a3/TPL4g48hFzX8qkFkbEF7ouYgI7AK+KiGd0XcgADwWWjfBzuAOwfITrA1gCnBwRq7ouZIA9gb+IiL27LmSAvYEdRvg1XgIsGeH6ADYH3h4RV3VdyAAPAe4Y4edwD2D7Ea4PRnZfd+I94dVjWloCxIK1wOq5KajaqNYFkMAoP3cA6xjd+tb0/juq9UF5jVczujWO+ntwHXc9h6NoAaNd33qj/Bono/0cru39d1TrW28E61uwttkSkNloOyqTE+eooimLiNWZubjrOgaJiEuBYzLzo13X0iYiTgJ2zMyDu66lTUQcBJyamVt3XcsgvSMPu2bm1V3X0iYiLgBOy8yju66lTUQcBxySmft2XUubiFgBXJaZy7quZZCIuA44NDPP6rqWNhFxOnBlZh7edS1tIuII4NjMXNl1LYOM6r4ugiOBE/rb7AMhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqGSAkSVI1A4QkSapmgJAkSdUMEJIkqZoBQpIkVTNASJKkagYISZJUzQAhSZKqLeq6AGlDE8ESYA9gZ2CbXvOtwI3AFcDFmWRH5alFBDvBjw+E4xdE8DLKj6vbgGuBnwG/zWRNp0VKI8YAIQ1BBPcHDgOeCDwQWDLB7DdFcBHwLeALmfxoitt4BHDM4Dm+ew/Y5i8ieBAlsAimfW4AAA40SURBVNwE/AH4CfCjTH4zle30bW8b4D8azZ/I5GM162lZ7wrgU43mkzM5aSbrrazh7sCTKa/X/sAOsA9wCsB7Wxa5JYLTgI9n8rW5qlMaZQYIaQYieCzwRuBRFYttATy8N70hgkuBDwPvz+TqCZbbHnj84IcPBNgcuPuAWi+i7CHfm8mtU6hzacv2zp3CctNZ73eHsN5JRfBU4DXAQZWLLgeeCzw3grOAF2Xy62HXJ80n9oGQpiGCrSP4d+BM6sJDm5XAscClEfz/9u49yM66vuP4+5sNJEhIQi7cIiQFCoWAVFFwcIhmuKipIO0wvUAHyqRKGVE6GPEGbS1FkXDp2EgtghcYKb1AS0AFiuUWKVNa0bGDg1SusYBcAixXQ/LtH88BN88+e3af3c15ns2+XzPPkPM75zznwznJnu/+nt/lkghmjTlgtf2BLwL3R3DMZnqNtjuV+sVD2RLgrk6PkDRp2QMh1RTB3sANwKIuD7sXeAh4EtgAzO8cvw7MHeI504EPAX9NMV5ic9kZuDqCT2WycjO+zkSyAdbfCzcuhg98A+ij6PFZBPwGEKXHzwWuieCATB7raVKpJSwgpFqu3gO4luLLpexhit/wr8tkbdWzI5gCHEhx7f14ii+nsfgYcGfxx0OvhB3vgH++CtiKomdjf+B9wB6l500Bzovg8UyuGGOGieoV4BrgKuB22HorYG1mLh/4oAgWAp8GPsymhcR84FzgxN7EldrFAkIasdnAB69icPGwATgTuCiTV7udIZONwN3A3RF8nuK6+pnAPqMM9b+vD8KMWPMS8FgmNw98QAQBHA2sAt5cev6qCO7I5KFRvv5E1E9R6F2cybrXGzuDOwfJ5GHgTzpjSFaV7j42glMz6d9saaWWcgyENGJfngp9C0uNLwK/ncm5wxUPZZlszORKil6CzwDrxylo+XUyk2uBd8KgmRgzKQYVThZXAntlcs7A4mEkMvkycEep+U0U0zekSccCQhqBYsDccX2l5gSOy+S6sZw7kw2ZfIFicN6DYznXMK/zc2B5xV3Hdi6tbPEyuSKTx8dwin+paCv36kiTwqT4oSGNg7Mr2lZlsnq8XiCTu4D9YPNND8zkNop1IQaaB+y5uV5zC/NURdvGnqeQWsACQhpGBIuB95Sa11EMrBtXmbyUyS/H+7wlt1W0Va4doUF2qmi7v+cppBawgJCGd3xF21dHuBhTG1VNO9yu5ykmpveVbj8M/KiJIFLTLCCk4R1V0fa3PU8xfqpmXz3f8xQTTATvB5aWms9xXxNNVhYQUhcRzAYWl5ofmODTHhdVtI1lYOEWL4JlFPuCDFwH4lrg0mYSSc1zHQipu7cyeBXCnuzbsDlE0AccVmruB+5rIE6rddaFWAqcRLHx1kCrgT+w90GTmQWE1F3V4MIf9DzF+DmewdMOb667hsWW6dkvwM+2juA+ilXDqlYbXUexb8nfdBYFkyYtCwipuwUVbVVT+Vovgj0p9tko+0qvs7TTtF+DtwWwV8WdP6R4767O5IXe5pLayTEQUnczKtpqrWDYBhEsBW4Hti/ddUsmNzUQaaLZH/gIcEoE85sOI7WBBYTU3bSKts25U+a4iGBWBG+P4EMR3Ar8O8UunAM9SXF9X8PrA94BnAc8GMHZEWzVcCapUV7CkLqr2p+iqleiKf8Y8XrG52bCtH0iOHMEz3saOKazUZQAOHU5XHMfPLMzxW6mO1Jsv/5u4Dh4Y7OtbSk2QHtXBMsyeaWRuFLDKgqI7xwQ8Vu/1/sow5oSEW3M9boZwCER8VLTQYawJzCrxe/hfsDW7ct36+7F98dAX1sWsXxW77OsOqjoRd/EgGJmJhS/KQ/jmYfgjy6E63aNoMv7vWT24EUrb1sc8Z7RfkYHAHPhsKPhe6W7bl0csbTpz347oA/iyFL7nbDHPbD6JNh3yYD2pXDvDRGLe7kmyHTgiIgo9ya1xQJgRvv+Hb/hENqdD1r7XfftA2DZJi0BWZqGdObzcE4bBwntAvxf0yG62Al4oXO00VyKS1ZPNh1kCNMprs9XrZLYoNO3hQtKxcKK5+CCBlah/P3p8PdzRv/8RzfAyn645GV4dQTTDxf0wdodN207rx8+Odqtq2cD02C3p+Dh0nnP7YdPN70ldh9Fr8MQP2cCuGkOHD79V20JHPYU3LK5lx9/3c4UY3Da2usxn2JvkKebDjKEGZ2jzeuetPS77rMz4K9mlhozS8eK7Oz/26YDWN90hmHyPQKc1HSOLvkuB25qOkeXfIcC65rOMThXHl3xb+TChrL8TkWWquNFyLWQP4a8AfIsyHdDblXz9eZUnPucMXzGZwM/hNy14ryfav6zZh7wyjDvyT4V2b/ew4zPAEuafq+65LsJuLzpHF3ynQQ80nSOYTK28rsOckX5775jIKTuyjtXAhzc8xRDW5bJdwEi4h7g+sw8a5zOXbW89TbjcN5tK9paPzAVIJOfdNaJ2HtA83ubyiM1yVkYUncPMLg79uDJMJUvk9dg0IZhbxqHU1cNQp0QBUTHA6XbOzkjQ5ORBYTURSYJ3Fhq7gM+3ECcJpTXvNhtHM5ZtbrnRFqcqzxWIxi8voa0xbOAkIZ3TUXbaRHj8tt42/24dHvvykfVU96cDCbWlthzS7cTdzPVJGQBIQ1vNYNHRc8HPtlAll77r9Lt3SPYZ4znLG9M9WgmT4zxnD0RQTC4AFqXrgWhScgCQhpGJuuBCyvu+mxniehxEcEOEdwcwb7jdc5x8J8VbSeP/nQHTadY0XGgu0Z/vp47lGLK9kATKb80biwgpJFZBQ+W107oA/4hYuyzMiJYAtzD4K22m/ZvFFMHBzpldEXO1IALd2Hw9uj/NLpo9USwdCyXnToDJc+ruOva0aeSJi4LCGkEMnkVTnwN2FC6az5wSwQndLq3a4lgQQRfpdirYpdxiDquiv9vVpWatwa+HVE373eOhHdtV2p8CPjX0ear6QzgpxEsj6jc42RIEUwF/o7BU3h/AVw1TvmkCcUCQhqxOzbCY1X7TGwDfBO4O4KjI5he8Zg3RDAtgvdHcAXFlMA/ZkRLUDfmfBi0Z8Yi4J4ITozonj2CvSK4EY4oX7oAOKNziahXFgCXAmsjuCiCd3TLH0FEcBiwhuqNx/4s0wGUmpxcSEqqZZfLINdTdGWXC/ADKbqzX47g+8B9FEuHrwd2oOitWAS8DboXGW2SSX8Ex1FsYDEw9w7AN4DzI7iBYtGtJ4BXKVZ1XAAcDryVwZctAC7J7M3liwrzgD/tHM9HsAaeXQsX9EVwMsVntTewhKGnrn4LuKQnaaUWsoCQasrkggjuBy7jVzs0DrQNxRfn4TVPvRG4nhau05/JnREcQ9FdP7t09zzgD2ue8hIqdgZryExgGcwC/hLgKyN4zjXA8s46IdKk5CUMaRQyWU0xne9iYKwbKT0NfAnYJ5MPZg4atNgKmdwIHATcPfqzvLiB4ov35M5Kl710HfDzMZ7jZeA04NjO+BBp0rKAkEYpk19k8hGKLu4zKFasHOmOkj+j6P4/FliQyWmZ/HSY5/ySYmXIgUcvxw+Qyf2ZHESxr+/3GHnx9BBccQss/EkmX9tsAbvI5GJgIXAEcAHFJZeR9iA8BnwOWJTJl+x5kLyEIY1ZZxGklcDKzoC8AyjGOsyh6N6fTVFYPEdROPwos/52vZlc3zln4zobeH03gm2AdwJvocg2h2KWxnMU0z8fBL6fydqIE84GjmooMgCZbABu7hwrIphJkf0t8Mwi+Nbp8NHLgJcoeoYeBe7M5P6mMkttZQEhjaPOF9QPOscWL5OXgVs6x4TTmUGxBlgTMXce8LHMj45hoSxp8vAShiRJqs0CQpIk1WYBIUmSarOAkCRJtVlASJKk2iwgJElSbRYQkiSpNgsISZJUmwWEJEmqzQJCkiTVZgEhSZJqs4CQJEm1WUBIkqTaLCAkSVJtFhCSJKk2CwhJklSbBYQkSarNAkKSJNVmASFJkmqzgJAkSbVZQEiSpNosICRJUm0WEJIkqbaAzE2bztgIKzc2E6erqcBrTYfoYiqwsXO0UV/nvxsaTTG0oMjY9s+47fn8Ozg2E+Ez3gDkcA9sSNs/4ymdo+2fcQvzfWIKnLdJp0NFAcEnMjm/h6lGJCLWZ+ZWTecYSkQ8Avx5Zn696SxVIuJyYKfMPLLpLFUi4lBgdWZu33SWoUTEK8Cumflk01mqRMQ9wPWZeVbTWapExNnAUZn5m01nqRIR84C1mTm96SxDiYhngGMy8/ams1SJiJuAxzPzhKazVImIk4DPZeZuTWcZSlu/6yJYAawc2OYlDEmSVJsFhCRJqs0CQpIk1WYBIUmSarOAkCRJtVlASJKk2iwgJElSbRYQkiSpNgsISZJUmwWEJEmqzQJCkiTVZgEhSZJqs4CQJEm1WUBIkqTaLCAkSVJtFhCSJKk2CwhJklSbBYQkSarNAkKSJNVmASFJkmqzgJAkSbVZQEiSpNosICRJUm0WEJIkqTYLCEmSVJsFhCRJqs0CQpIk1WYBIUmSarOAkCRJtVlASJKk2iwgJElSbRYQkiSpNgsISZJUmwWEJEmqzQJCkiTVZgEhSZJqm1rRtiSCjT1PMqyPT4ng9KZTDO0z28GB741g+6aTVLtoX3h5u/a+h2t2h9XT2psPYEUffPyUCF5oOkm1v5gPbz64ve/hpQfDo/Pbm++JbWFlX3vzAZwxHY7+3Qje3nSSap9fCNvMae97ePVB8N8t/jkILf6uW1JuCMhsIokkSZq4pgJ3wXMzYOq+sK2XNCRJ0hBe3Aiv3QuzXojMJCJmwLr/gNn7NR1NkiS11bP/A9sfkpn9UwEy84UI+puOJUmS2mx2f2b2A/w/oBAPkhhjz7QAAAAASUVORK5CYII=" + } + }, + "cell_type": "markdown", + "id": "9a9e8c44", + "metadata": {}, + "source": [ + "
\n", + "\n", + "
" ] }, { @@ -516,7 +596,7 @@ "metadata": {}, "source": [ "**Communication cost:** \n", - "- Each process broadcasts a message of length $N$ to $P-1$ processes per iteration. Thus, the **send cost** per process is $O(N P)$ per iteration (if we use send/receive instead of broadcast).\n", + "- One process broadcasts a message of length $N$ to $P-1$ processes per iteration. Thus, the **send cost** is $O(N P)$ per iteration (if we use send/receive instead of broadcast).\n", "- $P-1$ processes receive one message of length $N$ per iteration. Hence, the **receive cost** is $O(N)$ per iteration at each process. " ] }, @@ -752,9 +832,8 @@ "source": [ "### Is this implementation correct?\n", "\n", - "- Point-to-point messages are *non-overtaking* (i.e. FIFO order) according to section 3.5 of the MPI standard 4.0\n", - "\n", - "- Unfortunately this is not enough in this case" + "Point-to-point messages are *non-overtaking* (i.e. FIFO order) between the specified sender and receiver according to section 3.5 of the MPI standard 4.0.\n", + "Unfortunately this is not enough in this case. The messages can still arrive in the wrong order if messages from different processes overtake each other." ] }, {