mirror of
https://github.com/fverdugo/XM_40017.git
synced 2026-11-15 01:47:17 +01:00
3 files changed
+1229
-908
No files matched your search
+1
-1
@@ -126,7 +126,7 @@ makedocs(;
|
||||
"Jacobi method" => "jacobi_method.md",
|
||||
"All pairs of shortest paths" => "asp.md",
|
||||
"Gaussian elimination" => "LEQ.md",
|
||||
#"Traveling salesperson problem" => "tsp.md",
|
||||
"Traveling salesperson problem" => "tsp.md",
|
||||
#"Partial differential equations" => "pdes.md",
|
||||
],
|
||||
"Solutions" => "solutions_for_all_notebooks.md",
|
||||
|
||||
+1032
-784
File diff suppressed because it is too large.
Load diff
|
Before Width: | Height: | Size: 6.8 MiB After Width: | Height: | Size: 6.8 MiB |
+196
-123
@@ -22,8 +22,8 @@
|
||||
"In this notebook, we will learn\n",
|
||||
"\n",
|
||||
"- How to parallelize the solution of the traveling sales person problem\n",
|
||||
"- How to fix dynamic load imbalance\n",
|
||||
"- The concept of search overhead\n"
|
||||
"- The concept of search overhead\n",
|
||||
"- A dynamic load balancing method\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -51,9 +51,17 @@
|
||||
" \"It's not correct. Keep trying! 💪\"\n",
|
||||
" end |> println\n",
|
||||
"end\n",
|
||||
"tsp_check_2(answer) = answer_checker(answer, 2)\n",
|
||||
"tsp_check_2(answer) = answer_checker(answer, 4)\n",
|
||||
"tsp_check_3(answer) = answer_checker(answer, \"d\")\n",
|
||||
"tsp_check_4(answer) = answer_checker(answer, \"a\")"
|
||||
"tsp_check_4(answer) = answer_checker(answer, \"a\")\n",
|
||||
"function q_superlinear_answer(bool)\n",
|
||||
" bool || return\n",
|
||||
" msg = \"\"\"\n",
|
||||
" Negative search overhead can explain the superlinear speedup in this algorithm. The optimal speedup (speedup equal to the numer of processors) assumes that the work done in the sequental and parallel algorithm is the same. If the parallel code does less work, it is possible to go beyond the optimal speedup. Cache effects are not likely to have a positive impact here. Even large search spaces can be represented with rather small distance matrices. Moreover, we are not partitioning the distance matrix.\n",
|
||||
" \"\"\"\n",
|
||||
" println(msg)\n",
|
||||
"end\n",
|
||||
"println(\"🥳 Well done!\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -64,9 +72,16 @@
|
||||
"## The traveling sales person (TSP) problem\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"In this notebook we will study another algorithm that works with graphs, the [traveling sales person (TSP) problem](https://en.wikipedia.org/wiki/Travelling_salesman_problem). The classical formulation of this problem is as follows (quoted from Wikipedia) \"Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once?\" This problem as applications in combinatorial optimization, theoretical computer science, and operations research. It is very expensive problem to solve (NP-hard problem) which often needs parallel computing.\n",
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<b>Note:</b> There are two key variations of this problem. One in which the sales person returns to the initial city, and another in which the sales person does not return to the initial city. We will consider the second variant for simplicity.\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"### Problem statement\n",
|
||||
"\n",
|
||||
"Given a graph $G$ with a distance table $C$ and an initial node (i.e. a city) in the graph, compute the shortest route that visits all cities exactly once, without returning to the initial city."
|
||||
"Our version of the TSP problem can be formalized as follows. Given a graph $G$ with a distance table $C$ and an initial node in the graph, compute the shortest route that visits all nodes exactly once, without returning to the initial node. The nodes on the graph can be interpreted as the \"cities\", and the solution is the optimal route for the traveling salesperson to visit all cities. The following figure shows a simple TSP problem and its solution."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -86,14 +101,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c303dddf",
|
||||
"id": "fd4f87fc",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Sequential algorithm (branch and bound)\n",
|
||||
"\n",
|
||||
"The sequential algorithm finds a shortest path by traversing the paths tree of the problem. The root of this tree is the initial city. The children of each node in the graph are the neighbour cities that have not been visited on the path so far. When all neighbour cities are already visited, the city becomes a leaf node in the tree.\n",
|
||||
"\n",
|
||||
"The possile solutions of the problem are the paths from the root of the tree to a leaf node. Note that we assume the children are sorted using the **nearest city first heuristic**. This allows to quickly find a minimum bound for the distance which will be used to prune the remaining paths (see next section). "
|
||||
"A well known method to solve this problem is based on a [branch and bound](https://en.wikipedia.org/wiki/Branch_and_bound) strategy. It consisting in organizing all possible routes in a tree-like structure (this is the \"branch\" part). The root of this tree is the initial city. The children of each node in the graph are the neighbor cities that have not been visited in the path so far. When all neighbor cities are already visited, the city becomes a leaf node in the tree. See figure below for the tree associated with our TSP problem example. The TSP problem consists now in finding which is the \"shortest\" branch in this tree. The tree data structure is just a convenient way of organizing all possible routes in order to search for the shortest one. We refer to it as the *search tree* or the *search space*."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -113,10 +126,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9da5f5ae",
|
||||
"id": "c303dddf",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Of course, visiting all paths in the tree is impractical for moderate and large numbers of cities. The number of possible paths might be up to $O(N!)$. Therefore, an essential part of the algorithm is to bound the search by remembering the current minimum distance. "
|
||||
"### Nearest city first heuristic\n",
|
||||
"\n",
|
||||
"When building the search tree we are free to choose any order when defining the children of a node. A clever order is using the *nearest city first heuristic*. I.e., we sort the children according to how far they are from the current node, in ascending order. This allows to quickly find a minimum bound for the distance which will be used to prune the remaining paths (see next section). The figure above used the nearest city first heuristic. In blue you can see the distance between cities. The first child is always the one with the shortest distance."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -126,9 +141,9 @@
|
||||
"source": [
|
||||
"### Pruning the search tree\n",
|
||||
"\n",
|
||||
"The algorithm keeps track of the best solution of all paths visited so far. This allows to skip searching paths that already exceed this value. \n",
|
||||
"The basic idea of the algorithm is to loop over all possible routes (all branches in the search tree) and find find the one with the shortest distance. One can optimize this process by \"pruning\" the search tree. We keep track of the best solution of all paths visited so far, which allows us to skip searching paths that already exceed this value. This is the \"bound\" part of the branch and bound strategy. \n",
|
||||
"\n",
|
||||
"For example, in the following graph only 3 out of 6 possible routes need to be visted when we cut off the search after the minimum distance is exceeded. (The grey nodes are the ones we don't visit because the minimum distance had been exceeded at the previous node already.)"
|
||||
"For example, in the following graph only 3 out of 6 possible routes need to be fully traversed to find the shortest route. In particular, we do not need to fully traverse the second branch/route (figure below left). when visiting the third city in this branch the current distance is already equal to the full previous route. It means that the solution will not be in this part of the tree for sure. In figure below (right), the gray nodes are the ones we do not visit because the minimum distance had been exceeded before completing the route."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -148,37 +163,14 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f3c78a1d",
|
||||
"id": "9da5f5ae",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that it is not necessary that the graph be fully connected. Variations of this algorithm work for sparse graphs or directed graphs as well. \n",
|
||||
"### Computation complexity\n",
|
||||
"\n",
|
||||
"In the previous example, the shortest route was also the leftmost path in the graph. Although it is more likely that the shortest route be found in the left part of the graph when using the nearest city first heuristic, the solution can be anywhere in the search tree."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "85d771de",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b> Example:</b> Look at the following graph and its corresponding search tree. If $x\\leq 15$, the shortest route is the leftmost branch of the search tree. If $x > 16$, the route is situated on the right side of the search tree. \n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"fig-tsp-question-1.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAACQQAAAJCCAYAAACcIlijAAAACXBIWXMAAB7CAAAewgFu0HU+AAAAGXRFWHRTb2Z0d2FyZQB3d3cuaW5rc2NhcGUub3Jnm+48GgAAIABJREFUeJzs3XeYnVW1+PHvOjNJCL0mEyJSFFBEUbGRhBJIQtGr/iwI0rwWmvWqeMWLF5Frv16v/doRUARUVARJIYmkAYK9ASoqJQUIEBJSZuas3x9n0DPvnJlMOTNnyvfzPPPIu/Z+914H+eM9511778hMJEmSJEmSJEkaziI4Hri4TsPdm8nL6zSWJEnSmBLBVOCVwJHAwcBuwK7Aw8Ba4A7gp8C1mfyxDvPtApwCHNcx3yRgAvAQ8BdgCfCdTG4f6FzSaBIWBEmSJEmSJEmShrsI3g78b52G+3MmT63TWJIkSWNCBFOADwOvBcb34pYy8CPgvZnc0Y/5moB3AxcA2/filuuBN2fy177OJY1GpUYnIEmSJEmSJElSL+ze6AQkSZLGqghmAb8EXkfvioGgUo/wcuDWiL7tzhjBROBa4KP0rhgI4ATgtgim92UuabRqbnQCkiRJkiRJkiT1ggVBkiRJDRDB0cCPqRzTVfR34BfAY8BOwCHAkwt9dgSujuAlmcztxXwBfAc4vkbzPcCvgfXAvsDz6LwRym7AdRFMz+R3W5tLGs0sCJIkSZIkSZIkjQTFgqDFwBX9HGvdwFKRJEkaGyLYB7iGrsVAtwLvzmRJjXteAHwCOKIq3AxcGcEzMrlvK9OeC7y0EFsLnA18L5Ny1Vz7A5+mc/HQTsC3Izg0k7atzCWNWpGZjc5BkiRJkiRJkqQeRbAYOLIq9F+ZvL9B6UiSJI0JEXwXeGUh/B3gjEy29HBfM3A58JpC01cyObOH+7YF7gYmVYUfBqZl8sdu7mnqmOukQtPrMvlmd3NJo11p610kSZIkSZIkSWq4PQrXDzUkC0mSpDEigmfQtRjoN2ylGAigY2ee10OX3YBeF8FOPdx6Fp2LgQD+rbtioI652qnsHnRPoel9HcVC0phkQZAkSZIkSZIkaSQoHhn2YEOykCRJGjtOrhH7wNaKgZ6QyePAFwvhccCsHm57W+H6TuDSXsz1KJVjyqodABy3tXul0cqCIEmSJEmSJEnSsBZBCdi1ELYgSJIkaXAVi2nWA9f3cYxFNWLPrtUxgoOAfQrhr2WSvZzrW8CmQuyEXt4rjToWBEmSJEmSJEmShrtdgOZCzIIgSZKkQRJBAAcWwr/L7FJwszX31ogVjwR7wvE1Yr0uQMpkLXBLL8aUxoTiFyhJkiRJkiRJ0ggSwW7AcmCPQtN1mZzWyzH+E3hHIbwFmJnJH6r67UelOGew/LybFeC71Yg9NIh5SJIkbVUEh1D7SKrvZvLnXo7xUuDphXA78NlMNg8wxYEYB7y+ELu/H+PU2qRkXDd9jy5cr4d/Pov20q3AkVXX+0awXyZ/6eM40ohnQZAkSZIkSZIkjWCZPBTBO4DrgKhqOjWCqzK5tqf7IzgAOB/YptD0nupioA4fAU4caM49aKbyAqyo1iryBwYxD0mSpN64C7iCrgU9x0cwc2tHXXU8h11J1+ewi4vFQBF8D9hxgPl259eZvKs6kMkW4Oo6jN1SI7a2m75PK1z/KrPms2FPft7NuBYEacyxIEiSJEmSJEmSRrhMfhLBV4AzC02fiWBhJhtq3ddxFMRX6PoSaiHwyfpn2m/FHYI2ZbIeIIJ9gBOA5wNTgO2ovGS6D1gEzMvk0aFLVZIkjRWZPB7BKcDNwPiqpiOBfwW+3t29Hc9hX6Drc9htwMU1bjmS2rsm1kN3O/bUw2E1Yn8sBiIYD+xdCP+9H/PdUyO2fz/GkUa8WttzSZIkSZIkSZJGnncCdxRi+wAf6OGe1wNHFGJrgTMyKdcts4HbvXD9YAT7RfAD4M/A54HXAccCM4CXAucAVwErI/h4BLsOYb6SJGmMyOQXwPtrNH0iouYuh084HTimEFsPnJxJa73yGwZeUSO2vEbsKUBTIdafI8ruqxGzIEhjkgVBkiRJkiRJkjQKdOwCdCp0eYH0jgieXezf8YLq4zWGOiuTe7uZZgPw8CD+dWePwvXOwK+Bl7H137knAucBv4tgxlb6SpIk9cd/U9mZsNqudLPjYgS7d9xT9LZM/lTn3BomgsOgy/PXLzL5fY3uT6oRW9mPaWvds1c/xpFGPI8MkyRJkiRJkqRRIpPbIrgQ+HBVuBn4cgSHZdJeFf8f6LJrztcz+W4P47+eyq5CQ624Q9D2/RijBbgxgtMyuaoOOUmSJAGQSTmCM4BfAbtUNZ0awaWZzC/c8t90fb75bibf6GGad9L1eLF66U/hTY86jgD7vxpNn+7mlu1qxNb3dd5MNkfQSudj0Prz7CiNeBYESZIkSZIkSdLo8nHgODofBfZ84FzgswARzAFOKdz3J+DtQ5FgPxR3CHrCw8B3geuBvwOPdfR9PnAS8KJC//HANyO4N7PmURWSJEn9ksk9EbwJuhRXfzmCgzt2cySCI6kcF1btPuDsrYx/ad2SHWQRBPAl4FmFptuAy7q5rVZB0KZ+prCZzgVB2/ZzHGlE88gwSZIkSZIkSRpFOnYBOh14tND0XxE8KYKJwBcKbW3AqZl9X4U9RIor6KHykumpmZyZyQ8y+Xkmd2WyPJNPZ3IY8CpgXeG+bYArIgZthb0kSRqjMvkedCnc2Qd4H0DH88eXgahqLwOnZ/LQUOQ4RD4MvK4QWwucnEm5m3tqFe1s6ef8xUKiWsVG0qhnQZAkSZIkSZIkjTKZ/I3KjkDVdgQ+A7wfeEqh7aJMbhmK3PqpuiCoDJybydmZrO3ppo6XckdBZUV+lSezlVX4kiRJ/fRW4O5C7LwIDqZSGHRAoe2TmSwcksyGQAQfAd5bCG8BXpPJn3q4dWKNWH8Lgor31RpbGvUiMxudgyRJkiRJkiRpEETwLeC1hXA70FR1vRQ4qmNnoWEpgh35Z87tmV12/dna/W+ishq/2p8y2b8e+UmSJFWLYDrwUzo/c/0GOJDKEaZP+AXwosx+F74MGxGMo7IL5RsLTW3AiZlcs5X7zwL+rxA+OZPv9COXNXQ+cvZ3mRzc13Gkkc4dgiRJkiRJkiRp9Hoz8LdCrPrF1KPAacO5GAggk3WZPNzx16dioA7fAP5aiD01gv0Gnp0kSVJnmSyjcmxWtWfSuRjoceCUUVIMtBNwHV2LgbZQ+Yw9FgN1KO7oCDChnykVj4atNbY06jU3OgFJkiRJkiRJ0uDI5JEITgcW0rkQ6AlvzuxSKNOtCJ4JTK5TerXcmEndt7XPpC2Ca4B/KzRNB/5S7/kkSZKADwKzgRd10/6uTP4whPkMigimAj8Gnl1oWg+8OpMbejnU4zVi9SoIWt/PcaQRzYIgSZIkSZIkSRrFMrkpgmuBlxea/pDJt/o43AXAifXJrKZmGLTdim6uEWsZpLkkSdIY11GQfBqVY8G2LzT/GPhSX8aL4HZg5zqlV3RrJif39aYIngdcS9dnqpXACZn8sg/D1drFZ7t+5DQOGFcI1yo2kkY9C4IkSZIkSZIkaRSL4EXAS2o0PT2CN2by1aHOqUFW1YjtMeRZSJKksWQCtd/J70mlaKUvx4XtDexWj6RquKevN0QwB/gusEOh6U/A8Zn8qY9D3lcj1p/i7T1rxO7txzjSiFdqdAKSJEmSJEmSpMERwQ7A5XS/OPRTEew/hCk10qYasc1DnoUkSRoTIpgAfJuux1cBPBf4wJAmVEcRvAG4jq7FQLcA0/pRDATwZ6BciE3txzi1CoLu6sc40ojnDkGSJEmSJEmSNHp9BnhKIdYONHX88/bA5RHMyKS1F+P9Bbi9jvkV5SCOXWtF/YODOJ8kSRrbPgE8q4f2f49gbiY/HaqEBiqCAD4MvLdG8/eBUzPZ2J+xM9kYwb3Ak6vCe/VjqFpFRBYEaUyyIEiSJEmSJEmSRqEIXg28rhD+PvBH4H1VsRcAFwIXbG3MTM4Hzq9TilsVwUTgtEL4L5ks6MdwT6sRq3U0hSRJ0oBEcDzwlkJ4MZWdgXbsuC4Bl0ZwSCaP9GLYV1I5ZmwwPLy1DhGMB74BvLZG8yeB92R22eGnr/5I54KgQyIo9XHc59SI3TGwtKSRKTIHc8GFJEmSJEmSJGmoRbAX8Ctgl6rwo8BBwNqOtgOq2tqBozJZOmRJ9kIEJWAdsF1V+PZMntePsX4AvKwqlMCUTFYPLEtJkqR/imAS8GtgclX4MeAZwEuBzxVu+U4mJw9Rev0SQRPwHeBVhaYy8NZMvlCned4LfKQQPiiTP/RhjAXAMVWhe4EnZw7qTpTSsFRqdAKSJEmSJEmSpPrpKKK5lM7FQAD/nsn9mWwCzqLz8VxNwGUR7DREafZKx2rw3xbCz4pg976ME8FU4IRC+LcWA0mSpHrqOFLra3QuBgJ4Xyb3AF8EVhTaTorg1KHIbwD+j67FQO3AGfUqBurwkxqx2b29OYIdgBcVx7QYSGOVBUGSJEmSJEmSNLqcBxxViC0BvvzERSaLqbysqrYP8PlBzKu/ritcj6PrUWhb80G6HrHx5VodJUmSBuAc4CWF2AqoFM10FDu/CdhS6PP5CPYZ9Oz6IYJzgDcWwmXglEwur/N0v6brka7FuXtyEp13lgS4fkAZSSOYR4ZJkiRJkiRJ0igRwaHAcmB8VXgz8OxM/ljouwvwe6ClMMxrM7liUBPtgwieDNxN5wWujwDPyeSvvbj/RODKQng1sG8mG+uVpyRJGtsiOAi4DZhYFd4CHJrZecfDCD4IvL8wxFIqR7i2D2qifRDBAVSKdCYUmt6TyScGac7zgQ8Xwq/K5HtbuW874BfA/lXhvwIHZNJa1ySlEcIdgiRJkiRJkiRpFOh4CfItOhcDAXyoWAwEkMnDwNtqDPWFjiKcYSGTv1M5pqLazsCCCJ7V070RvB5qrlx/m8VAkiSpXiKYQOU5bGKh6ePFYqAOH4IuLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "d1df7878",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:fig-tsp-question-1.png\" align=\"left\" width=\"800\"/>\n",
|
||||
"</div>"
|
||||
"The total number of routes we need to traverse is $O(N!)$, where $N$ is the number of cities. This comes from the fact that the number of possible routes is equal to the number of possible permutations of $N$ cities. Thus the cost of the algorithm is $O(N!)$, which becomes expensive very quickly when $N$ grows.\n",
|
||||
"\n",
|
||||
"In practice, however, we will not need to traverse all $O(N!)$ possible routes to find the shortest one since we consider pruning. The nearest city first heuristic also makes more likely that the shortest route is among the first routes to be traversed (left part of the tree), thus speeding the process. However, the solution can be anywhere in the search tree, and the number of routes to be traversed is $O(N!)$ in the worse case scenario."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -188,7 +180,12 @@
|
||||
"source": [
|
||||
"## Serial implementation\n",
|
||||
"\n",
|
||||
"Let's implement the serial algorithm. First, we sort the neighbours according to their distance. "
|
||||
"Let's implement the serial algorithm.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<b>Note:</b> The implementation of this algorithm is rather challenging. Try to understand the key ideas (the explanations) instead of all code details. Having the complete functional implementation is useful to analyze the actual performance of our parallel implementation at the end of the notebook, and to check if it is consistent with the theory.\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -196,7 +193,9 @@
|
||||
"id": "f2b70f85",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Nearest-city first heuristic"
|
||||
"### Nearest-city first heuristic\n",
|
||||
"\n",
|
||||
"The first step is preprocessing the distance table to create a new data structure that takes into account the nearest city first heuristic. This is done in the following function."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -217,27 +216,12 @@
|
||||
"end"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "2eeecdd6",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"C = [\n",
|
||||
" 0 2 3 2\n",
|
||||
" 2 0 4 1\n",
|
||||
" 3 4 0 3\n",
|
||||
" 2 1 3 0 \n",
|
||||
"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f769627c",
|
||||
"id": "9fc1a398",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The data structure we will use for the connections table is a matrix of tuples of the form (destination, distance). The tuples are sorted by their distance in ascending order (per start city). "
|
||||
"Execute the next cell to understand the output of `sort_neighbors`."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -247,15 +231,21 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"C = [\n",
|
||||
" 0 2 3 2\n",
|
||||
" 2 0 4 1\n",
|
||||
" 3 4 0 3\n",
|
||||
" 2 1 3 0 \n",
|
||||
"]\n",
|
||||
"C_sorted = sort_neighbors(C)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "51ed8312",
|
||||
"id": "a63dc266",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The connections matrix can be indexed by a city. This returns a `Vector{Tuple}}` of all the destinations and their corresponding distances. "
|
||||
"The output is a vector of vector of tuples. The outer vector is indexed by a city id, for instance:"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -271,6 +261,14 @@
|
||||
"C_sorted[city]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f769627c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
" This returns a vector of tuples that contains information about the connections to this city of the form (destination, distance). In this case, city 3 is connected to city 3 at distance 0 (itself), then with city 1 at distance 3, then with city 4 at distance 3, and finally with city 2 at distance 4. Note that the connections are sorted by their distance in ascending order (here is where the nearest city first heuristic is used). "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "44025bd5",
|
||||
@@ -284,7 +282,7 @@
|
||||
"id": "6c91a99f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Next, we write an algorithm that traverses the whole search tree and prints all the possible paths. The tree is traversed in depth-first order. Before we go to a neighbouring city, we also have to verify that it has not been visited on this path yet. If we reach a leaf node, we print the complete path and continue searching. "
|
||||
"Next, we write an algorithm that traverses the whole search tree and prints all the possible paths. To this end, the tree is traversed in [depth-first order](https://en.wikipedia.org/wiki/Depth-first_search) using a recursive function call. Before we go to a neighbouring city, we also have to verify that it has not been visited on this path yet. If we reach a leaf node, we print the complete path and continue searching. "
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -333,7 +331,7 @@
|
||||
" end\n",
|
||||
" return nothing\n",
|
||||
" else\n",
|
||||
" println(path)\n",
|
||||
" println(\"I just completed route $path\")\n",
|
||||
" return nothing\n",
|
||||
" end\n",
|
||||
"end"
|
||||
@@ -357,7 +355,7 @@
|
||||
"source": [
|
||||
"### Serial implementation without pruning\n",
|
||||
"\n",
|
||||
"Now, we add the computation of the minimum distance. At each leaf node, we update the minimum distance. Furthermore, as we add another node to our path, we update the distance of the current path. That makes it necessary to include two more parameters in our recursive algorithm: `distance`, the distance of the current path, and `min_distance`, the best minimum distance found so far. "
|
||||
"Now, we know how to traverse all possible routes. We just need a minor modification of the code below to solve the TSP problem (without pruning). We add a new variable called `min_distance` that keeps track of the distance of the shortest route so-far. This variable is updated at the end of each route, i.e., when a leaf node is visited. After traversing all routes, `min_distance` will contain the distance of the shortest route (the solution of the ASP problem)."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -375,6 +373,16 @@
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9ac4875c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<b>Note:</b> We could further modify the function so that we also return a vector containing the cities in the shortest route. However, in this notebook, we will only return the distance of the shortest route (a single value) for simplicity.\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
@@ -382,6 +390,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"verbose::Bool = true\n",
|
||||
"function tsp_serial_no_prune(C_sorted,city)\n",
|
||||
" num_cities = length(C_sorted)\n",
|
||||
" path=zeros(Int,num_cities)\n",
|
||||
@@ -411,7 +420,7 @@
|
||||
" else\n",
|
||||
" # Set new minimum distance in leaf nodes\n",
|
||||
" min_distance = min(distance,min_distance)\n",
|
||||
" #@show path, distance, min_distance\n",
|
||||
" verbose && println(\"I just completed route $path. Min distance so far is $min_distance\")\n",
|
||||
" return min_distance\n",
|
||||
" end\n",
|
||||
"end"
|
||||
@@ -425,6 +434,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"city = 1\n",
|
||||
"verbose = true\n",
|
||||
"min_distance = tsp_serial_no_prune(C_sorted,city)"
|
||||
]
|
||||
},
|
||||
@@ -435,7 +445,7 @@
|
||||
"source": [
|
||||
"### Final serial implementation\n",
|
||||
"\n",
|
||||
"Finally, we add the pruning to our algorithm. Anytime the current distance exceeds the minimum distance, the search in this path is aborted and continued with another path. "
|
||||
"Finally, we add the pruning to our algorithm. Anytime the current distance exceeds the minimum distance, the search in this path is aborted and continued with another path. By running the function below, you will see that only three routes will be traversed thanks to pruning as shown in next figure."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -472,6 +482,7 @@
|
||||
"function tsp_serial_recursive!(C_sorted,hops,path,distance,min_distance)\n",
|
||||
" # Prune this path if its distance is too high already\n",
|
||||
" if distance >= min_distance\n",
|
||||
" verbose && println(\"I am pruning at $(view(path,1:hops))\")\n",
|
||||
" return min_distance\n",
|
||||
" end\n",
|
||||
" num_cities = length(C_sorted)\n",
|
||||
@@ -493,7 +504,7 @@
|
||||
" else\n",
|
||||
" # Set new minimum distance in leaf nodes\n",
|
||||
" min_distance = min(distance,min_distance)\n",
|
||||
" #@show path, distance, min_distance\n",
|
||||
" verbose && println(\"I just completed route $path. Min distance so far is $min_distance\")\n",
|
||||
" return min_distance\n",
|
||||
" end\n",
|
||||
"end"
|
||||
@@ -507,6 +518,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"city = 1\n",
|
||||
"verbose = true\n",
|
||||
"min_distance = tsp_serial(C_sorted,city)"
|
||||
]
|
||||
},
|
||||
@@ -526,13 +538,14 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"n = 12 # It is safe to test up to n=12\n",
|
||||
"n = 11 # It is safe to test up to n=11 on a laptop\n",
|
||||
"using Random\n",
|
||||
"using Test\n",
|
||||
"Random.seed!(1)\n",
|
||||
"C = rand(1:10,n,n)\n",
|
||||
"C_sorted = sort_neighbors(C)\n",
|
||||
"city = 1\n",
|
||||
"verbose = false\n",
|
||||
"@time min_no_prune = tsp_serial_no_prune(C_sorted,city)\n",
|
||||
"@time min_prune = tsp_serial(C_sorted,city)\n",
|
||||
"@test min_no_prune == min_prune"
|
||||
@@ -543,7 +556,7 @@
|
||||
"id": "6088ddc9",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can observe that, especially for larger numbers of cities (n=11 or n=12), the performance of the algorithm with pruning is much better than the performance of the algorithm without pruning. "
|
||||
"You can observe that, especially for larger numbers of cities (n=11), the performance of the algorithm with pruning is much better than the performance of the algorithm without pruning. "
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -556,11 +569,13 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c6375465",
|
||||
"id": "732a3ffb",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Where can we extract parallelism ?\n",
|
||||
"Unlike the previous algorithms we studied, in this problem we don't know beforehand how much work is performed since we don't know where the pruning cuts off part of the search tree. Still, we want to divide the workload among multiple processes to enhance the performance. "
|
||||
"\n",
|
||||
"All branches of the search tree can be traversed in parallel. Let us discuss how we can distribute these branches over several processes.\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -585,7 +600,7 @@
|
||||
"source": [
|
||||
"### Option 1\n",
|
||||
"\n",
|
||||
"The first idea how to parallelize the TSP algorithm is to assign a branch of our search tree to each process. However, as mentioned in an earlier section, the number of branches in the search tree can be up to $O(N!)$. This would require an unfeasibly large amount of proecesses which each do only very little work. "
|
||||
"We can (at least in theory) assign a branch of our search tree to each process. However, as mentioned in an earlier section, the number of branches in the search tree can be up to $O(N!)$. This would require an unfeasibly large amount of processors which each do only very little work. Thus, we skip this option as it is impractical."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -609,7 +624,8 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Option 2\n",
|
||||
"Instead of assigning one branch per worker, we can assign a fixed number of branches to each worker. This way, each worker can perform the pruning within their own subtree and less workers are needed. \n"
|
||||
"\n",
|
||||
"Instead of assigning one branch per worker, we can assign a fixed number of branches to each worker. This would be a good strategy if we do not consider pruning. However, it is not efficient if we include pruning (which is essential in this algorithm). "
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -632,13 +648,14 @@
|
||||
"id": "e3af7def",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Performance issues\n",
|
||||
"### Performance issues: Load balance\n",
|
||||
"\n",
|
||||
"#### Load balancing\n",
|
||||
"However, this approach has a problem with load balancing. Since we don't know beforehand how much pruning can be done in each subtree, some workers might end up doing less work than others. This uneven distribution of workload leads to some workers being idle, which impairs the speedup. \n",
|
||||
"Pruning is essential in this algorithm but makes challenging to evenly distribute the work over available processors. Image that we assign the same number of branches per worker and that the workers use pruning locally to speed up the solution process. It is not possible to know in advance how many branches will be fully traversed by each worker since pruning depends on the actual values in the input distance matrix (runtime values). It might happen that a worker can prune many branches and finishes fast, whereas other workers are not able to prune so many branches and they need more time to finish. This is a clear example of bad load balance. We will explain later a strategy to fix it.\n",
|
||||
"\n",
|
||||
"#### Search overhead \n",
|
||||
"Another disadvantage of this kind of parallel search is that the pruning is now less effective. The workers each run their own version of the search algorithm and keep track of their local minimum distances. This means that less nodes will be pruned in the parallel version than in the serial version. This is called **search overhead**."
|
||||
"\n",
|
||||
"### Performance issues: Search overhead \n",
|
||||
"\n",
|
||||
"Another disadvantage of this kind of parallel search is that the pruning is now less effective. The workers each run their own version of the search algorithm and keep track of their local minimum distances. This means that less nodes will be pruned in the parallel version than in the serial version. The parallel code might search more routes than the sequential ones. This is called *search overhead*."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -647,7 +664,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question:</b> How many nodes are pruned in total when we assign two branches to each worker? Look at the illustration below.\n",
|
||||
"<b>Question:</b> How routes are fully traversed in total when we assign two branches to each worker? Look at the illustration below. Assume that each worker does pruning locally and independently of the other workers.\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
@@ -682,33 +699,42 @@
|
||||
"id": "d0bc4fdd",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In this example, the parallel algorithm prunes less nodes than the serial version because not all workers are able to use the global minimum distance as a pruning bound."
|
||||
"In this example, the parallel algorithm traverses more routes (1 more) then the serial version because not all workers are able to use the global minimum distance as a pruning bound. Remember that the sequential code only traverses 3 routes completely. See figure:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g26375.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1oAAAHpCAYAAACbY01sAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N15fFxl9cfxz7mTZtKWRQrIItAmadkqyCaI7IsisikKgsiOFOVHwQLNTFrgKs3MpC1Vyr6oQBWxZRGogFAoImURQUCRpSRpWcsiFeiSSTNzfn/cDEymycwkmZk7k5z369WX5s6duac0mcy5z/N8H8EYY4wpnBpgD2BXYOuuP1sB6wAju/73f8BKYAXQCrza9edp4AUgWfKqjTHGGGOMMabMbAZMAhYAqwAdwJ//AncAJ+I1ZsYYY4wxxhgzZAhwFHAf0MnAmqve/nwK3Iw3OmaMMcYYY4wxg1YAOAH4F8Vprnr78wCwTwn+fsYYY0xBiN8FGGOMqRhfBa4hvxGmj4F/4629agU+xJtW2I43JXAksDHeGq5tgO3x1nflcjvwM+CtPtZujDHGGGOMMWVlBHAVkCD7qNMTwIXAbngjX30xHDgIaAJez3GdT4FzsJuFxhhjjDHGmAq1HdmnCS4HongjU4UiwF7ATUBHlmv/CdiggNc1xhhjjDHGmKI7Gi+CvbcGKwysV+QaRgNXA2t6qaMNGF/kGowxxhhjjDGmIE6h9+ZmLrBJievZEXi8l3o+AvYucT3GGGOMMcYY0ycX0nND8z5wqI91CXA+PU8nXAl8w7/SPhd5PbJXZGlke7/rMMYY4y9bSGyMMSbdGcD1rP37YRHwA+Dtkle0tq8B84AtMo6vAA4Enil5RV2u+8d1wz4a9dFLQD1wRyARCE8eN7nFr3qMMcb4xxotY4wxKUcBd7B2YuDdwHF40ezlYkvgL3hhHek+xAvSeK3kFQGRtshEUbk87dAa4LcBCVwyuXbyMj9qMsYY4w9rtIwxxgCMBZ5l7XCLOcBpQGfJK8ptQ+BBYJeM4/8C9gBWl7KYWEtsfRVdjLc/WKYViv5KAjIrPDq8vJR1GWOM8YfjdwHGGGN8FwT+yNpN1nzKt8kC+C9wCN6myOl2AC5f+/TikmEyUpC/4q0Zy7SOIFNJsCTSGmma+erMjUpdnzHGmNKyES1jjDHT8QIw0j0NHECJR4X6qRb4O5DZvHwHb9pjScVaYjuo6EXAMVlOWwn82qlymhu2aninRKUZY4wpIWu0jDFmaBsP/BMYlnZsOd50vCUFeP31u15rR6Am7XgH8MsCvH7KoXgjcOkzNd7EW8O1soDXyVvTkqYDnKQTxZvG2JtVKDc4SWdmw7iGt0pVmzHGmOKzRssYY4YuAf4K7JNx/LvAn/rxeusAOwG7pv3Zlp6nqa8A1u3HNbKZAVyQcSyGt7GyL1RVokuiR4mKC3wly6lrgNsEuSxUF3qhNNUZY4wpJmu0jDFm6DoMbxQo3e1kn/LWm58DU8l/7W8xGq3hwEt4UwlT2oE64N0CX6vPoi3Rg1W0SZDdc5y6SJDmhtqG+SLS03ovY4wxFcAaLWOMKQPNS5q304Tur6L7iSM3h8aE7i/BZRcBX0/7egWwPd6Uu766HJjYh/OL0WgBHMna67JmAJOLcK1+iS2JHapJvQjYM9t5gjwP/HJEYsQfJ46bGC9NdcYYYwrFGi1jjPFBdGm0jiQHo+wN7I+3L1TK7HBd+Nwil7Af8GjGsYFMs8vWaC3BS+JLH2kqVqMF8DjeXlrp19oKb+1Z2Yi2RA/GIYxyYI5TP1D0hmQyee3UsVP70wQbY4zxgTVaxhhTAtHXo2MlIAep6gF4jdUmWU5/IVwX3qnIJf0OOCHt64FOsUs1Wu/i7ceV+vM08D4QBUJp5xez0ToUuC/j2NnA1UW63oA0tTXt5KgzCW9T6GFZTk0Cjwgy26YVGmNM+bNGyxhjiiCyOLKxBGR/4GBBvqFoba7npEkGhwU3nrTlpI+KVN66wDJgRNqx64CzBvCadcAnwIe9PF7KRgvgBbykw5SnyDFVz2/TWqeNrtKq81T0dHL9t1H+o47ewDDmNG7R+N/SVGiMMaYvrNEyxpgCcF9y16mpqdlPHT0I5WDgy/T9PbZT0eccdR4NaOCyC8de+H4RSgU4Gbgp49ieeM1IsZS60ZoEXJZxbGtgcRGvWRBum/uFIMEJKD/Fm/KYTRzhThLcGKoPLbRRLmOMKR/WaBljTD+4C92q4JbBrxDg4K7Gal+guo8vk0B4HmUR8Hhc4g+5te7/Cl/tWv6AN00t5VW8GPZiKnWjtSleqEdV2rFzgCuLeM2CmqtzAy1LW46UpJyt6IHk+J2toi2SlF87w5ybbRNkY4zxnzVaxhiTB1WVyJLIVxx1DkI5CGFfYGQfXyYBPIfyiATkr9Iujzds2/BpEcrNRoB38BqRlFKk8pW60YK1UxXvBL5X5GsWRaQlsrUgp4nIBEW/kOP0JPAkwi3xVfFb3fHuilLUaIwxpjtrtIwxphfNbzRvnuxM7gUcjHIYwpf68TKtwAJgQXBY8OEirrvK1/Z4e02l+zZQ7Dh5PxqtS/H29kpZDmyE14hUJHexu17QCf4I4Qxg5zyesgK4I6nJOWvq1ix0xa3Yv7sxxlQaa7SMMaZLZHFkY6mSA0XlIODgPgZYpLwLPIzwsNPpLGgY1/BWgcscqOOBW9O+TgBfwPtAXkx+NFoH4TW56cYAS4t83ZKIvh7dFeEMhB8C6+XxlLcQ5klCbm+ob3jS1nMZY0xxWaNljBmy5urcQEtLy05d66yOwAuEcPr4MivxpmktIMGCUH3ouTL/AOsCl6R93QKMLcF1/Wi0NsFLV0z3TeChIl+3pGYsmzFyzco1x4rI6XTfPyybt1DuEOT29rr2J2ykyxhjCs8aLWPMkJK2UfDBeB+61+/jS3QivEDSmw4Yb48/5o53OwpfadHcijeqlXIfcFgJrutHowXedMH0NU0VFYjRV7GW2Fbq6PEoZ5B/A/0hcL8g89qXtt/vHuB2FrFEY4wZMqzRMsYMajOWzRjZubJzT+Bg4AiE7fvxMp+ts4on4n9xx7mfFLTI0lqIt2FyylXA/5Xgun41Ws8Cu6R9HQGmlOC6vou+Ht0Vh5OAH+KtTcuHNV3GGFMg1mgZYwYVV10n2BLceYCx6581VlqtjwyyDWH/Aeya9nUT3QMjisWvRiuzsbwCmFiC65YNt82tCSaDhyEcgzd6uU6eT31PRe8UlT+NTIz868RxE+NFLNMYYwYda7SMMRVvetv0TTuTnYeKyCF4I1cb9uX5gvxP0YcRHgQeDNeGlxSjzjLxKt7GvSlhIFaC6/rVaN0NHJn29U3AqSW4blly29yaGq35hqLHAEeRX4gGwGpgkYjMJ8ldofrQG8Wr0hhjBgdrtIwxFSdj1Ko/IRZJhH9W8DqrgWgF0tMUzwdmleC6fjVac4Fj0r7+A95UuiEvo+k6kr6tV2wF5qPcO2r5qL9O2G3CmuJUaYwxlcsaLWNMRZj56syN1gxbcwDeiNWRdN9wNx/vAY8JMr96WPX8MtjPyi//Ar6c9vUlwC9KcF2/Gq37gW+lfX0DcGYJrltRZi+eHVwVWPXN/jRdin4kyMMIC5KSvHfKmCnvFq9SY4ypHNZoGWPKUg9rrfYHqvrwEquBRRUUu14qTwJfS/t6BjC5BNf1q9F6nO6R55cBF5TguhVr9uLZwRXOim+IyPfx1nTlG6QB3r5sfxfkPhFZsHrJ6n9YoIYxZqiyRssYUzYib0U2lLh8E+HbwCHAxn18iX8L8oAgD66W1X9za932IpRZ6TJHeG4BTi7Bdf1qtF4DxqV9fTFwaQmuOyj0ME336/Tts0Ol7TNnjDEFY42WMcY3qiqxltgu4sihin4b2B0I9OElVgAPi8j9JLnfFujn5Srgp2lfP4W3xq3Y/Gi0qvE+6KePhJ6At5eY6YfI4sjGTsA5VNHD6d8+dB8AjyIs6JTOhy4ac1Fb4as0xpjyYI2WMaakZr05a3i8I74XwhHA0cAWfXyJVmCBIPNHJEY8aJHTfTYRuDzt6+XAqBJc149Ga1vg5YxjX8WLuDcDNFfnBlpbW/dU9HAcDka7bRuQr3eBxxEWJKoSf5665dS3C12nMcb4xRotY0zRzXh9xhcTTuJbXXfBDyX/fXwgba2Vonc31ja+Upwqh4xDgAcyjm0NLC7ydf1otE7Gi3NPUbwRmE+LfN0hafri6fWdVZ2HicqhwD7AyD6+hAL/VvQxR5zHJSCPNWzV8E7hKzXGmNKwRssYUxSxxbHxWqWH93Ndx2cbBsdXx+93x7srilLk0PQF4EO6T9E8C7iuyNf1o9G6GTgp7esXga8U+ZoGcBe6VcEtg19JC7PZBwj246XeBR4XkUWa0MdtjZcxppJYo2WMKYgBTglsp2v6kDp6b+Poxv8Up0rT5Rlgt7Svb6f7XlPFUOpGS4A3gS+lHbscOK+I1zS9cN9xRwRXB78OHNw1zXBn+rb3Xcr7wN8RHpekLNpg+QZP2x5exphyZY2WMabfmhc3b6FVepiqHgEcCAzvw9PfAeYLMr+9pv1hd3N3VXGqND1opnuk+ypgM+CTIl6z1I3WfsCjGceOAu4p4jVNnlL74qnqQSJyEDC2ny/1CcoihEVJJ/nEmpVrnrERcGNMubBGyxjTJwOaEqj8B7hXkPkNdQ2LbAqQb3YHns44djrwmyJes9SN1o14f6eUT/CaSWvoy1CsJbZV0kkeIMi+KHvjrRvsj07gX4o+4eA85SScJyePm9xSwFKNMSZv1mgZY7JyX3Kra0bUHJBMJr8rIkcAm/fh6auBhxHudQLOfFvYXjaCQBte45Hyd2CPIl6zlI3WKGBJxuv/HvhRka5nCqyptWmTAIHdVXUvhL3xbg4M6+fLvQc8g/Is8Hh8ePwJG0E3xpSCNVrGmLXMenPW8I41HQcreowgRyj6hT48/QO8VLt7nQ7ngYZtGyzhrXx8EW8Prf8DLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "f6329f5d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g26375.png\" align=\"left\" width=\"300\"/>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "95643cd8",
|
||||
"id": "1c8e5eb7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question:</b> The previous example described positive search overhead. There is also negative search overhead, resulting in superlinear speedups. Can there be negative search overhead in this parallel TSP algorithm? (Provided the workers communicate the minimum distance with each other)\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
" a) No, because we use the nearest city first heuristic. \n",
|
||||
" b) No, because each worker has to search the whole subtree before the algorithm completes. \n",
|
||||
" c) Yes, because the parallel algorithm does not need to search the whole search tree.\n",
|
||||
" d) Yes, because the global minimum distance can be found more quickly, enabling the parallel version to do more pruning."
|
||||
"In order to minimize search overhead, workers need to collaboratively keep track of a global minimum distance. However, this needs to be done carefully to avoid race conditions. We show how to do this later in the notebook."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "fff58498",
|
||||
"cell_type": "markdown",
|
||||
"id": "5e4cee1a",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"answer = \"x\" # Replace x with a,b,c or d\n",
|
||||
"tsp_check_3(answer)"
|
||||
"### Negative search overhead\n",
|
||||
"\n",
|
||||
"The parallel algorithm might search more branches than the sequential one when we parallelize the pruning process. However, it is also possible that parallel algorithm searches less branches that the sequential one for particular cases. Imagine that the optimal route is on the right side of the tree (or the last route in the tree in the limit case). The parallel algorithm will need less work than the sequential one in this case. The last workers might find the optimal route very quickly and inform the other workers about the optimal minimum, which can then prune branches very effectively. Whereas the sequential algorithm will need to traverse many branches in order to reach the optimal one. If the parallel code does less searches than the sequential one, we way that the search overhead is negative. \n",
|
||||
"\n",
|
||||
"Negative search overhead is very good for parallel speedups, but it depends on the input values. We cannot rely on it to speed up the parallel execution of the algorithm. \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -717,7 +743,16 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Option 3: Dynamic load balancing with replicated workers model\n",
|
||||
"In our parallel implementation, we will use a coordinator process and several worker processes. The coordinator process (or _master_) searches the tree up to a certain maximum depth _maxhops_. When _maxhops_ is reached, the coordinator creates a job and delegates it to a worker. The workers repeatedly get work from the master and execute it. This is an example of **dynamic load balancing**: the load is distributed among the workers during runtime."
|
||||
"\n",
|
||||
"In this third option, we explain a strategy to improve load balance based using the [*replicated workers model*](https://en.wikipedia.org/wiki/Thread_pool) also known as *worker pool* or *thread pool*. In this model, the main processes (aka master or coordinator process) sends jobs to a job queue. Then, workers take one available job from the queue, run it, and take a new job when they are done. In this process, workers never wait for other workers thus fixing the load balance problem. It does not matter if there are some jobs that are larger than others as long as there are enough jobs to keep the workers busy. The main limiting factor of this model is the number of jobs and speed in which the main process is able to generate jobs and send them to the queue. This is an example of **dynamic load balancing**: the load is distributed among the workers at runtime.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"In our parallel implementation, we will use a coordinator process and several worker processes. The coordinator process will search the tree up to a certain maximum depth given by a number of hops/levels _maxhops_. When _maxhops_ is reached, the coordinator will stop searching the tree and will let any available worker to continue searching in the subtree. In the figure below, the master process will only visit the nodes in the top green box. The worker processes will search in parallel the subtrees below.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -745,28 +780,19 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "81219a27",
|
||||
"id": "9e345393",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
" <b>Question:</b> To find the right <i> maxhops </i> level is a tradeoff between...\n",
|
||||
"</div>\n",
|
||||
" \n",
|
||||
" a) Communication overhead (large maxhops) and load imbalance (small maxhops). \n",
|
||||
" b) Search overhead (large maxhops) and load imbalance (small maxhops). \n",
|
||||
" c) the number of workers (large maxhops) and the job size (small maxhops). \n",
|
||||
" d) buffer for the job queue (large maxhops) and idle time of the coordinator process (small maxhops)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0cf1ec88",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"answer=\"x\" #Replace x with a,b,c, or d\n",
|
||||
"tsp_check_4(answer)"
|
||||
"### Performance impact of maxhops\n",
|
||||
"\n",
|
||||
"We introduced a new parameter `maxhops`. Which is then the optimal value for it? When choosing `maxhops`, there is a trade off between load balance and communication overhead.\n",
|
||||
"\n",
|
||||
"- A small `maxhops` will reduce the number of jobs communicated to the workers (less communication), but reducing the number of jobs is bad for load balance. In the limit, we might generate even less jobs than the number of workers.\n",
|
||||
"\n",
|
||||
"- A large `maxhops` will increase the number of parallel jobs, improving dynamic load balance, but it will lead to more communication.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"The optimal value of `maxhops` will depend on the given system, the number of workers, problem size, and also the particular input values. It is not possible to determine it in advance."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -776,6 +802,8 @@
|
||||
"source": [
|
||||
"## Implementation of the parallel algorithm \n",
|
||||
"\n",
|
||||
"We will implement this algorithm using the task-based programming model provided by Distributed.jl as it is convenient to implement the replicated workers model.\n",
|
||||
"\n",
|
||||
"First, let's add our worker processes. "
|
||||
]
|
||||
},
|
||||
@@ -937,7 +965,7 @@
|
||||
"\n",
|
||||
"### Simplified example\n",
|
||||
"\n",
|
||||
"We will demonstrate how the workers communicate the minimum distance with each other with a short example. Each worker generates a random value and updates a globally shared minimum. The variable for the global minimum is stored in a `RemoteChannel`. The buffer size of the channel is 1, such that only one channel can take and put new values to the channel at a time."
|
||||
"We will demonstrate how the workers communicate the minimum distance with each other with a short example. Each worker generates a random value and updates a globally shared minimum. The variable for the global minimum is stored in a `RemoteChannel`. The buffer size of the channel is 1, such that only one worker can take and put new values to the channel at a time, thus solving the race condition problem."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1087,7 +1115,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Testing the parallel implementation\n",
|
||||
"Next, we will test the correctness and performance of our parallel implementation by comparing the results of the parallel algorithm to the results of the serial algorithm for multiple problem instances. "
|
||||
"Next, we will test the correctness and performance of our parallel implementation by comparing the results of the parallel algorithm to the results of the serial algorithm for multiple problem instances. Run it for different values of `n` and `max_hops`. Try to explain the impact of these values on the parallel efficiency."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1097,12 +1125,13 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"n = 18 # Safe to run up to 18\n",
|
||||
"n = 18 # Safe to run up to 18 on a laptop\n",
|
||||
"using Random\n",
|
||||
"Random.seed!(1)\n",
|
||||
"C = rand(1:10,n,n)\n",
|
||||
"C_sorted = sort_neighbors(C)\n",
|
||||
"city = 1\n",
|
||||
"verbose = false\n",
|
||||
"T1 = @elapsed min_serial = tsp_serial(C_sorted,city)\n",
|
||||
"max_hops = 2\n",
|
||||
"P = nworkers()\n",
|
||||
@@ -1115,6 +1144,50 @@
|
||||
"@test min_serial == min_dist"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "92e68978",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Super-linear speedup\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9a724509",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question:</b> For some values of `n` and `max_hops` the parallel efficiency can be above 100% (super-linear speedup). For example with `n=18` and `max_hops=2`, I get super-linear speedup on my laptop for some runs. Explain a possible cause for super-linear speedup in this algorithm."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "eebe7e9a",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"uncover = false\n",
|
||||
"q_superlinear_answer(uncover)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "19835531",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Summary\n",
|
||||
"\n",
|
||||
"- We studied the solution of the TSP problem using a branch and bound strategy\n",
|
||||
"- The problem is $O(N!)$ complex in the worse case scenario, where $N$ is the number of cities.\n",
|
||||
"- Luckily, the compute time can be drastically reduced in practice using the nearest city first heuristic and branch pruning.\n",
|
||||
"- Pruning, however, introduces load imbalance in the parallel code. To this fix this, one needs a dynamic load balancing strategy as the actual work per worker depends on the input matrix (runtime values).\n",
|
||||
"- A replicated workers model is useful to distribute work dynamically. However, it introduces a trade-off between load balance and communication depending on the value of `maxhops`.\n",
|
||||
"- The parallel code might suffer from positive search overhead (if the optimal route is on the left of the tree) or it can benefit from negative search overhead (if the optimal route is on the right of the tree).\n",
|
||||
"- In some cases, it is possible to observe super-linear speedup thanks to negative search overhead.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c789dc7a",
|
||||
@@ -1130,15 +1203,15 @@
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Julia 1.9.1",
|
||||
"display_name": "Julia 1.10.0",
|
||||
"language": "julia",
|
||||
"name": "julia-1.9"
|
||||
"name": "julia-1.10"
|
||||
},
|
||||
"language_info": {
|
||||
"file_extension": ".jl",
|
||||
"mimetype": "application/julia",
|
||||
"name": "julia",
|
||||
"version": "1.9.1"
|
||||
"version": "1.10.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
Reference in new issue
Block a user