mirror of
https://github.com/fverdugo/XM_40017.git
synced 2026-11-17 15:37:12 +01:00
6 files changed
+27635
-166
No files matched your search
+1
-1
@@ -123,8 +123,8 @@ makedocs(;
|
||||
"Matrix-matrix multiplication"=>"matrix_matrix.md",
|
||||
"Jacobi method" => "jacobi_method.md",
|
||||
"All pairs of shortest paths" => "asp.md",
|
||||
# "Solutions" => "solutions.md",
|
||||
],
|
||||
"Solutions" => "solutions_for_all_notebooks.md",
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# Solutions
|
||||
|
||||
## Julia Basics
|
||||
|
||||
### NB1-Q1
|
||||
|
||||
In the first, line we assign a variable to a value. In the second line, we assign another variable to the same value. Thus,we have 2 variables associated with the same value. In line 3, we associate `y` to a new value (re-assignment). Thus, we have 2 variables associated with 2 different values. Variable `x` is still associated with its original value. Thus, the value at the final line is `x=1`.
|
||||
|
||||
### NB1-Q2
|
||||
|
||||
It will be `1` for very similar reasons as in the previous questions: we are reassigning a local variable, not the global variable defined outside the function.
|
||||
|
||||
### NB1-Q3
|
||||
|
||||
It will be `6`. In the returned function `f2`, `x` is equal to `2`. Thus, when calling `f2(3)` we compute `2*3`.
|
||||
|
||||
|
||||
### Exercise 1
|
||||
|
||||
```julia
|
||||
function ex1(a)
|
||||
j = 1
|
||||
m = a[j]
|
||||
for (i,ai) in enumerate(a)
|
||||
if m < ai
|
||||
m = ai
|
||||
j = i
|
||||
end
|
||||
end
|
||||
(m,j)
|
||||
end
|
||||
```
|
||||
|
||||
### Exercise 2
|
||||
|
||||
```julia
|
||||
ex2(f,g) = x -> f(x) + g(x)
|
||||
```
|
||||
|
||||
### Exercise 3
|
||||
|
||||
```julia
|
||||
using GLMakie
|
||||
max_iters = 100
|
||||
n = 1000
|
||||
x = LinRange(-1.7,0.7,n)
|
||||
y = LinRange(-1.2,1.2,n)
|
||||
heatmap(x,y,(i,j)->mandel(i,j,max_iters))
|
||||
```
|
||||
|
||||
## Asynchronous programming in Julia
|
||||
|
||||
|
||||
### NB2-Q1
|
||||
|
||||
Evaluating `compute_π(100_000_000)` takes about 0.25 seconds. Thus, the loop would take about 2.5 seconds since we are calling the function 10 times.
|
||||
|
||||
### NB2-Q2
|
||||
|
||||
The time in doing the loop will be almost zero since the loop just schedules 10 tasks, which should be very fast.
|
||||
|
||||
### NB2-Q3
|
||||
|
||||
It will take 2.5 seconds, like in question 1. The `@sync` macro forces to wait for all tasks we have generated with the `@async` macro. Since we have created 10 tasks and each of them takes about 0.25 seconds, the total time will be about 2.5 seconds.
|
||||
|
||||
### NB2-Q4
|
||||
|
||||
It will take about 3 seconds. The channel has buffer size 4, thus the call to `put!`will not block. The call to `take!` will not block neither since there is a value stored in the channel. The taken value is 3 and therefore we will wait for 3 seconds.
|
||||
|
||||
### NB2-Q5
|
||||
|
||||
The channel is not buffered and therefore the call to `put!` will block. The cell will run forever, since there is no other task that calls `take!` on this channel.
|
||||
|
||||
|
||||
+325
-142
@@ -2,172 +2,116 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9c32b051",
|
||||
"id": "46d0dd15",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Solving Linear Equations\n",
|
||||
"<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/39/VU_logo.png/800px-VU_logo.png?20161029201021\" width=\"350\">\n",
|
||||
"\n",
|
||||
"## Serial Algorithm\n",
|
||||
"To demonstrate the algorithm, we will consider a simple system of linear equations $Ax = b$:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 66,
|
||||
"id": "2b369b73",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"A = [1.0 4.0 5.0 8.0 1.0; \n",
|
||||
" 2.0 -1.0 4.0 3.0 0.0; \n",
|
||||
" 7.0 6.0 3.0 -4.0 5.0; \n",
|
||||
" -3.0 4.0 2.0 2.0 2.0; \n",
|
||||
" 0.0 -4.0 2.0 1.0 2.0]\n",
|
||||
"### Programming large-scale parallel systems\n",
|
||||
"\n",
|
||||
"b = [61.0; 24.0; 37.0; 29.0; 12.0];"
|
||||
"# Gaussian elimination"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "53124eb8",
|
||||
"id": "ff0fbd76",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The code in the following cell converts the general problem $Ax=b$ to the upper triangular equation system $Ux=y$. Note that this function assumes that the pivots are all nonzero. This function will be erroneos if any of the diagonal entries are zero!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 67,
|
||||
"id": "7a7b926a",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"convert_to_upper_triangular! (generic function with 1 method)"
|
||||
]
|
||||
},
|
||||
"execution_count": 67,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"function convert_to_upper_triangular!(A,b)\n",
|
||||
" n = size(A,1)\n",
|
||||
" # Upper Triangularization: convert Ax=b to Ux=y\n",
|
||||
" for k in 1:n\n",
|
||||
" for j in k+1:n\n",
|
||||
" # Divide by pivot\n",
|
||||
" A[k,j] = A[k,j] / A[k,k]\n",
|
||||
" end\n",
|
||||
" b[k] = b[k] / A[k,k]\n",
|
||||
" A[k,k] = 1\n",
|
||||
" # Substract lower rows\n",
|
||||
" for i in k+1:n \n",
|
||||
" for j in k+1:n\n",
|
||||
" A[i,j]=A[i,j] - A[i,k] * A[k,j]\n",
|
||||
" end\n",
|
||||
" b[i] = b[i] - A[i,k] * b[k]\n",
|
||||
" A[i,k] = 0\n",
|
||||
" end\n",
|
||||
" end\n",
|
||||
" return A, b #U,y\n",
|
||||
"end\n"
|
||||
"## Contents\n",
|
||||
"\n",
|
||||
"In this notebook, we will learn\n",
|
||||
"\n",
|
||||
"- How to parallelize Gaussian elimination\n",
|
||||
"- How the data partition can create (or solve) load imbalances"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "78ef1849",
|
||||
"id": "8dcee319",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The function in the following cell solves the upper triangular equation system using backwards substitution. Note that the function alters the input values. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 68,
|
||||
"id": "5a134433",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"solve_upper_triangular! (generic function with 1 method)"
|
||||
]
|
||||
},
|
||||
"execution_count": 68,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"function solve_upper_triangular!(U,y)\n",
|
||||
" n = size(U,1)\n",
|
||||
" for step in reverse(1:n)\n",
|
||||
" if U[step,step] == 0\n",
|
||||
" if y[step] != 0\n",
|
||||
" return \"No solution\"\n",
|
||||
" else\n",
|
||||
" return \"Infinity solutions\"\n",
|
||||
" end\n",
|
||||
" else\n",
|
||||
" # Backwards substitution\n",
|
||||
" y[step] = y[step] / U[step,step]\n",
|
||||
" end\n",
|
||||
" for row in reverse(1:step-1)\n",
|
||||
" y[row] -= U[row,step] * y[step]\n",
|
||||
" end\n",
|
||||
" end\n",
|
||||
" return y \n",
|
||||
"end"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 69,
|
||||
"id": "b92332f7",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"5-element Vector{Float64}:\n",
|
||||
" 1.0000000000000009\n",
|
||||
" 1.999999999999999\n",
|
||||
" 2.9999999999999964\n",
|
||||
" 4.000000000000002\n",
|
||||
" 5.000000000000005"
|
||||
]
|
||||
},
|
||||
"execution_count": 69,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"U,y = convert_to_upper_triangular!(A,b)\n",
|
||||
"sol = solve_upper_triangular!(U,y)"
|
||||
"## Gaussian elimination\n",
|
||||
"\n",
|
||||
"[Gaussian elimination](https://en.wikipedia.org/wiki/Gaussian_elimination) is provably one of the first algorithms you have learned to solve linear equations like this one:\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\left[\n",
|
||||
"\\begin{matrix}\n",
|
||||
"1 & 3 & 1 \\\\\n",
|
||||
"1 & 2 & -1 \\\\\n",
|
||||
"3 & 11 & 5 \\\\\n",
|
||||
"\\end{matrix}\n",
|
||||
"\\right]\n",
|
||||
"\\left[\n",
|
||||
"\\begin{matrix}\n",
|
||||
"x \\\\\n",
|
||||
"y \\\\\n",
|
||||
"z \\\\\n",
|
||||
"\\end{matrix}\n",
|
||||
"\\right]\n",
|
||||
"=\n",
|
||||
"\\left[\n",
|
||||
"\\begin{matrix}\n",
|
||||
"9 \\\\\n",
|
||||
"1 \\\\\n",
|
||||
"35 \\\\\n",
|
||||
"\\end{matrix}\n",
|
||||
"\\right]\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"The value of $x$, $y$, and $z$ can be found by creating an *augmented matrix* and applying Gaussian elimination to it:\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\left[\n",
|
||||
"\\begin{matrix}\n",
|
||||
"1 & 3 & 1 & 9 \\\\\n",
|
||||
"1 & 2 & -1 & 1 \\\\\n",
|
||||
"3 & 11 & 5 & 35 \\\\\n",
|
||||
"\\end{matrix}\n",
|
||||
"\\right]\n",
|
||||
"\\rightarrow\n",
|
||||
"\\left[\n",
|
||||
"\\begin{matrix}\n",
|
||||
"1 & 3 & 1 & 9 \\\\\n",
|
||||
"0 & -1 & -2 & -8 \\\\\n",
|
||||
"0 & 2 & 2 & 8 \\\\\n",
|
||||
"\\end{matrix}\n",
|
||||
"\\right]\n",
|
||||
"\\rightarrow\n",
|
||||
"\\left[\n",
|
||||
"\\begin{matrix}\n",
|
||||
"1 & 3 & 1 & 9 \\\\\n",
|
||||
"0 & 1 & 2 & 8 \\\\\n",
|
||||
"0 & 0 & 1 & 4 \\\\\n",
|
||||
"\\end{matrix}\n",
|
||||
"\\right]\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"The result is an upper diagonal matrix with ones on the diagonal. From this matrix the values $x$, $y$, and $z$ can be found via [backward substitution](https://en.wikipedia.org/wiki/Triangular_matrix).\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "962ec4a9",
|
||||
"id": "94c10106",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We can test if the obtained solution is correct using `@test`:"
|
||||
"### Serial implementation\n",
|
||||
"\n",
|
||||
"Gaussian elimination can be implemented as shown in the following function. Note that the result is overwritten in-place on the input matrix."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 70,
|
||||
"id": "0e336c85",
|
||||
"id": "e4070214",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"\u001b[32m\u001b[1mTest Passed\u001b[22m\u001b[39m"
|
||||
"gaussian_elimination! (generic function with 2 methods)"
|
||||
]
|
||||
},
|
||||
"execution_count": 70,
|
||||
@@ -176,22 +120,261 @@
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"using Test\n",
|
||||
"@test sol ≈ [1.0; 2.0; 3.0; 4.0; 5.0]"
|
||||
"function gaussian_elimination!(B)\n",
|
||||
" n,m = size(B)\n",
|
||||
" @inbounds for k in 1:n\n",
|
||||
" for t in (k+1):m\n",
|
||||
" B[k,t] = B[k,t]/B[k,k]\n",
|
||||
" end\n",
|
||||
" B[k,k] = 1\n",
|
||||
" for i in (k+1):n \n",
|
||||
" for j in (k+1):m\n",
|
||||
" B[i,j] = B[i,j] - B[i,k]*B[k,j]\n",
|
||||
" end\n",
|
||||
" B[i,k] = 0\n",
|
||||
" end\n",
|
||||
" end\n",
|
||||
" B\n",
|
||||
"end"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cb7f89e5",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let us test the function with the example above:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "28dff449",
|
||||
"execution_count": 68,
|
||||
"id": "eb30df0d",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"3×4 Matrix{Float64}:\n",
|
||||
" 1.0 3.0 1.0 9.0\n",
|
||||
" 1.0 2.0 -1.0 1.0\n",
|
||||
" 3.0 11.0 5.0 35.0"
|
||||
]
|
||||
},
|
||||
"execution_count": 68,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"A = Float64[1 3 1; 1 2 -1; 3 11 5]\n",
|
||||
"b = Float64[9,1,35]\n",
|
||||
"B = [A b]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 71,
|
||||
"id": "52bfada2",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"3×4 Matrix{Float64}:\n",
|
||||
" 1.0 3.0 1.0 9.0\n",
|
||||
" 0.0 1.0 2.0 8.0\n",
|
||||
" 0.0 0.0 1.0 4.0"
|
||||
]
|
||||
},
|
||||
"execution_count": 71,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"gaussian_elimination!(B)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "563aa0d0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We get the same result as shown before (as expected)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "39f2e8ef",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Parallelization\n",
|
||||
"\n",
|
||||
"Gaussian elimination is expensive and thus it makes sense to try to seed-up this computation by using several processors. It requires $(2n^3)/3$ operation, where $n$ is the number of rows in the input matrix. Thus, the time complexity is $O(n^3)$, which rapidly grows with $n$.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1b1a6469",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Where can we extract parallelism?\n",
|
||||
"\n",
|
||||
"```julia\n",
|
||||
"n,m = size(B)\n",
|
||||
"for k in 1:n\n",
|
||||
" for t in (k+1):m\n",
|
||||
" B[k,t] = B[k,t]/B[k,k]\n",
|
||||
" end\n",
|
||||
" B[k,k] = 1\n",
|
||||
" for i in (k+1):n \n",
|
||||
" for j in (k+1):m\n",
|
||||
" B[i,j] = B[i,j] - B[i,k]*B[k,j]\n",
|
||||
" end\n",
|
||||
" B[i,k] = 0\n",
|
||||
" end\n",
|
||||
"end\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"- The loop over k cannot be parallelized (the state of B at iteration k depends on the state at iteration k-1)\n",
|
||||
"- Loops over t, i, and j can be parallelized\n",
|
||||
"- BUT the loop over t needs to be done before loop over i and j"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a67e0aad",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Data partition\n",
|
||||
"\n",
|
||||
"Let us start considering a simple 1D block partition. We assume that each process contains only a portion of the input matrix consisting in a block of consecutive rows. In this algorithm, the data stored in a process does not correspond with the data updated at a given iteration over the outer loop over k. The data updated is only a subset of the data stored, which leads to load imbalances (as we will see in a second).\n",
|
||||
"\n",
|
||||
"Consider next figure. Let's find out which is the data updated and the data used by CPU 3 at iteration $k$.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g20491.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABS8AAAJNCAYAAADQ04CWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13uCRlmbDx+0xgYAYYcnLIGQQERIwY1xyWNec1YN5VV1fd/VDXtLrmHBFRMeKacA0YMICCogSVICBIziBxcn9/PKc5VW9XV1f16e6qOdy/6+oLuqfCc6qrqt963gSSNH4fBjqZ1yOaDecu6WZmjv8VE9zvEuBs4Ibp14sGLP8n8ufKwrFGp3Hakvx3+atmw1mnPIn8sXv3GPbxVWauy8+NYfuSpPbaADg489plwvs/ILPvrUqWm08+zr3HH5rGaBPy3+eyZsNZpywjf+w2GfH210+2v3jE25+1BU0HILYEPlZhuTuI5MMtwI3AX4DTgUvGF5okzdqRzBQ0LwI+32Askma8C3gyMA94LnAM8IsmA5IkTcyuwGmZ9/9LVJxNwsOBH03//0pgH+CaPssuIR/nacAh4wtNY3YY8J3M+/cAr2solnXNq4DXZN4/EfjmCLe/EvgMcOD0+/cn+2ucycvmLSYeHoZ1HXAKceJ+g0huttVGwP/LvP8b8ImGYpEADgcOzbz/LHB+Q7HMRXsA/5Z5/yZgxYj38d9E8gXgJiIho9F4C7Bo+v9vA97WYCwavTOArwFPB6aAjxAF1jVNBiXVdCDw0wHLLGemAcBNRCOA84A/TL/+Ns4AJeWsT77hzkeBCxuKRdKMtUSu5vvT7/+V6Jnzp8YiUuvsSL5r2GxeNxEPl0sm+hdUty35eE9uNhxNUFu7jX+SfFwPbzacsWqi2/hXM/u8kGoVZnW7ja/OLOsD6Ghlz5lrh1jfbuPDm0S3cYD9icJqdz/PGdN+pHE5hNmXny8nyil3n3Dsde0C/Djzekuz4Wgdd3fy18E3JrTfN2T2eQuw+YDlNyYf5+8GLL8n+evk/5Uvrhp2In9sh6nUfjyTKd/MRe8lf+z+aUz7OSmzjx+PaR9DseVlO50FrEo+mwcsBTad/u+8dKXpz48kHj6eBvxmjDFKUpl7kG9V/h4i0TjIZcQ4TF2dUQaliVoD/DXzfpJjraqas4D/Ax47/f7NRKXDysYikiZvO+BfgFcA3yV6DPy1dI1mbAQ8LPP+1qYCkYa0Cfkuwp8Crh/xPpaSv05Gvf27sg3JH9s7mgpEY/VO4HvT//8w4CHAz5oLZ4bJy3Z6NFEL3M8SYhDVewPPJFpOZO0A/JxIHHx3DPFJ0iD/wUwly9+BL1Zc75HjCUcNuIEYU0vt9jFmkpe7EJWfX2guHGnWjiJaFKeWTr+2J8ZiThsCTAFPIB7WjgC+MsYYpbui1xINcSAa6nygwjqrgOMy7+1ivm67gvz3eVZTgaiv7xM94bq9Ed4B3Ke5cNQmRd3G71ZzGw8kxq5Kt7Ocdg1obLfxuy67jTdvkt3GdyIKm939fXSM+7Lb+PjMttu4hjepbuMQCZyLMvs6fYz7kkatqNv4oOFGILqiPoyoWFtesI21wMvGEO9sHEA+xm81G47WcZPuNr4RMd5sd3/HlS8+tHuR/7u+Oqb93BWl54yNpCZrUt3GIX7/svt64Bj3VVlR12Ote35BtML8XPL5IuBLzEy4IEmT8DLyLfuPaSgOSYOtJd8y+h7EbKDSXHYz8BPg2USF23eSf+9OYvUwJI3C84lu412fbioQSQN9Cbg98/7VTQWSZbfxuWM58EJga6LbedfuRCKhSrP8rG2AfYhWoBsRY5fdAFxKtMoY9YzBs7ElsC+wjIi1Q8R6GRHrXXk8jl2IGZ+3IwriVxEDXV8zou1vBxxEtKpdRbToO4/Rt4BbjxiAe1dgK6JVxS3E93wG8V23zTwiCbCMOEcXAlcDFzPTUnq2pojjvwuwBXE8Lp/eflNjYc0HnpV5/zfg9w3FMlsbE8d3G6Kb0+1EC8TfE9/lqC0khgG5G3FtrQLOZXAr9c2ILpA7EeMRzSNaN1w5Heu6Pi7afOCexDHZhvj9uY4Yk25cMyBuSFy/2xMPW38H/kLMTFzUHXVd9w3gjZn3zwV+2VAs0qRdBRxOTOyRnYBiHnA0sBf5hzhJ9WVbMl8J/LSpQCQN9HfgeOCp0+8fR5SJL20sIkxezjVriaTB+eRnbnsl0W13Tcm6i4kxrx4LPIg4OftZTgza+h5ibM1BTiMe/Ocnnx9E+bglPyPGHEqtDzyKmK3sgcDOJdtYQTyAvR/4YYVYZ+vLwKGZ9/ehXqLwHCJRB5GQK2ui/Rjie+36JPGdQIxZ9p/AfgXrrSEKDEcyeMa+fg4jZrk8jOIW3L8nHgDSlgx17Et0nXwo0QWkrAXxRcQDxseIpE2ZZwP/Nf3/WyT/dgzlye5/BP44YPt7ETMpPppIWha5mhhL678ZrjvuQmJCgZcSQ0+kVgBfB95EJEsn6aFEMrvrW9RL1P6QqHTp2pPeiX6+RyTrIH9f2Y7ye8o5zIzt18884vp5EXBfirsfdogk1geJa75qMusUZs6J5cQ5zvRn/w/4Z2I8tqzfE4m7rIVEa6B/JO4Re5bsczUxedsHgW9XiPU44t4MMb5y16aUH9uL6G2htBn5e8xpzBSCqtiDuE89lpkxslKXEOf6O4nkfRXbkE8I/474ziHOoXcCTyF+a1LXEJWBHyS+wzZ6CfDvyWf/DXy2ZJ2zgAuA3abfP4mYvOSuXPmnu5YO8Hbi9+c5mc+3J66FOkM37ETct/ciKjA3JH6XbyDuWb8iyhJl5fKs7v1v4+Tz9eh/b2R6n4OSrsuIMtZexN+6IVFx1m0EcBJRIVplwr1R2Jj87/qgMl3WFPmWfauJiu4q5hNl9oOI++BGxLPRTcBtRFnqXOL3ok5MWfOIOQseQPzWbElMjnY1cDZwAlExN1v7Ec9Jy4jjcSWRcDgR+PMItj+MQ4nf9K5vUb3slH6va4iW0/2W2Sj5t1FcJ10bE+Xc/Yjvb1PiWrmauK5/Tb1rpTshb1d6zs4H7k+cl8uIRkrdpFLZc+2O5O9BGxF/543M3IPOYvb3oIWUH9uVxPWTrrNh5v1y6pc15hO9Th9AlOm2II7LdcS99cfT7+tYj3y5N41rHlHOPYRoZLBken9nAD9idA2DxiU97lD8/WR9g5ly+zxirpV3jT40rUtGMeZl6m0F2yzrArY+caNM16nyOpbiB7ysa4fc9vF9tnfdkNv7Fr0/aKN2YrLP7Wqunx136aIByz412dc7iQTf16h2PFYzk8Sragp4H1HgqLKPzxKVJHXHvHxjxe2nr2uJ5HuZVwy57Q69SaSs9YguZqsK1uv3+jv5Gbmr2J5oUVxl+7cwk6yb1JiXH0tiGJQsTP0pWb8oeVj1709fg8byuzuRlKyzzVPJJ2vLXJlZr1tQfiDl97Q/FGzn7CH//p/Rm7BPnTzkts8r2NaWyTK/GrDvrilicPCVNfZ/A/D0ittflqzbbWH4D8QDSJX9/YbyAvtsDTPm5RTxO5BdbxWRiK/iE8m6jy5fXGqFYce87Gcx8RCa3d7FlA+1NY8oZ39setkq95C/Af/KTIV12baHuSd3KK6wmCKSdB8kKiyqbOcKokJkgwGxjsKfk32nDR/KbJKse2qFdRYSs19fRrVjsXZ6u0+pEdcSooJy0O/LaqICcffizQx0CFFJWLaPE5lp2DDJMS9nM/b9xsm6RQ0vNqDa91f0+niFGPYBvsngcsl1xLUy6Nm4a+dk/W5r1CmiEuWvffaTjiU/j0jkfZT8GNZlr0uIBk5Vhper+tyXvo4p2Nbjk2XqVAxtQEwIOigXsJJoJFLWwCn15GQb/5P5txcSx6vf/lYQ99Rx5xmGHfNyK6IBRXbdvzHTiKKfJcTf1l1nUAMe3QXsSO8FMNvk5bb03mTeWbL84oIYOkQm/m9ETePF9L9h/5jyQt2ok5dFcdxO3FTOJW7aKwqW6RAPnIMKirNxYrK/SScv08TRbURB8I9E7XHRMXlbuuESH+mzjeuJGrxziFqq7L+9i/qFlncX7GMt8UBxPtF9s98P10rgISXbHkfycilR4Cha53ai8HHe9P+n/74GePGA49G17fTfXnRs/kYUWi8lf/3fQXR/nVTyMlvQWk1vS8JBmkpePpj8QPLZ19XE93cZ+QmCstdqlQJSmrw8mOjSXTfmSwuWWz4d37lEC8miCSg6xDWa1rxmNZ28nE/MdN3vWrpg+u8vqiRYSyQDBilKXt6f3t+NbuujfhUS42zNXzd5uR5RmZhd51bqVR48JVl/nBNtSaMy6uQlRMVuus2ymVbTBESd16+I1lT9jDp5udkstvd7yntmjcIkk5ebTS8zzLGoOhP9vYnhfOpsewXVK+O6Xk5x+aTfb+ljmWzyMls2XEG+hdsgTSYvp4jnpKrHtvs6iyhrDFKUvJwPfH7A9j+WbGeHWfz9JxOtF8u0IXm5B1G+rbP/5eRb0pcpSl4uInpYVd3fKeRbCY/aMMnL3emtqDqd6jmKXybr1kkIaw7akd4Tf7bJS4gH1Ow2f1KybDd5eSvRau8ZxNiCU8ly6xPNpb9TEPN/lGx/R2JMvnsn6/x++vN+r3430pXED+83iRvSHvQmT9cjWjQVtUIcZ3PnE5N9TTJ5+cfM/19KdIPM1vzNBx5Jb3KoQ7UH3GcXrHcq0dIxe/yXAM9jpuXC2oJ9Vk1e/pF4iLgfxbVZOxI1nGkriSuIQmmRjZk5x9IfpOdQfk4W1U5OAd9PtrMK+AzxUJUteC8gkmTpebKS/HADRaaI6zi73hqiC+sOybK7EIWxbmHjfPJJ5XElL3dK4hume1KV5OUyZr6TbIHycsq/v34Fyb3oTSJeQiTC0vvx5sRDQtqC4tQ+sWZlk5fLybeg/AHwRCJBvYBI/D2AaBGSupQo/P8fURu8L73DwCwgzqnPEudJNtajS2LcjpnjlT0mN1B+bIseLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "6d8b79ba",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g20491.png\" align=\"left\" width=\"450\"/>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "edb211d0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"By looking into the code above, you can see that `B[i,j]` is updated at iteration k, if and only if i and j are greater or equal than k. Thus, from all entries owned by CPU3, only the ones fulfilling this condition will be updated. See the entries highlighted in the next figure."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g20853.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABRAAAAJNCAYAAABJK1kjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13vCx1efjxz7mNC1x6r9JBlCKiYlQkorFgEjUaNWoSozEkwTTRFFGx/IwtakyiUWNsEWxILETBiqCCIYAoIIiAFOm93rq/P56znpnvzszO7NndmXvu5/167Qt275TnzM7Mfuf5NpCkyfsg0Mu8jmo1mg3TauaO/xVT3O8WwM+A22dfLxmy/JXMxblysqFpwvYmf93/T7vhrFdeRv7YnTCBfXyVuevyvRPYviSp2kbAIzOvfaa8/4My+96xYrkZ8nEeOPnQNEFbkP8+d203nPXKruSP3VZj3v7yZPubjHn787ak7QDE9sC/1ljuAeBu4B7gDuBy4Hzg2smFJknz9ibmCsSXAie3GIukOW8DvjP7/8cBnyDKFZKk6dgdOC/z/jTgmVPa95HEb8AMsAY4GLixZNll5OO8CDhkksFpoh4PfCXz/j3A37QUy/rmL4HjM++fC5wyxu2vAj4MPGL2fee+GxOI7dsUeN481r8FOAf4AnHy3jOOoCZkC+DvM++vIlqmSW35XeCwzPsPEuelxuNg4M8z719LFFLHZQnwlsz7m4F3j3H7G7p/JB4sIFqpvaPFWDR+ZxKtEJ8OLAb+BXhcqxFpfbUfURatspJ8RfhdwGXABUTi+spJBigpZxnw78z9xn+QqOSV1K51wD8Q5TOAVwIfBX7cWkTqnD3Jd1Oaz+sOorXPplP9C+rblXy8Z7Ybjqaoq12YP0Y+riNbjWay2ujC/KXMPn/CXEG1SpMuzMvJf3+XjBypiqxj7tiO8nBvF+bRTaMLM8ATkv381oT2o4Xtocy/DHsj8H7mWl101c7A1zOvt7cbjtZz+5K/Dr5SvfjY/HVmn/cDOwxZfiPycf5oyPJ7k79OTpxHrMrbjfyxfesI2ziG/Pdp5Xt97yR/7H5nQvs5K7OPb0xoHyOxBWI3XUQ87GctIlrwbTX730UF620JvA74feD5wLkTjFGSqjwW+M3M+7cTP4LDXJtZLr0Pav2ymnzisaxrlNpzFnA20Z0J4M3EA+y61iLShmoH4E9nX18lEhyXtRpRsU2AJ2fe1/ldk7pkM6KFU99HgJsmsI/sddLlHnLrm03JH1vLygvTPxJDGgAcPfv6ZnvhzDGB2E1PA26o+PcVxKCaRwAvIgbAzXoI0brvd5g78SRpmrKF01uAz9Rc74kTiEXtuIZohaBu+zfmEogHE+Nvfam9cLRAfJziVuSbExXhuxITMSwuWObpwK8TQ2D856QClDZQfwNsO/v/a4F31VhnHfC5zPtfjDsoTdUN5L/PYS1KNX1fJbot9/M8/4+OJBDVvqIuzDs13MZRxIWfbucB8uO7tc0uzBsuuzC3b5pdmA8g3/11UmPn2YV5subbhVmjm1YXZoiuabdk9vWdCe5LC1NRF+Y6M1OuIBKFHyHKrEVdm189gXjnYx/y8Z3Rbjhaz027C/Mm5O/3k6osOpT83/WFCe1nQ3QADg/Tpml1YYZokd+55+eibrBa/3yHaI348eTz5cBJxEC5kjQtx5Ef7/BjLcUhabiV5GdHfyLRElGatHuBbxMJ890obqn+dhybUxqX32eu9SHAh9oKRNJQJwH3Zd7/dVuBZNmFeeF4APgjYEfgqZnP9weOBd7XcHs7El1LdiHGsVgL3EaMT3YBMcV4V2xPxLorEWuPmDG0H+uD7YXWun2I2RF3Ym6Q8v8lah/HYVeilnFn4iH0BuCnRNfFcdqIqHHbC9gOWErM5ng7cCFw/Zj3Nw6LiWOzKxHzYmKMmauIcU7HMW7SImI4gz2BbYhr9HrimNxXsd4kLQNekHl/Ketvy8AtiVbcO8z+/33EtXMe47uGspYBhxDX687EffZiho9nuw3wMOIBfDMieXs7cS6cTwyQvj5bChxOHJPtiXvNLcDPiPvNJKwgJnPYnehyeSdxLvdb+y80nydm+uv7A+BVLcWiDdOtxG/HueQH9J8BPkz0WrmrhbikheTPMv9/K/C1tgKRNNRdwJeZe656JlEuHfdzdiMmEBeWdcDvEd0Ts11H/hL4V6oHRd+EmPDgmUTz2F0rln2A6IP/Lup1Qz6fGO8mPd8eDfy8Yr2vE8nP1HLgGUSN9BOBPSq2sXI2xnfNbm/SPkskdPoOJ2bHrusK5lpuXU0MmFrmWcA/Zd7/C/De2fVfBPw9kVhNrSFmc3ot8d2M4knEjGqPo7gl8/8Cb2R+Y3AeBDxvdl+Porol7c+JLlDvZ/gDxsuJYwOR2Ms6meqE89OBy4ds/2HA380uu03JMjcAnyIGyL19yPaKLANeA7yCSBqlHgQ+TUyqdN0I25+PZ5D/u09tuP6ZzN1/VhOJ49Q3ies+ndV5H6rvKf8H/O6Q/S8GXkK0iDmC4t/JdcQ5/m5iDJm6CaUfE/daiO/9UbP/vxPRRfUlRAIw60wGuywsIyqKfpu4B+5Tsc/VxCQZ7yaux2GxnsbcMc8e392oPrY/mY0na3eidVHfmURFV10PJ47L04kkXpGriBrad1I/ubAPcHrm/beJ+wLEefVaYiKy9LuAuHbfQdxrulSRlvUa4E8y73vEmKSfrVjnbOBmIkELUZZ4DVF5KE3Te4hrNJvo2J4Yt+0NDbazG3GPPWD2/1cQ98N+BfPZRGVb3XN8S+KemN6LllLdXXsVwyv0dibKxAcQ980Vs3H1K4K+R/x+rakZ63xtTn5syjtpVnGSPR5riQrfOhYBjyHK0fsS9+BNZ/d/P1Euvoz4/b2tQTxZM0Tl0JFEWWPb2RhvJCqJTmc8FYQPI367diOOx43EeXcmUYnchkPJj5v/RZqdU3W+1/4y6XWyjOrrZCX1Kzs3I56PDibK8VsR18pNxLVyNs3+rv5EpX1ryE/6shj4NeKZbheikc1dRNfhquec3Zm7B/UbuawknguvJSYx+xH170Flx3aUe9BS4j7T9yDxfN/EYuJ6fQJxTLYjzolbiDLhGTSv9BkW13JiAplDiO9iUyIR/iMiGX5zw/1N2xIGy5bDfiM+z1wCcRHwYkabeVsLyDjGQEy9tWCbj6tYflOiG0nR2DPDXh8nLuYqt4+47bLEw50NtpF9fW72b52ks5N9blu9+IDsmGPDElUvSfb1RuK7OIV6x2M18aDcxCKiNWs2zqrXB4gfmKZjIL6l5vbT143MTQZQ5vgRt90jEhplls/+nWsabO8OmnfN2ov4Ya6z/buA35hdb1pjIH40ieFJDde/MrNu0QD8ENfGKN/f2UP2fRiR5Guyze8ymIguc09mvX4h56lU39O+W7CdayuWr3r9D/EQXKVoPN06r/MKtrV3wf7rWEwkEZpcSzdT/1o6MFm3P+bUb1H/9+qb5Au54zbKGIiLicrC7HoriWRgHScn6/5as5C1ARt1DMQyy4gWFtnt3UR1ReIMcc6+j6jsqHMdX0e0tB1WjoVIboxyb/x0yfYeRVQCX1ZzOzcRlZ+TLsdC3M+z+y6qTCmzLFm3TrJsCfBXDH7nw35zXtQgro2JSpEbhmx3LfDfFFde1vEI4Jwh+zhrdjmY7hiI6dhtTcqfGyXrFk26sZh6313R6z9qxHAA8Sy3csi2bicq8jeu+bdVjdH/IsrvJx9MtjNDPG//C/mybNXrWqJbap170Kqa20xf/1WwrWOSZd5dsEyZ5cDfkh9Ls+i1mrj/7dVg289JtpGd4OdlxH2wbH+riPt/k/vVKEYdA3E7Bu8NvyAqG6psSiRS++tc3DxkLTSTSCDuwmCC5y0Vy29eEEOPyIb/gugidjXlN66vUj2e5rgTiEXJq/uJgsdPiRYpZT8uZzHZlrdtJxA/nHx2L3Gj+TGRTCo6Jq+rGdsMg4nA/qtf+3Mp+ZtcP66mCcR/LtjHWuKH4/LZ120lsaykOok4iQTi1sS5VXYd/Xw25qIHkDVEd8E6diPO76JjczXRQiFNLt1LJEymkUCcIV84f5D6Bbi+thKITyOf4Mu+biQe8q4njnX675cT991h0gTi4xm8XoruWamihOMDxHf/U+IYlm33fKoLqm0nEJdRXglyH9Ft+Try53P2WqrTwrEogfhUBhOWq4jfr7JE5ufSDY9R0wTiJsRDb3adO4gJKup6RbJ+VblByhp3AhGKf6ufXLH8dgXL1339kOH38HEmENNkTJPXRcSzwyRNM4G4BeXlp2Gv/64Z02HE80yTba8C/rDm9vteTvFvU9HrQSJZMs0E4qWZ/awh3+pumDYTiDNE6+O6x7b/uphoBThMUQJxEdGzqWr76fiROw1Zvup1DtESuUoXEoj7EOWwJvtfCby05vaLEojLZv+Guvv7IfP//akySgJxXwaP24UM/877zkzWbZKU1QI0iQQiDD5gn16xbD+BeA9R0Hkh8eCXdg/cGHgK0Rc/jblqlrw9iBP915J1zp39vOy1Q8n21hEFuc8TSbR9GUxgbkQkqT5fEOubK2KdrzYTiNmWU1cT3X83yiy/hPjBuJTBY/IbDJc+0PaIrgJPIH+urCAKUbfOLrOW+BHPrnfUkH31E4gXAq8nzp2i1j57ELVg/X31X9dQ3uVxC+bOsTRR8QKqz8mi1g+LiNZI2e2sIlpfHka+G9BS4gEoPU8eZK42usyigvVWEwPMp0MO7Eskk/vn08XkkyCTSiCmiZn/HWEbdRKIuxHfR/rQegXV31/ZD/UhDM4AeiUxhEJ6P96OaCmRJrC/Tf67LpJNIN6Z/K1fJLoA7zi7ne2J66RowOQ7iXPmi0Sh7ICCfS8lEpT/xWClS9WYuLsyd7yy611L9bEtevgeJYH4j8k6/cLgU5O/cWtisp60gmo18Ngh+0jP0x9ntnMHcc/ZP7P8UuIeeW5BbE+r8TeNokkCsahW+xqqLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "f18f33da",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g20853.png\" align=\"left\" width=\"450\"/>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1eced940",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Data dependencies\n",
|
||||
"\n",
|
||||
"At iteration k, CPU3 needs part of row k to update its entries. The process owning row k needs to broadcast this entries to all other processes that require them. This communication pattern is almost identical to the one previously studied in Floyd's algorithm."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g21211.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABRAAAAJNCAYAAABJK1kjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13vCV1efjxz91dlmWpCyxSpQoI0lExKqLYUaMRW9QYolGSoFGjRmPB9jOKBltib1jATtBgwYqAiiJFRASRjvTett/fH8893pnvmZkzc+49Z2Yvn/frdV5wzk557pyZOd95vg0kafQ+DkxmXoe0Gs1900qmj/8lY9zvxsAfgVumXi8csPylTMe5fLShacR2In/df6fdcNYqLyZ/7N40gn18l+nr8gMj2L4kqdq6wAGZ1y5j3v9emX1vWbHcBPk49xh9aBqhjcl/n9u2G85aZVvyx27JLG9/UbL9xbO8/Rlb0HYAYgvgv2ssdy9wB3AncCtwMXA2cNXoQpOkGXs70wXiPwAntBiLpGnvBn469f9HAZ8nyhWSpPG4P3BW5v3JwFPGtO+Did+ACWAVsDdwXcmyC8nH+Vtgn1EGp5F6BPB/mffvB17dUixrm1cAr828Pxz4xixufwXwSWC/qfed+25MILZvfeBZM1j/RuCXwDeJk/fO2QhqRDYG3pB5fxnRMk1qy7OB/TPvP06cl5odewP/knn/RqKQOlsWAO/MvL8BOHYWt39f95/EgwVExdV7WoxFs+9UohXik4D5wIeJh4rJNoPSWmlXoixaZTn5ivDbgYuAc4jE9aWjDFBSzkLgY0z/xn8cuLC9cCRNWQP8B1E+A3g58Fng/NYiUufsSL6b0kxetxKtfdYf619Q37bk4z213XA0Rl3twvw58nEd3Go0o9VGF+ZvZfZ5ATCvxjpNujAvIv/9/X7oSFVkDdPHdpjEul2YhzeOLswAj0z289cj2o/mtgcy8zLsdcBHmG510VVbAz/IvKxY0Uw8gPx18H/Vi8+aV2X2eQ9wvwHLr0s+zvMGLL8z+evkrTOIVXnbkT+27xpiG4eR/z6tfK/vGPLH7pkj2s9pmX38cET7GIotELvpt8TDftY8ogXfkqn/Fj2IbwK8Gfg74DnAmSOMUZKqPAx4aub9e4iE1CBXMd0CKr0Pau2yknyrorKuUWrPacDpRMtDgHcA36betSrNpvsB/zT1+i6R4Lio1YiKLQYem3lvi12tbTYkWjj1fBq4fgT7yF4nXe4ht7ZZn/yxtaw8N/0nMaQBwKFTrx+1F840E4jd9ETg2op/34AYVPMg4PnEALhZ2xOt+57J9IknSeOULZzeCHyl5nqPGkEsasdVRCsEddv/MJ1A3ItI/J/UXjiaI46juBX5RkRF+LbERAzzC5Z5EvBoYgiMz4wqQOk+6lXA5lP/vxp4X4111gBfy7y/YraD0lhdS/77HNSiVOP3XaLbci/P8//oSAJR7SvqwrxVw20cQlz46XbuJT++W9vswnzfZRfm9o2zC/Pu5Lu/HjOi/diFebRm2oVZwxtXF2aIrmk34m+zhlfUhbnOzJQbEInCTxNl1qKuza8tXbsdu5CP75R2w9FabtxdmBeTv99/a0T72Zf83/XNEe3nvmh3HB6mTePqwgzRIr9zz891xqNS9/2UaI14XPL5IuB4YqBcSRqXo5gemBsiUSupm5YTZYWeg3F2TY3HXcBPiIT5dhS3VH8P8LRxBiXNYX/HdOtDgE+0FYikgY4H7s68f1VbgWTZhXnuuBf4B2BL4AmZz3cDjgQ+1HB7WxJdS7YhxrFYDdxMdEk7h5hivCu2IGLdloh1EriF6ViXtRda63YhZkfciulByn9N1D7Ohm2JWsatiYfQa4E/AFfO0vZ71iVq3HYClgLrELM53gKcC1wzy/ubDfOJY7MtEfN8YoyZy4hxTmdj3KR5xHAGOwKbEdfoNcQxubtivVFaCDw38/5C1t6WgZsQrbjvN/X/dxPXzlnM3jWUtZBI3GxFXFMriMlnBo1nuxmwJ/EAviGRvL2FOBfOJgZIX5utAxxIHJMtiHvNjcAfifvNKGxATOZwf6LL5W3Eudxr7T/XfB14Reb9i4BXtxSL7ptuIn47ziQ/oP8E8EmiZeztLcQlzSX/nPn/m4DvtRWIpIFuJ8al7j1XPYUol872c3YjJhDnljXA3xLdE7NdR/4V+G+qB0VfTIx79BSieey2FcveS/TBfx/1ujqdTYx3k55vDwH+VLHeD4jkZ2oR8GSiRvpRwA4V21g+FeP7prY3al8lEjo9BxKzY9d1CdMtty4nBkwt83TgvzLvPwx8YGr95wNvIBKrqVXEbE5vJL6bYTyGmFHt4RS3ZP418DZmNgbnXsDhxDF4MNUtaf9EdIH6CIMfMF5CHBuIxF7WCVQnnJ8EXDxg+3sCr59adrOSZa4FvkQMkHvLgO0VWQi8DngpkTRKLQO+TEyqdPUQ25+JJ5P/u09suP6pTN9/VhKJ49SPiOt+Ivl8F6rvKb8Bnj1g//OBFxItYg6i+HdyDXGOH0uMIVM3oXQ+ca+F+N4fPPX/WxFdVF9IJACzTqW/y8JCoqLor4l74C4V+1xJTJJxLHE9Dor1ZKaPefb4bkv1sf0d/TP4bke0kO85lajoqutBxHF5EpHEK3IZcd0eQ/3kwi7A9zPvf0LcFyDOqzcSE5Gl3wXEtXsMca/pUkVa1uuAl2XeTxJjkn61Yp0ziAqO3kyczyO6jq4eRYBShfcT12g20bEFkdA+usF2tiPusbtP/f8GxP2wV8F8OlHZVvcc34S4J6b3onWo7q69gsEVelsTZeLdiYfDDabi6lUEnUH8fq2qGetMbUR+bMrbaFZxkj0eq4kK3zrmAQ8lytEPIO7B60/t/x6iXHwR8ft7c4N4siaIyqGDid+1zadivI6oJPo+s1NBuCfx27UdcTyuI867U4lK5DbsS37c/JNodk7V+V57y6TXyUKqr5Pl1K/s3JB4NtibKMcvIa6V64lr5XSa/V29iUp7VpGf9GU+8FfEM902RCOb24muw1XPOfdn+h7Ua+SynHguvIqYxOw86t+Dyo7tMPegdYj7TM8y4vm+ifnE9fpI4pgsJc6JG4ky4Sk0r/QZFNciYgKZfYjvYn0iEX4ekQy/oeH+xm0B/WXLQb8RX2c6gTgPeAHDzbytOWQ2xkBMvatgmw+vWH59ohtJ0dgzg17HERdzlVuG3HZZ4uG2BtvIvr429beO0unJPjevXrxPdsyxQYmqFyb7ehvxXXyDesdjJfGg3MQ8ojVrNs6q10eJH5imYyC+s+b209d1TE8GUOY1Q257kkholFk09XeuarC9W2neNWsn4oe5zvZvBx4/td64xkD8bBLDYxquf2lm3aIB+CGujWG+v9MH7Ht/IsnXZJs/oz8RXebOzHq9Qs7jqb6n/axgO1dVLF/1+g7xEFylaDzdOq+zCra1U8H+65hPJBGaXEs30p/ALLNHsm5vzKmnUf/36kfkC7mzbZgxEOcTlYXZ9ZYTFYt1nJCs+1fNQtZ92LBjIJZZSLSwyG7veqorEieIc/ZDRGVHnev4auDfGFyOhUhuDHNv/HLJ9h5MVAJfVHM71xMVAaO87/Scley7qDKlzMJk3TrJsgXAK+n/zgf95jy/QVzrEZUr1w7Y7mrgfymuvKxjP+CXA/Zx2tRyMN4xEN+b7KtJ+XPdZN2iSTfmU++7K3p9qkYMuxPPcssHbOsWoiJ/vZp/WzpGf7bc9XzK7ycfT7YzQTxvf5h8WbbqdRXRLbXOPWhFzW2mry8WbOuwZJljC5Ypswj4d/JjaRa9VhL3v50abPtvkm1kJ/h5MXEfLNvfCuL+3+R+NYxhx0BcSv+94QqisqHK+kQitbfOBc1D1lwzigTiNvQneN5ZsfxGBTFMEtnwK4guYpdTfuP6LtXjac52ArEoeXUPUfD4A9EipezH5TRG2/K27QTiJ5PP7iJuNOcTyaSiY/LmmrFN0J8I7L16tT8Xkr/J9eJqmkD8YME+VhM/HBdPvW4uiWU51UnEUSQQNyXOrbLr6E9TMRc9gKwiugvWsR1xfhcdm8uJFgppcukuImEyjgTiBPnC+TLqF+B62kogPpF8gi/7uo54yLuGONbpv19M3HcHSROIj6B84oDsPStVlHC8l/ju/0Acw/Q67L3Oprqg2nYCcSHllSB3E92WryZ/PmevpRfX2EdRAvEJ9CcsVxC/X2WJzK+lG55FTROIi4mH3uw6txITVNT10mT9qnKDlDXbCUQo/q1+bMXySwuWr/v6FYPv4bOZQEyTMU1ev6XZA/kwxplA3IhI2AxzLP63Zkz7E88zTba9Avj7mtvveQnFv01Fr2VEsmScCcQLM/tZRb7V3SBtJhAniNbHdY9t73UB0QpwkKIE4jyiZ1PV9tPxI7casHzV65dES+QqXUgg7kKUw5rsfzlwRM3tFyUQF079DXX39ytm/vtTZZgE4gPoP27nMvg77zk1WXfUvwHquFEkEKH/Afv7Fcv2Eoh3EgWd5wE70989cD3gcURf/DTmqlnydiBO9L9K1jlz6vOy1/0KtgWRZLuHaNL7QuKiTBOY6xJJqq8XxPqOilhnqs0EYrbl1OXAs4jj0LOA+MH4Pf3H5PEMlj7QThJdBR5J/lzZgChE3TS1zGriRzy73iED9tVLIJ4LvIU4d4pq3XcgasF6++q9rqS8y+PGTJ9jaaLiuVSfk0WtH+YRrZGy21lBtL7cn3w3oHWIB6D0PFnGdG10mXkF660kBphPhxx4AJFM7p1PF5BPgowqgZgmZn49xDbqJBC3I76P9KH1Eqq/v7If6n3oT+RdSgyhkN6PlxItJdIE9k/If9dFsgnE25K/9SSiBd2WU9vZgrhOigZMvo04Z04iCmW7F+x7HSJB+UX6K12qxsTdlunjlV3vKqqPbdHD9zAJxP9M1ukVBp+Q/I2bEpP1pBVUK4GHDdhHep6en9nOrcQ9Z7fM8usQ98gzC2J7Yo2/aRhNEohFLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "410eb1ba",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g21211.png\" align=\"left\" width=\"450\"/>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d04f70fa",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Load imbalance\n",
|
||||
"\n",
|
||||
"Do all processes process the same amount of data at a given iteration? This answer is no. To understand this let us find out the data updated by CPU1 in the figure below. At iteration k, CPU 1 has not data to process since k is larger than all row ids that CPU 1 owns. This is in contrast to CPU 2 and CPU 3 which have data to process. As a result, CPU 1 is idle (doing nothing), which is waisting computational resources that could be otherwise used."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g23603.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABS8AAAJNCAYAAADQ04CWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N133Cxlefj/z55z6OVQFfEoTRBFQUDsYo29Ye/GgkY00fhLlChKLIl+kxh7ibGLlSj2rmBDUJSigqhI7+3QOXV/f1y7PDP3zs7ObJs5D5/367Uvzi5Trmd3yj3X3UCSZu+9QDfzemSz4dwqXcvC93/hHPe7BXA6cFXv9dIRy/+e/LGy0Uyj0yztSP63/Fmz4WxQnkr+u/uPGezjCyycl5+YwfYlSe21GXBg5rXHnPe/X2bftylZbin5OO8y+9A0Q9uQ/z1XNBvOBmUF+e9umylvf9Nk+5tPefsTW9Z0AGJH4AMVlruRSD5cB6wE/gT8Fjh/dqFJ0sSOYKGgeTbwqQZjkbTgHcDTgCXAC4BPAj9pMiBJ0tzsAZyUef8V4Clz2vcjgO/1/r0auCtw2ZBltyAf50nAQbMLTTN2MPC1zPv/BF7bUCwbmlcD/1/m/VOI83ZaVgP/C+zfe//fyf4aZ/KyeZsTDw/jugI4gThw/49IbrbVVsAbMu/PBT7UUCwSwCHAvTPvPwb8uaFYFqO9gNdk3r8JWDXlffw7kXyBqNh5x5S3f2v2ZmCT3r9vAN7aYCyavlOALwLPAjrA+4gC67omg5Jq2h/40YhlbibfAOBq4EyiEcBvifKopPnYlHzDnQ8AZzUUi6QF64lczbd77/+B6Jnz+8YiUuvsQr5r2CSvlcTD5RZz/Ququx35eH/RbDiao7Z2G/8w+bge0Ww4M9VEt/EvZPZ5FtUqzOp2G1+bWdYH0OnKHjOXj7G+3cbHN49u4wD7EoXV/n6eP6P9SLNyEJOXny8kyil3m3Psde0O/CDzenOz4WgDdzfy58GX57TfwzP7vA7YfsTyW5OP89cjlr8z+fPkDeWLq4ZdyX+341RqP4H5lG8Wo/8i/909eUb7+XlmHz+Y0T7GYsvLdjoNWJN8tgRYDmzb+++SdKXe50cQDx/PBH45wxglqcw9yLcq/08i0TjKBcQ4TH3daQaluVoH/DXz/qKmAtFQpwHfAh7Xe38kUemwurGIpPnbGfh74JXA14keA38tXaMZWwEPz7y/vqlApDFtQ76L8EeAK6e8j+Xkz5Npb//WbEvy3+1NTQWimXo78M3evx8OPBT4cXPhLDB52U6Pobxl1JbAAcB9gOcQLSey7ggcRyQOvj6D+CRplH9hoZLlGuAzFdd71GzCUQOuYv4TAKi+D7CQvNydqPz8dHPhSBP7KNGiOLW897oDMRZz2hCgAzyReFg7FPj8DGOUbo3+iWiIA9FQ578rrLMGODrz3i7mG7aLyP+epzUViIb6NtETrt8b4d+A+zYXjtqkqNv47Wtu40HE2FXpdm6mXQMa22381stu482bZ7fxXYnCZn9/75/hvuw2PjuTdhvX+ObVbRwigXN2Zl8nz3Bf0rQVdRsfNdwIRFfUhxMVazcXbGM9cNgM4p3EfuRjPKbZcLSBm3e38a2I8Wb7+zu6fPGx3Yv83/WFGe3n1ig9ZmwkNV/z6jYOcf/L7utBM9xXZUVdj7Xh+QnRCvMTyeebAJ9lYcIFSZqHw8i37P9kQ3FIGm09+ZbR9yBmA5UWs2uBHwLPIyrcvpb8//4kVn8z37CkRetFRLfxvo80FYikkT4L3Jh5/49NBZJlt/HF42bgJcBtiW7nfXsSiYR31dzeTsBdiVagWxFjl10FnE+0ypj2jMGT2BHYB1hBxNolYr2AiPXWPB7H7sSMzzsTBfFLiIGuL5vS9m9PzPR5O6Kl3UXEDJ7TbgG3MTEA9x7AbYhWFdcRv/MpxG/dNkuIJMAK4hjdCLiUaOF0KtMZy7FDDCGxO7AD8X1cSHwnTY2FtRR4bub9ucBvGoplUlsT3+9ORDenG4kWiL8hfstp24gYBuT2xDm7Bvgjo1upb0d0gdyVGFZkCdG64eJerBv6uGhLgXsS38lOxP3nCmJMulnNgLglcf7egXjYugb4EzEzcVF31A3d/wFvzLx/AfDThmKR5u0S4BBiYo/sBBRLgI8T5Y8bC9aTVF22JfPFwI+aCkTSSNcA3wCe0Xv/eKJMfH5jEWHycrFZTyQN/kx+5rZXEd1215Wsuzkx5tXjgAcTB+cwNxODtv4nMbbmKCcRD/5Lk88PoHzckh8TYw6lNgUeTcxW9iBgt5JtrCIewN4JfK9CrJP6HHDvzPv7Ui9ReAaRqINIyJU10X4s8bv2fZj4TSDGLHs9cPeC9dYRBYYjGD1j3zAHA28BHkhxC+7fEA8AaUuGOvYhuk4+jOgCUtaC+GziAeMDRNKmzPOAf+39e4fk/32S8mT3k4Dfjdj+3sRMio8hkpZFLiXG0vp3xuuOuxExocDLiaEnUquALwFvAs4ZY/uTeBiRzO47hnqJ2u8SlS59d2Zwop9vEsk6yF9Xdqb8mnIGC2P7DbOEOH9eCtyP4u6HXSKJ9W7inK+azDqBhWPiZuIYp/fZG4C/JcZjy/oNkbjL2ojo7vgk4hpx55J9riUmb3s38NUKsR5NXJsBtsh8vi3l3+3Z5Adxh0iqZq8xJ7FQCKpiL+I69TgWxshKnUcc628nkvdV7EQ+Ifxr4jeHOIbeDjyduNekLiMqA99N/IZt9HfAPyef/TvwsZJ1TgP+Atyp9/6pxOQlt+bKP926dIG3Efef52c+X0FM5vP/amxrV+K6vXdv/S2J+/JVxDXrZ0RZoqxcntW//m2dfL4xw6+N9PY5Kum6gihj7U2U/bckKs76jQB+TlSIVplwbxq2Jn9fH1Wmy+qQb9m3lqjormIpUWY/gLgObkU8G60EbiDKUn8k7hd1YspaAhxIlJ13Ju79q4ky4enA94mKuUndnXhOWkF8HxcTCYfjmF2F3yj3Ju7pfcdQveyU/q7riJbTw5bZKvl/0zhP+rYmyrl3J36/bYlz5VLivD6eeudKf0LevvSYXQo8gDguVxCNlPpJpe+WbHcX8tegrYi/82oWrkGnMfk1aCPKv9vVxPmTrrNl5v3N1C9rLCV6nT6QKNPtQHwvVxDX1h/03texMflybxrXZkQ5t9/IYIve/k4lfotpNQyalWUMnhtFv0/W/7FQbl9C5JnePv3QtCGZxpiXqbcVbLOsC9hmRMucdJ0qr6MofsDLunzMbX9jyPauGHN7x5C/WM7Csck+d665fnbcpbNHLPuMZF9vJxJ8X6Ta97GWhSReVR1icO31FffxMeJiWXfMyzdW3H76upxIvpd55Zjb7jKYRMramBjbcU3BesNe15CfkbuKOxAtiqts/zoWknXzGvPyA0kMo5KFqd8n6xclD6v+/elr1Fh+dyOSknW2eSL5ZG2ZizPr9QvKD6L8mlbUavX0Mf/+HzOYsE/9Ysxtn1mwrR2TZX42Yt99HWJw8NU19n8V8KyK21+RrNtvYfg3xANIlf39kvIC+6TGGfOyQ9wHsuutIRLxVXwoWfcx5YtLrTDumJfDbE48hGa3dy7lQ20tIcrZHyCSXFWuIecC/8BChXXZtse5JncprrDoEEm6dxMVFlW2cxFRIbLZiFin4Q/JvtOGD2W2SdY9scI6GxF/2wVU+y7W97ZbpyJuc6KCctT9ZS1Rgbhn8WZGOoioJCzbx7EsNGyY55iXk4x9v3WyblHDi82o9vsVvT5YIYa7Al9hdLnkCuJ4GvVs3Ldbsn6/NWqHaGzx1yH7SceSX0Ik8t5Pfgzrstd5RAOnKsPLVX3uS1+fLNjWE5Jl6ozpvRnRQGRULmA10UikrIFTKi13ZSusXsLgfSHd33sYTA5O27hjXu5INKDIrnsuC40ohtmCSHr312mq8kMtsguDJ8CkycudGbzIlGXJtyiIoUtk4s8lahrPYfgF+weUF+qmnbwsiuNG4iL8R+KivapgmS7xwDmqoDiJY5P9zTt5mSaObiAKgr8jao+LvpO3phsu8b4h27iSqME7g6ilyv6/d1C/0PIfBftYT9w4/kx03xx241oNPLRk27NIXi4nChxF69xIFD7O7P07/f/rgJeN+D76btf724u+m3OJQuv55M//m4jur/NKXmYLWmsZbEk4SlPJy4eQH0g++7qU+P0uID9BUPZcrVJASpOXBzK64qgo5vMLlru5F98fiRaSRRNQdIlztKwSp+nk5VJiputh59Jfen9/USXBeiIZMEpR8vIBDN43+q2PhlVIlLV6mFTd5OXGRGVidp3rqVd58PRk/VlOtCVNy7STlxAVu+k2y2ZaTRMQdV4/I1pTDTPt5OW2E2zvN5T3zJqGeSYvt2Xwgb7qq+okMPcmylx1tr2K6pVxfa+guHwy7F76OOabvMyWDVeRb+E2SpPJyw7xnFT1u+2/fkeUNUYpSl4uBT41YvsfSLZzhwn+/l8QrRfLtCF5uRfVK1z6r5vJt6QvU5S83IToYVV1fyeSbyU8beMkL/cknp2z651C9RzFT5N1d68XshabXRg88CdNXkI8wGa3+cOSZfvJy+uJVnvPJsYW7CTLbUo0l/56Qcz/UrL9XYgD/T7JOr/pfT7sNexCupq48X6FuCDtxWDydGOiRVNRK8R3lMQ6qWOTfc0zefm7zL/PJ7pBZmv+lgKPYjA51KXaA+7zCtY7kWjpmP3+twBeyEIN1fqCfVZNXv6OeIi4P8W1WbsQNZxpbdhFRJfVIluzcIylN6TnU35MFtVOdoBvJ9tZA/wv8VCVLXgvI5Jk6XGymvxwA0U6xHmcXW8d0YX1jsmyuxOFsX5h48/kk8qzSl7umsT3hzG2USV5uYKF3yRboLyQ8t9vWEFybwaTiOcRibD0erw98ZCQtqA4cUisWdnk5c3kW1B+B3gKkaBeRiT+Hgi8tmA75xOF/28RtcH7MDgMzDLimPoYcZxkY/14SYw7s/B9Line truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "3988b2ae",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g23603.png\" align=\"left\" width=\"450\"/>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c1957040",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Addressing the load imbalance\n",
|
||||
"\n",
|
||||
"For this algorithm, a cyclic distribution mitigates the load imbalance.\n",
|
||||
"\n",
|
||||
"- Cyclic partitions are often useful in algorithms with *predictable* load imbalance\n",
|
||||
"- A cyclic partition is a form of *static* load balancing\n",
|
||||
"- Cyclic partition are not suitable for all communication patters. E.g., algorithms that require communication between nearest-neighbors like Jacobi."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {
|
||||
"g23933.png": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABL4AAAI1CAYAAADYRTMtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N13nB1V3cfx79yyfTeFFBICREoISEcQgyFggAQjIoYAtoCAIFIUIYB0EDQ+BFFUiiI8CfKoIE0REanSQ9UQSkCSkJCerXfbbfP8cbKyuXtuttzdOyeTz/v1mhdk9s7Mb+49O/fsd2bOSAA2V+Mk+TnTnwKoo8ZSxxMB1LE5OVZd37MrAq0IAAAAYTZHXfufk3q47EOWZbcegBqBARELugBscaol7StphOVnL0ha3k/bmSapopfLNEmql7Ra0gpJ7f1UCwAAAMKjStLekkZZfvaypCX9tJ0pMicYeyOhjfuzbf1UCwBstgi+MJA6OgWfkrTfhv+OkxTJ8/rjJN3TT9u+SdJ2BSyflbRU0tsygdw/JT0vKV14aQAAANhMVGjj/ux+ksZLiuZ5/cmS7uinbf9U0m4FLO/L9GffkenPPiPpWUmpwksDgM0HwRf621hJV8l0DnZR/k6B6yKSPrFh+vyGeetkbiW8SdKCgOoCAADAwBot6VqZkGtXbb5/M3kyffOxkqZumFcr6X6Z/uxrgVQFAEWW78oboK/GSpopc3Zqcw298hkm6duS/iVzn/suwZYDAACAATBa0kmS9tDmG3rlM1TSKZJelfSopN2DLQcABh7BF9B7nswYYv+WdMGGfwMAAACbk8MlvS7pSvF3IYAQ4wCHYkrIjC3wM5lBN4vtVJmQyjbFJG0laUdJB8pc2XW7pIWbWF+JpJ9I+oOk+IBVDQAAAFc0S3pO0o2S1gSw/e8pf382KnNF146SDpD0LUm3yZyszScm82TpBySVDljV2NydINO2Ok9BtH+gT8J26S7ckZA5g/Rqp+ldmUHjJenLkgYHU5pVRmbMg1pJH0h6SdKtG342XtIZMp2Hcsuyx2347wkyg4gCAABg89ci6Q1t3J99W6bfKEmHyf6k8qBkJdVtmD6QecLkbRt+tpOk02X6tJWWZY+SGcv2S/p4/4AOiaALAApB8IX+9m+Z8b06h1ybu3ckfVfSDTKdh8mW1xwnE/TNLmJdAAAA6H/vyYzv1Tnk2ty9L2mWzJ0Xv9bHD2/q7AsyD6m6tIh1AcCA41ZH9LdamU5CWEKvzpZImiLTWbC5WubJPwAAANh8NUh6U+EJvTr7SObqrhvy/PwHkvYtXjkAMPC44gvonYzMJeJDJM3I+Vlc0nUyZ8tcNFrSzjJjmbVLWi7pQ5nL4V3lSdpW0khJw2WOWWs3TEslpYpUR6mk7WTeu602/LtR0mpJi2TeTwAAgM1BVtJ5Mv3Zk3J+FpH0U0mHFLekbg2TtLU+7ou1S2qS6Q8uDbCugTRcpg+8JezzUEmj9PG+pmT29UOZiw8YTgYFIfgCei8r6TRJEyRtk/Ozz8vc6vlWsYvKY5Skc2RuxdzB8vOspH9K+j9JcyUli1daXhGZ8PBoSUfK7INNo6R/SHpI0h8ltfZzHZ+QCTcPk/RZ2cd3k0wYukjS45IekfR3Sel+rmVTviLpGMv8x/XxOHUAAACd+ZLOkjRRZjD8ziZJ2l9mjLCgRGT6YF/Y8N9N3VWRkDRfpl94v8yQKza/kAmSOrtE5tbWQl2lrjVeLzNucE9FZJ60OU3d73OTzD4/Juk+mb7oQLpIXa8EPE19f2CZJxOuHiWzr7tvmGfTLOkVmX19QOZqTADYLCyV+cLtPOVeQdXf6z+lH9cvmadE5m7DV/5Lx/vbOMu2/9Tp59+R+VK01WibFkj6dB/qqLGs64k+rEcyHa2Fvai5Y1opaWYft5lrjKS7ZMKr3tbhy9xC8KVutnGsZbkr+lDrxTLhZe665sk8dRQAAAwcW5/lmwO8/u/24/qljx/OlDv9pp+30xtflBk6pS/9sKykp2X6p7musbz++n6od4zMFUqd19sg+0ME8jlaZlzhvu7zU7Lvc2dzLMtO6mF9D1mW3bqHy+Y6XNK/LOvr6fS8ugaYAOCkMARfJTK33OVu54N+3k4+mwq+fmb5WU+mtHr/OfRX8HWpzNVTff0S9CX9ToU9inuGTEelkBp8mbN+m1Jo8BWTdEuebV+r/GfMAABA/wlD8BWVtMyynVUq/njQlTIn7wrth/nqeleGJG2vric216qwvqMkXW7Z/q96uGylpDsty/dlGt3NtoIOvkqVv//a22m3Xm4bWzhudQT6LilzOfW3cuZ/QmYsrf64bLovzlXXTlFS5szZcplb9rZX18vaJdP5uUtSm6S/DGCNua6RudQ8n5Uy9/inZb7Ux8oe7nxN0iCZW/96e7vh2ZJ+nme9kjmb9qGkNTKX1A/ZUEuxzzhVytzaOS1nfkbSmeL2RgAA0HMZSffI9B87GylpL5mnlhdDjaSHJR20idc0yfRl18mEckNk+t35hqPItXTDNo7qNG+YpOkyw370RVT2k+v5HobV2aAN9UzYxGsK3WdXlEt6UOZqr3yaZULYdRv+PUSmz9+bK+cAwClhuOJLkk60bMeXGXdpoNmu+Pq3TGjV8e96mTG+hlqW303SbbLfKlcv+5kym0Kv+JqWp4asTIizl2WZ7WWuqmq1LOfLnHnrjRPy1OBLek3SN2QGGLXZRmYMtT/KfGH7GrgrvkbKjLeRu2xC7j5UAQCAsArDFV+SCX5sfaDck7sDJSYzRqqthpRMiDRJ9os2YpL2lnSBzDhQm7riSzLjx+Zu46kCap9mWd8LPVguJulRy7Id+3yrNr3P+6jrPrt6xZcn6V7L8r5M8DpX0mTZh+mIStpDJph9QR/317niC8BmISzB186W7fiSfjwA28plC746T2/LPBGxO1+SuSIsd/n7e1hHIcHXcJlL6XOXb5T5AuzOrpIWW5ZPyQzK2hM7yD4WWlLS6erdZf6DZW7ZPLub1/Ul+NpF5jba3OVWq+f7CgAA+k9Ygq9Rlu34MoPBF8Mlebb/pjY9wLvNfjJDf+R7OFJE0n9ytpOVNL7XVRsPqmvdJ/Vgucssy/kyY+72tpb9ZIKl7oKooIKvsy3L+jJ3x9hOcG/KHjJ3p+zSy+UAIBBhCb482ceEumsAtpVrU8HXKpmronrqxDzrmdiDZQsJvv7HsmxKPQu9Ouws+1hrj/dw+Qcsy7bLnBEcKL0Nvj4rc9l37jLvyv60TgAAMPDCEnxJ0grLth4coG11tp3sV/C/LPsdC/3hAsv2+jLIvW1Q+zpJFd0st702vkOjY5ovc3vfQAki+Bohc0LbFvAxQD2KptgDFgJh40tab5nf3aXGA+1CmfCvp+ZK+ptl/un9U45VuaSTLfNvVM9DK8mcLZplmX+ozKORN2Vvmafo5LpC9vcjCMfKPJ57q5z5z8uMg1GshykAAIDwWmeZV4z+7IWSynLmNcqcEK8doG3eLnOSs7OZ6v0g9yer662Id0pq6Wa5iyzbapDZ57pe1uC6cyVV58xrlXS8zF0LQFEQfAGFs31B5R7gi+ktmSfi9NZFlnnTNXBnno5V1zCnUdKVfVjXXJnHInfmqfvg7lTLvAUyV6K54FyZscNyO4T3SzpM9k4qAABAbwXRny2T9HXL/MslLRnA7a6TGdC/s45B7nuqr4Pal8s8jCnXZerdSevNQVT2k9yzZf5eAYqG4AsonK2jEOSTVu6SuRKtt/6truFRmXp2u2NffM4y7x6Z8bZ6y5f0v5b5h3az3BTLvF/KjPUQJE/STyT9VF2P07+QCQ1bi10UAAAIrSD6s5+RGTKjs4SkOwZ4u5J0s2VebwbzP1LmNs3OnpMZl2xTJqhroNgkcxI3bPaSudWxs6R4AjkCQPAFFM4WkgT5u/WXApa1jeVwQAHr25QDLfPuK2B9uWfuJDMg6qA8rx8taaeceRmZK6yCVC4zKOsFOfOzks6XeUpn0MEcAAAIlyD6swdb5v1Z5g6Agfa8up7wnaSeDyxvC8l6EujY9vlBFWefi802ftij4hZHBIDgCyic7VbAoK7GaZN5mmNfvW6ZNxBPDKyR/Wksrxawzo9kBvXvLCLpU3lef5Bl3kKZMRaCMkzSY5K+nDO/XdJX1beBVwEAALpj6892N1ZVoWx9secHeJud3ZLzb089u+prG0mfz5lXK3PisjtB73MxTbDMC+u+wnEEX0DhBlvmJYpehfGepHQByy+0zBuIgU1HyHQuOlujws8A/dsyL98TZ2z79VqB2y/EDjKXyOd2EuokHaHgr0QDAADhZQu+mgd4m0H3xe5S1yutejLI/SnqOqj9PPXsxHfQ+1xMW9K+wnEEX0Dhcgdol8wjoYNQ6NNvbMsPxOD2trCwP57cY3vCZr76bY/IDnKw+JmSxuXMWyrps5L+WfxyAADAFsTWLxro/mzQfbEmmacwdtbdIPd9HdS+Q9D7XExb0r7CcQRfQGG2lT1YWV7sQjboy8DwndnGF7CFVIWyrbPQ2iX7bYq9Cb7q+6GG/tSmYG+9BAAA4TdU0hjL/IHuz9r6YsXu9+Te7ihJp23i9VPVdVD7f6rnQ424sM/FsiXtKxxH8AUU5jN55i8oahWbn748dbIncm+f7G0NvVm+vz0n86SbznaR6UyNLXo1AABgS/EZ2ftAtiEk+tNA9Qd7401Jz+TMmyRptzyvt4VivXlKoQv7DGxxCL6AwtgGbZSkV4paxcdyHwndW7YnINoeb10o25VVhdaebx35bqG0zc/3BMhi+IekL6nr+BA7yIRfOxe9IgAAsCXIdyK3kIcO9YStLzYQdxp052bLPNvtjLZB7ddJurcX23Jln4vBNgRJWPcVjiP4AvouIhNU5Fop6a0i19LBdklxb9jGKxuI2/9s67Rtu7eGWeblC+5sHY/h/VBDIf4maZq6PhxhW0lPS/pk0SsCAABhl/s0acn0n4IIvoLoi90r85ClzmyD3NsGLine truncated
|
||||
}
|
||||
},
|
||||
"cell_type": "markdown",
|
||||
"id": "c6450ee1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div>\n",
|
||||
"<img src=\"attachment:g23933.png\" align=\"left\" width=\"400\"/>\n",
|
||||
"</div>"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Julia 1.9.1",
|
||||
"display_name": "Julia 1.9.0",
|
||||
"language": "julia",
|
||||
"name": "julia-1.9"
|
||||
},
|
||||
@@ -199,7 +382,7 @@
|
||||
"file_extension": ".jl",
|
||||
"mimetype": "application/julia",
|
||||
"name": "julia",
|
||||
"version": "1.9.1"
|
||||
"version": "1.9.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
+27122
-5
File diff suppressed because it is too large.
Load diff
|
Before Width: | Height: | Size: 3.6 MiB After Width: | Height: | Size: 4.8 MiB |
+110
-7
@@ -37,7 +37,7 @@
|
||||
"\n",
|
||||
"### Creating a task\n",
|
||||
"\n",
|
||||
"A task is a piece of computation work that can be run asynchronously (i.e., that can be run in the background). To create a task, we first need to create a function that represents the work to be done in the task. In next cell, we generate a task that generates and sums two matrices."
|
||||
"Technically, a task in Julia is a *symmetric co-routine*. More informally, a task is a piece of computation work that can be started (scheduled) at some point in the future, and that can be interrupted and resumed. To create a task, we first need to create a function that represents the work to be done in the task. In next cell, we generate a task that generates and sums two matrices."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -181,7 +181,7 @@
|
||||
"source": [
|
||||
"### Tasks do not run in parallel\n",
|
||||
"\n",
|
||||
"It is also important to note that tasks do not run in parallel. We were able to run code while previous tasks was running because the task was idling most of the time. If the task does actual work, the current process will be busy running this task and it is likely that we cannot run other code at the same time. Let's illustrate this with an example. The following code computes an approximation of $\\pi$ using [Leibniz formula](https://en.wikipedia.org/wiki/Leibniz_formula_for_pi). The quality of the approximation increases with the value of `n`."
|
||||
"It is also important to note that tasks do not run in parallel. We were able to run code while previous tasks was running because the task was idling most of the time in the sleep function. If the task does actual work, the current process will be busy running this task and preventing to run other tasks. Let's illustrate this with an example. The following code computes an approximation of $\\pi$ using [Leibniz formula](https://en.wikipedia.org/wiki/Leibniz_formula_for_pi). The quality of the approximation increases with the value of `n`."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -265,6 +265,109 @@
|
||||
"1+1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a700fe69",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### `yield`\n",
|
||||
"\n",
|
||||
"If tasks do not run in parallel, what is the purpose of tasks? Tasks are handy since they can be interrupted and to switch control to other tasks. This is achieved via function `yield`. When we call yield, we provide the opportunity to switch to another task. The function below is a variation of function `compute_π` in which we yield every 1000 iterations. At the call to yield we allow other tasks to take over. Without this call to yield, once we start function `compute_π` we cannot start any other tasks until this function finishes."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9260c065",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"function compute_π_yield(n)\n",
|
||||
" s = 1.0\n",
|
||||
" for i in 1:n\n",
|
||||
" s += (isodd(i) ? -1 : 1) / (i*2+1)\n",
|
||||
" if mod(i,1000) == 0\n",
|
||||
" yield()\n",
|
||||
" end\n",
|
||||
" end\n",
|
||||
" 4*s\n",
|
||||
"end"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "69fd4131",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can check this behavior experimentally with the two following cells. The next one creates and schedules a task that computes pi with the function `compute_π_yield`. Note that you can run the 2nd cell bellow while this task is running since we call to yield often inside `compute_π_yield`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "a85f3f39",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"fun = () -> compute_π_yield(3_000_000_000)\n",
|
||||
"t = Task(fun)\n",
|
||||
"schedule(t)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "24e23e88",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"1+1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "66ca10d6",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example: Implementing function sleep\n",
|
||||
"\n",
|
||||
"Using yield, we can implement our own version of the sleep function as follows:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "beed2b29",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"function mysleep(secs)\n",
|
||||
" final_time = time() + secs\n",
|
||||
" while time() < final_time\n",
|
||||
" yield()\n",
|
||||
" end\n",
|
||||
" nothing\n",
|
||||
"end"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5c44f6c1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can check that it behaves as expected."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "73c13bfb",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"@time mysleep(3)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "25048665",
|
||||
@@ -602,7 +705,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question (Q1):</b> How long will the compute time of next cell be? \n",
|
||||
"<b>Question (NB2-Q1):</b> How long will the compute time of next cell be? \n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
" a) 10*t\n",
|
||||
@@ -629,7 +732,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question (Q2):</b> How long will the compute time of next cell be? \n",
|
||||
"<b>Question (NB2-Q2):</b> How long will the compute time of next cell be? \n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
" a) 10*t\n",
|
||||
@@ -657,7 +760,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question (Q3):</b> How long will the compute time of next cell be? \n",
|
||||
"<b>Question (NB2-Q3):</b> How long will the compute time of next cell be? \n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
" a) 10*t\n",
|
||||
@@ -684,7 +787,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question (Q4):</b> How long will the compute time of the 2nd cell be? \n",
|
||||
"<b>Question (NB2-Q4):</b> How long will the compute time of the 2nd cell be? \n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
" a) infinity\n",
|
||||
@@ -724,7 +827,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question (Q5):</b> How long will the compute time of the 2nd cell be? \n",
|
||||
"<b>Question (NB2-Q5):</b> How long will the compute time of the 2nd cell be? \n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
" a) infinity\n",
|
||||
|
||||
@@ -449,7 +449,7 @@
|
||||
"source": [
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question:</b> What will be the value of `x` in the last line ? (Think your answer before executing next cell to find out the result) \n",
|
||||
"<b>Question (NB1-Q1):</b> What will be the value of `x` in the last line ? (Think your answer before executing next cell to find out the result) \n",
|
||||
"</div>\n",
|
||||
"\n"
|
||||
]
|
||||
@@ -557,7 +557,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question:</b> What will be the value of `x` in the last line ?\n",
|
||||
"<b>Question (NB1-Q2):</b> What will be the value of `x` in the last line ?\n",
|
||||
"</div>\n"
|
||||
]
|
||||
},
|
||||
@@ -1030,7 +1030,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div class=\"alert alert-block alert-success\">\n",
|
||||
"<b>Question:</b> Which will be the value of `x` below? \n",
|
||||
"<b>Question (NB1-Q3):</b> Which will be the value of `x` below? \n",
|
||||
"</div>\n",
|
||||
"\n"
|
||||
]
|
||||
@@ -1645,14 +1645,6 @@
|
||||
"\n",
|
||||
"This notebook is part of the course [Programming Large Scale Parallel Systems](https://www.francescverdugo.com/XM_40017/) at Vrije Universiteit Amsterdam and may be used under a [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) license."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "f8d92f25",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
Reference in new issue
Block a user