\n",
@@ -814,7 +803,7 @@
},
{
"cell_type": "markdown",
- "id": "db099c49",
+ "id": "4d456fed",
"metadata": {},
"source": [
"### Data dependencies"
@@ -822,7 +811,7 @@
},
{
"cell_type": "markdown",
- "id": "b5290fcf",
+ "id": "67b65ea6",
"metadata": {},
"source": [
"
\n",
@@ -833,7 +822,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "ac898985",
+ "id": "2825385a",
"metadata": {},
"outputs": [],
"source": [
@@ -842,12 +831,12 @@
},
{
"cell_type": "markdown",
- "id": "4f8dbc8c",
+ "id": "429faa32",
"metadata": {},
"source": [
"### Implementation\n",
"\n",
- "These are the main steps of the implementation of algorithm 2:\n",
+ "These are the main steps of the implementation of algorithm 3:\n",
"\n",
"1. The worker receives the corresponding rows A[rows,:] and matrix B from the master process\n",
"2. The worker computes the product of A[rows,:] times B\n",
@@ -861,7 +850,7 @@
}
},
"cell_type": "markdown",
- "id": "188ce727",
+ "id": "c14ebcb3",
"metadata": {},
"source": [
"
\n",
@@ -871,7 +860,7 @@
},
{
"cell_type": "markdown",
- "id": "e5e4eafa",
+ "id": "7d7952c1",
"metadata": {},
"source": [
"The implementation of this variant is let as an exercise (see below)."
@@ -879,7 +868,7 @@
},
{
"cell_type": "markdown",
- "id": "dce52b5b",
+ "id": "0323d6d8",
"metadata": {},
"source": [
"### Communication overhead\n",
@@ -894,7 +883,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "d6dbbf50",
+ "id": "50b8bf53",
"metadata": {},
"outputs": [],
"source": [
@@ -903,7 +892,7 @@
},
{
"cell_type": "markdown",
- "id": "ba449065",
+ "id": "3f0d99e6",
"metadata": {},
"source": [
"In this case, the ratio between communication and computation is O(P/N). If the matrix size N is much larger than the number of workers P, then the communication overhead O(P/N) would be negligible. This opens the door to an scalable implementation."
@@ -911,7 +900,7 @@
},
{
"cell_type": "markdown",
- "id": "706cb6ea",
+ "id": "7bb65f2e",
"metadata": {},
"source": [
"## Summary\n",
@@ -934,7 +923,7 @@
},
{
"cell_type": "markdown",
- "id": "8a1048b3",
+ "id": "8b83e744",
"metadata": {},
"source": [
"## Exercises"
@@ -942,7 +931,7 @@
},
{
"cell_type": "markdown",
- "id": "3e3a0c49",
+ "id": "a628a1df",
"metadata": {},
"source": [
"### Implementation of algorithm 3\n",
@@ -953,7 +942,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "c2941c87",
+ "id": "8e50b923",
"metadata": {},
"outputs": [],
"source": [
@@ -984,7 +973,7 @@
},
{
"cell_type": "markdown",
- "id": "9810107f",
+ "id": "4506dcfb",
"metadata": {},
"source": [
"Use test-driven development to implement the algorithm. Use this test:"
@@ -993,7 +982,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "d2df240f",
+ "id": "28cde36a",
"metadata": {},
"outputs": [],
"source": [
@@ -1009,7 +998,7 @@
},
{
"cell_type": "markdown",
- "id": "968d8237",
+ "id": "03952b0b",
"metadata": {},
"source": [
"Measure the performance of your implementation by running next cell. Do you get close to the optimal speedup?"
@@ -1018,7 +1007,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "c0d43911",
+ "id": "b3aa2b7c",
"metadata": {},
"outputs": [],
"source": [
@@ -1055,21 +1044,10 @@
},
{
"cell_type": "code",
- "execution_count": 10,
+ "execution_count": null,
"id": "023b20d1",
"metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "matmul_dist_1_v2! (generic function with 1 method)"
- ]
- },
- "execution_count": 10,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "outputs": [],
"source": [
"function matmul_dist_1_v2!(C, A, B)\n",
" m = size(C,1)\n",
@@ -1107,21 +1085,10 @@
},
{
"cell_type": "code",
- "execution_count": 11,
+ "execution_count": null,
"id": "c1d3595b",
"metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "\u001b[32m\u001b[1mTest Passed\u001b[22m\u001b[39m"
- ]
- },
- "execution_count": 11,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "outputs": [],
"source": [
"using Test\n",
"N = 50\n",
@@ -1133,7 +1100,7 @@
},
{
"cell_type": "markdown",
- "id": "b49ee366",
+ "id": "ab609c18",
"metadata": {},
"source": [
"Run the next cell to check the performance of this implementation. Note that we are far away from the optimal speed up. Why? To answer this question compute the theoretical communication over computation ratio for this implementation and reason about the obtained result. Hint: the number of times a worker is spawned in this implementation is N^3/P on average."
@@ -1141,20 +1108,10 @@
},
{
"cell_type": "code",
- "execution_count": 18,
- "id": "9a4c526c",
+ "execution_count": null,
+ "id": "d7d31710",
"metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Speedup = 0.0012612337176382187\n",
- "Optimal speedup = 4\n",
- "Efficiency = 0.03153084294095547%\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"N = 100\n",
"A = rand(N,N)\n",
@@ -1172,7 +1129,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "4f2d0d9b",
+ "id": "cd31d955",
"metadata": {},
"outputs": [],
"source": []