From 208fed1dacd97a47e5aa84ef8911f7ba01bb5edb Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Tue, 1 Oct 2024 05:36:40 +0000 Subject: [PATCH] build based on 20c92dc --- dev/.documenter-siteinfo.json | 2 +- dev/LEQ.ipynb | 6 +++--- dev/LEQ/index.html | 2 +- dev/LEQ_src/index.html | 6 +++--- dev/asp.ipynb | 4 ++-- dev/asp/index.html | 2 +- dev/asp_src/index.html | 4 ++-- dev/getting_started_with_julia/index.html | 2 +- dev/index.html | 2 +- dev/jacobi_2D/index.html | 2 +- dev/jacobi_method.ipynb | 20 ++++++++++---------- dev/jacobi_method/index.html | 2 +- dev/jacobi_method_src/index.html | 20 ++++++++++---------- dev/julia_async.ipynb | 8 ++++---- dev/julia_async/index.html | 2 +- dev/julia_async_src/index.html | 8 ++++---- dev/julia_basics.ipynb | 2 +- dev/julia_basics/index.html | 2 +- dev/julia_basics_src/index.html | 2 +- dev/julia_distributed.ipynb | 8 ++++---- dev/julia_distributed/index.html | 2 +- dev/julia_distributed_src/index.html | 8 ++++---- dev/julia_intro/index.html | 2 +- dev/julia_jacobi/index.html | 2 +- dev/julia_mpi.ipynb | 12 ++++++------ dev/julia_mpi/index.html | 2 +- dev/julia_mpi_src/index.html | 12 ++++++------ dev/julia_tutorial/index.html | 2 +- dev/matrix_matrix.ipynb | 4 ++-- dev/matrix_matrix/index.html | 2 +- dev/matrix_matrix_src/index.html | 4 ++-- dev/mpi_collectives/index.html | 2 +- dev/mpi_tutorial/index.html | 2 +- dev/notebook-hello/index.html | 2 +- dev/pdes/index.html | 2 +- dev/solutions/index.html | 2 +- dev/solutions_for_all_notebooks/index.html | 2 +- dev/tsp/index.html | 2 +- 38 files changed, 86 insertions(+), 86 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index cd18f67..11a06d9 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-30T15:25:09","documenter_version":"1.7.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-10-01T05:36:31","documenter_version":"1.7.0"}} \ No newline at end of file diff --git a/dev/LEQ.ipynb b/dev/LEQ.ipynb index 5444139..93a5af1 100644 --- a/dev/LEQ.ipynb +++ b/dev/LEQ.ipynb @@ -103,7 +103,7 @@ "### Problem statement\n", "\n", "Let us consider a system of linear equations written in matrix form $Ax=b$, where $A$ is a nonsingular square matrix, and $x$ and $b$ are vectors. $A$ and $b$ are given, and $x$ is unknown. The goal of Gaussian elimination is to transform the system $Ax=b$, into a new system $Ux=c$ such that\n", - "- both system have the same solution vector $x$,\n", + "- both systems have the same solution vector $x$,\n", "- the matrix $U$ of the new system is *upper triangular* with unit diagonal, namely $U_{ii} = 1$ and $U_{ij} = 0$ for $i>j$.\n", "\n", "\n", @@ -398,7 +398,7 @@ "source": [ "### Data partition\n", "\n", - "Let start considering a row-wise block partition, as we did in previous algorithms.\n", + "Let's start considering a row-wise block partition, as we did in previous algorithms.\n", "\n", "In the figure below, we use different colors to illustrate which entries are assigned to a CPU. All entries with the same color are assigned to the same CPU." ] @@ -454,7 +454,7 @@ "Definition: *Load imbalance*: is the problem when work is not equally distributed over all processes and consequently some processes do more work than others.\n", "\n", "\n", - "Having processors waiting for others is a waist of computational resources and affects negatively parallel speedups. The optimal speedup (speedup equal to the number of processors) assumes that the work is perfectly parallel and that it is evenly distributed. If there is load imbalance, the last assumption is not true anymore and the speedup will be suboptimal.\n" + "Having processors waiting for others is a waste of computational resources and affects negatively parallel speedups. The optimal speedup (speedup equal to the number of processors) assumes that the work is perfectly parallel and that it is evenly distributed. If there is load imbalance, the last assumption is not true anymore and the speedup will be suboptimal.\n" ] }, { diff --git a/dev/LEQ/index.html b/dev/LEQ/index.html index 26dcf4d..0ceb0b1 100644 --- a/dev/LEQ/index.html +++ b/dev/LEQ/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/LEQ_src/index.html b/dev/LEQ_src/index.html index f7f1c08..76a4c48 100644 --- a/dev/LEQ_src/index.html +++ b/dev/LEQ_src/index.html @@ -7618,7 +7618,7 @@ $$

This is just a small example with three unknowns, but practical applications need to solve linear equations with large number of unknowns. Parallel processing is needed in these cases.

Problem statement

Let us consider a system of linear equations written in matrix form $Ax=b$, where $A$ is a nonsingular square matrix, and $x$ and $b$ are vectors. $A$ and $b$ are given, and $x$ is unknown. The goal of Gaussian elimination is to transform the system $Ax=b$, into a new system $Ux=c$ such that

For the particular system shown above, the transformed one is the following:

@@ -7933,7 +7933,7 @@ d) only the first inner loop
@@ -7986,7 +7986,7 @@ d) only the first inner loop
Definition: *Load imbalance*: is the problem when work is not equally distributed over all processes and consequently some processes do more work than others.
-

Having processors waiting for others is a waist of computational resources and affects negatively parallel speedups. The optimal speedup (speedup equal to the number of processors) assumes that the work is perfectly parallel and that it is evenly distributed. If there is load imbalance, the last assumption is not true anymore and the speedup will be suboptimal.

+

Having processors waiting for others is a waste of computational resources and affects negatively parallel speedups. The optimal speedup (speedup equal to the number of processors) assumes that the work is perfectly parallel and that it is evenly distributed. If there is load imbalance, the last assumption is not true anymore and the speedup will be suboptimal.

diff --git a/dev/asp.ipynb b/dev/asp.ipynb index 01e080b..0568933 100644 --- a/dev/asp.ipynb +++ b/dev/asp.ipynb @@ -57,7 +57,7 @@ "function q1_answer(bool)\n", " bool || return\n", " msg = \"\"\"\n", - " The we can change the loop order over i and j without changing the result. Rememeber:\n", + " Then we can change the loop order over i and j without changing the result. Remember:\n", " \n", " C[i,j] = min(C[i,j],C[i,k]+C[k,j])\n", " \n", @@ -788,7 +788,7 @@ " if rank == 0\n", " N = size(C,1)\n", " if mod(N,P) !=0\n", - " println(\"N not multplie of P\")\n", + " println(\"N not multiple of P\")\n", " MPI.Abort(comm,-1)\n", " end\n", " Nref = Ref(N)\n", diff --git a/dev/asp/index.html b/dev/asp/index.html index 896609b..a7f07d3 100644 --- a/dev/asp/index.html +++ b/dev/asp/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/asp_src/index.html b/dev/asp_src/index.html index 97fb875..e5fd0fa 100644 --- a/dev/asp_src/index.html +++ b/dev/asp_src/index.html @@ -7573,7 +7573,7 @@ a.anchor-link { function q1_answer(bool) bool || return msg = """ - The we can change the loop order over i and j without changing the result. Rememeber: + Then we can change the loop order over i and j without changing the result. Remember: C[i,j] = min(C[i,j],C[i,k]+C[k,j]) @@ -8351,7 +8351,7 @@ send the pieces to all other ranks. This is done in the function below. We start if rank == 0 N = size(C,1) if mod(N,P) !=0 - println("N not multplie of P") + println("N not multiple of P") MPI.Abort(comm,-1) end Nref = Ref(N) diff --git a/dev/getting_started_with_julia/index.html b/dev/getting_started_with_julia/index.html index 2c838c8..3215517 100644 --- a/dev/getting_started_with_julia/index.html +++ b/dev/getting_started_with_julia/index.html @@ -15,4 +15,4 @@ DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195"

Copy the contents of previous code block into a file called Project.toml and place it in an empty folder named newproject. It is important that the file is named Project.toml. You can create a new folder from the REPL with

julia> mkdir("newproject")

To install all the packages registered in this file you need to activate the folder containing your Project.toml file

(@v1.10) pkg> activate newproject

and then instantiating it

(newproject) pkg> instantiate

The instantiate command will download and install all listed packages and their dependencies in just one click.

Getting help in package mode

You can get help about a particular package operator by writing help in front of it

(@v1.10) pkg> help activate

You can get an overview of all package commands by typing help alone

(@v1.10) pkg> help

Package operations in Julia code

In some situations it is required to use package commands in Julia code, e.g., to automatize installation and deployment of Julia applications. This can be done using the Pkg package. For instance

julia> using Pkg
 julia> Pkg.status()

is equivalent to calling status in package mode.

(@v1.10) pkg> status

Creating you own package

In many situations, it is useful to create your own package, for instance, when working with a large code base, when you want to reduce compilation latency using Revise.jl, or if you want to eventually register your package and share it with others.

The simplest way of generating a package (called MyPackage) is as follows. Open Julia, go to package mode, and type

(@v1.10) pkg> generate MyPackage

This will crate a minimal package consisting of a new folder MyPackage with two files:

Tip

This approach only generates a very minimal package. To create a more sophisticated package skeleton (including unit testing, code coverage, readme file, licence, etc.) use PkgTemplates.jl or BestieTemplate.jl. The later one is developed in Amsterdam at the Netherlands eScience Center.

You can add dependencies to the package by activating the MyPackage folder in package mode and adding new dependencies as always:

(@v1.10) pkg> activate MyPackage
 (MyPackage) pkg> add MPI

This will add MPI to your package dependencies.

Using your own package

To use your package you first need to add it to a package environment of your choice. This is done by changing to package mode and typing develop followed by the path to the folder containing the package. For instance:

(@v1.10) pkg> develop MyPackage
Note

You do not need to "develop" your package if you activated the package folder MyPackage.

Now, we can go back to standard Julia mode and use it as any other package:

using MyPackage
-MyPackage.greet()

Here, we just called the example function defined in MyPackage/src/MyPackage.jl.

Conclusion

We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:

+MyPackage.greet()

Here, we just called the example function defined in MyPackage/src/MyPackage.jl.

Conclusion

We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:

diff --git a/dev/index.html b/dev/index.html index 6061ecf..f31cd79 100644 --- a/dev/index.html +++ b/dev/index.html @@ -2,4 +2,4 @@ Home · XM_40017

Programming Large-Scale Parallel Systems (XM_40017)

Welcome to the interactive lecture notes of the Programming Large-Scale Parallel Systems course at VU Amsterdam!

What

This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam. We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems. Further information about the course is found in the study guide (click here) and our Canvas page (for registered students).

Note

Material will be added incrementally to the website as the course advances.

Warning

This page will eventually contain only a part of the course material. The rest will be available on Canvas. In particular, the material in this public webpage does not fully cover all topics in the final exam.

How to use this page

You have two main ways of studying the notebooks:

  • Download the notebooks and run them locally on your computer (recommended). At each notebook page you will find a green box with links to download the notebook.
  • You also have the static version of the notebooks displayed in this webpage for quick reference.

How to run the notebooks locally

To run a notebook locally follow these steps:

  • Install Julia (if not done already). More information in Getting started.
  • Download the notebook.
  • Launch Julia. More information in Getting started.
  • Execute these commands in the Julia command line:
julia> using Pkg
 julia> Pkg.add("IJulia")
 julia> using IJulia
-julia> notebook()
  • These commands will open a jupyter in your web browser. Navigate in jupyter to the notebook file you have downloaded and open it.

Authors

This material is created by Francesc Verdugo with the help of Gelieza Kötterheinrich. Part of the notebooks are based on the course slides by Henri Bal.

License

All material on this page that is original to this course may be used under a CC BY 4.0 license.

Acknowledgment

This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project "Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course" funded by the "Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning".

+julia> notebook()

Authors

This material is created by Francesc Verdugo with the help of Gelieza Kötterheinrich. Part of the notebooks are based on the course slides by Henri Bal.

License

All material on this page that is original to this course may be used under a CC BY 4.0 license.

Acknowledgment

This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project "Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course" funded by the "Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning".

diff --git a/dev/jacobi_2D/index.html b/dev/jacobi_2D/index.html index a634d30..7140c1f 100644 --- a/dev/jacobi_2D/index.html +++ b/dev/jacobi_2D/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/jacobi_method.ipynb b/dev/jacobi_method.ipynb index 598079d..4592f08 100644 --- a/dev/jacobi_method.ipynb +++ b/dev/jacobi_method.ipynb @@ -27,7 +27,7 @@ "\n", "In this notebook, we will learn\n", "\n", - "- How to paralleize the Jacobi method\n", + "- How to parallelize the Jacobi method\n", "- How the data partition can impact the performance of a distributed algorithm\n", "- How to use latency hiding to improve parallel performance\n", "\n" @@ -452,7 +452,7 @@ "- We need to get remote entries from 2 neighbors (2 messages per iteration)\n", "- We need to communicate 1 entry per message\n", "- Thus, communication complexity is $O(1)$\n", - "- Communication/computation ration is $O(P/N)$, making the algorithm potentially scalable if $P< per iteration | Communication
per worker | Computation
per worker | Ratio communication/
computation |\n", "|---|---|---|---|---|\n", - "| 1d block | 2 | O(N) | N²/P | O(P/N) |\n", - "| 2d block | 4 | O(N/√P) | N²/P | O(√P/N) |\n", - "| 2d cyclic | 4 |O(N²/P) | N²/P | O(1) |" + "| 1D block | 2 | O(N) | N²/P | O(P/N) |\n", + "| 2D block | 4 | O(N/√P) | N²/P | O(√P/N) |\n", + "| 2D cyclic | 4 |O(N²/P) | N²/P | O(1) |" ] }, { @@ -862,9 +862,9 @@ "\n", "\n", "\n", - "- Both 1d and 2d block partitions are potentially scalable if $P< + diff --git a/dev/jacobi_method_src/index.html b/dev/jacobi_method_src/index.html index 6beca03..d2bdf5f 100644 --- a/dev/jacobi_method_src/index.html +++ b/dev/jacobi_method_src/index.html @@ -7543,7 +7543,7 @@ a.anchor-link { @@ -8232,7 +8232,7 @@ a.anchor-link { end
    -
  • The outer loop cannot be parallelized (like in the 1d case).
    • +
  • The outer loop cannot be parallelized (like in the 1D case).
  • The two inner loops are trivially parallel
@@ -8245,7 +8245,7 @@ a.anchor-link { diff --git a/dev/julia_async_src/index.html b/dev/julia_async_src/index.html index ce44461..ea35bca 100644 --- a/dev/julia_async_src/index.html +++ b/dev/julia_async_src/index.html @@ -7603,7 +7603,7 @@ a.anchor-link { @@ -7910,7 +7910,7 @@ a.anchor-link { @@ -7944,7 +7944,7 @@ a.anchor-link { @@ -7985,7 +7985,7 @@ a.anchor-link { diff --git a/dev/julia_basics.ipynb b/dev/julia_basics.ipynb index 85e87d4..7b78065 100644 --- a/dev/julia_basics.ipynb +++ b/dev/julia_basics.ipynb @@ -37,7 +37,7 @@ "source": [ "## Using Jupyter notebooks in Julia\n", "\n", - "We are going to use Jupyter notebooks in this and other lectures. You provably have worked with notebooks (in Python). If not, here are the basic concepts you need to know to follow the lessons.\n", + "We are going to use Jupyter notebooks in this and other lectures. You probably have worked with notebooks (in Python). If not, here are the basic concepts you need to know to follow the lessons.\n", "\n", "
\n", "Tip: Did you know that Jupyter stands for Julia, Python and R?\n", diff --git a/dev/julia_basics/index.html b/dev/julia_basics/index.html index 5f2d3a3..6c1683a 100644 --- a/dev/julia_basics/index.html +++ b/dev/julia_basics/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); -
+ diff --git a/dev/julia_basics_src/index.html b/dev/julia_basics_src/index.html index 4283e60..84196b4 100644 --- a/dev/julia_basics_src/index.html +++ b/dev/julia_basics_src/index.html @@ -7547,7 +7547,7 @@ a.anchor-link { + diff --git a/dev/julia_distributed_src/index.html b/dev/julia_distributed_src/index.html index 31d7efc..fdc96ed 100644 --- a/dev/julia_distributed_src/index.html +++ b/dev/julia_distributed_src/index.html @@ -7671,7 +7671,7 @@ a.anchor-link { @@ -7813,7 +7813,7 @@ a.anchor-link { diff --git a/dev/julia_intro/index.html b/dev/julia_intro/index.html index d68243a..050aff9 100644 --- a/dev/julia_intro/index.html +++ b/dev/julia_intro/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_jacobi/index.html b/dev/julia_jacobi/index.html index c166c71..5219efe 100644 --- a/dev/julia_jacobi/index.html +++ b/dev/julia_jacobi/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_mpi.ipynb b/dev/julia_mpi.ipynb index 899424e..281e386 100644 --- a/dev/julia_mpi.ipynb +++ b/dev/julia_mpi.ipynb @@ -167,7 +167,7 @@ "```julia\n", "using MPI\n", "MPI.Init()\n", - "# Your MPI programm here\n", + "# Your MPI program here\n", "MPI.Finalize() # Optional\n", "```\n", "\n", @@ -176,7 +176,7 @@ "```julia\n", "using MPI\n", "MPI.Init(finalize_atexit=false)\n", - "# Your MPI programm here\n", + "# Your MPI program here\n", "MPI.Finalize() # Mandatory\n", "```\n", "\n", @@ -186,7 +186,7 @@ "#include \n", "int main(int argc, char** argv) {\n", " MPI_Init(NULL, NULL);\n", - " /* Your MPI Programm here */\n", + " /* Your MPI Program here */\n", " MPI_Finalize();\n", "}\n", "```\n", @@ -612,7 +612,7 @@ "id": "4b455f98", "metadata": {}, "source": [ - "So, the full MPI program needs to be in the source file passed to Julia or the quote block. In practice, long MPI programms are written as Julia packages using several files, which are then loaded by each MPI process. For our simple example, we just need to include the definition of `foo` inside the quote block." + "So, the full MPI program needs to be in the source file passed to Julia or the quote block. In practice, long MPI programs are written as Julia packages using several files, which are then loaded by each MPI process. For our simple example, we just need to include the definition of `foo` inside the quote block." ] }, { @@ -920,7 +920,7 @@ " source = MPI.ANY_SOURCE\n", " tag = MPI.ANY_TAG\n", " status = MPI.Probe(comm,MPI.Status; source, tag)\n", - " count = MPI.Get_count(status,Int) # Get incomming message length\n", + " count = MPI.Get_count(status,Int) # Get incoming message length\n", " println(\"I am about to receive $count integers.\")\n", " rcvbuf = zeros(Int,count) # Allocate \n", " MPI.Recv!(rcvbuf, comm, MPI.Status; source, tag)\n", @@ -973,7 +973,7 @@ " if rank == 3\n", " rcvbuf = zeros(Int,5)\n", " MPI.Recv!(rcvbuf, comm, MPI.Status; source=2, tag=0)\n", - " # recvbuf will have the incomming message fore sure. Recv! has returned.\n", + " # recvbuf will have the incoming message fore sure. Recv! has returned.\n", " @show rcvbuf\n", " end\n", "end\n", diff --git a/dev/julia_mpi/index.html b/dev/julia_mpi/index.html index 84c0600..f13bebd 100644 --- a/dev/julia_mpi/index.html +++ b/dev/julia_mpi/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_mpi_src/index.html b/dev/julia_mpi_src/index.html index ed98d1d..dbb153c 100644 --- a/dev/julia_mpi_src/index.html +++ b/dev/julia_mpi_src/index.html @@ -7692,20 +7692,20 @@ a.anchor-link {

In Julia (option 1, recommended):

using MPI
 MPI.Init()
-# Your MPI programm here
+# Your MPI program here
 MPI.Finalize() # Optional

In Julia (option 2, advanced):

using MPI
 MPI.Init(finalize_atexit=false)
-# Your MPI programm here
+# Your MPI program here
 MPI.Finalize() # Mandatory

In C:

#include <mpi.h>
 int main(int argc, char** argv) {
     MPI_Init(NULL, NULL);
-    /* Your MPI Programm here */
+    /* Your MPI Program here */
     MPI_Finalize();
 }
@@ -8216,7 +8216,7 @@ a.anchor-link { @@ -8547,7 +8547,7 @@ a.anchor-link { source = MPI.ANY_SOURCE tag = MPI.ANY_TAG status = MPI.Probe(comm,MPI.Status; source, tag) - count = MPI.Get_count(status,Int) # Get incomming message length + count = MPI.Get_count(status,Int) # Get incoming message length println("I am about to receive $count integers.") rcvbuf = zeros(Int,count) # Allocate MPI.Recv!(rcvbuf, comm, MPI.Status; source, tag) @@ -8608,7 +8608,7 @@ a.anchor-link { if rank == 3 rcvbuf = zeros(Int,5) MPI.Recv!(rcvbuf, comm, MPI.Status; source=2, tag=0) - # recvbuf will have the incomming message fore sure. Recv! has returned. + # recvbuf will have the incoming message fore sure. Recv! has returned. @show rcvbuf end end diff --git a/dev/julia_tutorial/index.html b/dev/julia_tutorial/index.html index 22bbc68..d3b2ac2 100644 --- a/dev/julia_tutorial/index.html +++ b/dev/julia_tutorial/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/matrix_matrix.ipynb b/dev/matrix_matrix.ipynb index 955abd9..1448e87 100644 --- a/dev/matrix_matrix.ipynb +++ b/dev/matrix_matrix.ipynb @@ -293,7 +293,7 @@ "## Where can we exploit parallelism?\n", "\n", "\n", - "The matrix-matrix multiplication is an example of [embarrassingly parallel algorithm](https://en.wikipedia.org/wiki/Embarrassingly_parallel). An embarrassingly parallel (also known as trivially parallel) algorithm is an algorithm that can be split in parallel tasks with no (or very few) dependences between them. Such algorithms are typically easy to parallelize.\n", + "The matrix-matrix multiplication is an example of [embarrassingly parallel algorithm](https://en.wikipedia.org/wiki/Embarrassingly_parallel). An embarrassingly parallel (also known as trivially parallel) algorithm is an algorithm that can be split in parallel tasks with no (or very few) dependencies between them. Such algorithms are typically easy to parallelize.\n", "\n", "Which parts of an algorithm are completely independent and thus trivially parallel? To answer this question, it is useful to inspect the for loops, which are potential sources of parallelism. If the iterations are independent of each other, then they are trivial to parallelize. An easy check to find out if the iterations are dependent or not is to change their order (for instance changing `for j in 1:n` by `for j in n:-1:1`, i.e. doing the loop in reverse). If the result changes, then the iterations are not independent.\n", "\n", @@ -314,7 +314,7 @@ "Note that:\n", "\n", "- Loops over `i` and `j` are trivially parallel.\n", - "- The loop over `k` is not trivially parallel. The accumulation into the reduction variable `Cij` introduces extra dependences. In addition, remember that the addition of floating point numbers is not strictly associative due to rounding errors. Thus, the result of this loop may change with the loop order when using floating point numbers. In any case, this loop can also be parallelized, but it requires a parallel *fold* or a parallel *reduction*.\n", + "- The loop over `k` is not trivially parallel. The accumulation into the reduction variable `Cij` introduces extra dependencies. In addition, remember that the addition of floating point numbers is not strictly associative due to rounding errors. Thus, the result of this loop may change with the loop order when using floating point numbers. In any case, this loop can also be parallelized, but it requires a parallel *fold* or a parallel *reduction*.\n", "\n" ] }, diff --git a/dev/matrix_matrix/index.html b/dev/matrix_matrix/index.html index fb3e849..fb59883 100644 --- a/dev/matrix_matrix/index.html +++ b/dev/matrix_matrix/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/matrix_matrix_src/index.html b/dev/matrix_matrix_src/index.html index 6d768dc..8c9fb1c 100644 --- a/dev/matrix_matrix_src/index.html +++ b/dev/matrix_matrix_src/index.html @@ -7855,7 +7855,7 @@ d) O(N³) + diff --git a/dev/mpi_tutorial/index.html b/dev/mpi_tutorial/index.html index 4bcb366..97da59f 100644 --- a/dev/mpi_tutorial/index.html +++ b/dev/mpi_tutorial/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/notebook-hello/index.html b/dev/notebook-hello/index.html index c829706..0fc9075 100644 --- a/dev/notebook-hello/index.html +++ b/dev/notebook-hello/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/pdes/index.html b/dev/pdes/index.html index 1c57129..13503df 100644 --- a/dev/pdes/index.html +++ b/dev/pdes/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/solutions/index.html b/dev/solutions/index.html index e095033..c482029 100644 --- a/dev/solutions/index.html +++ b/dev/solutions/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/solutions_for_all_notebooks/index.html b/dev/solutions_for_all_notebooks/index.html index 11e19c4..51e26d2 100644 --- a/dev/solutions_for_all_notebooks/index.html +++ b/dev/solutions_for_all_notebooks/index.html @@ -333,4 +333,4 @@ end

« Traveling salesperson problem

Powered by Documenter.jl and the Julia Programming Language.

+end diff --git a/dev/tsp/index.html b/dev/tsp/index.html index a24accb..1c15dac 100644 --- a/dev/tsp/index.html +++ b/dev/tsp/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - +