Merge pull request #72 from jvdtoorn/fix/mpi_collectives-deadlock-comm_dup

Change julia kernel version for MPI collectives notebook

docs/make.jl

@@ -13,7 +13,7 @@ EditURL = "https://github.com/fverdugo/XM_40017/blob/main/notebooks/SCRIPT_NAME.
     <div class="admonition-body">
         <ul>
             <li>
-                Download this notebook and run it locally on your machine [recommended]. Click <a href="https://www.francescverdugo.com/XM_40017/dev/SCRIPT_NAME.ipynb" download>here</a>.
+                Download this notebook and run it locally on your machine [highly recommended]. Click <a href="https://www.francescverdugo.com/XM_40017/dev/SCRIPT_NAME.ipynb" download>here</a>.
             </li>
         </ul>
     </div>
@@ -119,13 +119,15 @@ makedocs(;
                          "Julia Basics" => "julia_basics.md",
                          "Asynchronous programming in Julia" => "julia_async.md",
                          "Distributed computing in Julia" => "julia_distributed.md",
-                         "Distributed computing with MPI" => "mpi_tutorial.md",
+                         #"Distributed computing with MPI" => "mpi_tutorial.md",
                          "Matrix-matrix multiplication"=>"matrix_matrix.md",
+                         "MPI (point-to-point)" => "julia_mpi.md",
+                         "MPI (collectives)" => "mpi_collectives.md",
                          "Jacobi method" => "jacobi_method.md",
                          "All pairs of shortest paths" => "asp.md",
                          "Gaussian elimination" => "LEQ.md",
                          "Traveling salesperson problem" => "tsp.md",
-                         "Partial differential equations" => "pdes.md",
+                         #"Partial differential equations" => "pdes.md",
                          ],
           "Solutions" => "solutions_for_all_notebooks.md",
           ],

docs/src/getting_started_with_julia.md

@@ -124,12 +124,23 @@ $ julia --version
 
 If this runs without error and you see a version number, you are good to go!
 
+You can also run julia code from the terminal using the `-e` flag:
+
+```
+$ julia -e 'println("Hello, world!")'
+```
 
 !!! note
     In this tutorial, when a code snipped starts with `$`, it should be run in the terminal. Otherwise, the code is to be run in the Julia REPL.
 
 !!! tip
-    Avoid calling Julia code from the terminal, use the Julia REPL instead! Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch. This can be time consuming for large projects. In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster. Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code from the terminal.
+    Avoid calling Julia code from the terminal, use the Julia REPL instead!
+    Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch.
+    This can be time consuming for large projects.
+    In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster.
+    Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code
+    from the terminal. Visit this link ([Julia workflow tips](https://docs.julialang.org/en/v1/manual/workflow-tips/))
+    from the official Julia documentation for further information about how to develop Julia code effectivelly.
 
 ### Running parallel code
 
@@ -199,11 +210,11 @@ To install a package, we need to enter *package* mode. Remember that we entered
 ```julia
 julia> ]
 ```
-At this point, the prompt should have changed to `(@v1.8) pkg>` indicating that we are in package mode. The text between the parentheses indicates which is the active *project*, i.e., where packages are going to be installed. In this case, we are working with the global project associated with our Julia installation (which is Julia 1.8 in this example, but it can be another version in your case).
+At this point, the prompt should have changed to `(@v1.11) pkg>` indicating that we are in package mode. The text between the parentheses indicates which is the active *project*, i.e., where packages are going to be installed. In this case, we are working with the global project associated with our Julia installation (which is Julia 1.11 in this example, but it can be another version in your case).
 
 To install the MPI package, type
 ```julia
-(@v1.8) pkg> add MPI
+(@v1.11) pkg> add MPI
 ```
 Congrats, you have installed MPI!
 
@@ -211,7 +222,8 @@ Congrats, you have installed MPI!
     Many Julia package names end with `.jl`. This is just a way of signaling that a package is written in Julia. When using such packages, the `.jl` needs to be omitted. In this case, we have installed the `MPI.jl` package even though we have only typed `MPI` in the REPL.
 
 !!! note
-    The package you have installed is the Julia interface to MPI, called  `MPI.jl`. Note that it is not a MPI library by itself. It is just a thin wrapper between MPI and Julia. To use this interface, you need an actual MPI library installed in your system such as OpenMPI or MPICH. Julia downloads and installs a MPI library for you, but it is also possible to use a MPI library already available in your system. This is useful, e.g., when running on HPC clusters. See the [documentation](https://juliaparallel.org/MPI.jl/stable/configuration/) of `MPI.jl` for further details.
+    The package you have installed is the Julia interface to MPI, called  `MPI.jl`. Note that it is not an MPI library by itself. It is just a thin wrapper between MPI and Julia. To use this interface, you need an actual MPI library installed in your system such as OpenMPI or MPICH. Julia downloads and installs an MPI library for you, but it is also possible to use an MPI library already available in your system. This is useful, e.g., when running on HPC clusters. See the [documentation](https://juliaparallel.org/MPI.jl/stable/configuration/) of `MPI.jl` for further details.
+
 
 To check that the package was installed properly, exit package mode by pressing the backspace key several times, and run it again
 
@@ -230,7 +242,7 @@ $ mpiexec -np 4 julia hello_mpi.jl
 But it will probably not work since the version of `mpiexec` needs to match with the MPI version we are using from Julia. Don't worry if you could not make it work! A more elegant way to run MPI code is from the Julia REPL directly, by using these commands:
 ```julia
 julia> using MPI
-julia> mpiexec(cmd->run(`$cmd -np 4 julia hello_mpi.jl`))
+julia> run(`$(mpiexec()) -np 4 julia hello_mpi.jl`);
 ```
 
 Now, you should see output from 4 ranks.
@@ -241,9 +253,9 @@ Now, you should see output from 4 ranks.
 
 We have installed the `MPI` package globally and it will be available in all Julia sessions. However, in some situations, we want to work with different versions of the same package or to install packages in an isolated way to avoid potential conflicts with other packages. This can be done by using local projects.
 
-A project is simply a folder in the hard disk. To use a particular folder as your project, you need to *activate* it. This is done by entering package mode and using the `activate` command followed by the path to the folder you want to activate.
+A project is simply a folder in your file system. To use a particular folder as your project, you need to *activate* it. This is done by entering package mode and using the `activate` command followed by the path to the folder you want to activate.
 ```julia
-(@v1.8) pkg> activate .
+(@v1.11) pkg> activate .
 ```
  The previous command will activate the current working directory. Note that the dot `.` is indeed the path to the current folder.
 
@@ -253,7 +265,7 @@ The prompt has changed to `(lessons) pkg>` indicating that we are in the project
     You can activate a project directly when opening Julia from the terminal using the `--project` flag. The command `$ julia --project=.` will open Julia and activate a project in the current directory. You can also achieve the same effect by setting the environment variable  `JULIA_PROJECT` with the path of the folder you want to activate.
 
 !!! note
-    The active project folder and the current working directory are two independent concepts! For instance,  `(@v1.8) pkg> activate folderB` and then `julia> cd("folderA")`, will activate the project in `folderB` and change the current working directory to `folderA`.
+    The active project folder and the current working directory are two independent concepts! For instance,  `(@v1.11) pkg> activate folderB` and then `julia> cd("folderA")`, will activate the project in `folderB` and change the current working directory to `folderA`.
 
 At this point all package-related operations will be local to the new project. For instance, install the `DataFrames` package.
 
@@ -271,7 +283,7 @@ Now, we can return to the global project to check that `DataFrames` has not been
 ```julia
 (lessons) pkg> activate
 ```
-The prompt is again `(@v1.8) pkg>`
+The prompt is again `(@v1.11) pkg>`
 
 Now, try to use `DataFrames`.
 
@@ -295,13 +307,13 @@ In other words, `Project.toml` contains the packages relevant for the user, wher
 You can see the path to the current `Project.toml` file by using the `status` operator (or `st` in its short form) while in package mode
 
 ```julia
-(@v1.8) pkg> status
+(@v1.11) pkg> status
 ```
 
 The information about the `Manifest.toml` can be inspected by passing the `-m` flag.
 
 ```julia
-(@v1.8) pkg> status -m
+(@v1.11) pkg> status -m
 ```
 
 ### Installing packages from a project file
@@ -325,7 +337,7 @@ julia> mkdir("newproject")
 
 To install all the packages registered in this file you need to activate the folder containing your `Project.toml` file
 ```julia
-(@v1.8) pkg> activate newproject
+(@v1.11) pkg> activate newproject
 ```
 and then *instantiating* it
 ```julia
@@ -339,12 +351,12 @@ The instantiate command will download and install all listed packages and their
 You can get help about a particular package operator by writing `help` in front of it
 
 ```julia
-(@v1.8) pkg> help activate
+(@v1.11) pkg> help activate
 ```
 
 You can get an overview of all package commands by typing `help` alone
 ```julia
-(@v1.8) pkg> help
+(@v1.11) pkg> help
 ```
 
 ### Package operations in Julia code
@@ -357,12 +369,65 @@ julia> Pkg.status()
 ```
 is equivalent to calling `status` in package mode.
 ```julia
-(@v1.8) pkg> status
+(@v1.11) pkg> status
 ```
 
+### Creating you own package
+
+In many situations, it is useful to create your own package, for instance, when working with a large code base, when you want to reduce compilation latency using [`Revise.jl`](https://github.com/timholy/Revise.jl),
+or if you want to eventually [register your package](https://github.com/JuliaRegistries/Registrator.jl) and share it with others.
+
+The simplest way of generating a package (called `MyPackage`) is as follows. Open Julia, go to package mode, and type
+
+```julia
+(@v1.11) pkg> generate MyPackage
+```
+
+This will crate a minimal package consisting of a new folder `MyPackage` with two files:
+
+* `MyPackage/Project.toml`: Project file defining the direct dependencies of your package.
+* `MyPackage/src/MyPackage.jl`: Main source file of your package. You can split your code in several files if needed, and include them in the package main file using function `include`.
+
+!!! tip
+    This approach only generates a very minimal package. To create a more sophisticated package skeleton (including unit testing, code coverage, readme file, licence, etc.) use
+    [`PkgTemplates.jl`](https://github.com/JuliaCI/PkgTemplates.jl) or [`BestieTemplate.jl`](https://github.com/JuliaBesties/BestieTemplate.jl). The later one is developed in Amsterdam at the
+    [Netherlands eScience Center](https://www.esciencecenter.nl/).
+
+You can add dependencies to the package by activating the `MyPackage` folder in package mode and adding new dependencies as always:
+
+```julia
+(@v1.11) pkg> activate MyPackage
+(MyPackage) pkg> add MPI
+```
+
+This will add MPI to your package dependencies.
+
+### Using your own package
+
+To use your package you first need to add it to a package environment of your choice. This is done by changing to package mode and typing `develop ` followed by the path to the folder containing the package. For instance:
+
+```julia
+(@v1.11) pkg> develop MyPackage
+```
+
+!!! note
+    You do not need to "develop" your package if you activated the package folder `MyPackage`.
+
+Now, we can go back to standard Julia mode and use it as any other package:
+
+```julia
+using MyPackage
+MyPackage.greet()
+```
+
+Here, we just called the example function defined in `MyPackage/src/MyPackage.jl`.
+
+
 ## Conclusion
 
-We have learned the basics of how to work with Julia. If you want to further dig into the topics we have covered here, you can take a look at the following links:
+We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages.
+This knowledge will allow you to follow the course effectively!
+If you want to further dig into the topics we have covered here, you can take a look at the following links:
 
 - [Julia Manual](https://docs.julialang.org/en/v1/manual/getting-started/)
 - [Package manager](https://pkgdocs.julialang.org/v1/getting-started/)

docs/src/index.md

@@ -10,7 +10,7 @@ Welcome to the interactive lecture notes of the [Programming Large-Scale Paralle
 This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam.
 We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems.
 Further information about the course is found in the study guide
-([click here](https://studiegids.vu.nl/EN/courses/2023-2024/XM_40017#/)) and our Canvas page (for registered students).
+([click here](https://studiegids.vu.nl/en/vakken/2025-2026/XM_40017#/)) and our Canvas page (for registered students).
 
 !!! note
     Material will be added incrementally to the website as the course advances.

docs/src/solutions_for_all_notebooks.md

@@ -2,19 +2,6 @@
 
 ## Julia Basics
 
-### NB1-Q1
-
-In the first, line we assign a variable to a value. In the second line, we assign another variable to the same value. Thus,we have 2 variables associated with the same value. In line 3, we associate `y` to a new value (re-assignment). Thus, we have 2 variables associated with 2 different values. Variable `x` is still associated with its original value. Thus, the value at the final line is `x=1`.
-
-### NB1-Q2
-
-It will be `1` for very similar reasons as in the previous questions: we are reassigning a local variable, not the global variable defined outside the function.
-
-### NB1-Q3
-
-It will be `6`. In the returned function `f2`, `x` is equal to `2`. Thus, when calling `f2(3)` we compute `2*3`.
-
-
 ### Exercise 1
 
 ```julia
@@ -40,94 +27,33 @@ ex2(f,g) = x -> f(x) + g(x)
 ### Exercise 3
 
 ```julia
-using GLMakie
-max_iters = 100
 n = 1000
 x = LinRange(-1.7,0.7,n)
 y = LinRange(-1.2,1.2,n)
-heatmap(x,y,(i,j)->mandel(i,j,max_iters))
+values = zeros(n,n)
+for j in 1:n
+    for i in 1:n
+        values[i,j] = surprise(x[i],y[j])
+    end
+end
+using GLMakie
+heatmap(x,y,values)
 ```
 
 ## Asynchronous programming in Julia
 
-### NB2-Q1
-
-Evaluating `compute_π(100_000_000)` takes about 0.25 seconds. Thus, the loop would take about 2.5 seconds since we are calling the function 10 times.
-
-### NB2-Q2
-
-The time in doing the loop will be almost zero since the loop just schedules 10 tasks, which should be very fast.
-
-### NB2-Q3
-
-It will take 2.5 seconds, like in question 1. The `@sync` macro forces to wait for all tasks we have generated with the `@async` macro. Since we have created 10 tasks and each of them takes about 0.25 seconds, the total time will be about 2.5 seconds.
-
-### NB2-Q4
-
-It will take about 3 seconds. The channel has buffer size 4, thus the call to `put!`will not block. The call to `take!` will not block neither since there is a value stored in the channel. The taken value is 3 and therefore we will wait for 3 seconds. 
-
-### NB2-Q5
-
-The channel is not buffered and therefore the call to `put!` will block. The cell will run forever, since there is no other task that calls `take!` on this channel. 
-
 ## Distributed computing in Julia
 
-### NB3-Q1
-
-We send the matrix (16 entries) and then we receive back the result (1 extra integer). Thus, the total number of transferred integers in 17.
-
-### NB3-Q2
-
-Even though we only use a single entry of the matrix in the remote worker, the entire matrix is captured and sent to the worker. Thus, we will transfer 17 integers like in Question 1.
-
-### NB3-Q3
-
-The value of `x` will still be zero since the worker receives a copy of the matrix and it modifies this copy, not the original one.
-
-### NB3-Q4
-
-In this case, the code `a[2]=2` is executed in the main process. Since the matrix is already in the main process, it is not needed to create and send a copy of it. Thus, the code modifies the original matrix and the value of `x` will be 2. 
-
-## Distributed computing with MPI
-
 ### Exercise 1
 
-```julia
-using MPI
-MPI.Init()
-comm = MPI.Comm_dup(MPI.COMM_WORLD)
-rank = MPI.Comm_rank(comm)
-nranks = MPI.Comm_size(comm)
-buffer = Ref(0)
-if rank == 0
-    msg = 2
-    buffer[] = msg
-    println("msg = $(buffer[])")
-    MPI.Send(buffer,comm;dest=rank+1,tag=0)
-    MPI.Recv!(buffer,comm;source=nranks-1,tag=0)
-    println("msg = $(buffer[])")
-else
-    dest = if (rank != nranks-1)
-        rank+1
-    else
-        0
-    end
-    MPI.Recv!(buffer,comm;source=rank-1,tag=0)
-    buffer[] += 1
-    println("msg = $(buffer[])")
-    MPI.Send(buffer,comm;dest,tag=0)
-end
-```
-
-### Exercise 2
-
 ```julia
 f = () -> Channel{Int}(1)
-chnls = [ RemoteChannel(f,w) for w in workers() ]
-@sync for (iw,w) in enumerate(workers())
+worker_ids = workers()
+chnls = [ RemoteChannel(f,w) for w in worker_ids ]
+@sync for (iw,w) in enumerate(worker_ids)
     @spawnat w begin
         chnl_snd = chnls[iw]
-        if w == 2
+        if iw == 1
             chnl_rcv = chnls[end]
             msg = 2
             println("msg = $msg")
@@ -145,21 +71,26 @@ chnls = [ RemoteChannel(f,w) for w in workers() ]
 end
 ```
 
-This is another possible solution.
+This is another possible solution that does not use remote channels.
 
 ```julia
-@everywhere function work(msg)
+@everywhere function work(msg,iw,worker_ids)
     println("msg = $msg")
-    if myid() != nprocs()
-        next = myid() + 1
-        @fetchfrom next work(msg+1)
+    if iw < length(worker_ids)
+        inext = iw+1
+        next = worker_ids[iw+1]
+        @fetchfrom next work(msg+1,inext,worker_ids)
     else
-        @fetchfrom 2 println("msg = $msg")
+        @fetchfrom worker_ids[1] println("msg = $msg")
     end
+    return nothing
 end
 msg = 2
-@fetchfrom 2 work(msg)
+iw = 1
+worker_ids = workers()
+@fetchfrom worker_ids[iw] work(msg,iw,worker_ids)
 ```
+
 ## Matrix-matrix multiplication
 
 ### Exercise 1
@@ -209,62 +140,306 @@ end
 end
 ```
 
+## MPI (Point-to-point)
+
+### Exercise 1
+
+```julia
+function matmul_mpi_3!(C,A,B)
+    comm = MPI.COMM_WORLD
+    rank = MPI.Comm_rank(comm)
+    P = MPI.Comm_size(comm)
+    if  rank == 0
+        N = size(A,1)
+        myB = B
+        for dest in 1:(P-1)
+            MPI.Send(B,comm;dest)
+        end
+    else
+        source = 0
+        status = MPI.Probe(comm,MPI.Status;source)
+        count = MPI.Get_count(status,eltype(B))
+        N = Int(sqrt(count))
+        myB = zeros(N,N)
+        MPI.Recv!(myB,comm;source)
+    end
+    L = div(N,P)
+    myA = zeros(L,N)
+    if  rank == 0
+        lb = L*rank+1
+        ub = L*(rank+1)
+        myA[:,:] = view(A,lb:ub,:)
+        for dest in 1:(P-1)
+            lb = L*dest+1
+            ub = L*(dest+1)
+            MPI.Send(view(A,lb:ub,:),comm;dest)
+        end
+    else
+        source = 0
+        MPI.Recv!(myA,comm;source)
+    end
+    myC = myA*myB
+    if rank == 0
+        lb = L*rank+1
+        ub = L*(rank+1)
+        C[lb:ub,:] = myC
+        for source in 1:(P-1)
+            lb = L*source+1
+            ub = L*(source+1)
+            MPI.Recv!(view(C,lb:ub,:),comm;source)
+        end
+    else
+        dest = 0
+        MPI.Send(myC,comm;dest)
+    end
+    C
+end
+```
+
 ### Exercise 2
 
-At each call to @spawnat we will communicate O(N) and compute O(N) in a worker process just like in algorithm 1. However, we will do this work N^2/P times on average at each worker. Thus, the total communication and computation on a worker will be O(N^3/P) for both communication and computation.  Thus, the communication over computation ratio will still be O(1) and thus the communication will dominate in practice, making the algorithm inefficient.
+```julia
+using MPI
+MPI.Init()
+comm = MPI.COMM_WORLD
+rank = MPI.Comm_rank(comm)
+nranks = MPI.Comm_size(comm)
+buffer = Ref(0)
+if rank == 0
+    msg = 2
+    buffer[] = msg
+    println("msg = $(buffer[])")
+    MPI.Send(buffer,comm;dest=rank+1,tag=0)
+    MPI.Recv!(buffer,comm;source=nranks-1,tag=0)
+    println("msg = $(buffer[])")
+else
+    dest = if (rank != nranks-1)
+        rank+1
+    else
+        0
+    end
+    MPI.Recv!(buffer,comm;source=rank-1,tag=0)
+    buffer[] += 1
+    println("msg = $(buffer[])")
+    MPI.Send(buffer,comm;dest,tag=0)
+end
+```
+## MPI (collectives)
+
+### Exercise 1
+
+```julia
+function matmul_mpi_3!(C,A,B)
+    comm = MPI.COMM_WORLD
+    rank = MPI.Comm_rank(comm)
+    P = MPI.Comm_size(comm)
+    root = 0
+    if  rank == root
+        N = size(A,1)
+        Nref = Ref(N)
+    else
+        Nref = Ref(0)
+    end
+    MPI.Bcast!(Nref,comm;root)
+    N = Nref[]
+    if  rank == root
+        myB = B
+    else
+        myB = zeros(N,N)
+    end
+    MPI.Bcast!(myB,comm;root)
+    L = div(N,P)
+    # Tricky part
+    # Julia works "col major"
+    myAt = zeros(N,L)
+    At = collect(transpose(A))
+    MPI.Scatter!(At,myAt,comm;root)
+    myCt = transpose(myB)*myAt
+    Ct = similar(C)
+    MPI.Gather!(myCt,Ct,comm;root)
+    C .= transpose(Ct)
+    C
+end
+```
+
+This other solution uses a column partition instead of a row partition.
+It is more natural to work with column partitions in Julia if possible since matrices are
+in "col major" format. Note that we do not need all the auxiliary transposes anymore.
+
+```julia
+function matmul_mpi_3!(C,A,B)
+    comm = MPI.COMM_WORLD
+    rank = MPI.Comm_rank(comm)
+    P = MPI.Comm_size(comm)
+    root = 0
+    if  rank == root
+        N = size(A,1)
+        Nref = Ref(N)
+    else
+        Nref = Ref(0)
+    end
+    MPI.Bcast!(Nref,comm;root)
+    N = Nref[]
+    if  rank == root
+        myA = A
+    else
+        myA = zeros(N,N)
+    end
+    MPI.Bcast!(myA,comm;root)
+    L = div(N,P)
+    myB = zeros(N,L)
+    MPI.Scatter!(B,myB,comm;root)
+    myC = myA*myB
+    MPI.Gather!(myC,C,comm;root)
+    C
+end
+```
 
 ## Jacobi method
 
 ### Exercise 1
 
 ```julia
-@everywhere workers() begin
-    using MPI
-    comm = MPI.Comm_dup(MPI.COMM_WORLD)
-    function jacobi_mpi(n,niters)
-        nranks = MPI.Comm_size(comm)
-        rank = MPI.Comm_rank(comm)
-        if mod(n,nranks) != 0
-            println("n must be a multiple of nranks")
-            MPI.Abort(comm,1)
+function jacobi_mpi(n,niters)
+    u, u_new = init(n,comm)
+    load = length(u)-2
+    rank = MPI.Comm_rank(comm)
+    nranks = MPI.Comm_size(comm)
+    nreqs = 2*((rank != 0) + (rank != (nranks-1)))
+    reqs = MPI.MultiRequest(nreqs)
+    for t in 1:niters
+        ireq = 0
+        if rank != 0
+            neig_rank = rank-1
+            u_snd = view(u,2:2)
+            u_rcv = view(u,1:1)
+            dest = neig_rank
+            source = neig_rank
+            ireq += 1
+            MPI.Isend(u_snd,comm,reqs[ireq];dest)
+            ireq += 1
+            MPI.Irecv!(u_rcv,comm,reqs[ireq];source)
         end
-        n_own = div(n,nranks)
-        u = zeros(n_own+2)
-        u[1] = -1
-        u[end] = 1
-        u_new = copy(u)
-        for t in 1:niters
-            reqs = MPI.Request[]
-            if rank != 0
-                neig_rank = rank-1
-                req = MPI.Isend(view(u,2:2),comm,dest=neig_rank,tag=0)
-                push!(reqs,req)
-                req = MPI.Irecv!(view(u,1:1),comm,source=neig_rank,tag=0)
-                push!(reqs,req)
-            end
-            if rank != (nranks-1)
-                neig_rank = rank+1
-                s = n_own+1
-                r = n_own+2
-                req = MPI.Isend(view(u,s:s),comm,dest=neig_rank,tag=0)
-                push!(reqs,req)
-                req = MPI.Irecv!(view(u,r:r),comm,source=neig_rank,tag=0)
-                push!(reqs,req)
-            end
-            for i in 3:n_own
-                u_new[i] = 0.5*(u[i-1]+u[i+1])
-            end
-            MPI.Waitall(reqs)
-            for i in (2,n_own+1)
-                u_new[i] = 0.5*(u[i-1]+u[i+1])
-            end
-            u, u_new = u_new, u
+        if rank != (nranks-1)
+            neig_rank = rank+1
+            u_snd = view(u,(load+1):(load+1))
+            u_rcv = view(u,(load+2):(load+2))
+            dest = neig_rank
+            source = neig_rank
+            ireq += 1
+            MPI.Isend(u_snd,comm,reqs[ireq];dest)
+            ireq += 1
+            MPI.Irecv!(u_rcv,comm,reqs[ireq];source)
         end
-        return u
+        # Upload interior cells
+        for i in 3:load
+            u_new[i] = 0.5*(u[i-1]+u[i+1])
+        end
+        # Wait for the communications to finish
+        MPI.Waitall(reqs)
+        # Update boundaries
+        for i in (2,load+1)
+            u_new[i] = 0.5*(u[i-1]+u[i+1])
+        end
+        u, u_new = u_new, u
     end
+    return u
+end
+```
+
+### Exercise 2
+
+```julia
+function jacobi_mpi(n,niters,tol,comm) # new tol arg
+    u, u_new = init(n,comm)
+    load = length(u)-2
+    rank = MPI.Comm_rank(comm)
+    nranks = MPI.Comm_size(comm)
+    nreqs = 2*((rank != 0) + (rank != (nranks-1)))
+    reqs = MPI.MultiRequest(nreqs)
+    for t in 1:niters
+        ireq = 0
+        if rank != 0
+            neig_rank = rank-1
+            u_snd = view(u,2:2)
+            u_rcv = view(u,1:1)
+            dest = neig_rank
+            source = neig_rank
+            ireq += 1
+            MPI.Isend(u_snd,comm,reqs[ireq];dest)
+            ireq += 1
+            MPI.Irecv!(u_rcv,comm,reqs[ireq];source)
+        end
+        if rank != (nranks-1)
+            neig_rank = rank+1
+            u_snd = view(u,(load+1):(load+1))
+            u_rcv = view(u,(load+2):(load+2))
+            dest = neig_rank
+            source = neig_rank
+            ireq += 1
+            MPI.Isend(u_snd,comm,reqs[ireq];dest)
+            ireq += 1
+            MPI.Irecv!(u_rcv,comm,reqs[ireq];source)
+        end
+        MPI.Waitall(reqs)
+        # Compute the max diff in the current
+        # rank while doing the local update
+        mydiff = 0.0
+        for i in 2:load+1
+            u_new[i] = 0.5*(u[i-1]+u[i+1])
+            diff_i = abs(u_new[i] - u[i])
+            mydiff = max(mydiff,diff_i)
+        end
+        # Now we need to find the global diff
+        diff_ref = Ref(mydiff)
+        MPI.Allreduce!(diff_ref,max,comm)
+        diff = diff_ref[]
+        # If global diff below tol, stop!
+        if diff < tol
+            return u_new
+        end
+        u, u_new = u_new, u
+    end
+    return u
+end
+```
+
+## All pairs of shortest paths
+
+### Exercise 1
+
+```julia
+function floyd_iterations!(myC,comm)
+    L = size(myC,1)
+    N = size(myC,2)
+    rank = MPI.Comm_rank(comm)
+    P = MPI.Comm_size(comm)
+    lb = L*rank+1
+    ub = L*(rank+1)
+    C_k = similar(myC,N)
+    for k in 1:N
+        if (lb<=k) && (k<=ub)
+            # If I have the row, fill in the buffer
+            myk = (k-lb)+1
+            C_k[:] = view(myC,myk,:)
+        end
+        # We need to find out the owner of row k.
+        # Easy since N is a multiple of P
+        root = div(k-1,L)
+        MPI.Bcast!(C_k,comm;root)
+        # Now, we have the data dependencies and
+        # we can do the updates locally
+        for j in 1:N
+            for i in 1:L
+                myC[i,j] = min(myC[i,j],myC[i,k]+C_k[j])
+            end
+        end
+    end
+    myC
 end
 ```
 
 
 
 
+

notebooks/figures/fig_mpi.svg

@@ -365,6 +365,84 @@
          id="path8527-8"
          inkscape:connector-curvature="0" />
     </marker>
+    <marker
+       style="overflow:visible"
+       id="marker8529-3-3"
+       refX="0"
+       refY="0"
+       orient="auto"
+       inkscape:stockid="Arrow2Mend"
+       inkscape:isstock="true"
+       inkscape:collect="always">
+      <path
+         transform="scale(-0.6)"
+         d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+         style="fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#0000ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+         id="path8527-8-6"
+         inkscape:connector-curvature="0" />
+    </marker>
+    <marker
+       inkscape:isstock="true"
+       inkscape:stockid="Arrow2Mend"
+       orient="auto"
+       refY="0"
+       refX="0"
+       id="marker4031-1"
+       style="overflow:visible"
+       inkscape:collect="always">
+      <path
+         inkscape:connector-curvature="0"
+         id="path4029-0"
+         style="fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#0000ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+         d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+         transform="scale(-0.6)" />
+    </marker>
+    <marker
+       style="overflow:visible"
+       id="marker4223-6"
+       refX="0"
+       refY="0"
+       orient="auto"
+       inkscape:stockid="Arrow2Mend"
+       inkscape:isstock="true">
+      <path
+         transform="scale(-0.6)"
+         d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+         style="fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#0000ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+         id="path4221-3"
+         inkscape:connector-curvature="0" />
+    </marker>
+    <marker
+       inkscape:isstock="true"
+       inkscape:stockid="Arrow2Mend"
+       orient="auto"
+       refY="0"
+       refX="0"
+       id="marker4031-5"
+       style="overflow:visible"
+       inkscape:collect="always">
+      <path
+         inkscape:connector-curvature="0"
+         id="path4029-3"
+         style="fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#0000ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+         d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+         transform="scale(-0.6)" />
+    </marker>
+    <marker
+       style="overflow:visible"
+       id="marker4223-5"
+       refX="0"
+       refY="0"
+       orient="auto"
+       inkscape:stockid="Arrow2Mend"
+       inkscape:isstock="true">
+      <path
+         transform="scale(-0.6)"
+         d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z"
+         style="fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#0000ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1"
+         id="path4221-6"
+         inkscape:connector-curvature="0" />
+    </marker>
   </defs>
   <sodipodi:namedview
      id="base"
@@ -373,9 +451,9 @@
      borderopacity="1.0"
      inkscape:pageopacity="0.0"
      inkscape:pageshadow="2"
-     inkscape:zoom="0.70000001"
-     inkscape:cx="4005.3666"
-     inkscape:cy="58.38566"
+     inkscape:zoom="0.98994951"
+     inkscape:cx="6863.501"
+     inkscape:cy="332.85381"
      inkscape:document-units="mm"
      inkscape:current-layer="layer1"
      inkscape:document-rotation="0"
@@ -2678,5 +2756,439 @@
            y="194.96161"
            x="1153.2484">20</tspan></text>
     </g>
+    <g
+       transform="translate(192.3794,-112.09032)"
+       id="g5148-2"
+       inkscape:export-filename="/home/francesc/Downloads/g5148.png"
+       inkscape:export-xdpi="200"
+       inkscape:export-ydpi="200">
+      <path
+         style="fill:none;stroke:#000000;stroke-width:0.529167;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+         d="m 1359.9626,79.585912 h 20.9409"
+         id="path978-3-6-3-0"
+         sodipodi:nodetypes="cc"
+         inkscape:connector-curvature="0" />
+      <path
+         inkscape:connector-curvature="0"
+         sodipodi:nodetypes="cc"
+         id="path1168-6-1-6-6"
+         d="m 1314.5268,79.585912 h 20.9408"
+         style="fill:none;stroke:#000000;stroke-width:0.529167;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+      <g
+         transform="translate(1155.5468,183.31702)"
+         id="g1148-7-5-75-1">
+        <path
+           inkscape:connector-curvature="0"
+           id="path1063-5-5-3-5"
+           d="m 129.89807,-136.23112 12.99038,7.5 17.32051,-10 -12.99038,-7.5 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="cccc"
+           id="path1065-3-4-5-5"
+           d="m 142.88845,-128.73112 v 39.999996 l 17.32051,-10 v -39.999996"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="ccc"
+           id="path1067-56-7-62-4"
+           d="m 129.89807,-136.23112 v 39.999996 l 12.99038,7.5"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1069-2-6-9-7"
+           d="m 147.21857,-126.23112 v 15 l 4.33013,-2.5 v -15 l -4.33013,2.5"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1071-91-5-1-6"
+           d="m 145.48652,-96.231123 v 1.999999 l 12.99038,-7.499996 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1073-2-6-2-5"
+           d="m 145.48652,-100.23111 v 2.000003 l 12.99038,-7.500013 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1075-7-9-7-6"
+           d="m 145.48652,-104.23111 v 2 l 12.99038,-7.50001 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+      </g>
+      <g
+         transform="translate(1163.1327,183.31701)"
+         id="g1157-09-3-0-9">
+        <path
+           inkscape:connector-curvature="0"
+           id="path1086-3-7-93-3"
+           d="m 169.05642,-136.23112 12.99038,7.5 17.32051,-10 -12.99038,-7.5 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="cccc"
+           id="path1088-6-4-6-7"
+           d="m 182.0468,-128.73112 v 39.999996 l 17.32051,-10 v -39.999996"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="ccc"
+           id="path1090-0-5-0-4"
+           d="m 169.05642,-136.23112 v 39.999996 l 12.99038,7.5"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1092-6-2-6-5"
+           d="m 186.37692,-126.23112 v 15 l 4.33013,-2.5 v -15 l -4.33013,2.5"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1094-2-5-2-2"
+           d="m 184.64487,-96.231123 v 1.999999 l 12.99038,-7.499996 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1096-6-4-6-5"
+           d="m 184.64487,-100.23111 v 2.000003 l 12.99038,-7.500013 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1098-1-7-1-4"
+           d="m 184.64487,-104.23111 v 2 l 12.99038,-7.50001 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+      </g>
+      <text
+         y="15.986771"
+         x="1244.8425"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1130-2);stroke-width:1.87007797"
+         id="text1128-7-8-4-8-7"
+         xml:space="preserve"><tspan
+           style="stroke-width:1.87007797"
+           id="tspan78-79-4-7-4"
+           y="103.6842"
+           x="1289.8303">worker 1</tspan></text>
+      <text
+         y="15.986771"
+         x="1294.8483"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1184);stroke-width:1.87007797"
+         id="text1182-0-0-2-3"
+         xml:space="preserve"><tspan
+           style="stroke-width:1.87007797"
+           id="tspan83-2-7-0-0"
+           y="103.6842"
+           x="1339.8361">worker 2</tspan></text>
+      <g
+         id="g970-7-6-3-8"
+         transform="translate(1209.6376,183.31701)">
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 169.05642,-136.23112 12.99038,7.5 17.32051,-10 -12.99038,-7.5 z"
+           id="path956-5-8-7-6"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 182.0468,-128.73112 v 39.999996 l 17.32051,-10 v -39.999996"
+           id="path958-92-8-5-8"
+           sodipodi:nodetypes="cccc"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 169.05642,-136.23112 v 39.999996 l 12.99038,7.5"
+           id="path960-2-4-9-8"
+           sodipodi:nodetypes="ccc"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 186.37692,-126.23112 v 15 l 4.33013,-2.5 v -15 l -4.33013,2.5"
+           id="path962-8-3-2-4"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 184.64487,-96.231123 v 1.999999 l 12.99038,-7.499996 v -2 z"
+           id="path964-9-1-2-3"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 184.64487,-100.23111 v 2.000003 l 12.99038,-7.500013 v -2 z"
+           id="path966-7-4-8-1"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 184.64487,-104.23111 v 2 l 12.99038,-7.50001 v -2 z"
+           id="path968-3-9-9-4"
+           inkscape:connector-curvature="0" />
+      </g>
+      <text
+         xml:space="preserve"
+         id="text976-6-2-7-9"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1184);stroke-width:1.87007797"
+         x="1342.9568"
+         y="15.986771"><tspan
+           x="1387.9445"
+           y="103.6842"
+           id="tspan974-1-0-3-2"
+           style="stroke-width:1.87007797">worker 3</tspan></text>
+      <text
+         y="-54.098297"
+         x="1247.6248"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1130-2);fill:#0000ff;stroke-width:1.87007797"
+         id="text1128-7-8-7-3-7-0-06"
+         xml:space="preserve"><tspan
+           style="fill:#0000ff;stroke-width:1.87007797"
+           id="tspan78-79-8-2-1-3-8"
+           y="33.599152"
+           x="1292.6124">msg</tspan></text>
+      <path
+         sodipodi:nodetypes="cc"
+         inkscape:connector-curvature="0"
+         id="path1386-9"
+         d="m 1304.0316,27.773394 c 16.0706,-14.103005 27.5564,-13.085868 41.886,-0.615601"
+         style="fill:none;stroke:#0000ff;stroke-width:0.76499999;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker8529-3-3)" />
+      <text
+         y="-54.561871"
+         x="1290.4174"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1130-2);fill:#0000ff;stroke-width:1.87007797"
+         id="text1128-7-8-7-3-7-0-0-2"
+         xml:space="preserve"><tspan
+           style="fill:#0000ff;stroke-width:1.87007797"
+           id="tspan78-79-8-2-1-3-6-6"
+           y="33.135578"
+           x="1335.405">msg+=1</tspan></text>
+      <text
+         y="-53.871326"
+         x="1338.3951"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1130-2);fill:#0000ff;stroke-width:1.87007797"
+         id="text1128-7-8-7-3-7-0-0-3-6"
+         xml:space="preserve"><tspan
+           style="fill:#0000ff;stroke-width:1.87007797"
+           id="tspan78-79-8-2-1-3-6-2-4"
+           y="33.826126"
+           x="1383.3828">msg+=1</tspan></text>
+      <path
+         style="fill:none;stroke:#0000ff;stroke-width:0.76499999;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker4031-1)"
+         d="m 1354.3024,27.773394 c 16.0706,-14.103005 27.5564,-13.085868 41.886,-0.615601"
+         id="path4027-9"
+         inkscape:connector-curvature="0"
+         sodipodi:nodetypes="cc" />
+      <path
+         sodipodi:nodetypes="cc"
+         inkscape:connector-curvature="0"
+         id="path4219-5"
+         d="m 1402.8962,28.340358 c -14.7223,-30.9560949 -88.632,-34.6967981 -104.63,-1.371553"
+         style="fill:none;stroke:#0000ff;stroke-width:0.76499999;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker4223-6)" />
+    </g>
+    <g
+       id="g2038"
+       inkscape:export-xdpi="200"
+       inkscape:export-ydpi="200">
+      <path
+         style="fill:none;stroke:#000000;stroke-width:0.529167;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+         d="m 1861.8989,152.34405 h 20.9409"
+         id="path978-3-6-3-1"
+         sodipodi:nodetypes="cc"
+         inkscape:connector-curvature="0" />
+      <path
+         inkscape:connector-curvature="0"
+         sodipodi:nodetypes="cc"
+         id="path1168-6-1-6-2"
+         d="m 1816.4631,152.34405 h 20.9408"
+         style="fill:none;stroke:#000000;stroke-width:0.529167;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+      <g
+         transform="translate(1657.4831,256.07516)"
+         id="g1148-7-5-75-7">
+        <path
+           inkscape:connector-curvature="0"
+           id="path1063-5-5-3-0"
+           d="m 129.89807,-136.23112 12.99038,7.5 17.32051,-10 -12.99038,-7.5 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="cccc"
+           id="path1065-3-4-5-9"
+           d="m 142.88845,-128.73112 v 39.999996 l 17.32051,-10 v -39.999996"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="ccc"
+           id="path1067-56-7-62-3"
+           d="m 129.89807,-136.23112 v 39.999996 l 12.99038,7.5"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1069-2-6-9-6"
+           d="m 147.21857,-126.23112 v 15 l 4.33013,-2.5 v -15 l -4.33013,2.5"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1071-91-5-1-0"
+           d="m 145.48652,-96.231123 v 1.999999 l 12.99038,-7.499996 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1073-2-6-2-6"
+           d="m 145.48652,-100.23111 v 2.000003 l 12.99038,-7.500013 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1075-7-9-7-2"
+           d="m 145.48652,-104.23111 v 2 l 12.99038,-7.50001 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+      </g>
+      <g
+         transform="translate(1665.069,256.07515)"
+         id="g1157-09-3-0-6">
+        <path
+           inkscape:connector-curvature="0"
+           id="path1086-3-7-93-1"
+           d="m 169.05642,-136.23112 12.99038,7.5 17.32051,-10 -12.99038,-7.5 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="cccc"
+           id="path1088-6-4-6-8"
+           d="m 182.0468,-128.73112 v 39.999996 l 17.32051,-10 v -39.999996"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="ccc"
+           id="path1090-0-5-0-7"
+           d="m 169.05642,-136.23112 v 39.999996 l 12.99038,7.5"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1092-6-2-6-9"
+           d="m 186.37692,-126.23112 v 15 l 4.33013,-2.5 v -15 l -4.33013,2.5"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1094-2-5-2-20"
+           d="m 184.64487,-96.231123 v 1.999999 l 12.99038,-7.499996 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1096-6-4-6-2"
+           d="m 184.64487,-100.23111 v 2.000003 l 12.99038,-7.500013 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+        <path
+           inkscape:connector-curvature="0"
+           id="path1098-1-7-1-3"
+           d="m 184.64487,-104.23111 v 2 l 12.99038,-7.50001 v -2 z"
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" />
+      </g>
+      <text
+         y="88.744911"
+         x="1746.7788"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1130-2);stroke-width:1.87007797"
+         id="text1128-7-8-4-8-75"
+         xml:space="preserve"><tspan
+           style="stroke-width:1.87007797"
+           id="tspan78-79-4-7-9"
+           y="176.44234"
+           x="1791.7666">rank 0<tspan
+   id="tspan76-2-3-9-2"
+   style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Hack;-inkscape-font-specification:Hack;stroke-width:1.87007797" /></tspan></text>
+      <text
+         y="88.744911"
+         x="1796.7845"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1184);stroke-width:1.87007797"
+         id="text1182-0-0-2-2"
+         xml:space="preserve"><tspan
+           style="stroke-width:1.87007797"
+           id="tspan83-2-7-0-8"
+           y="176.44234"
+           x="1841.7723">rank 1<tspan
+   id="tspan81-3-8-2-9"
+   style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Hack;-inkscape-font-specification:Hack;stroke-width:1.87007797" /></tspan></text>
+      <g
+         id="g970-7-6-3-7"
+         transform="translate(1711.5739,256.07515)">
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 169.05642,-136.23112 12.99038,7.5 17.32051,-10 -12.99038,-7.5 z"
+           id="path956-5-8-7-3"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 182.0468,-128.73112 v 39.999996 l 17.32051,-10 v -39.999996"
+           id="path958-92-8-5-6"
+           sodipodi:nodetypes="cccc"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.79374999;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 169.05642,-136.23112 v 39.999996 l 12.99038,7.5"
+           id="path960-2-4-9-1"
+           sodipodi:nodetypes="ccc"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 186.37692,-126.23112 v 15 l 4.33013,-2.5 v -15 l -4.33013,2.5"
+           id="path962-8-3-2-2"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 184.64487,-96.231123 v 1.999999 l 12.99038,-7.499996 v -2 z"
+           id="path964-9-1-2-9"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 184.64487,-100.23111 v 2.000003 l 12.99038,-7.500013 v -2 z"
+           id="path966-7-4-8-3"
+           inkscape:connector-curvature="0" />
+        <path
+           style="fill:#e6e6e6;stroke:#000000;stroke-width:0.529167;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+           d="m 184.64487,-104.23111 v 2 l 12.99038,-7.50001 v -2 z"
+           id="path968-3-9-9-1"
+           inkscape:connector-curvature="0" />
+      </g>
+      <text
+         xml:space="preserve"
+         id="text976-6-2-7-94"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1184);stroke-width:1.87007797"
+         x="1844.8931"
+         y="88.744911"><tspan
+           x="1889.8807"
+           y="176.44234"
+           id="tspan974-1-0-3-7"
+           style="stroke-width:1.87007797">rank 2</tspan></text>
+      <text
+         y="-2.3611109"
+         x="1822.4244"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1130-2);fill:#0000ff;stroke-width:1.87007797"
+         id="text1128-7-8-7-3-7-0-0-3-61"
+         xml:space="preserve"><tspan
+           style="fill:#0000ff;stroke-width:1.87007797"
+           id="tspan78-79-8-2-1-3-6-2-0"
+           y="85.336334"
+           x="1867.4121">msg</tspan></text>
+      <path
+         style="fill:none;stroke:#0000ff;stroke-width:0.76499999;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker4031-5)"
+         d="m 1854.3678,99.061558 c 16.0706,-14.103008 27.5564,-13.085871 41.886,-0.615604"
+         id="path4027-6"
+         inkscape:connector-curvature="0"
+         sodipodi:nodetypes="cc" />
+      <text
+         y="18.408323"
+         x="1793.9163"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1130-2);fill:#0000ff;stroke-width:1.87007797"
+         id="text1128-7-8-7-3-7-0-0-3-61-2"
+         xml:space="preserve"><tspan
+           style="fill:#0000ff;stroke-width:1.87007797"
+           id="tspan78-79-8-2-1-3-6-2-0-0"
+           y="106.10577"
+           x="1838.9039">MPI_Send</tspan></text>
+      <text
+         xml:space="preserve"
+         id="text1992"
+         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.93749809px;line-height:1.25;font-family:Hack;-inkscape-font-specification:Hack;letter-spacing:0px;word-spacing:0px;white-space:pre;shape-inside:url(#rect1130-2);fill:#0000ff;stroke-width:1.87007797"
+         x="1841.223"
+         y="18.408323"><tspan
+           x="1886.2107"
+           y="106.10577"
+           id="tspan1990"
+           style="fill:#0000ff;stroke-width:1.87007797">MPI_Recv</tspan><tspan
+           id="tspan1994"
+           x="1886.2107"
+           y="106.10577"
+           style="fill:#0000ff;stroke-width:1.87007797" /></text>
+    </g>
   </g>
 </svg>

notebooks/julia_async.ipynb

@@ -28,6 +28,56 @@
     "Understanding these concepts is important to learn distributed computing later."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "cde5ee75",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-block alert-info\">\n",
+    "<b>Note:</b> Do not forget to execute the next cell before starting this notebook! \n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0b0496c7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "function why_q1()\n",
+    "    msg = \"\"\"\n",
+    "    Evaluating compute_π(100_000_000) takes about 0.25 seconds on the teacher's laptop. Thus, the loop would take about 2.5 seconds since we are calling the function 10 times.\n",
+    "    \"\"\"\n",
+    "    println(msg)\n",
+    "end\n",
+    "function why_q2()\n",
+    "    msg = \"\"\"\n",
+    "    The time in doing the loop will be O(1) since the loop just schedules 10 tasks, which should be a (small) constant time independent of n.\n",
+    "    \"\"\"\n",
+    "    println(msg)\n",
+    "end\n",
+    "function why_q3()\n",
+    "    msg = \"\"\"\n",
+    "    It will take 2.5 seconds, like in question 1. The @sync macro forces to wait for all tasks we have generated with the @async macro. Since we have created 10 tasks and each of them takes about 0.25 seconds, the total time will be about 2.5 seconds.\n",
+    "    \"\"\"\n",
+    "    println(msg)\n",
+    "end\n",
+    "function why_q4()\n",
+    "    msg = \"\"\"\n",
+    "    It will take about 3 seconds. The channel has buffer size 4, thus the call to put!will not block. The call to take! will not block neither since there is a value stored in the channel. The taken value is 3 and therefore we will wait for 3 seconds.\n",
+    "    \"\"\"\n",
+    "    println(msg)\n",
+    "end\n",
+    "function why_q5()\n",
+    "    msg = \"\"\"\n",
+    "    The channel is not buffered and therefore the call to put! will block. The cell will run forever, since there is no other task that calls take! on this channel.\n",
+    "    \"\"\"\n",
+    "    println(msg)\n",
+    "end\n",
+    "println(\"🥳 Well done! \")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "caf64254",
@@ -37,7 +87,7 @@
     "\n",
     "### Creating  a task\n",
     "\n",
-    "Technically, a task in Julia is a *symmetric co-routine*. More informally, a task is a piece of computation work that can be started (scheduled) at some point in the future, and that can be interrupted and resumed.  To create a task, we first need to create a function that represents the work to be done in the task. In next cell, we generate a task that generates and sums two matrices."
+    "Technically, a task in Julia is a *symmetric* [*co-routine*](https://en.wikipedia.org/wiki/Coroutine). More informally, a task is a piece of computational work that can be started (scheduled) at some point in the future, and that can be interrupted and resumed.  To create a task, we first need to create a function that represents the work to be done in the task. In the next cell, we generate a task that generates and sums two matrices."
    ]
   },
   {
@@ -233,7 +283,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "fun = () -> compute_π(4_000_000_000)\n",
+    "fun = () -> @show compute_π(4_000_000_000)\n",
     "t = Task(fun)"
    ]
   },
@@ -272,7 +322,7 @@
    "source": [
     "### `yield`\n",
     "\n",
-    "If tasks do not run in parallel, what is the purpose of tasks? Tasks are handy since they can be interrupted and to switch control to other tasks. This is achieved via function `yield`. When we call yield, we provide the opportunity to switch to another task. The function below is a variation of function `compute_π` in which we yield every 1000 iterations. At the call to yield we allow other tasks to take over. Without this call to yield, once we start function `compute_π` we cannot start any other tasks until this function finishes."
+    "If tasks do not run in parallel, what is the purpose of tasks? Tasks are handy since they can be interrupted and to switch control to other tasks. This is achieved via function `yield`. When we call `yield`, we provide the opportunity to switch to another task. The function below is a variation of function `compute_π` in which we `yield` every 1000 iterations. At the call to `yield` we allow other tasks to take over. Without this call to `yield`, once we start function `compute_π` we cannot start any other tasks until this function finishes."
    ]
   },
   {
@@ -299,7 +349,7 @@
    "id": "69fd4131",
    "metadata": {},
    "source": [
-    "You can check this behavior experimentally with the two following cells. The next one creates and schedules a task that computes pi with the function `compute_π_yield`. Note that you can run the 2nd cell bellow while this task is running since we call to yield often inside  `compute_π_yield`."
+    "You can check this behavior experimentally with the two following cells. The next one creates and schedules a task that computes pi with the function `compute_π_yield`. Note that you can run the 2nd cell bellow while this task is running since we call to `yield` often inside  `compute_π_yield`."
    ]
   },
   {
@@ -309,7 +359,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "fun = () -> compute_π_yield(3_000_000_000)\n",
+    "fun = () -> @show compute_π_yield(3_000_000_000)\n",
     "t = Task(fun)\n",
     "schedule(t)"
    ]
@@ -331,7 +381,7 @@
    "source": [
     "### Example: Implementing function sleep\n",
     "\n",
-    "Using yield, we can implement our own version of the sleep function as follows:"
+    "Using `yield`, we can implement our own version of the sleep function as follows:"
    ]
   },
   {
@@ -434,7 +484,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "fun = () -> a + b\n",
+    "fun = () -> add(a, b)\n",
     "t = Task(fun)\n",
     "schedule(t)"
    ]
@@ -679,6 +729,19 @@
     "take!(chnl)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "8a1ef849",
+   "metadata": {},
+   "source": [
+    "In summary:\n",
+    "\n",
+    "- `put!` will wait for a `take!` if there is not space left in the channel's buffer.\n",
+    "- `take!` will wait for a `put!` if there is no data to be consumed in the channel.\n",
+    "- `put!` will raise an error if the channel is closed.\n",
+    "- `take!` will raise an error if the channel is closed *and* empty."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "9ddd66ca",
@@ -696,7 +759,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "t = @elapsed compute_π(100_000_000)"
+    "n = 140_000_000\n",
+    "t = @elapsed @show compute_π(n)"
    ]
   },
   {
@@ -711,7 +775,7 @@
     "    a) 10*t\n",
     "    b) t\n",
     "    c) 0.1*t\n",
-    "    d) near 0*t \n"
+    "    d) O(1), i.e. time independent from n \n"
    ]
   },
   {
@@ -721,11 +785,22 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "n = 140_000_000\n",
     "@time for i in 1:10\n",
-    "    compute_π(100_000_000)\n",
+    "    @show compute_π(n)\n",
     "end"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d6b8382e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "why_q1()"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "5f19d38c",
@@ -738,7 +813,7 @@
     "    a) 10*t\n",
     "    b) t\n",
     "    c) 0.1*t\n",
-    "    d) near 0*t \n",
+    "    d) O(1) \n",
     "\n"
    ]
   },
@@ -749,11 +824,22 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "n = 140_000_000\n",
     "@time for i in 1:10\n",
-    "    @async compute_π(100_000_000)\n",
+    "    @async @show compute_π(n)\n",
     "end"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "edff9747",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "why_q2()"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "5041c355",
@@ -766,7 +852,7 @@
     "    a) 10*t\n",
     "    b) t\n",
     "    c) 0.1*t\n",
-    "    d) near 0*t \n"
+    "    d) O(1) \n"
    ]
   },
   {
@@ -776,11 +862,22 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "n = 140_000_000\n",
     "@time @sync for i in 1:10\n",
-    "    @async compute_π(100_000_000)\n",
+    "    @async @show compute_π(n)\n",
     "end"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "87bc7c5c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "why_q3()"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "841b690e",
@@ -792,7 +889,7 @@
     "\n",
     "    a) infinity\n",
     "    b) 1 second\n",
-    "    c) near 0 seconds\n",
+    "    c) less than 1 seconds\n",
     "    d) 3 seconds"
    ]
   },
@@ -821,6 +918,16 @@
     "end"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a18a0a7d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "why_q4()"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "df663f11",
@@ -832,7 +939,7 @@
     "\n",
     "    a) infinity\n",
     "    b) 1 second\n",
-    "    c) near 0 seconds\n",
+    "    c) less than 1 seconds\n",
     "    d) 3 seconds"
    ]
   },
@@ -860,6 +967,42 @@
     "end"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d8923fae",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "why_q5()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0ee77abe",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-block alert-info\">\n",
+    "<b>Note:</b> If for some reason a cell keeps running forever, we can stop it with Kernel > Interrupt or Kernel > Restart (see tabs above).\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dfab0c90",
+   "metadata": {},
+   "source": [
+    "## Summary\n",
+    "\n",
+    "In order to start \"thinking in parallel\" you first need to be familiar with concepts of asynchronous programming, in particular tasks. In this notebook, we have seen the basics of working with tasks. Some key points to remember:\n",
+    "\n",
+    "- How to create, schedule, and fetch from a task.\n",
+    "- Tasks run asynchronously, but not in parallel. You can have a single core CPU and still be able to work with several tasks.\n",
+    "- Channels are used to communicate data between tasks.\n",
+    "- Adding data (`put!`) or taking data (`take!`) from a channel might wait depending on the channel state. Be careful to avoid dead locks.\n",
+    "\n"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "a5d3730b",
@@ -873,15 +1016,15 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Julia 1.9.0",
+   "display_name": "Julia 1.11.6",
    "language": "julia",
-   "name": "julia-1.9"
+   "name": "julia-1.11"
   },
   "language_info": {
    "file_extension": ".jl",
    "mimetype": "application/julia",
    "name": "julia",
-   "version": "1.9.0"
+   "version": "1.11.6"
   }
  },
  "nbformat": 4,

notebooks/mpi_tutorial.ipynb

@@ -1036,7 +1036,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Julia 1.9.1",
+   "display_name": "Julia 1.9.0",
    "language": "julia",
    "name": "julia-1.9"
   },
@@ -1044,7 +1044,7 @@
    "file_extension": ".jl",
    "mimetype": "application/julia",
    "name": "julia",
-   "version": "1.9.1"
+   "version": "1.9.0"
   }
  },
  "nbformat": 4,

notebooks/pdes.ipynb

@@ -1534,7 +1534,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Julia 1.9.1",
+   "display_name": "Julia 1.9.0",
    "language": "julia",
    "name": "julia-1.9"
   },
@@ -1542,7 +1542,7 @@
    "file_extension": ".jl",
    "mimetype": "application/julia",
    "name": "julia",
-   "version": "1.9.1"
+   "version": "1.9.0"
   }
  },
  "nbformat": 4,