diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 6f07d3d..25a619f 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.4","generation_timestamp":"2024-08-26T11:01:02","documenter_version":"1.6.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.4","generation_timestamp":"2024-08-27T07:47:06","documenter_version":"1.6.0"}} \ No newline at end of file diff --git a/dev/LEQ/index.html b/dev/LEQ/index.html index 4c0642e..0f45aac 100644 --- a/dev/LEQ/index.html +++ b/dev/LEQ/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/asp/index.html b/dev/asp/index.html index 8d2bf3e..9f73bdd 100644 --- a/dev/asp/index.html +++ b/dev/asp/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/getting_started_with_julia/index.html b/dev/getting_started_with_julia/index.html index f5c8da7..ffd9a64 100644 --- a/dev/getting_started_with_julia/index.html +++ b/dev/getting_started_with_julia/index.html @@ -15,4 +15,4 @@ DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195"

Copy the contents of previous code block into a file called Project.toml and place it in an empty folder named newproject. It is important that the file is named Project.toml. You can create a new folder from the REPL with

julia> mkdir("newproject")

To install all the packages registered in this file you need to activate the folder containing your Project.toml file

(@v1.10) pkg> activate newproject

and then instantiating it

(newproject) pkg> instantiate

The instantiate command will download and install all listed packages and their dependencies in just one click.

Getting help in package mode

You can get help about a particular package operator by writing help in front of it

(@v1.10) pkg> help activate

You can get an overview of all package commands by typing help alone

(@v1.10) pkg> help

Package operations in Julia code

In some situations it is required to use package commands in Julia code, e.g., to automatize installation and deployment of Julia applications. This can be done using the Pkg package. For instance

julia> using Pkg
 julia> Pkg.status()

is equivalent to calling status in package mode.

(@v1.10) pkg> status

Creating you own package

In many situations is useful to create your own package, for instance, when working with a large code base, when you want to reduce compilation latency using Revise.jl, or if you want to eventually register your package and share it with others.

The simplest way of generating a package (called MyPackage) is as follows. Open Julia go to package mode and type

(@v1.10) pkg> generate MyPackage

This will crate a minimal package consisting of a new folder MyPackage with two files:

Tip

This approach only generates a very minimal package. To create a more sophisticated package skeleton (including unit testing, code coverage, readme file, licence, etc.) use PkgTemplates.jl or BestieTemplate.jl. The later one is developed in Amsterdam at the Netherlands eScience Center.

You can add dependencies to the package by activating the MyPackage folder in package mode and adding new dependencies as always:

(@v1.10) pkg> activate MyPackage
 (MyPackage) pkg> add MPI

This will add MPI to your package dependencies.

Using your own package

To use your package you first need to add it to a package environment of your choice. This is done with develop path/to/the/package/folder in package mode. For instance:

(@v1.10) pkg> develop MyPackage
Note

You do not need to "develop" your package if you activated the package folder MyPackage.

Now, we can go back to standard Julia mode and use it as any other package:

using MyPackage
-MyPackage.greet()

Here, we just called the example function defined in MyPackage/src/MyPackage.jl.

Conclusion

We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:

+MyPackage.greet()

Here, we just called the example function defined in MyPackage/src/MyPackage.jl.

Conclusion

We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:

diff --git a/dev/index.html b/dev/index.html index 67ac256..f8ddc6a 100644 --- a/dev/index.html +++ b/dev/index.html @@ -2,4 +2,4 @@ Home · XM_40017

Programming Large-Scale Parallel Systems (XM_40017)

Welcome to the interactive lecture notes of the Programming Large-Scale Parallel Systems course at VU Amsterdam!

What

This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam. We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems. Further information about the course is found in the study guide (click here) and our Canvas page (for registered students).

Note

Material will be added incrementally to the website as the course advances.

Warning

This page will eventually contain only a part of the course material. The rest will be available on Canvas. In particular, the material in this public webpage does not fully cover all topics in the final exam.

How to use this page

You have two main ways of studying the notebooks:

  • Download the notebooks and run them locally on your computer (recommended). At each notebook page you will find a green box with links to download the notebook.
  • You also have the static version of the notebooks displayed in this webpage for quick reference.

How to run the notebooks locally

To run a notebook locally follow these steps:

  • Install Julia (if not done already). More information in Getting started.
  • Download the notebook.
  • Launch Julia. More information in Getting started.
  • Execute these commands in the Julia command line:
julia> using Pkg
 julia> Pkg.add("IJulia")
 julia> using IJulia
-julia> notebook()
  • These commands will open a jupyter in your web browser. Navigate in jupyter to the notebook file you have downloaded and open it.

Authors

This material is created by Francesc Verdugo with the help of Gelieza Kötterheinrich. Part of the notebooks are based on the course slides by Henri Bal.

License

All material on this page that is original to this course may be used under a CC BY 4.0 license.

Acknowledgment

This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project "Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course" funded by the "Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning".

+julia> notebook()

Authors

This material is created by Francesc Verdugo with the help of Gelieza Kötterheinrich. Part of the notebooks are based on the course slides by Henri Bal.

License

All material on this page that is original to this course may be used under a CC BY 4.0 license.

Acknowledgment

This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project "Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course" funded by the "Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning".

diff --git a/dev/jacobi_2D/index.html b/dev/jacobi_2D/index.html index ef08b5c..ffd089d 100644 --- a/dev/jacobi_2D/index.html +++ b/dev/jacobi_2D/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/jacobi_method/index.html b/dev/jacobi_method/index.html index e098802..a2cb28d 100644 --- a/dev/jacobi_method/index.html +++ b/dev/jacobi_method/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_async/index.html b/dev/julia_async/index.html index c375c9d..b6ed650 100644 --- a/dev/julia_async/index.html +++ b/dev/julia_async/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_basics/index.html b/dev/julia_basics/index.html index ff5de5e..ccbf30c 100644 --- a/dev/julia_basics/index.html +++ b/dev/julia_basics/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_distributed/index.html b/dev/julia_distributed/index.html index 7bece87..7721757 100644 --- a/dev/julia_distributed/index.html +++ b/dev/julia_distributed/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_intro/index.html b/dev/julia_intro/index.html index f1a5d99..98341fe 100644 --- a/dev/julia_intro/index.html +++ b/dev/julia_intro/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_jacobi/index.html b/dev/julia_jacobi/index.html index b64b2b3..14b075a 100644 --- a/dev/julia_jacobi/index.html +++ b/dev/julia_jacobi/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_mpi/index.html b/dev/julia_mpi/index.html index a78ff61..f9095cd 100644 --- a/dev/julia_mpi/index.html +++ b/dev/julia_mpi/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/julia_tutorial/index.html b/dev/julia_tutorial/index.html index 81ba83e..4d68e75 100644 --- a/dev/julia_tutorial/index.html +++ b/dev/julia_tutorial/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/matrix_matrix.ipynb b/dev/matrix_matrix.ipynb index eb7dd6c..5f1fd22 100644 --- a/dev/matrix_matrix.ipynb +++ b/dev/matrix_matrix.ipynb @@ -55,18 +55,10 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "id": "2f8ba040", "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "🥳 Well done! \n" - ] - } - ], + "outputs": [], "source": [ "using Distributed\n", "using BenchmarkTools\n", @@ -290,7 +282,7 @@ "\n", "The matrix-matrix multiplication is an example of [embarrassingly parallel algorithm](https://en.wikipedia.org/wiki/Embarrassingly_parallel). An embarrassingly parallel (also known as trivially parallel) algorithm is an algorithm that can be split in parallel tasks with no (or very few) dependences between them. Such algorithms are typically easy to parallelize.\n", "\n", - "Which parts of an algorithm are completely independent and thus trivially parallel? To answer this question, it is useful to inspect the for loops, which are potential sources parallelism. If the iterations are independent of each other, then they are trivial to parallelize. An easy check to find out if the iterations are dependent or not is to change their order (for instance changing `for j in 1:n` by `for j in n:-1:1`, i.e. doing the loop in reverse). If the result changes, then the iterations are not independent.\n", + "Which parts of an algorithm are completely independent and thus trivially parallel? To answer this question, it is useful to inspect the for loops, which are potential sources of parallelism. If the iterations are independent of each other, then they are trivial to parallelize. An easy check to find out if the iterations are dependent or not is to change their order (for instance changing `for j in 1:n` by `for j in n:-1:1`, i.e. doing the loop in reverse). If the result changes, then the iterations are not independent.\n", "\n", "Look at the three nested loops in the sequential implementation of the matrix-matrix product:\n", "\n", @@ -320,7 +312,7 @@ "source": [ "### Parallel algorithms\n", "\n", - "Parallelizing the loops over `i` and `j` means that all the entries of matrix C can be potentially computed in parallel. However, *which it the most efficient solution to solve all these entries in parallel in a distributed system?* To find this we will consider different parallelization strategies:\n", + "The loops over `i` and `j` are trivially parallel implies that all the entries of matrix C can be potentially computed in parallel. However, *which it the most efficient solution to solve all these entries in parallel in a distributed system?* To find this we will consider different parallelization strategies:\n", "\n", "- Algorithm 1: each worker computes a single entry of C\n", "- Algorithm 2: each worker computes a single row of C\n", @@ -352,7 +344,7 @@ "source": [ "### Data dependencies\n", "\n", - "Moving data through the network is expensive and reducing data movement is one of the key points in a distributed algorithm. To this end, we need to determine which is the minimum data needed by a worker to perform its computations. These are called the *data dependencies*. This will give us later information about the performance of the parallel algorithm.\n", + "Moving data through the network is expensive and reducing data movement is one of the key points to design efficient distributed algorithms. To this end, we need to determine which is the minimum data needed by a worker to perform its computations. These are called the *data dependencies*. This will give us later information about the performance of the parallel algorithm.\n", "\n", "In algorithm 1, each worker computes only an entry of the result matrix C." ] @@ -403,7 +395,7 @@ "Taking into account the data dependencies, the parallel algorithm 1 can be efficiently implemented following these steps from the worker perspective:\n", "\n", "1. The worker receives the data dependencies, i.e., the corresponding row A[i,:] and column B[:,j] from the master process\n", - "2. The worker computes the dot product of A[i,:] and B[:,j]\n", + "2. The worker computes the dot product of A[i,:] and B[:,j] locally\n", "3. The worker sends back the result of C[i,j] to the master process" ] }, @@ -423,7 +415,7 @@ "id": "9d22ccea", "metadata": {}, "source": [ - "A possible implementation of this algorithm in Julia is as follows:" + "A possible implementation of this algorithm in Julia is as follows. Try to understand why `@sync` and `@async` are needed here." ] }, { @@ -483,7 +475,8 @@ "A = rand(N,N)\n", "B = rand(N,N)\n", "C = similar(A)\n", - "@test matmul_dist_1!(C,A,B) ≈ A*B" + "matmul_dist_1!(C,A,B)\n", + "@test C ≈ A*B" ] }, { diff --git a/dev/matrix_matrix/index.html b/dev/matrix_matrix/index.html index b08f514..9061142 100644 --- a/dev/matrix_matrix/index.html +++ b/dev/matrix_matrix/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/matrix_matrix_src/index.html b/dev/matrix_matrix_src/index.html index e3be326..2417331 100644 --- a/dev/matrix_matrix_src/index.html +++ b/dev/matrix_matrix_src/index.html @@ -7578,12 +7578,12 @@ a.anchor-link { -
+
diff --git a/dev/mpi_collectives/index.html b/dev/mpi_collectives/index.html index ca6d77b..0ba6836 100644 --- a/dev/mpi_collectives/index.html +++ b/dev/mpi_collectives/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); -
+ diff --git a/dev/mpi_tutorial/index.html b/dev/mpi_tutorial/index.html index e4d146b..de11550 100644 --- a/dev/mpi_tutorial/index.html +++ b/dev/mpi_tutorial/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/notebook-hello/index.html b/dev/notebook-hello/index.html index 109c017..b0f1204 100644 --- a/dev/notebook-hello/index.html +++ b/dev/notebook-hello/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/pdes/index.html b/dev/pdes/index.html index 9a9fc42..4a2d8d2 100644 --- a/dev/pdes/index.html +++ b/dev/pdes/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/solutions/index.html b/dev/solutions/index.html index 9c144fa..9104328 100644 --- a/dev/solutions/index.html +++ b/dev/solutions/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - + diff --git a/dev/solutions_for_all_notebooks/index.html b/dev/solutions_for_all_notebooks/index.html index 1b271e2..29b2daf 100644 --- a/dev/solutions_for_all_notebooks/index.html +++ b/dev/solutions_for_all_notebooks/index.html @@ -172,4 +172,4 @@ end

« Jacobi method

Powered by Documenter.jl and the Julia Programming Language.

+end diff --git a/dev/tsp/index.html b/dev/tsp/index.html index 580836b..05feb20 100644 --- a/dev/tsp/index.html +++ b/dev/tsp/index.html @@ -14,4 +14,4 @@ var myIframe = document.getElementById("notebook"); iFrameResize({log:true}, myIframe); }); - +