seb/XM_40017
1
0
Fork 0
mirror of https://github.com/fverdugo/XM_40017.git synced 2025-11-09 00:24:25 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Remove generated files from index

Browse Source
This commit is contained in:
Gelieza K 2023-08-15 11:39:53 +02:00
parent 14b4706526
commit 95c4608380
39 changed files with 0 additions and 220836 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16611
docs/src/notebook-html/jacobi_2D.html
View File

File diff suppressed because one or more lines are too long

16332
docs/src/notebook-html/jacobi_method.html
View File

File diff suppressed because one or more lines are too long

16211
docs/src/notebook-html/julia_async.html
View File

File diff suppressed because one or more lines are too long

17402
docs/src/notebook-html/julia_basics.html
View File

File diff suppressed because one or more lines are too long

16937
docs/src/notebook-html/julia_distributed.html
View File

File diff suppressed because one or more lines are too long

16142
docs/src/notebook-html/julia_intro.html
View File

File diff suppressed because one or more lines are too long

15919
docs/src/notebook-html/julia_jacobi.html
View File

File diff suppressed because one or more lines are too long

15559
docs/src/notebook-html/julia_tutorial.html
View File

File diff suppressed because one or more lines are too long

16584
docs/src/notebook-html/matrix_matrix.html
View File

File diff suppressed because one or more lines are too long

15559
docs/src/notebook-html/mpi_tutorial.html
View File

File diff suppressed because one or more lines are too long

15345
docs/src/notebook-html/notebook-hello.html
View File

File diff suppressed because one or more lines are too long

15376
docs/src/notebook-html/solutions.html
View File

File diff suppressed because one or more lines are too long

15456
docs/src/notebook-html/tsp.html
View File

File diff suppressed because one or more lines are too long

2126
docs/src/notebooks/jacobi_2D.ipynb
View File

File diff suppressed because one or more lines are too long

30
docs/src/notebooks/jacobi_2D.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/jacobi_2D.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="jacobi_2D.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/jacobi_2D.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/jacobi_2D.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

978
docs/src/notebooks/jacobi_method.ipynb
View File

File diff suppressed because one or more lines are too long

30
docs/src/notebooks/jacobi_method.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/jacobi_method.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="jacobi_method.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/jacobi_method.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/jacobi_method.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

794
docs/src/notebooks/julia_async.ipynb
View File

@ -1,794 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "f64b009a",
"metadata": {},
"source": [
"<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/39/VU_logo.png/800px-VU_logo.png?20161029201021\" width=\"350\">\n",
"\n",
"### Programming large-scale parallel systems\n",
"\n",
"\n",
"# Asynchronous programming in Julia\n"
]
},
{
"cell_type": "markdown",
"id": "bf68ad38",
"metadata": {},
"source": [
"## Contents\n",
"\n",
"In this notebook, we will learn the basics of asynchronous programming in Julia. In particular, we will learn about:\n",
"\n",
"- Tasks\n",
"- Channels\n",
"\n",
"Understanding these concepts is important to learn later distributed computing."
]
},
{
"cell_type": "markdown",
"id": "caf64254",
"metadata": {},
"source": [
"## Tasks\n",
"\n",
"### Creating a task\n",
"\n",
"A task is a piece of computation work that can be run asynchronously (i.e., that can be run in the background). To create a task, we first need to create a function that represents the work to be done in the task. In next cell, we generate a task that generates and sums two matrices."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fe668cb1",
"metadata": {},
"outputs": [],
"source": [
"function work()\n",
" println(\"Starting work\")\n",
" sleep(7)\n",
" a = rand(3,3)\n",
" b = rand(3,3)\n",
" r = a + b\n",
" println(\"Finishing work\")\n",
" r\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67ee0328",
"metadata": {},
"outputs": [],
"source": [
"t = Task(work)"
]
},
{
"cell_type": "markdown",
"id": "e459c5c2",
"metadata": {},
"source": [
"### Scheduling a task\n",
"\n",
"The task has been created, but the corresponding work has not started. Note that we do not see any output from function `work` yet. To run the task we need to schedule it."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8778c199",
"metadata": {},
"outputs": [],
"source": [
"schedule(t)"
]
},
{
"cell_type": "markdown",
"id": "f1fb9283",
"metadata": {},
"source": [
"### Fetching the task result\n",
"\n",
"The task has been executed, but we do not see the result. To get the result we need to fetch it."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0c7b626e",
"metadata": {},
"outputs": [],
"source": [
"fetch(t)"
]
},
{
"cell_type": "markdown",
"id": "fedbbd71",
"metadata": {},
"source": [
"### Tasks run asynchronously\n",
"\n",
"It is important to note that tasks run asynchronously. To illustrate this let's create and schedule a new task."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4ccc996c",
"metadata": {},
"outputs": [],
"source": [
"t = Task(work)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "015bea27",
"metadata": {},
"outputs": [],
"source": [
"schedule(t)"
]
},
{
"cell_type": "markdown",
"id": "5ec0718e",
"metadata": {},
"source": [
"Note that while the task is running we can execute Julia code. To check this, execute the next two cells while the task is running."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a70fcbe8",
"metadata": {},
"outputs": [],
"source": [
"sin(4π)*exp(-0.1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6def444b",
"metadata": {},
"outputs": [],
"source": [
"1 + 1"
]
},
{
"cell_type": "markdown",
"id": "d483d4d0",
"metadata": {},
"source": [
"How is this possible? Tasks run in the brackground and this particular task is sleeping for most of the time. Thus, it is possible to use the current Julia process for other operations while the tasks is sleeping."
]
},
{
"cell_type": "markdown",
"id": "910323fd",
"metadata": {},
"source": [
"### Tasks do not run in parallel\n",
"\n",
"It is also important to note that tasks do not run in parallel. We were able to run code while previous tasks was running because the task was idling most of the time. If the task does actual work, the current process will be busy running this task and it is likely that we cannot run other code at the same time. Let's illustrate this with an example. The following code computes an approximation of $\\pi$ using [Leibniz formula](https://en.wikipedia.org/wiki/Leibniz_formula_for_pi). The quality of the approximation increases with the value of `n`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b53ac640",
"metadata": {},
"outputs": [],
"source": [
"function compute_π(n)\n",
" s = 1.0\n",
" for i in 1:n\n",
" s += (isodd(i) ? -1 : 1) / (i*2+1)\n",
" end\n",
" 4*s\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "7614df94",
"metadata": {},
"source": [
" Call this function with a large number. Note that it will take some time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "63f2aec3",
"metadata": {},
"outputs": [],
"source": [
"compute_π(4_000_000_000)"
]
},
{
"cell_type": "markdown",
"id": "840ff590",
"metadata": {},
"source": [
"Create a task that performs this computation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f16ade6c",
"metadata": {},
"outputs": [],
"source": [
"fun = () -> compute_π(4_000_000_000)\n",
"t = Task(fun)"
]
},
{
"cell_type": "markdown",
"id": "60c2567c",
"metadata": {},
"source": [
"Schedule the tasks and then try to execute the 2nd cell bellow. Note that the current process will be busy running the task."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1f28a388",
"metadata": {},
"outputs": [],
"source": [
"schedule(t)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "52e060e7",
"metadata": {},
"outputs": [],
"source": [
"1+1"
]
},
{
"cell_type": "markdown",
"id": "25048665",
"metadata": {},
"source": [
"### Tasks take a function with no arguments\n",
"\n",
"This function needs to have zero arguments, but it can capture variables if needed. If we try to create a task with a function that has arguments, it will result in an error when we schedule it."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "87397749",
"metadata": {},
"outputs": [],
"source": [
"add(a,b) = a + b"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e99766c7",
"metadata": {},
"outputs": [],
"source": [
"t = Task(add)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1a785bae",
"metadata": {},
"outputs": [],
"source": [
"schedule(t)"
]
},
{
"cell_type": "markdown",
"id": "ebfce725",
"metadata": {},
"source": [
"If we need, we can capture variables in the function to be run by the task as shown in the next cells."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "06f54fa0",
"metadata": {},
"outputs": [],
"source": [
"a = rand(3,3)\n",
"b = rand(3,3);"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4c9e586e",
"metadata": {},
"outputs": [],
"source": [
"fun = () -> a + b\n",
"t = Task(fun)\n",
"schedule(t)"
]
},
{
"cell_type": "markdown",
"id": "cd829a64",
"metadata": {},
"source": [
"### Useful macro: `@async`\n",
"\n",
"So far, we have created tasks using low-level functions, but there are more convenient ways of creating and scheduling tasks. For instance using the `@async` macro. This macro is used to run a piece of code asynchronously. Under the hood it puts the code in an anonymous function, creates a task, and schedules it. For instance, the next cell is equivalent to previous one."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9b5152d1",
"metadata": {},
"outputs": [],
"source": [
"@async a + b"
]
},
{
"cell_type": "markdown",
"id": "ac3262d8",
"metadata": {},
"source": [
"### Another useful macro: `@sync`\n",
"\n",
"This macro is used to wait for all the tasks created with `@async` in a given block of code. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef2e49f9",
"metadata": {},
"outputs": [],
"source": [
"@sync begin\n",
" @async sleep(3)\n",
" @async sleep(4)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "98f0d685",
"metadata": {},
"source": [
"## Channels\n",
"\n",
"### Sending data between tasks\n",
"\n",
"Julia provides channels as a way to send data between tasks. A channel is like a FIFO queue in which tasks can put and take values from. In next example, we create a channel and a task that puts five values into the channel. Finally, the task closes the channel."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b88d5308",
"metadata": {},
"outputs": [],
"source": [
"chnl = Channel{Int}()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3421c7a6",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"@async begin\n",
" for i in 1:5\n",
" put!(chnl,i)\n",
" end\n",
" close(chnl)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "1df0508c",
"metadata": {},
"source": [
"By executing next cell several times, we will get the values from the channel. We are indeed communicating values from two different tasks. If we execute the cell more than 5 times, it will raise an error since the channel is closed. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "64b9436e",
"metadata": {},
"outputs": [],
"source": [
"take!(chnl)"
]
},
{
"cell_type": "markdown",
"id": "757a1b07",
"metadata": {},
"source": [
"### Channels are iterable\n",
"\n",
"Instead of taking values from a channel until an error occurs, we can also iterate over the channel in a for loop until the channel is closed."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2fc22dfa",
"metadata": {},
"outputs": [],
"source": [
"chnl = Channel{Int}()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fa62a4df",
"metadata": {},
"outputs": [],
"source": [
"@async begin\n",
" for i in 1:5\n",
" put!(chnl,i)\n",
" end\n",
" close(chnl)\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e511e19b",
"metadata": {},
"outputs": [],
"source": [
"for i in chnl\n",
" @show i\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "b1a2a557",
"metadata": {},
"source": [
"### Calls to `put!` and `take!` are blocking\n",
"\n",
"Note that `put!` and `take!` are blocking operations. Calling `put!` blocks the tasks until another task calls `take!` and viceversa. Thus, we need at least 2 tasks for this to work. If we call `put!` and `take!` from the same task, it will result in a dead lock. We have added a print statement to previous example. Run it again and note how `put!` blocks until we call `take!`. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f34373ca",
"metadata": {},
"outputs": [],
"source": [
"chnl = Channel{Int}()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bfde2ecd",
"metadata": {},
"outputs": [],
"source": [
"@async begin\n",
" for i in 1:5\n",
" put!(chnl,i)\n",
" println(\"I have put $i\")\n",
" end\n",
" close(chnl)\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5830659d",
"metadata": {},
"outputs": [],
"source": [
"take!(chnl)"
]
},
{
"cell_type": "markdown",
"id": "006140bd",
"metadata": {},
"source": [
"### Buffered channels\n",
"\n",
"We can be a bit more flexible and use a buffered channel. In this case, `put!` will block only if the channel is full and `take!` will block if the channel is empty. We repeat previous example, but with a buffered channel of size 2. Note that we can call `put!` until the channel is full. At this point, we need to wait to until we call `take!` which removes an item from the channel, making room for a new item."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dfe06b5f",
"metadata": {},
"outputs": [],
"source": [
"buffer_size = 2\n",
"chnl = Channel{Int}(buffer_size)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6289bc2e",
"metadata": {},
"outputs": [],
"source": [
"@async begin\n",
" for i in 1:5\n",
" put!(chnl,i)\n",
" println(\"I have put $i\")\n",
" end\n",
" close(chnl)\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2a87cd5f",
"metadata": {},
"outputs": [],
"source": [
"take!(chnl)"
]
},
{
"cell_type": "markdown",
"id": "9ddd66ca",
"metadata": {},
"source": [
"## Questions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dd45ae08",
"metadata": {},
"outputs": [],
"source": [
"t = @elapsed compute_π(100_000_000)"
]
},
{
"cell_type": "markdown",
"id": "2a23b1c3",
"metadata": {},
"source": [
"<div class=\"alert alert-block alert-success\">\n",
"<b>Question (Q1):</b> How long will the compute time of next cell be? \n",
"</div>\n",
"\n",
" a) 10*t\n",
" b) t\n",
" c) 0.1*t\n",
" d) near 0*t \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "18d6cfe3",
"metadata": {},
"outputs": [],
"source": [
"@time for i in 1:10\n",
" compute_π(100_000_000)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "5f19d38c",
"metadata": {},
"source": [
"<div class=\"alert alert-block alert-success\">\n",
"<b>Question (Q2):</b> How long will the compute time of next cell be? \n",
"</div>\n",
"\n",
" a) 10*t\n",
" b) t\n",
" c) 0.1*t\n",
" d) near 0*t \n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dac0c92a",
"metadata": {},
"outputs": [],
"source": [
"@time for i in 1:10\n",
" @async compute_π(100_000_000)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "5041c355",
"metadata": {},
"source": [
"<div class=\"alert alert-block alert-success\">\n",
"<b>Question (Q3):</b> How long will the compute time of next cell be? \n",
"</div>\n",
"\n",
" a) 10*t\n",
" b) t\n",
" c) 0.1*t\n",
" d) near 0*t \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c06dc4a5",
"metadata": {},
"outputs": [],
"source": [
"@time @sync for i in 1:10\n",
" @async compute_π(100_000_000)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "841b690e",
"metadata": {},
"source": [
"<div class=\"alert alert-block alert-success\">\n",
"<b>Question (Q4):</b> How long will the compute time of the 2nd cell be? \n",
"</div>\n",
"\n",
" a) infinity\n",
" b) 1 second\n",
" c) near 0 seconds\n",
" d) 3 seconds"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6ac116bb",
"metadata": {},
"outputs": [],
"source": [
"buffer_size = 4\n",
"chnl = Channel{Int}(buffer_size)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "25363a90",
"metadata": {},
"outputs": [],
"source": [
"@time begin\n",
" put!(chnl,3)\n",
" i = take!(chnl)\n",
" sleep(i)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "df663f11",
"metadata": {},
"source": [
"<div class=\"alert alert-block alert-success\">\n",
"<b>Question (Q5):</b> How long will the compute time of the 2nd cell be? \n",
"</div>\n",
"\n",
" a) infinity\n",
" b) 1 second\n",
" c) near 0 seconds\n",
" d) 3 seconds"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9abeed40",
"metadata": {},
"outputs": [],
"source": [
"chnl = Channel{Int}()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8e8428ce",
"metadata": {},
"outputs": [],
"source": [
"@time begin\n",
" put!(chnl,3)\n",
" i = take!(chnl)\n",
" sleep(i)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "a5d3730b",
"metadata": {},
"source": [
"# License\n",
"\n",
"TODO: replace link to website\n",
"\n",
"This notebook is part of the course [Programming Large Scale Parallel Systems](http://localhost:8000/) at Vrije Universiteit Amsterdam and may be used under a [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) license."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f9863011",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.9.1",
"language": "julia",
"name": "julia-1.9"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

30
docs/src/notebooks/julia_async.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/julia_async.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="julia_async.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/julia_async.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/julia_async.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

1625
docs/src/notebooks/julia_basics.ipynb
View File

File diff suppressed because it is too large Load Diff

30
docs/src/notebooks/julia_basics.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/julia_basics.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="julia_basics.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/julia_basics.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/julia_basics.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

1335
docs/src/notebooks/julia_distributed.ipynb
View File

File diff suppressed because one or more lines are too long

30
docs/src/notebooks/julia_distributed.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/julia_distributed.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="julia_distributed.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/julia_distributed.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/julia_distributed.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

746
docs/src/notebooks/julia_intro.ipynb
View File

@ -1,746 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "ae2a0512",
"metadata": {},
"source": [
"\n",
"# Is Julia fast?"
]
},
{
"cell_type": "markdown",
"id": "e040636a",
"metadata": {},
"source": [
"## Contents\n",
"\n",
"With this notebook, you will learn\n",
"\n",
"- Two basic julia concepts related performance:\n",
" - type-inference\n",
" - JIT compilation\n",
"- Some Julia syntax\n",
"- Some useful Julia packages"
]
},
{
"cell_type": "markdown",
"id": "beef4d5e",
"metadata": {},
"source": [
"## Using Jupyter notebooks in Julia\n",
"\n",
"We are going to use Jupyter notebooks in this and other lectures. You provably have worked with notebooks (in Python). If not, here are the basic concepts you need to know to follow the lessons.\n",
"\n",
"<div class=\"alert alert-block alert-info\">\n",
"<b>Tip:</b> Did you know that Jupyter stands for Julia, Python and R?\n",
"</div>\n",
"\n",
"### How to start a Jupyter nootebook in Julia\n",
"\n",
"To run a Julia Jupyther notebook, open a Julia REPL and type\n",
"\n",
"```julia\n",
"julia> ]\n",
"pkg> add IJulia\n",
"julia> using IJulia\n",
"julia> notebook()\n",
"```\n",
"A new browser window will open. Navigate to the corresponding notebook and open it.\n",
"\n",
"<div class=\"alert alert-block alert-warning\">\n",
"<b>Warning:</b> Make sure that the notebook is using the same Julia version as the one you used to launch `IJulia`. If it is not the same, go to Kernel > Change Kernel and choose the right version.\n",
"</div>\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "7cdd5195",
"metadata": {},
"source": [
"### Running a cell\n",
"\n",
"To run a cell, click on a cell and press `Shift` + `Enter`. You can also use the \"Run\" button in the toolbar above."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ec0f9041",
"metadata": {},
"outputs": [],
"source": [
"1+3\n",
"4*5"
]
},
{
"cell_type": "markdown",
"id": "a8474251",
"metadata": {},
"source": [
"As you can see from the output of previous cell, the value of the last line is displayed. We can suppress the output with a semicolon. Try it. Execute next cell."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fb3eaec4",
"metadata": {},
"outputs": [],
"source": [
"1+3\n",
"4*5;"
]
},
{
"cell_type": "markdown",
"id": "0823876a",
"metadata": {},
"source": [
"### Cell order is important\n",
"\n",
"Running the two cells below in reverse order won't work (try it). "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bc1da6cb",
"metadata": {},
"outputs": [],
"source": [
"foo() = \"Well done!\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ddea57d8",
"metadata": {},
"outputs": [],
"source": [
"foo()"
]
},
{
"cell_type": "markdown",
"id": "629189d9",
"metadata": {},
"source": [
"### REPL modes\n",
"\n",
"This is particular to Julia notebooks. You can use package, help, and shell mode just like in the Julia REPL."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "defa851f",
"metadata": {},
"outputs": [],
"source": [
"] add MPI"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1e6fbe66",
"metadata": {},
"outputs": [],
"source": [
"? print"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "123c0f6a",
"metadata": {},
"outputs": [],
"source": [
"; ls"
]
},
{
"cell_type": "markdown",
"id": "44c03fde",
"metadata": {},
"source": [
"## How fast is Julia code?"
]
},
{
"cell_type": "markdown",
"id": "f1e716db",
"metadata": {},
"source": [
"NB. Most of the examples below are taken from the lecture by S.G. Johnson at MIT. See here:\n",
"https://github.com/mitmath/18S096/blob/master/lectures/lecture1/Boxes-and-registers.ipynb"
]
},
{
"cell_type": "markdown",
"id": "6c088d9e",
"metadata": {},
"source": [
"### Example\n",
"\n",
"Sum entries of a given array $a = [a_1,a_2,...,a_n]$\n",
"\n",
" $$s = \\sum_{i=1}^n a_i$$\n",
"\n",
"### "
]
},
{
"cell_type": "markdown",
"id": "34e3c7dd",
"metadata": {},
"source": [
"### Hand-written sum function"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a672edac",
"metadata": {
"code_folding": []
},
"outputs": [],
"source": [
"function sum_hand(a)\n",
" s = zero(eltype(a))\n",
" for ai in a\n",
" s += ai\n",
" end\n",
" s\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "0494a1b3",
"metadata": {},
"source": [
"### Test it\n",
"\n",
"The Julia macro `@test` which is provided in the `Test` package is useful to write (unit) tests in Julia."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ebac7b9e",
"metadata": {},
"outputs": [],
"source": [
"using Test"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "aa5505d4",
"metadata": {},
"outputs": [],
"source": [
"a = rand(5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fc11923c",
"metadata": {},
"outputs": [],
"source": [
"@test sum_hand(a) ≈ sum(a)"
]
},
{
"cell_type": "markdown",
"id": "e23f3ddf",
"metadata": {},
"source": [
"## Benchmarking\n",
"\n",
"In Julia, the most straight-forward way of measuring the computation time of a piece of code is with the macro `@time`. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "caff0307",
"metadata": {},
"outputs": [],
"source": [
"a = rand(10^7);"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6d073236",
"metadata": {},
"outputs": [],
"source": [
"@time sum_hand(a)"
]
},
{
"cell_type": "markdown",
"id": "443ac8da",
"metadata": {},
"source": [
"Note that `@time` also measures the compile time of a function if it's the first call to that function. So make sure to run `@time` twice on a freshly compiled function in order to get a more meaningful result."
]
},
{
"cell_type": "markdown",
"id": "c664522e",
"metadata": {},
"source": [
"A part of getting rid of compilation time, one typically wants to measure the runtime several times and compute sole. To do this we can call our code in a for-loop and gather the runtimes using the Julia macro `@elapsed`. This measures the runtime of an expression in seconds, just as the `@time` macro, only `@elapsed` discards the result of the computation and returns the elapsed time instead."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bba864c1",
"metadata": {},
"outputs": [],
"source": [
"@elapsed sum_hand(a)"
]
},
{
"cell_type": "markdown",
"id": "70317d33",
"metadata": {},
"source": [
"## BenchmarkTools\n",
"\n",
"The `BenchmarkTools` extension package provides useful macros for sampling runtimes automatically. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "424ffe03",
"metadata": {},
"outputs": [],
"source": [
"using BenchmarkTools"
]
},
{
"cell_type": "markdown",
"id": "079ef7d0",
"metadata": {},
"source": [
"First of all, the `@benchmark` macro runs the code multiple times and gives out a lot of details: the minimum and maximum time, mean time, median time, number of samples taken, memory allocations, etc. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b5bb9a12",
"metadata": {},
"outputs": [],
"source": [
"bch_sum_hand = @benchmark sum_hand($a)"
]
},
{
"cell_type": "markdown",
"id": "c77d454b",
"metadata": {},
"source": [
"For quick sanity checks, one can use the `@btime` macro, which is a convenience wrapper around `@benchmark`. It returns only the minimum execution time and memory allocations. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fee5493d",
"metadata": {},
"outputs": [],
"source": [
"@btime sum_hand($a)"
]
},
{
"cell_type": "markdown",
"id": "3a1a774c",
"metadata": {},
"source": [
"Similar to the `@elapsed` macro, `BenchmarkTool`'s `@belapsed` discards the return value of the function and instead returns the minimum runtime in seconds. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8e0cdb8c",
"metadata": {},
"outputs": [],
"source": [
"@belapsed sum_hand($a)"
]
},
{
"cell_type": "markdown",
"id": "fb530b87",
"metadata": {},
"source": [
"As opposed to `@time` and `@elapsed`, `@btime` and `@belapsed` run the code several times and return the minimum runtime, thus eliminating possible compilation times from the measurement. "
]
},
{
"cell_type": "markdown",
"id": "782be14f",
"metadata": {},
"source": [
"### Built-in sum function"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "07df4939",
"metadata": {},
"outputs": [],
"source": [
"bch_sum = @benchmark sum($a)"
]
},
{
"cell_type": "markdown",
"id": "18972fa7",
"metadata": {},
"source": [
"### Hand-written sum in Python\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "038702c0",
"metadata": {},
"outputs": [],
"source": [
"using PyCall"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8e1dcbfa",
"metadata": {},
"outputs": [],
"source": [
"py\"\"\"\n",
"def sum_py_hand(A):\n",
" s = 0.0\n",
" for a in A:\n",
" s += a\n",
" return s\n",
"\"\"\"\n",
"sum_py_hand = py\"sum_py_hand\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "60ca4517",
"metadata": {},
"outputs": [],
"source": [
"@test sum(a) ≈ sum_py_hand(a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f009076",
"metadata": {},
"outputs": [],
"source": [
"bch_sum_py_hand = @benchmark sum_py_hand($a)"
]
},
{
"cell_type": "markdown",
"id": "22d29afa",
"metadata": {},
"source": [
"### Numpy sum "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0f7ffb8e",
"metadata": {},
"outputs": [],
"source": [
"using Conda"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e667fc80",
"metadata": {},
"outputs": [],
"source": [
"numpy = pyimport(\"numpy\")\n",
"sum_numpy = numpy[\"sum\"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ee0361df",
"metadata": {},
"outputs": [],
"source": [
"@test sum_numpy(a) ≈ sum(a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d2f4e9ea",
"metadata": {},
"outputs": [],
"source": [
"bch_sum_numpy = @benchmark sum_numpy($a)"
]
},
{
"cell_type": "markdown",
"id": "d7c3cbd6",
"metadata": {},
"source": [
"### Sumary of the results\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5ce4c97a",
"metadata": {},
"outputs": [],
"source": [
"timings = [bch_sum_hand,bch_sum,bch_sum_py_hand,bch_sum_numpy]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9d5caabd",
"metadata": {},
"outputs": [],
"source": [
"methods = [\"sum_hand\",\"sum\",\"sum_py_hand\",\"sum_numpy\"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8b7a3661",
"metadata": {},
"outputs": [],
"source": [
"using DataFrames"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "82b11610",
"metadata": {},
"outputs": [],
"source": [
"df = DataFrame(method=methods,time=timings)"
]
},
{
"cell_type": "markdown",
"id": "f03c4281",
"metadata": {},
"source": [
"### Improving the hand-written sum in Julia\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7696719c",
"metadata": {},
"outputs": [],
"source": [
"# ✍️ Exercise 3\n",
"function sum_hand_fast(a)\n",
" s = 0.0\n",
" @simd for ai in a\n",
" s += ai\n",
" end\n",
" s\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0b461036",
"metadata": {},
"outputs": [],
"source": [
"@test sum_hand_fast(a) ≈ sum(a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fc228cba",
"metadata": {},
"outputs": [],
"source": [
"@benchmark sum_hand_fast($a)"
]
},
{
"cell_type": "markdown",
"id": "328f9128",
"metadata": {},
"source": [
"## Conlcusions so far\n",
"\n",
"- Julia code (for loops) are much faster than in Python\n",
"- Julia code can be as fast as optimized C code"
]
},
{
"cell_type": "markdown",
"id": "f9ce5464",
"metadata": {},
"source": [
"## Why Julia is fast?\n",
"\n",
"- Julia is a compiled language (like C, C++, Fortran)\n",
"- Julia is JIT compiled (C, C++, Fortran are AOT compiled)\n",
"- Type declarations are optional in Julia\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "aa5254e4",
"metadata": {},
"source": [
"# Conclusion: Why we use Julia in this course\n",
"\n",
"- Julia code is fast (it can be as fast as C)\n",
"- Julia is a high-level language with simpler syntax than C \n",
"- Julia supports different parallel programming models\n",
"\n",
"We will look into the third point in a later section of this course. \n"
]
},
{
"cell_type": "markdown",
"id": "d4efafe2",
"metadata": {},
"source": [
"## Solution to the exercises\n",
"\n",
"### Solution to Exercise 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "17f73c0f",
"metadata": {},
"outputs": [],
"source": [
"function sum_hand(a)\n",
" s = 0.0\n",
" for ai in a\n",
" s += ai\n",
" end\n",
" s\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "d12d6400",
"metadata": {},
"source": [
"### Solution to Exercise 2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e39094da",
"metadata": {},
"outputs": [],
"source": [
"using Statistics\n",
"\n",
"a = rand(10^7)\n",
"num_it = 15\n",
"runtimes = zeros(num_it)\n",
"for i in 1:num_it\n",
" runtimes[i] = @elapsed sum_hand(a)\n",
"end\n",
"@show mean(runtimes) \n",
"@show std(runtimes)\n",
"@show minimum(runtimes)\n",
"@show maximum(runtimes);"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7943f1cc",
"metadata": {},
"outputs": [],
"source": [
"# ✍️ Exercise 3\n",
"function sum_hand_fast(a)\n",
" s = 0.0\n",
" @simd for ai in a\n",
" s += ai\n",
" end\n",
" s\n",
"end"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.9.0",
"language": "julia",
"name": "julia-1.9"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.9.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

30
docs/src/notebooks/julia_intro.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/julia_intro.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="julia_intro.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/julia_intro.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/julia_intro.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

652
docs/src/notebooks/julia_jacobi.ipynb
View File

File diff suppressed because one or more lines are too long

30
docs/src/notebooks/julia_jacobi.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/julia_jacobi.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="julia_jacobi.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/julia_jacobi.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/julia_jacobi.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

523
docs/src/notebooks/julia_tutorial.ipynb
View File

@ -1,523 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/39/VU_logo.png/800px-VU_logo.png?20161029201021\" width=\"350\">\n",
"\n",
"### Programming large-scale parallel systems\n",
"### Lectures on Julia for HPC\n",
"\n",
"\n",
"# Tutorial: Getting started with Julia\n",
"\n",
"by Francesc Verdugo (VU Amsterdam)\n",
"\n",
"Version fall 2022"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contents\n",
"\n",
"- Using the Julia REPL\n",
"- Running serial and parallel code\n",
"- Installing and managing packages"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introduction\n",
"\n",
"Julia has its own way of running code and using packages. Many educational sources about Julia assume that you have this basic knowledge, which can be confusing to new users. In this lesson, we will learn these basic skills so that you can start learning more on Julia."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Preliminaries\n",
"\n",
"This is a tutorial. To follow it:\n",
"\n",
"- Download and install Julia from https://julialang.org/\n",
"- Download and install VSCode and its Julia extension. How to: https://www.julia-vscode.org/docs/dev/gettingstarted/\n",
"- Continue reading this document \n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Julia REPL\n",
"\n",
"There are several ways of opening Julia depending on your operating system and your IDE, but it is usually as simple as launching the Julia app. With VSCode, open a folder (File > Open Folder). Then, press `Ctrl`+`Shift`+`P` to open the command bar, and execute `Julia: Start REPL`. If this does not work, make sure you have the Julia extension for VSCode installed. Independently of the method you use, opening Julia results in a window with some text ending with:\n",
"\n",
"```\n",
"julia> \n",
"```\n",
"\n",
"You have just opened the Julia *read-evaluate-print loop*, or simply the Julia *REPL*. Congrats! You will spend most of time using the REPL, when working in Julia. The REPL is a console waiting for user input. Just as in other consoles, the string of text right before the input area (`julia>` in the case) is called the *command prompt* or simply the *prompt*.\n",
"\n",
"\n",
"The usage is as follows:\n",
"- You write some input\n",
"- press enter\n",
"- you get the output\n",
"\n",
"For instance, try this\n",
"\n",
"```julia\n",
"julia> 1 + 1\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Hello world!\n",
"\n",
"A \"Hello world\" example looks like this in Julia\n",
"\n",
"```julia\n",
"julia> println(\"Hello, world!\")\n",
"```\n",
"\n",
"Try to run it in the REPL."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Help mode\n",
"\n",
"\n",
"\n",
"Curious about what function `println` does? Enter into *help* mode to look into the documentation. This is done by typing a question mark (`?`) into the inut field:\n",
"\n",
"```julia\n",
"julia> ?\n",
"```\n",
"\n",
"After typing `?`, the command prompt changes to `help?>`. It means we are in help mode. Now, we can type a function name to see its documentation.\n",
"\n",
"```julia\n",
"help?> println\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Package and shell modes\n",
"\n",
"The REPL comes with two more modes, namely *package* and *shell* modes. To enter package mode type\n",
"\n",
"```julia\n",
"julia> ]\n",
"```\n",
"\n",
"Package mode is used to install and manage packages. We are going to discuss the package mode in greater detail later. To return back to normal mode press the backspace key several times.\n",
"\n",
"To enter shell mode type semicolon (`;`)\n",
"```julia\n",
"julia> ;\n",
"```\n",
"The prompt should have changed to `shell>` indicating that we are in shell mode. Now you can type commands that you would normally do on your system command line. For instance, \n",
"\n",
"```julia\n",
"shell> ls\n",
"```\n",
"will display the contents of the current folder in Mac or Linux. Using shell mode in Windows is not straightforward, and thus not recommended for beginners.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Running more complex code\n",
"\n",
"Real-world Julia programs are not typed in the REPL in practice. They are written in one or more files and *included* in the REPL. To try this, create a new file called `hello.jl`, write the code of the \"Hello world\" example above, and save it. If you are using VSCode, you can create the file using File > New File > Julia File. Once the file is saved with the name `hello.jl`, execute it as follows\n",
"\n",
"```julia\n",
"julia> include(\"hello.jl\")\n",
"```\n",
"\n",
"<div class=\"alert alert-block alert-warning\">\n",
"<b>Warning:</b> Make sure that the file <code>\"hello.jl\"</code> is located in the current working directory of your Julia session. You can query the current directory with function <code>pwd()</code>. You can change to another directory with function <code>cd()</code> if needed. Also, make sure that the file extension is <code>.jl</code>.\n",
"</div>\n",
"\n",
"The recommended way of running Julia code is using the REPL as we did. But it is also possible to run code directly from the system command line. To this end, open a terminal and call Julia followed buy the path to the file containing the code you want to execute.\n",
"\n",
"```\n",
"$ julia hello.jl\n",
"```\n",
"\n",
"Previous line assumes that you have Julia properly installed in the system and that is usable from the terminal. In UNIX systems (Linux and Mac), the Julia binary needs to be in one of the directories listed in the `PATH` environment variable. To check that Julia is properly installed, you can use\n",
"\n",
"```\n",
"$ julia --version\n",
"```\n",
"\n",
"If this runs without error and you see a version number, you are good to go!\n",
"\n",
"\n",
"<div class=\"alert alert-block alert-info\">\n",
"<b>Tip:</b> In this tutorial, when a code snipped starts with <code>$</code>, it should be run in the terminal. Otherwise, the code is to be run in the Julia REPL.\n",
"</div>\n",
"\n",
"<div class=\"alert alert-block alert-info\">\n",
"<b>Tip:</b> Avoid calling Julia code from the terminal, use the Julia REPL instead! Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch. This can be time consuming for large projects. In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster. Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code from the terminal.\n",
"</div>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Running parallel code\n",
"\n",
"\n",
"Since we are in a parallel computing course, let's run a parallel \"hello world\" example in Julia. Open a Julia REPL and write \n",
"\n",
"```julia\n",
"julia> using Distributed\n",
"julia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")\n",
"```\n",
"\n",
"Here, we are using the `Distributed` package, which is part of the Julia standard library that provides distributed memory parallel support. The code prints the process id and the number of processes in the current Julia session.\n",
"\n",
"You will provably only see output from 1 proces. We need to add more processes to run the example in parallel. This is done with the `addprocs` function.\n",
"\n",
"```julia\n",
"julia> addprocs(3)\n",
"```\n",
"We have added 3 new processes, plus the old one, we have 4 processes. Run the code again.\n",
"\n",
"```julia\n",
"julia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")\n",
"```\n",
"\n",
"Now, you should see output from 4 processes.\n",
"\n",
"It is possible to specify the number of processes when starting Julia from the terminal with the `-p` argument (useful, e.g., when running in a cluster). If you launch Julia from the terminal as\n",
"\n",
"```\n",
"$ julia -p 3\n",
"```\n",
"and then run \n",
"\n",
"```julia\n",
"julia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")\n",
"```\n",
"\n",
"You should get output from 4 processes as before.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Installing packages\n",
"\n",
"One of the most useful features of Julia is its package manager. It allows one to install Julia packages in a straightforward and platform independent way. To illustrate this, let us consider the following parallel \"Hello world\" example.\n",
"\n",
"\n",
"Copy the following block of code into a new file named `\"hello_mpi.jl\"`\n",
"\n",
"```julia\n",
"# file hello_mpi.jl\n",
"using MPI\n",
"MPI.Init()\n",
"comm = MPI.COMM_WORLD\n",
"rank = MPI.Comm_rank(comm)\n",
"nranks = MPI.Comm_size(comm)\n",
"println(\"Hello world, I am rank $rank of $nranks\")\n",
"```\n",
"\n",
"As you can see from this example, one can access MPI from Julia in a clean way, without type annotations and other complexities of C/C++ code.\n",
"\n",
"Now, run the file from the REPL\n",
"```julia\n",
"julia> incude(\"hello_mpi.jl\")\n",
"```\n",
"\n",
"It provably didn't work, right? Read the error message and note that the MPI package needs to be installed to run this code.\n",
"\n",
"To install a package, we need to enter *package* mode. Remember that we entered into help mode by typing `?`. Package mode is activated by typing `]`\n",
"```julia\n",
"julia> ]\n",
"```\n",
"At this point, the promp should have changed to `(@v1.8) pkg>` indicating that we are in package mode. The text between parenthesis indicates which is the active *project*, i.e., where packages are going to be installed. In this case, we are working with the global project associated with our Julia installation (which is Julia 1.8 in this example, but it can be another version in your case).\n",
"\n",
"To install the MPI package, type\n",
"```julia\n",
"(@v1.8) pkg> add MPI\n",
"```\n",
"Congrats, you have installed MPI!\n",
"\n",
"<div class=\"alert alert-block alert-info\">\n",
" <b>Tip:</b> Many Julia package names end with <code>.jl</code>. This is just a way of signaling that a package is written in Julia. When using such packages, the <code>.jl</code> needs to be ommited. In this case, we have isntalled the <code>MPI.jl</code> package even though we have only typed <code>MPI</code> in the REPL.\n",
"</div>\n",
"\n",
"<div class=\"alert alert-block alert-info\">\n",
" <b>Tip:</b>\n",
"The package you have installed it is the Julia interface to MPI, called <code>MPI.jl</code>. Note that it is not a MPI library by itself. It is just a thin wrapper between MPI and Julia. To use this interface, you need an actual MPI library installed in your system such as OpenMPI or MPICH. Julia downloads and installs a MPI library for you, but it is also possible to use a MPI library already available in your system. This is useful, e.g., when running on HPC clusters. See the documentation of <code>MPI.jl</code> for further details.\n",
"</div>\n",
"\n",
"To check that the package was installed properly, exit package mode by pressing the backspace key several times, and run it again\n",
"\n",
"```julia\n",
"julia> incude(\"hello_mpi.jl\")\n",
"```\n",
"\n",
"Now, it should work, but you provably get output from a single MPI rank only.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Running MPI code\n",
"\n",
"To run MPI applications in parallel, you need a launcher like `mpiexec`. MPI codes written in Julia are not an exception to this rule. From the system terminal, you can run\n",
"```\n",
"$ mpiexec -np 4 julia hello_mpi.jl\n",
"```\n",
"But it will provably don't work since the version of `mpiexec` needs to match with the MPI version we are using from Julia. You can find the path to the `mpiexec` binary you need to use with these commands\n",
"\n",
"```julia\n",
"julia> using MPI\n",
"julia> MPI.mpiexec_path\n",
"```\n",
"\n",
"and then try again\n",
"```\n",
"$ /path/to/my/mpiexec -np 4 julia hello_mpi.jl\n",
"```\n",
"with your particular path.\n",
"\n",
"However, this is not very convenient. Don't worry if you could not make it work! A more elegant way to run MPI code is from the Julia REPL directly, by using these commands:\n",
"```julia\n",
"julia> using MPI\n",
"julia> mpiexec(cmd->run(`$cmd -np 4 julia hello_mpi.jl`))\n",
"```\n",
"\n",
"Now, you should see output from 4 ranks."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Installing packages locally\n",
"\n",
"We have installed the `MPI` package globally and it will be available in all Julia sessions. However, in some situations, we want to work with different versions of the same package or to install packages in an isolated way to avoid potential conflicts with other packages. This can be done by using local projects.\n",
"\n",
"A project is simply a folder in the hard disk. To use a particular folder as your project, you need to *activate* it. This is done by entering package mode and using the `activate` command followed by the path to the folder you want to activate.\n",
"```julia\n",
"(@v1.8) pkg> activate .\n",
"```\n",
" Previous command will activate the current working directory. Note that the dot `.` is indeed the path to the current folder.\n",
" \n",
"The prompt has changed to `(lessons) pkg>` indicating that we are in the project within the `lessons` folder. The particular folder name can be different in your case.\n",
"\n",
"<div class=\"alert alert-block alert-info\">\n",
" <b>Tip:</b> You can activate a project directly when opening Julia from the terminal using the <code>--project</code> flag. The command <code>$ julia --project=.</code> will open Julia and activate a project in the current directory. You can also achieve the same effect by setting the environment variable <code>JULIA_PROJECT</code> with the path of the folder you want to activate.\n",
" </div>\n",
" \n",
" \n",
" <div class=\"alert alert-block alert-info\">\n",
" <b>Tip:</b> The active project folder and the current working directory are two independent concepts! For instance, <code>(@v1.8) pkg> activate folderB</code> and then <code>julia> cd(\"folderA\")</code>, will activate the project in <code>folderB</code> and change the current working directory to <code>folderA</code>.\n",
" \n",
"</div>\n",
"\n",
"At this point all package-related operations will be local to the new project. For instance, install the `DataFrames` package.\n",
"\n",
"```julia\n",
"(lessons) pkg> add DataFrames\n",
"```\n",
"Use the package to check that it is installed\n",
"\n",
"```julia\n",
"julia> using DataFrames\n",
"julia> DataFrame(a=[1,2],b=[3,4])\n",
"```\n",
"Now, we can return to the global project to check that `DataFrames` has not been installed there. To return to the global environment, use `activate` without a folder name.\n",
"\n",
"```julia\n",
"(lessons) pkg> activate\n",
"```\n",
"The prompt is again `(@v1.8) pkg>`\n",
"\n",
"Now, try to use `DataFrames`.\n",
"\n",
"```julia\n",
"julia> using DataFrames\n",
"julia> DataFrame(a=[1,2],b=[3,4])\n",
"```\n",
"You should get an error or a warning unless you already had `DataFrames` installed globally.\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Project and Manifest files\n",
"\n",
"The information about a project is stored in two files `Project.toml` and `Manifest.toml`.\n",
"\n",
"- `Project.toml` contains the packages explicitly installed (the direct dependencies)\n",
"\n",
"- `Manifest.toml` contains direct and indirect dependencies along with the concrete version of each package.\n",
"\n",
"\n",
"In other words, `Project.toml` contains the packages relevant for the user, whereas `Manifest.toml` is the detailed snapshot of all dependencies. The `Manifest.toml` can be used to reproduce the same envinonment in another machine.\n",
"\n",
"You can see the path to the current `Project.toml` file by using the `status` operator (or `st` in its short form) while in package mode\n",
"\n",
"```julia\n",
"(@v1.8) pkg> status\n",
"```\n",
"\n",
"The information about the `Manifest.toml` can be inspected by passing the `-m` flag.\n",
"\n",
"```julia\n",
"(@v1.8) pkg> status -m\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Installing packages from a project file\n",
"\n",
"Project files can be used to install lists of packages defined by others. E.g., to install all the dependencies of a Julia application.\n",
"\n",
"Assume that a colleague has sent to you a `Project.toml` file with this content:\n",
"\n",
"```\n",
"[deps]\n",
"BenchmarkTools = \"6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf\"\n",
"DataFrames = \"a93c6f00-e57d-5684-b7b6-d8193f3e46c0\"\n",
"MPI = \"da04e1cc-30fd-572f-bb4f-1f8673147195\"\n",
"```\n",
"\n",
"Copy the contents of previous code block into a file called `Project.toml` and place it in an empty folder named `newproject`. It is important that the file is named `Project.toml`. You can create a new folder from the REPL with \n",
"\n",
"```julia\n",
"julia> mkdir(\"newproject\")\n",
"```\n",
"\n",
"To install all the packages registered in this file you need to activate the folder containing your `Project.toml` file\n",
"```julia\n",
"(@v1.8) pkg> activate newproject\n",
"```\n",
"and then *instantiating* it\n",
"```julia\n",
"(newproject) pkg> instantiate\n",
"```\n",
"\n",
"The instantiate command will download and install all listed packages and their dependencies in just one click."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Getting help in package mode\n",
"\n",
"You can get help about a particular package operator by writing `help` in front of it\n",
"\n",
"```julia\n",
"(@v1.8) pkg> help activate\n",
"```\n",
"\n",
"You can get an overview of all package commands by typing `help` alone\n",
"```julia\n",
"(@v1.8) pkg> help\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Package operations in Julia code\n",
"\n",
"In some situations it is required to use package commands in Julia code, e.g., to automatize installation and deployment of Julia applications. This can be done using the `Pkg` package. For instance\n",
"\n",
"```julia\n",
"julia> using Pkg\n",
"julia> Pkg.status()\n",
"```\n",
"is equivalent to call `status` in package mode.\n",
"```julia\n",
"(@v1.8) pkg> status\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Conclusion\n",
"\n",
"We have learned the basics of how to work with Julia. Now, you should be ready to start learning more on the language. If you want to further dig into the topics we have covered here, you can take a look and the following links\n",
"\n",
"- Julia Manual https://docs.julialang.org/en/v1/manual/getting-started/\n",
"- Package manager https://pkgdocs.julialang.org/v1/getting-started/\n",
"\n",
"If you want to interact with the Julia community on discourse, sign in at https://discourse.julialang.org/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# License\n",
"\n",
"TODO: replace link to website\n",
"\n",
"This notebook is part of the course [Programming Large Scale Parallel Systems](http://localhost:8000/) at Vrije Universiteit Amsterdam and may be used under a [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) license."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.8.5",
"language": "julia",
"name": "julia-1.8"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

30
docs/src/notebooks/julia_tutorial.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/julia_tutorial.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="julia_tutorial.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/julia_tutorial.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/julia_tutorial.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

1148
docs/src/notebooks/matrix_matrix.ipynb
View File

File diff suppressed because one or more lines are too long

30
docs/src/notebooks/matrix_matrix.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/matrix_matrix.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="matrix_matrix.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/matrix_matrix.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/matrix_matrix.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

413
docs/src/notebooks/mpi_tutorial.ipynb
View File

@ -1,413 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "8d800917",
"metadata": {},
"source": [
"# Tutorial: Using MPI in Julia\n",
"Message Passing Interface (MPI) is a standardized and portable library specification for communication between parallel processes in distributed memory systems. Julia offers a convenient way to work with MPI for creating efficient parallel and distributed applications. In this tutorial, you will learn how to use MPI from Julia to perform parallel computing tasks.\n",
"\n",
"## MPI launches separate Julia instances\n",
"When you run an MPI-enabled Julia script, MPI takes care of spawning multiple instances of the Julia executable, each acting as a separate process. These workers can communicate with each other using MPI communication functions. This enables parallel processing and distributed computation. Here's a summary of how it works:\n",
"\n",
"-- TODO: insert picture here --\n",
"\n",
"- **MPI Spawns Processes**: The `mpiexec` command launches multiple instances of the Julia executable, creating separate worker processes. In this example, 4 Julia workers are spawned.\n",
"\n",
"- **Worker Communication**: These workers can communicate with each other using MPI communication functions, allowing them to exchange data and coordinate actions.\n",
"\n",
"- **Parallel Tasks**: The workers execute parallel tasks simultaneously, working on different parts of the computation to potentially speed up the process.\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"## Installing MPI.jl and MPIClusterManagers Packages\n",
"To use MPI in Julia, you'll need the MPI.jl package, and if you intend to run MPI programs in a Jupyter Notebook, you'll also need the MPIClusterManagers package. These packages provide the necessary bindings to the MPI library and cluster management capabilities. To install the packages, open a terminal and run the following commands:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3cb5f151",
"metadata": {},
"outputs": [],
"source": [
"using Pkg\n",
"Pkg.add(\"MPI\")\n",
"Pkg.add(\"MPIClusterManagers\")"
]
},
{
"cell_type": "markdown",
"id": "ed45a4b2",
"metadata": {},
"source": [
"<div class=\"alert alert-block alert-info\">\n",
" <b>Tip:</b>\n",
"The package <code>MPI.jl</code> is the Julia interface to MPI. Note that it is not a MPI library by itself. It is just a thin wrapper between MPI and Julia. To use this interface, you need an actual MPI library installed in your system such as OpenMPI or MPICH. Julia downloads and installs a MPI library for you, but it is also possible to use a MPI library already available in your system. This is useful, e.g., when running on HPC clusters. See the documentation of <code>MPI.jl</code> for further details.\n",
"</div>"
]
},
{
"cell_type": "markdown",
"id": "7a36916e",
"metadata": {},
"source": [
"## Writing a HelloWorld MPI Program in Julia\n",
"Let's start by creating a simple MPI program that prints a message along with the rank of each worker. \n",
"\n",
"Create a new Julia script, for example, `mpi_hello_world.jl`:\n",
"\n",
"```julia\n",
"using MPI\n",
"\n",
"# Initialize MPI\n",
"MPI.Init()\n",
"\n",
"# Get the default communicator (MPI_COMM_WORLD) for all processes\n",
"comm = MPI.COMM_WORLD\n",
"\n",
"# Get the number of processes in this communicator\n",
"nranks = MPI.Comm_size(comm)\n",
"\n",
"# Get the rank of the current process within the communicator\n",
"rank = MPI.Comm_rank(comm)\n",
"\n",
"# Print a message with the rank of the current process\n",
"println(\"Hello, I am process $rank of $nranks processes!\")\n",
"\n",
"# Finalize MPI\n",
"MPI.Finalize()\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "6caa8d74",
"metadata": {},
"source": [
"### MPI Communicators\n",
"In MPI, a **communicator** is a context in which a group of processes can communicate with each other. `MPI_COMM_WORLD` is one of the MPI standard communicators, it represents all processes in the MPI program. Custom communicators can also be created to group processes based on specific requirements or logical divisions. \n",
"\n",
"The **rank** of a processor is a unique identifier assigned to each process within a communicator. It allows processes to distinguish and address each other in communication operations. "
]
},
{
"cell_type": "markdown",
"id": "19f41e38",
"metadata": {},
"source": [
"## Running the HelloWorld MPI Program\n",
"\n",
"To run MPI applications in parallel, you need a launcher like `mpiexec`. MPI codes written in Julia are not an exception to this rule. From the system terminal, you can run\n",
"```\n",
"$ mpiexec -np 4 mpi_hello_world.jl\n",
"```\n",
"In this command, `-np 4` specifies the desired number of processes. \n",
"But it will probably not work since the version of `mpiexec` needs to match with the MPI version we are using from Julia. You can find the path to the `mpiexec` binary you need to use with these commands\n",
"\n",
"```julia\n",
"julia> using MPI\n",
"julia> MPI.mpiexec_path\n",
"```\n",
"\n",
"and then try again\n",
"```\n",
"$ /path/to/my/mpiexec -np 4 julia mpi_hello_world.jl\n",
"```\n",
"with your particular path.\n",
"\n",
"However, this is not very convenient. Don't worry if you could not make it work! A more elegant way to run MPI code is from the Julia REPL directly, by using these commands:\n",
"```julia\n",
"julia> using MPI\n",
"julia> mpiexec(cmd->run(`$cmd -np 4 julia mpi_hello_world.jl`))\n",
"```\n",
"\n",
"Now, you should see output from 4 ranks.\n"
]
},
{
"cell_type": "markdown",
"id": "0592e58c",
"metadata": {},
"source": [
"## Running MPI Programs in Jupyter Notebook with MPIClusterManagers\n",
"If you want to run your MPI code from a Jupyter Notebook, you can do so using the `MPIClusterManagers` package.\n",
"\n",
"1. Load the packages and start an MPI cluster with the desired number of workers:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cf66dd39",
"metadata": {},
"outputs": [],
"source": [
"using MPIClusterManagers\n",
"# Distributed package is needed for addprocs()\n",
"using Distributed\n",
"\n",
"manager = MPIWorkerManager(4)\n",
"addprocs(manager)"
]
},
{
"cell_type": "markdown",
"id": "d40fe3ee",
"metadata": {},
"source": [
"2. Run your MPI code inside a `@mpi_do` block to execute it on the cluster workers:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0a51d1f2",
"metadata": {},
"outputs": [],
"source": [
"@mpi_do manager begin\n",
" using MPI\n",
" comm = MPI.COMM_WORLD\n",
" rank = MPI.Comm_rank(comm)\n",
" println(\"Hello from process $rank\")\n",
"end\n"
]
},
{
"cell_type": "markdown",
"id": "38ed88c1",
"metadata": {},
"source": [
"MPI is automatically initialized and finalized within the `@mpi_do` block.\n",
"\n",
"3. Remove processes when done:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e0b53cc1",
"metadata": {},
"outputs": [],
"source": [
"rmprocs(manager)"
]
},
{
"cell_type": "markdown",
"id": "5466a650",
"metadata": {},
"source": [
"## Point-to-Point Communication with MPI\n",
"MPI provides point-to-point communication using blocking send and receiving functions `MPI.send`, `MPI.recv`; or their non-blocking versions `MPI.Isend`, and `MPI.Irecv!`. These functions allow individual processes to send and receive data between each other.\n",
"\n",
"### Blocking communication\n",
"\n",
"Let's demonstrate how to send and receive with an example:\n",
"\n",
"```julia\n",
"using MPI\n",
"\n",
"MPI.Init()\n",
"\n",
"comm = MPI.COMM_WORLD\n",
"rank = MPI.Comm_rank(comm)\n",
"\n",
"# Send and receive messages using blocking MPI.send and MPI.recv\n",
"if rank == 0\n",
" data = \"Hello from process $rank !\"\n",
" MPI.send(data, comm, dest=1)\n",
"elseif rank == 1\n",
" received_data = MPI.recv(comm, source=0)\n",
" println(\"Process $rank received: $received_data\")\n",
"end\n",
"\n",
"MPI.Finalize()\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "d4dfe654",
"metadata": {},
"source": [
"In this example, process 0 sends a message using `MPI.send`, and process 1 receives it using `MPI.recv`.\n",
"\n",
"### Non-blocking communication\n",
"\n",
"To demonstrate asynchronous communication, let's modify the example using `MPI.Isend` and `MPI.Irecv!`:\n",
"\n",
"```julia\n",
"using MPI\n",
"\n",
"MPI.Init()\n",
"\n",
"comm = MPI.COMM_WORLD\n",
"rank = MPI.Comm_rank(comm)\n",
"\n",
"# Asynchronous communication using MPI.Isend and MPI.Irecv!\n",
"if rank == 0\n",
" data = \"Hello from process $rank !\"\n",
" request = MPI.Isend(data, comm, dest=1)\n",
" # Other computation can happen here\n",
" MPI.Wait(request)\n",
"elseif rank == 1\n",
" received_data = Array{UInt8}(undef, 50) # Preallocate buffer\n",
" request = MPI.Irecv!(received_data, comm, source=0)\n",
" # Other computation can happen here\n",
" MPI.Wait(request)\n",
" println(\"Process $rank received: $(String(received_data))\")\n",
"end\n",
"\n",
"MPI.Finalize()\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "024db538",
"metadata": {},
"source": [
"In this example, process 0 uses `MPI.Isend` to send the message asynchronously. This function returns immediately, allowing the sender process to continue its execution. However, the actual sending of data is done asynchronously in the background. Similar to `MPI.Isend`, `MPI.Irecv!` returns immediately, allowing the receiver process to continue executing. \n",
"\n",
"<div class=\"alert alert-block alert-warning\">\n",
"<b>Important:</b> In asynchronous communication, always use <code>MPI.Wait()</code> to ensure the communication is finished before accessing the send or receive buffer.\n",
"</div>\n",
"\n",
"\n",
"## Collective Communication with MPI\n",
"MPI provides collective communication functions for communication involving multiple processes. Let's explore some of these functions:\n",
"\n",
"- MPI.Gather: Gathers data from all processes to a single process.\n",
"- MPI.Scatter: Distributes data from one process to all processes.\n",
"- MPI.Bcast: Broadcasts data from one process to all processes.\n",
"- MPI.Barrier: Synchronizes all processes.\n",
"\n",
"\n",
"Let's illustrate the use of `MPI.Gather` and `MPI.Scatter` with an example:\n",
"\n",
"```julia\n",
"# TODO: check if this runs correctly\n",
"using MPI\n",
"using Random\n",
"\n",
"MPI.Init()\n",
"\n",
"comm = MPI.COMM_WORLD\n",
"rank = MPI.Comm_rank(comm)\n",
"size = MPI.Comm_size(comm)\n",
"\n",
"# Root processor generates random data\n",
"data = rand(rank == 0 ? size * 2 : 0)\n",
"\n",
"# Scatter data to all processes\n",
"local_data = Vector{Float64}(undef, 2)\n",
"MPI.Scatter!(data, local_data, comm, root=0)\n",
"\n",
"# Compute local average\n",
"local_average = sum(local_data) / length(local_data)\n",
"\n",
"# Gather local averages at the root processor\n",
"gathered_averages = Vector{Float64}(undef, size)\n",
"MPI.Gather!(local_average, gathered_averages, comm, root=0)\n",
"\n",
"if rank == 0\n",
" # Compute global average of sub-averages\n",
" global_average = sum(gathered_averages) / size\n",
" println(\"Global average: $global_average\")\n",
"end\n",
"\n",
"MPI.Finalize()\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e65cb53f",
"metadata": {},
"outputs": [],
"source": [
"using MPI\n",
"using Random\n",
"\n",
"# TODO: check if this runs correctly\n",
"\n",
"MPI.Init()\n",
"\n",
"comm = MPI.COMM_WORLD\n",
"rank = MPI.Comm_rank(comm)\n",
"size = MPI.Comm_size(comm)\n",
"\n",
"# Root processor generates random data\n",
"data = rand(rank == 0 ? size * 2 : 0)\n",
"\n",
"# Scatter data to all processes\n",
"local_data = Vector{Float64}(undef, 2)\n",
"MPI.Scatter!(data, local_data, comm, root=0)\n",
"\n",
"# Compute local average\n",
"local_average = sum(local_data) / length(local_data)\n",
"\n",
"# Gather local averages at the root processor\n",
"gathered_averages = Vector{Float64}(undef, size)\n",
"MPI.Gather!(local_average, gathered_averages, comm, root=0)\n",
"\n",
"if rank == 0\n",
" # Compute global average of sub-averages\n",
" global_average = sum(gathered_averages) / size\n",
" println(\"Global average: $global_average\")\n",
"end\n",
"\n",
"MPI.Finalize()"
]
},
{
"cell_type": "markdown",
"id": "dfd5da9e",
"metadata": {},
"source": [
"In this example, the root processor generates random data and then scatters it to all processes using MPI.Scatter. Each process calculates the average of its local data, and then the local averages are gathered using MPI.Gather. The root processor computes the global average of all sub-averages and prints it."
]
},
{
"cell_type": "markdown",
"id": "5e8f6e6a",
"metadata": {},
"source": [
"# License\n",
"\n",
"TODO: replace link to website\n",
"\n",
"This notebook is part of the course [Programming Large Scale Parallel Systems](http://localhost:8000/) at Vrije Universiteit Amsterdam and may be used under a [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) license."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c9364808",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.9.1",
"language": "julia",
"name": "julia-1.9"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

30
docs/src/notebooks/mpi_tutorial.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/mpi_tutorial.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="mpi_tutorial.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/mpi_tutorial.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/mpi_tutorial.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

187
docs/src/notebooks/notebook-hello.ipynb
View File

File diff suppressed because one or more lines are too long

30
docs/src/notebooks/notebook-hello.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/notebook-hello.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="notebook-hello.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/notebook-hello.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/notebook-hello.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

226
docs/src/notebooks/solutions.ipynb
View File

@ -1,226 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "f48b9a60",
"metadata": {},
"source": [
"# Solutions to Notebook Exercises\n",
"\n",
"## Julia Basics: Exercise 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a06fd02a",
"metadata": {},
"outputs": [],
"source": [
"function ex1(a)\n",
" j = 1\n",
" m = a[j]\n",
" for (i,ai) in enumerate(a)\n",
" if m < ai\n",
" m = ai\n",
" j = i\n",
" end\n",
" end\n",
" (m,j)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "175b6c35",
"metadata": {},
"source": [
"## Julia Basics: Exercise 2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb289acd",
"metadata": {},
"outputs": [],
"source": [
"ex2(f,g) = x -> f(x) + g(x) "
]
},
{
"cell_type": "markdown",
"id": "86250e27",
"metadata": {},
"source": [
"## Julia Basics: Exercise 3"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "41b537ab",
"metadata": {},
"outputs": [],
"source": [
"function compute_values(n,max_iters)\n",
" x = LinRange(-1.7,0.7,n)\n",
" y = LinRange(-1.2,1.2,n)\n",
" values = zeros(Int,n,n)\n",
" for j in 1:n\n",
" for i in 1:n\n",
" values[i,j] = mandel(x[i],y[j],max_iters)\n",
" end\n",
" end\n",
" values\n",
"end\n",
"values = compute_values(1000,10)\n",
"using GLMakie\n",
"heatmap(x,y,values)"
]
},
{
"cell_type": "markdown",
"id": "d6d12733",
"metadata": {},
"source": [
"## Matrix Multiplication : Exercise 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "be73e87a",
"metadata": {},
"outputs": [],
"source": [
"function matmul_dist_3!(C,A,B)\n",
" m = size(C,1)\n",
" n = size(C,2)\n",
" l = size(A,2)\n",
" @assert size(A,1) == m\n",
" @assert size(B,2) == n\n",
" @assert size(B,1) == l\n",
" @assert mod(m,nworkers()) == 0\n",
" # Implement here\n",
" nrows_w = div(m,nworkers())\n",
" @sync for (i,w) in enumerate(workers())\n",
" rows_w = (1:nrows_w) .+ (i-1)*nrows_w\n",
" Aw = A[rows_w,:]\n",
" ftr = @spawnat w begin\n",
" Cw = similar(Aw,nrows_w,n)\n",
" matmul_seq!(Cw,Aw,B)\n",
" Cw\n",
" end\n",
" @async C[rows_w,:] = fetch(ftr)\n",
" end\n",
" C\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "2d9f4813",
"metadata": {},
"source": [
"## Jacobi Method : Exercise 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cf3b1e72",
"metadata": {},
"outputs": [],
"source": [
"@mpi_do manager begin\n",
" using MPI\n",
" comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
" nw = MPI.Comm_size(comm)\n",
" iw = MPI.Comm_rank(comm)+1\n",
" function jacobi_mpi(n,niters)\n",
" if mod(n,nw) != 0\n",
" println(\"n must be a multiple of nw\")\n",
" MPI.Abort(comm,1)\n",
" end\n",
" n_own = div(n,nw)\n",
" u = zeros(n_own+2)\n",
" u[1] = -1\n",
" u[end] = 1\n",
" u_new = copy(u)\n",
" for t in 1:niters\n",
" reqs_snd = MPI.Request[]\n",
" reqs_rcv = MPI.Request[]\n",
" if iw != 1\n",
" neig_rank = (iw-1)-1\n",
" req = MPI.Isend(view(u,2:2),comm,dest=neig_rank,tag=0)\n",
" push!(reqs_snd,req)\n",
" req = MPI.Irecv!(view(u,1:1),comm,source=neig_rank,tag=0)\n",
" push!(reqs_rcv,req)\n",
" end\n",
" if iw != nw\n",
" neig_rank = (iw+1)-1\n",
" s = n_own+1\n",
" r = n_own+2\n",
" req = MPI.Isend(view(u,s:s),comm,dest=neig_rank,tag=0)\n",
" push!(reqs_snd,req)\n",
" req = MPI.Irecv!(view(u,r:r),comm,source=neig_rank,tag=0)\n",
" push!(reqs_rcv,req)\n",
" end\n",
" for i in 3:n_own\n",
" u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
" end\n",
" MPI.Waitall(reqs_rcv)\n",
" for i in (2,n_own+1)\n",
" u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
" end\n",
" MPI.Waitall(reqs_snd)\n",
" u, u_new = u_new, u\n",
" end\n",
" u\n",
" @show u\n",
" end\n",
" niters = 100\n",
" load = 4\n",
" n = load*nw\n",
" jacobi_mpi(n,niters)\n",
"end"
]
},
{
"cell_type": "markdown",
"id": "47d88e7a",
"metadata": {},
"source": [
"# License\n",
"\n",
"TODO: replace link to website\n",
"\n",
"This notebook is part of the course [Programming Large Scale Parallel Systems](http://localhost:8000/) at Vrije Universiteit Amsterdam and may be used under a [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) license."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "968304a6",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.9.1",
"language": "julia",
"name": "julia-1.9"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

30
docs/src/notebooks/solutions.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/solutions.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="solutions.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/solutions.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/solutions.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```

260
docs/src/notebooks/tsp.ipynb
View File

@ -1,260 +0,0 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "8850d90e",
"metadata": {},
"outputs": [],
"source": [
"using Distributed"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "5d4935ee",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4-element Vector{Int64}:\n",
" 2\n",
" 3\n",
" 4\n",
" 5"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"if procs() == workers()\n",
" addprocs(4)\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "4a2756ae",
"metadata": {},
"outputs": [],
"source": [
"@everywhere function visited(city,hops,path)\n",
" for i = 1:hops\n",
" if path[i] == city\n",
" return true\n",
" end\n",
" end\n",
" return false\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "39e9e667",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tsp_serial_impl (generic function with 1 method)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"function tsp_serial_impl(connections,hops,path,current_distance,min_distance)\n",
" num_cities = length(connections)\n",
" if hops == num_cities\n",
" if current_distance < min_distance\n",
" return current_distance\n",
" end\n",
" else\n",
" current_city = path[hops]\n",
" next_hops = hops + 1\n",
" for (next_city,distance_increment) in connections[current_city]\n",
" if !visited(next_city,hops,path)\n",
" path[next_hops] = next_city\n",
" next_distance = current_distance + distance_increment\n",
" if next_distance < min_distance\n",
" return tsp_serial_impl(connections,next_hops,path,next_distance,min_distance)\n",
" end\n",
" end\n",
" end \n",
" end\n",
" min_distance\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "83b58881",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tsp_serial (generic function with 1 method)"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"function tsp_serial(connections,city)\n",
" num_cities = length(connections)\n",
" path=zeros(Int,num_cities)\n",
" hops = 1\n",
" path[hops] = city\n",
" current_distance = 0\n",
" min_distance = typemax(Int)\n",
" min_distance = tsp_serial_impl(connections,hops,path,current_distance,min_distance)\n",
" (;path=path,distance=min_distance)\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "78095098",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(path = [1, 4, 5, 2, 3, 6], distance = 222)"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"connections = [\n",
" [(1,0),(4,39),(5,76), (6,78),(3,94),(2,97)],\n",
" [(2,0),(5,25),(4,58),(3,62),(1,97),(6,109)],\n",
" [(3,0),(6,58),(2,62),(4,68),(5,70),(1,94)],\n",
" [(4,0),(5,38),(1,39),(2,58),(3,68),(6,78)],\n",
" [(5,0),(2,25),(4,38),(3,70),(1,76),(6,104)],\n",
" [(6,0),(3,58),(1,78),(4,78),(5,104),(2,109)]\n",
"]\n",
"city = 1\n",
"tsp_serial(connections,city)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "03f0dd8e",
"metadata": {},
"outputs": [],
"source": [
"@everywhere function tsp_dist_impl(connections,hops,path,current_distance,min_distance,max_hops,jobs_chnl,ftr_result)\n",
" num_cities = length(connections)\n",
" if hops == num_cities\n",
" if current_distance < min_distance\n",
" if ftr_result !== nothing\n",
" @spawnat 1 begin\n",
" result = fetch(ftr_result)\n",
" result.path .= path\n",
" result.min_distance_ref[] = current_distance\n",
" end |> wait\n",
" end\n",
" return current_distance\n",
" end\n",
" elseif hops <= max_hops\n",
" current_city = path[hops]\n",
" next_hops = hops + 1\n",
" for (next_city,distance_increment) in connections[current_city]\n",
" if !visited(next_city,hops,path)\n",
" path[next_hops] = next_city\n",
" next_distance = current_distance + distance_increment\n",
" if next_distance < min_distance\n",
" return tsp_dist_impl(connections,next_hops,path,next_distance,min_distance,max_hops,jobs_chnl,ftr_result)\n",
" end\n",
" end\n",
" end \n",
" else\n",
" if jobs_channel !== nothing\n",
" put!(jobs_chnl,(;hops,path,current_distance))\n",
" end\n",
" end\n",
" min_distance\n",
"end\n",
"\n",
"function tsp_dist(connections,city)\n",
" max_hops = 2\n",
" num_cities = length(connections)\n",
" path=zeros(Int,num_cities)\n",
" hops = 1\n",
" path[hops] = city\n",
" current_distance = 0\n",
" min_distance = typemax(Int)\n",
" jobs_chnl = RemoteChannel(()->Channel{Any}(10))\n",
" ftr_result = @spawnat 1 (;path,min_distance_ref=Ref(min_distance))\n",
" task = @async begin\n",
" tsp_dist_impl(connections,hops,path,current_distance,min_distance,max_hops,jobs_chnl,nothing)\n",
" for w in workers()\n",
" put!(job_chnl,nothing)\n",
" end\n",
" end\n",
" @sync for w in workers()\n",
" @spawnat w begin\n",
" max_hops = typemax(Int)\n",
" jobs_channel = nothing\n",
" while true\n",
" job = take!(jobs_chnl)\n",
" if job == nothing\n",
" break\n",
" end\n",
" hobs = job.hobs\n",
" path = job.path\n",
" current_distance = job.current_distance\n",
" tsp_dist_impl(connections,hops,path,current_distance,min_distance,max_hops,jobs_chnl,ftr_result)\n",
" end\n",
" end\n",
" end \n",
" (;path=path,distance=min_distance)\n",
"end\n",
"city = 1\n",
"tsp_dist(connections,city)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "370a1205",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.9.0",
"language": "julia",
"name": "julia-1.9"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.9.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

30
docs/src/notebooks/tsp.md
View File

@ -1,30 +0,0 @@
```@meta
EditURL = "https://github.com/fverdugo/XM_40017/blob/main/docs/src/notebooks/tsp.ipynb"
```
```@raw html
<div class="admonition is-success">
<header class="admonition-header">Tip</header>
<div class="admonition-body">
<ul>
<li>
Download this notebook and run it locally on your machine [recommended]. Click <a href="tsp.ipynb" download>here</a>.
</li>
<li>
You can also run this notebook in the cloud using Binder. Click <a href="https://mybinder.org/v2/gh/fverdugo/XM_40017/gh-pages?filepath=dev/notebooks/tsp.ipynb">here</a>
.
</li>
</ul>
</div>
</div>
```
```@raw html
<iframe id="notebook" src="../notebook-html/tsp.html" style="width:100%"></iframe>
<script>
document.addEventListener('DOMContentLoaded', function(){
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
</script>
```