build based on 60a6d02

dev/.documenter-siteinfo.json

@@ -1 +1 @@
-{"documenter":{"julia_version":"1.10.4","generation_timestamp":"2024-08-27T08:33:26","documenter_version":"1.6.0"}}
+{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-02T11:57:03","documenter_version":"1.6.0"}}

dev/mpi_collectives.ipynb

@@ -39,7 +39,7 @@
    "source": [
     "## Collective communication\n",
     "\n",
-    "MPI provides collective communication functions for communication involving multiple processes. Some usual collective functions are:\n",
+    "MPI provides a set of routines for communication involving multiple processes. These are called *collective communication* operations. Some usual collective operations are:\n",
     "\n",
     "\n",
     "- `MPI_Barrier`: Synchronize all processes\n",
@@ -61,9 +61,9 @@
    "id": "4ffa5e56",
    "metadata": {},
    "source": [
-    "## Why collective primitives?\n",
+    "## Why collective operations?\n",
     "\n",
-    "Point-to-point communication primitives provide all the building blocks needed in parallel programs and could be used to implement the collective functions described above. Then, why does MPI provide collective communication directives? There are several reasons:\n",
+    "Point-to-point communication functions provide all the building blocks needed in parallel programs and could be used to implement the collective functions described above. Then, why does MPI provide collective communication functions? There are several reasons:\n",
     "\n",
     "- Ease of use: It is handy for users to have these functions readily available instead of having to implement them.\n",
     "- Performance: Library implementations typically use efficient algorithms (such as reduction trees).\n",
@@ -77,6 +77,8 @@
    "source": [
     "## Semantics of collective operations\n",
     "\n",
+    "These are key properties of collective operations:\n",
+    "\n",
     "\n",
     "- Completeness: All the collective communication directives above are *complete* operations. Thus, it is safe to use and reset the buffers once the function returns.\n",
     "- Standard mode: Collective directives are in standard mode only, like `MPI_Send`. Assuming that they block is erroneous, assuming that they do not block is also erroneous.\n",
@@ -84,7 +86,7 @@
     "\n",
     "\n",
     "<div class=\"alert alert-block alert-info\">\n",
-    "<b>Note:</b> Recent versions of the MPI standard also include non-blocking (incomplete) versions of collective operations (not covered in this notebook). A particularly funny one is the non-blocking barrier `MPI_Ibarrier`.\n",
+    "<b>Note:</b> Recent versions of the MPI standard also include non-blocking (i.e., incomplete) versions of collective operations (not covered in this notebook). A particularly funny one is the non-blocking barrier `MPI_Ibarrier`.\n",
     "</div>"
    ]
   },
@@ -145,7 +147,7 @@
    "source": [
     "## MPI_Reduce\n",
     "\n",
-    "Combines values provided by different processors according to a given reduction operation. The result is received in a single process (called the root process).\n",
+    "This function combines values provided by different processors according to a given reduction operation. The result is received in a single process (called the root process).\n",
     "\n",
     "In Julia:\n",
     "```julia\n",
@@ -702,7 +704,7 @@
     "When you write an MPI program it is very likely that you are going to use libraries that also use MPI to send messages. Ideally, these libraries should not interfere with application messages. Using tags to isolate the messages send by your application does not solve the problem. MPI communicators fix this problem as they provided an isolated communication context. For instance, `MPI_SEND` and `MPI_RECV` specify a communicator. `MPI_RECV` can only receive messages sent to same communicator. The same is also true for collective communication directives. If two libraries use different communicators, their message will never interfere. In particular it is recommended to never use the default communicator, `MPI_COMM_WORLD`, directly when working with other libraries. A new isolated communicator can be created with `MPI_Comm_dup`.\n",
     "\n",
     "\n",
-    "### Process groups\n",
+    "### Groups of processes\n",
     "\n",
     "On the other hand,  imagine that we want to use an MPI communication directive like `MPI_Gather`, but we only want a subset of the processes to participate in the operation. So far, we have used always the default communication `MPI_COMM_WORLD`, which represents all processes. Thus, by using this communicator, we are including all processes in the operation. We can create other communicators that contain only a subset of processes. To this end, we can use function `MPI_Comm_split`.\n"
    ]
@@ -793,7 +795,7 @@
     "\n",
     "There are two key parameters:\n",
     "\n",
-    "- `color`: all processes with the same color will be grouped in the same communicator.\n",
+    "- `color`: all processes with the same color will be grouped in the same new communicator.\n",
     "- `key`: The processes will be ranked in the new communicator according to key, breaking ties with the rank in the old communicator. \n",
     "\n"
    ]
@@ -872,6 +874,101 @@
     "run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "d465ebce",
+   "metadata": {},
+   "source": [
+    "Try to run the code without splitting the communicator. I.e., replace `newcomm = MPI.Comm_split(comm, color, key)` with `newcomm = comm`. Try to figure out what will happen before executing the code."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d334aea1",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
+    "\n",
+    "- MPI also defines operations involving several processes called, collective operations.\n",
+    "- These are provided both for convenience and performance.\n",
+    "- The semantics are equivalent to \"standard mode\" `MPI_Send`, but there are also non-blocking versions (not discussed in this notebook).\n",
+    "- Discovering message sizes is often done by communicating the message size, instead of using `MPI_Probe`.\n",
+    "- Finally, we discussed MPI communicators. They provide two key features: isolated communication context and creating groups of processes. They are useful, for instance, to combine different libraries using MPI in the same application, and to use collective operations in a subset of the processes.\n",
+    "\n",
+    "After learning this material and the previous MPI notebook, you have a solid basis to start implementing sophisticated parallel algorithms using MPI."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c6b23485",
+   "metadata": {},
+   "source": [
+    "## Exercises"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "90dc58bb",
+   "metadata": {},
+   "source": [
+    "### Exercise 1\n",
+    "\n",
+    "In the parallel implementation of the Jacobi method in previous notebook, we assumed that the method runs for a given number of iterations. However, other stopping criteria are used in practice. The following sequential code implements a version of Jacobi in which the method iterates until the norm of the difference between u and u_new is below a tolerance.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0fcb0cd6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "function jacobi_with_tol(n,tol)\n",
+    "    u = zeros(n+2)\n",
+    "    u[1] = -1\n",
+    "    u[end] = 1\n",
+    "    u_new = copy(u)\n",
+    "    increment = similar(u)\n",
+    "    while true\n",
+    "        for i in 2:(n+1)\n",
+    "            u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
+    "        end\n",
+    "        increment .= u_new .- u\n",
+    "        norm_increment = 0.0\n",
+    "        for i in 1:n\n",
+    "            increment_i = increment[i]\n",
+    "            norm_increment += increment_i*increment_i\n",
+    "        end\n",
+    "        norm_increment = sqrt(norm_increment)\n",
+    "        if norm_increment < tol*n\n",
+    "            return u_new\n",
+    "        end\n",
+    "        u, u_new = u_new, u\n",
+    "    end\n",
+    "    u\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "dbf0c3b8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "n = 10\n",
+    "tol = 1e-12\n",
+    "jacobi_with_tol(n,tol)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "aab1455e",
+   "metadata": {},
+   "source": [
+    "Implement a parallel version of this algorithm. Recommended: start with the parallel implementation given in the previous notebook (see function `jacobi_mpi`) and introduce the new stopping criteria. Think carefully about which MPI operations you need to use in this case."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "5e8f6e6a",

dev/mpi_collectives_src/index.html

@@ -7556,7 +7556,7 @@ a.anchor-link {
 </div>
 <div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
 </div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
-<h2 id="Collective-communication">Collective communication<a class="anchor-link" href="#Collective-communication">¶</a></h2><p>MPI provides collective communication functions for communication involving multiple processes. Some usual collective functions are:</p>
+<h2 id="Collective-communication">Collective communication<a class="anchor-link" href="#Collective-communication">¶</a></h2><p>MPI provides a set of routines for communication involving multiple processes. These are called <em>collective communication</em> operations. Some usual collective operations are:</p>
 <ul>
 <li><code>MPI_Barrier</code>: Synchronize all processes</li>
 <li><code>MPI_Bcase</code>: Send the same data from one to all processes</li>
@@ -7579,7 +7579,7 @@ a.anchor-link {
 </div>
 <div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
 </div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
-<h2 id="Why-collective-primitives?">Why collective primitives?<a class="anchor-link" href="#Why-collective-primitives?">¶</a></h2><p>Point-to-point communication primitives provide all the building blocks needed in parallel programs and could be used to implement the collective functions described above. Then, why does MPI provide collective communication directives? There are several reasons:</p>
+<h2 id="Why-collective-operations?">Why collective operations?<a class="anchor-link" href="#Why-collective-operations?">¶</a></h2><p>Point-to-point communication functions provide all the building blocks needed in parallel programs and could be used to implement the collective functions described above. Then, why does MPI provide collective communication functions? There are several reasons:</p>
 <ul>
 <li>Ease of use: It is handy for users to have these functions readily available instead of having to implement them.</li>
 <li>Performance: Library implementations typically use efficient algorithms (such as reduction trees).</li>
@@ -7595,13 +7595,14 @@ a.anchor-link {
 </div>
 <div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
 </div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
-<h2 id="Semantics-of-collective-operations">Semantics of collective operations<a class="anchor-link" href="#Semantics-of-collective-operations">¶</a></h2><ul>
+<h2 id="Semantics-of-collective-operations">Semantics of collective operations<a class="anchor-link" href="#Semantics-of-collective-operations">¶</a></h2><p>These are key properties of collective operations:</p>
+<ul>
 <li>Completeness: All the collective communication directives above are <em>complete</em> operations. Thus, it is safe to use and reset the buffers once the function returns.</li>
 <li>Standard mode: Collective directives are in standard mode only, like <code>MPI_Send</code>. Assuming that they block is erroneous, assuming that they do not block is also erroneous.</li>
 <li>Synchronization:  Completion of a call does not guarantee that other processes have completed the operation. A collective operation may or may not have the effect of synchronizing all processes, the only exception is <code>MPI_Barrier</code> of course.</li>
 </ul>
 <div class="alert alert-block alert-info">
-<b>Note:</b> Recent versions of the MPI standard also include non-blocking (incomplete) versions of collective operations (not covered in this notebook). A particularly funny one is the non-blocking barrier `MPI_Ibarrier`.
+<b>Note:</b> Recent versions of the MPI standard also include non-blocking (i.e., incomplete) versions of collective operations (not covered in this notebook). A particularly funny one is the non-blocking barrier `MPI_Ibarrier`.
 </div>
 </div>
 </div>
@@ -7665,7 +7666,7 @@ a.anchor-link {
 </div>
 <div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
 </div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
-<h2 id="MPI_Reduce">MPI_Reduce<a class="anchor-link" href="#MPI_Reduce">¶</a></h2><p>Combines values provided by different processors according to a given reduction operation. The result is received in a single process (called the root process).</p>
+<h2 id="MPI_Reduce">MPI_Reduce<a class="anchor-link" href="#MPI_Reduce">¶</a></h2><p>This function combines values provided by different processors according to a given reduction operation. The result is received in a single process (called the root process).</p>
 <p>In Julia:</p>
 <div class="highlight"><pre><span></span><span class="n">MPI</span><span class="o">.</span><span class="n">Reduce!</span><span class="p">(</span><span class="n">sendbuf</span><span class="p">,</span><span class="w"> </span><span class="n">recvbuf</span><span class="p">,</span><span class="w"> </span><span class="n">op</span><span class="p">,</span><span class="w"> </span><span class="n">comm</span><span class="p">;</span><span class="w"> </span><span class="n">root</span><span class="p">)</span>
 </pre></div>
@@ -8246,7 +8247,7 @@ a.anchor-link {
 <li>To define a group of processes</li>
 </ol>
 <h3 id="Communication-context">Communication context<a class="anchor-link" href="#Communication-context">¶</a></h3><p>When you write an MPI program it is very likely that you are going to use libraries that also use MPI to send messages. Ideally, these libraries should not interfere with application messages. Using tags to isolate the messages send by your application does not solve the problem. MPI communicators fix this problem as they provided an isolated communication context. For instance, <code>MPI_SEND</code> and <code>MPI_RECV</code> specify a communicator. <code>MPI_RECV</code> can only receive messages sent to same communicator. The same is also true for collective communication directives. If two libraries use different communicators, their message will never interfere. In particular it is recommended to never use the default communicator, <code>MPI_COMM_WORLD</code>, directly when working with other libraries. A new isolated communicator can be created with <code>MPI_Comm_dup</code>.</p>
-<h3 id="Process-groups">Process groups<a class="anchor-link" href="#Process-groups">¶</a></h3><p>On the other hand,  imagine that we want to use an MPI communication directive like <code>MPI_Gather</code>, but we only want a subset of the processes to participate in the operation. So far, we have used always the default communication <code>MPI_COMM_WORLD</code>, which represents all processes. Thus, by using this communicator, we are including all processes in the operation. We can create other communicators that contain only a subset of processes. To this end, we can use function <code>MPI_Comm_split</code>.</p>
+<h3 id="Groups-of-processes">Groups of processes<a class="anchor-link" href="#Groups-of-processes">¶</a></h3><p>On the other hand,  imagine that we want to use an MPI communication directive like <code>MPI_Gather</code>, but we only want a subset of the processes to participate in the operation. So far, we have used always the default communication <code>MPI_COMM_WORLD</code>, which represents all processes. Thus, by using this communicator, we are including all processes in the operation. We can create other communicators that contain only a subset of processes. To this end, we can use function <code>MPI_Comm_split</code>.</p>
 </div>
 </div>
 </div>
@@ -8330,7 +8331,7 @@ a.anchor-link {
 </pre></div>
 <p>There are two key parameters:</p>
 <ul>
-<li><code>color</code>: all processes with the same color will be grouped in the same communicator.</li>
+<li><code>color</code>: all processes with the same color will be grouped in the same new communicator.</li>
 <li><code>key</code>: The processes will be ranked in the new communicator according to key, breaking ties with the rank in the old communicator.</li>
 </ul>
 </div>
@@ -8421,6 +8422,121 @@ a.anchor-link {
 </div>
 </div>
 </div>
+<div class="jp-Cell jp-MarkdownCell jp-Notebook-cell" id="cell-id=d465ebce">
+<div class="jp-Cell-inputWrapper" tabindex="0">
+<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
+</div>
+<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
+</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
+<p>Try to run the code without splitting the communicator. I.e., replace <code>newcomm = MPI.Comm_split(comm, color, key)</code> with <code>newcomm = comm</code>. Try to figure out what will happen before executing the code.</p>
+</div>
+</div>
+</div>
+</div>
+<div class="jp-Cell jp-MarkdownCell jp-Notebook-cell" id="cell-id=d334aea1">
+<div class="jp-Cell-inputWrapper" tabindex="0">
+<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
+</div>
+<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
+</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
+<h2 id="Conclusion">Conclusion<a class="anchor-link" href="#Conclusion">¶</a></h2><ul>
+<li>MPI also defines operations involving several processes called, collective operations.</li>
+<li>These are provided both for convenience and performance.</li>
+<li>The semantics are equivalent to "standard mode" <code>MPI_Send</code>, but there are also non-blocking versions (not discussed in this notebook).</li>
+<li>Discovering message sizes is often done by communicating the message size, instead of using <code>MPI_Probe</code>.</li>
+<li>Finally, we discussed MPI communicators. They provide two key features: isolated communication context and creating groups of processes. They are useful, for instance, to combine different libraries using MPI in the same application, and to use collective operations in a subset of the processes.</li>
+</ul>
+<p>After learning this material and the previous MPI notebook, you have a solid basis to start implementing sophisticated parallel algorithms using MPI.</p>
+</div>
+</div>
+</div>
+</div>
+<div class="jp-Cell jp-MarkdownCell jp-Notebook-cell" id="cell-id=c6b23485">
+<div class="jp-Cell-inputWrapper" tabindex="0">
+<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
+</div>
+<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
+</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
+<h2 id="Exercises">Exercises<a class="anchor-link" href="#Exercises">¶</a></h2>
+</div>
+</div>
+</div>
+</div>
+<div class="jp-Cell jp-MarkdownCell jp-Notebook-cell" id="cell-id=90dc58bb">
+<div class="jp-Cell-inputWrapper" tabindex="0">
+<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
+</div>
+<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
+</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
+<h3 id="Exercise-1">Exercise 1<a class="anchor-link" href="#Exercise-1">¶</a></h3><p>In the parallel implementation of the Jacobi method in previous notebook, we assumed that the method runs for a given number of iterations. However, other stopping criteria are used in practice. The following sequential code implements a version of Jacobi in which the method iterates until the norm of the difference between u and u_new is below a tolerance.</p>
+</div>
+</div>
+</div>
+</div><div class="jp-Cell jp-CodeCell jp-Notebook-cell jp-mod-noOutputs" id="cell-id=0fcb0cd6">
+<div class="jp-Cell-inputWrapper" tabindex="0">
+<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
+</div>
+<div class="jp-InputArea jp-Cell-inputArea">
+<div class="jp-InputPrompt jp-InputArea-prompt">In [ ]:</div>
+<div class="jp-CodeMirrorEditor jp-Editor jp-InputArea-editor" data-type="inline">
+<div class="cm-editor cm-s-jupyter">
+<div class="highlight hl-julia"><pre><span></span><span class="k">function</span><span class="w"> </span><span class="n">jacobi_with_tol</span><span class="p">(</span><span class="n">n</span><span class="p">,</span><span class="n">tol</span><span class="p">)</span>
+<span class="w">    </span><span class="n">u</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">zeros</span><span class="p">(</span><span class="n">n</span><span class="o">+</span><span class="mi">2</span><span class="p">)</span>
+<span class="w">    </span><span class="n">u</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-</span><span class="mi">1</span>
+<span class="w">    </span><span class="n">u</span><span class="p">[</span><span class="k">end</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">1</span>
+<span class="w">    </span><span class="n">u_new</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">copy</span><span class="p">(</span><span class="n">u</span><span class="p">)</span>
+<span class="w">    </span><span class="n">increment</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">similar</span><span class="p">(</span><span class="n">u</span><span class="p">)</span>
+<span class="w">    </span><span class="k">while</span><span class="w"> </span><span class="nb">true</span>
+<span class="w">        </span><span class="k">for</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="mi">2</span><span class="o">:</span><span class="p">(</span><span class="n">n</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
+<span class="w">            </span><span class="n">u_new</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mf">0.5</span><span class="o">*</span><span class="p">(</span><span class="n">u</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span><span class="o">+</span><span class="n">u</span><span class="p">[</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">])</span>
+<span class="w">        </span><span class="k">end</span>
+<span class="w">        </span><span class="n">increment</span><span class="w"> </span><span class="o">.=</span><span class="w"> </span><span class="n">u_new</span><span class="w"> </span><span class="o">.-</span><span class="w"> </span><span class="n">u</span>
+<span class="w">        </span><span class="n">norm_increment</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mf">0.0</span>
+<span class="w">        </span><span class="k">for</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="mi">1</span><span class="o">:</span><span class="n">n</span>
+<span class="w">            </span><span class="n">increment_i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">increment</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
+<span class="w">            </span><span class="n">norm_increment</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="n">increment_i</span><span class="o">*</span><span class="n">increment_i</span>
+<span class="w">        </span><span class="k">end</span>
+<span class="w">        </span><span class="n">norm_increment</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">sqrt</span><span class="p">(</span><span class="n">norm_increment</span><span class="p">)</span>
+<span class="w">        </span><span class="k">if</span><span class="w"> </span><span class="n">norm_increment</span><span class="w"> </span><span class="o">&lt;</span><span class="w"> </span><span class="n">tol</span><span class="o">*</span><span class="n">n</span>
+<span class="w">            </span><span class="k">return</span><span class="w"> </span><span class="n">u_new</span>
+<span class="w">        </span><span class="k">end</span>
+<span class="w">        </span><span class="n">u</span><span class="p">,</span><span class="w"> </span><span class="n">u_new</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">u_new</span><span class="p">,</span><span class="w"> </span><span class="n">u</span>
+<span class="w">    </span><span class="k">end</span>
+<span class="w">    </span><span class="n">u</span>
+<span class="k">end</span>
+</pre></div>
+</div>
+</div>
+</div>
+</div>
+</div><div class="jp-Cell jp-CodeCell jp-Notebook-cell jp-mod-noOutputs" id="cell-id=dbf0c3b8">
+<div class="jp-Cell-inputWrapper" tabindex="0">
+<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
+</div>
+<div class="jp-InputArea jp-Cell-inputArea">
+<div class="jp-InputPrompt jp-InputArea-prompt">In [ ]:</div>
+<div class="jp-CodeMirrorEditor jp-Editor jp-InputArea-editor" data-type="inline">
+<div class="cm-editor cm-s-jupyter">
+<div class="highlight hl-julia"><pre><span></span><span class="n">n</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">10</span>
+<span class="n">tol</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mf">1e-12</span>
+<span class="n">jacobi_with_tol</span><span class="p">(</span><span class="n">n</span><span class="p">,</span><span class="n">tol</span><span class="p">)</span>
+</pre></div>
+</div>
+</div>
+</div>
+</div>
+</div>
+<div class="jp-Cell jp-MarkdownCell jp-Notebook-cell" id="cell-id=aab1455e">
+<div class="jp-Cell-inputWrapper" tabindex="0">
+<div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">
+</div>
+<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
+</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput" data-mime-type="text/markdown">
+<p>Implement a parallel version of this algorithm. Recommended: start with the parallel implementation given in the previous notebook (see function <code>jacobi_mpi</code>) and introduce the new stopping criteria. Think carefully about which MPI operations you need to use in this case.</p>
+</div>
+</div>
+</div>
+</div>
 <div class="jp-Cell jp-MarkdownCell jp-Notebook-cell" id="cell-id=5e8f6e6a">
 <div class="jp-Cell-inputWrapper" tabindex="0">
 <div class="jp-Collapser jp-InputCollapser jp-Cell-inputCollapser">