+- · XM_40017
Tip
- · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+- · XM_40017
Tip
diff --git a/dev/getting_started_with_julia/index.html b/dev/getting_started_with_julia/index.html
index 9a8c3b8..306b81a 100644
--- a/dev/getting_started_with_julia/index.html
+++ b/dev/getting_started_with_julia/index.html
@@ -1,5 +1,5 @@
-Getting started · XM_40017 Tip Warning Note Tip
Home · XM_40017 Note Warning
- · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
Getting started
Introduction
The programming of this course will be done using the Julia programming language. Thus, we start by explaining how to get up and running with Julia. After studying this page, you will be able to:
- Use the Julia REPL,
- Run serial and parallel code,
- Install and manage Julia packages.
Why Julia?
Courses related with high-performance computing (HPC) often use languages such as C, C++, or Fortran. We use Julia instead to make the course accessible to a wider set of students, including the ones that have no experience with C/C++ or Fortran, but are willing to learn parallel programming. Julia is a relatively new programming language specifically designed for scientific computing. It combines a high-level syntax close to interpreted languages like Python with the performance of compiled languages like C, C++, or Fortran. Thus, Julia will allow us to write efficient parallel algorithms with a syntax that is convenient in a teaching setting. In addition, Julia provides easy access to different programming models to write distributed algorithms, which will be useful to learn and experiment with them.
You can run the code in this link to learn how Julia compares to other languages (C and Python) in terms of performance.
Installing Julia
This is a tutorial-like page. Follow these steps before you continue reading the document.
- Download and install Julia from julialang.org;
- Follow the specific instructions for your operating system: Windows, MacOS, or Linux
- Download and install VSCode and its Julia extension;
The Julia REPL
Starting Julia
There are several ways of opening Julia depending on your operating system and your IDE, but it is usually as simple as launching the Julia app. With VSCode, open a folder (File > Open Folder). Then, press Ctrl+Shift+P to open the command bar, and execute Julia: Start REPL. If this does not work, make sure you have the Julia extension for VSCode installed. Independently of the method you use, opening Julia results in a window with some text ending with:
julia>You have just opened the Julia read-evaluate-print loop, or simply the Julia REPL. Congrats! You will spend most of time using the REPL, when working in Julia. The REPL is a console waiting for user input. Just as in other consoles, the string of text right before the input area (julia> in the case) is called the command prompt or simply the prompt.
Basic usage
The usage of the REPL is as follows:
- You write some input
- press enter
- you get the output
For instance, try this
julia> 1 + 1A "Hello world" example looks like this in Julia
julia> println("Hello, world!")Try to run it in the REPL.
Help mode
Curious about what the function println does? Enter into help mode to look into the documentation. This is done by typing a question mark (?) into the input field:
julia> ?After typing ?, the command prompt changes to help?>. It means we are in help mode. Now, we can type a function name to see its documentation.
help?> printlnPackage and shell modes
The REPL comes with two more modes, namely package and shell modes. To enter package mode type
julia> ]Package mode is used to install and manage packages. We are going to discuss the package mode in greater detail later. To return back to normal mode press the backspace key several times.
To enter shell mode type semicolon (;)
julia> ;The prompt should have changed to shell> indicating that we are in shell mode. Now you can type commands that you would normally do on your system command line. For instance,
shell> lswill display the contents of the current folder in Mac or Linux. Using shell mode in Windows is not straightforward, and thus not recommended for beginners.
Running Julia code
Running more complex code
Real-world Julia programs are not typed in the REPL in practice. They are written in one or more files and included in the REPL. To try this, create a new file called hello.jl, write the code of the "Hello world" example above, and save it. If you are using VSCode, you can create the file using File > New File > Julia File. Once the file is saved with the name hello.jl, execute it as follows
julia> include("hello.jl")Make sure that the file "hello.jl" is located in the current working directory of your Julia session. You can query the current directory with function pwd(). You can change to another directory with function cd() if needed. Also, make sure that the file extension is .jl.
The recommended way of running Julia code is using the REPL as we did. But it is also possible to run code directly from the system command line. To this end, open a terminal and call Julia followed by the path to the file containing the code you want to execute.
$ julia hello.jlThe previous line assumes that you have Julia properly installed in the system and that it's usable from the terminal. In UNIX systems (Linux and Mac), the Julia binary needs to be in one of the directories listed in the PATH environment variable. To check that Julia is properly installed, you can use
$ julia --versionIf this runs without error and you see a version number, you are good to go!
You can also run julia code from the terminal using the -e flag:
$ julia -e 'println("Hello, world!")'In this tutorial, when a code snipped starts with $, it should be run in the terminal. Otherwise, the code is to be run in the Julia REPL.
Avoid calling Julia code from the terminal, use the Julia REPL instead! Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch. This can be time consuming for large projects. In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster. Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code from the terminal. Visit this link (Julia workflow tips) from the official Julia documentation for further information about how to develop Julia code effectivelly.
Running parallel code
Since we are in a parallel computing course, let's run a parallel "Hello world" example in Julia. Open a Julia REPL and write
julia> using Distributed
+Getting started · XM_40017 Getting started
Introduction
The programming of this course will be done using the Julia programming language. Thus, we start by explaining how to get up and running with Julia. After studying this page, you will be able to:
- Use the Julia REPL,
- Run serial and parallel code,
- Install and manage Julia packages.
Why Julia?
Courses related with high-performance computing (HPC) often use languages such as C, C++, or Fortran. We use Julia instead to make the course accessible to a wider set of students, including the ones that have no experience with C/C++ or Fortran, but are willing to learn parallel programming. Julia is a relatively new programming language specifically designed for scientific computing. It combines a high-level syntax close to interpreted languages like Python with the performance of compiled languages like C, C++, or Fortran. Thus, Julia will allow us to write efficient parallel algorithms with a syntax that is convenient in a teaching setting. In addition, Julia provides easy access to different programming models to write distributed algorithms, which will be useful to learn and experiment with them.
Tip You can run the code in this link to learn how Julia compares to other languages (C and Python) in terms of performance.
Installing Julia
This is a tutorial-like page. Follow these steps before you continue reading the document.
- Download and install Julia from julialang.org;
- Follow the specific instructions for your operating system: Windows, MacOS, or Linux
- Download and install VSCode and its Julia extension;
The Julia REPL
Starting Julia
There are several ways of opening Julia depending on your operating system and your IDE, but it is usually as simple as launching the Julia app. With VSCode, open a folder (File > Open Folder). Then, press Ctrl+Shift+P to open the command bar, and execute Julia: Start REPL. If this does not work, make sure you have the Julia extension for VSCode installed. Independently of the method you use, opening Julia results in a window with some text ending with:
julia>
You have just opened the Julia read-evaluate-print loop, or simply the Julia REPL. Congrats! You will spend most of time using the REPL, when working in Julia. The REPL is a console waiting for user input. Just as in other consoles, the string of text right before the input area (julia> in the case) is called the command prompt or simply the prompt.
Basic usage
The usage of the REPL is as follows:
- You write some input
- press enter
- you get the output
For instance, try this
julia> 1 + 1
A "Hello world" example looks like this in Julia
julia> println("Hello, world!")
Try to run it in the REPL.
Help mode
Curious about what the function println does? Enter into help mode to look into the documentation. This is done by typing a question mark (?) into the input field:
julia> ?
After typing ?, the command prompt changes to help?>. It means we are in help mode. Now, we can type a function name to see its documentation.
help?> println
Package and shell modes
The REPL comes with two more modes, namely package and shell modes. To enter package mode type
julia> ]
Package mode is used to install and manage packages. We are going to discuss the package mode in greater detail later. To return back to normal mode press the backspace key several times.
To enter shell mode type semicolon (;)
julia> ;
The prompt should have changed to shell> indicating that we are in shell mode. Now you can type commands that you would normally do on your system command line. For instance,
shell> ls
will display the contents of the current folder in Mac or Linux. Using shell mode in Windows is not straightforward, and thus not recommended for beginners.
Running Julia code
Running more complex code
Real-world Julia programs are not typed in the REPL in practice. They are written in one or more files and included in the REPL. To try this, create a new file called hello.jl, write the code of the "Hello world" example above, and save it. If you are using VSCode, you can create the file using File > New File > Julia File. Once the file is saved with the name hello.jl, execute it as follows
julia> include("hello.jl")
Warning Make sure that the file "hello.jl" is located in the current working directory of your Julia session. You can query the current directory with function pwd(). You can change to another directory with function cd() if needed. Also, make sure that the file extension is .jl.
The recommended way of running Julia code is using the REPL as we did. But it is also possible to run code directly from the system command line. To this end, open a terminal and call Julia followed by the path to the file containing the code you want to execute.
$ julia hello.jl
The previous line assumes that you have Julia properly installed in the system and that it's usable from the terminal. In UNIX systems (Linux and Mac), the Julia binary needs to be in one of the directories listed in the PATH environment variable. To check that Julia is properly installed, you can use
$ julia --version
If this runs without error and you see a version number, you are good to go!
You can also run julia code from the terminal using the -e flag:
$ julia -e 'println("Hello, world!")'
Note In this tutorial, when a code snipped starts with $, it should be run in the terminal. Otherwise, the code is to be run in the Julia REPL.
Tip Avoid calling Julia code from the terminal, use the Julia REPL instead! Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch. This can be time consuming for large projects. In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster. Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code from the terminal. Visit this link (Julia workflow tips) from the official Julia documentation for further information about how to develop Julia code effectivelly.
Running parallel code
Since we are in a parallel computing course, let's run a parallel "Hello world" example in Julia. Open a Julia REPL and write
julia> using Distributed
julia> @everywhere println("Hello, world! I am proc $(myid()) from $(nprocs())")
Here, we are using the Distributed package, which is part of the Julia standard library that provides distributed memory parallel support. The code prints the process id and the number of processes in the current Julia session.
You will probably only see output from 1 process. We need to add more processes to run the example in parallel. This is done with the addprocs function.
julia> addprocs(3)
We have added 3 new processes. Plus the old one, we have 4 processes. Run the code again.
julia> @everywhere println("Hello, world! I am proc $(myid()) from $(nprocs())")
Now, you should see output from 4 processes.
It is possible to specify the number of processes when starting Julia from the terminal with the -p argument (useful, e.g., when running in a cluster). If you launch Julia from the terminal as
$ julia -p 3
and then run
julia> @everywhere println("Hello, world! I am proc $(myid()) from $(nprocs())")
You should get output from 4 processes as before.
Installing packages
One of the most useful features of Julia is its package manager. It allows one to install Julia packages in a straightforward and platform independent way. To illustrate this, let us consider the following parallel "Hello world" example. This example uses the Message Passing Interface (MPI). We will learn more about MPI later in the course.
Copy the following block of code into a new file named "hello_mpi.jl"
# file hello_mpi.jl
using MPI
MPI.Init()
@@ -15,4 +15,4 @@ DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195"
Copy the contents of previous code block into a file called Project.toml and place it in an empty folder named newproject. It is important that the file is named Project.toml. You can create a new folder from the REPL with
julia> mkdir("newproject")
To install all the packages registered in this file you need to activate the folder containing your Project.toml file
(@v1.10) pkg> activate newproject
and then instantiating it
(newproject) pkg> instantiate
The instantiate command will download and install all listed packages and their dependencies in just one click.
Getting help in package mode
You can get help about a particular package operator by writing help in front of it
(@v1.10) pkg> help activate
You can get an overview of all package commands by typing help alone
(@v1.10) pkg> help
Package operations in Julia code
In some situations it is required to use package commands in Julia code, e.g., to automatize installation and deployment of Julia applications. This can be done using the Pkg package. For instance
julia> using Pkg
julia> Pkg.status()
is equivalent to calling status in package mode.
(@v1.10) pkg> status
Creating you own package
In many situations, it is useful to create your own package, for instance, when working with a large code base, when you want to reduce compilation latency using Revise.jl, or if you want to eventually register your package and share it with others.
The simplest way of generating a package (called MyPackage) is as follows. Open Julia, go to package mode, and type
(@v1.10) pkg> generate MyPackage
This will crate a minimal package consisting of a new folder MyPackage with two files:
MyPackage/Project.toml: Project file defining the direct dependencies of your package.MyPackage/src/MyPackage.jl: Main source file of your package. You can split your code in several files if needed, and include them in the package main file using function include.
Tip This approach only generates a very minimal package. To create a more sophisticated package skeleton (including unit testing, code coverage, readme file, licence, etc.) use PkgTemplates.jl or BestieTemplate.jl. The later one is developed in Amsterdam at the Netherlands eScience Center.
You can add dependencies to the package by activating the MyPackage folder in package mode and adding new dependencies as always:
(@v1.10) pkg> activate MyPackage
(MyPackage) pkg> add MPI
This will add MPI to your package dependencies.
Using your own package
To use your package you first need to add it to a package environment of your choice. This is done by changing to package mode and typing develop followed by the path to the folder containing the package. For instance:
(@v1.10) pkg> develop MyPackage
Note You do not need to "develop" your package if you activated the package folder MyPackage.
Now, we can go back to standard Julia mode and use it as any other package:
using MyPackage
-MyPackage.greet()
Here, we just called the example function defined in MyPackage/src/MyPackage.jl.
Conclusion
We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Here, we just called the example function defined in MyPackage/src/MyPackage.jl.
Conclusion
We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
Programming Large-Scale Parallel Systems (XM_40017)
Welcome to the interactive lecture notes of the Programming Large-Scale Parallel Systems course at VU Amsterdam!
What
This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam. We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems. Further information about the course is found in the study guide (click here) and our Canvas page (for registered students).
Material will be added incrementally to the website as the course advances.
This page will eventually contain only a part of the course material. The rest will be available on Canvas. In particular, the material in this public webpage does not fully cover all topics in the final exam.
How to use this page
You have two main ways of studying the notebooks:
- Download the notebooks and run them locally on your computer (recommended). At each notebook page you will find a green box with links to download the notebook.
- You also have the static version of the notebooks displayed in this webpage for quick reference.
How to run the notebooks locally
To run a notebook locally follow these steps:
- Install Julia (if not done already). More information in Getting started.
- Download the notebook.
- Launch Julia. More information in Getting started.
- Execute these commands in the Julia command line:
julia> using Pkg
+Home · XM_40017 Programming Large-Scale Parallel Systems (XM_40017)
Welcome to the interactive lecture notes of the Programming Large-Scale Parallel Systems course at VU Amsterdam!
What
This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam. We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems. Further information about the course is found in the study guide (click here) and our Canvas page (for registered students).
Note Material will be added incrementally to the website as the course advances.
Warning This page will eventually contain only a part of the course material. The rest will be available on Canvas. In particular, the material in this public webpage does not fully cover all topics in the final exam.
How to use this page
You have two main ways of studying the notebooks:
- Download the notebooks and run them locally on your computer (recommended). At each notebook page you will find a green box with links to download the notebook.
- You also have the static version of the notebooks displayed in this webpage for quick reference.
How to run the notebooks locally
To run a notebook locally follow these steps:
- Install Julia (if not done already). More information in Getting started.
- Download the notebook.
- Launch Julia. More information in Getting started.
- Execute these commands in the Julia command line:
julia> using Pkg
julia> Pkg.add("IJulia")
julia> using IJulia
-julia> notebook()
- These commands will open a jupyter in your web browser. Navigate in jupyter to the notebook file you have downloaded and open it.
Authors
This material is created by Francesc Verdugo with the help of Gelieza Kötterheinrich. Part of the notebooks are based on the course slides by Henri Bal.
License
All material on this page that is original to this course may be used under a CC BY 4.0 license.
Acknowledgment
This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project "Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course" funded by the "Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning".
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
- These commands will open a jupyter in your web browser. Navigate in jupyter to the notebook file you have downloaded and open it.
Authors
This material is created by Francesc Verdugo with the help of Gelieza Kötterheinrich. Part of the notebooks are based on the course slides by Henri Bal.
License
All material on this page that is original to this course may be used under a CC BY 4.0 license.
Acknowledgment
This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project "Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course" funded by the "Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning".
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+- · XM_40017
Tip
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
\n",
- "Question: Which of the two loops in the Gauss-Seidel method are trivially parallelizable?\n",
- "
\n",
- "\n",
- "```julia\n",
- "for t in 1:niters\n",
- " for i in 2:(n+1)\n",
- " u[i] = 0.5*(u[i-1]+u[i+1])\n",
- " end\n",
- "end\n",
- "```\n",
- "\n",
- " a) Both of them\n",
- " b) The outer, but not the inner\n",
- " c) None of them\n",
- " d) The inner, but not the outer\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "4edad93f",
- "metadata": {
- "scrolled": true
- },
- "outputs": [],
- "source": [
- "answer = \"x\" # replace x with a, b, c or d\n",
- "gauss_seidel_1_check(answer)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "9df06442",
- "metadata": {},
- "source": [
- "## Parallelization of the Jacobi method\n",
- "\n",
- "Now, let us parallelize the Jacobi method."
- ]
- },
{
"cell_type": "markdown",
"id": "97a2d7d5",
@@ -521,203 +466,16 @@
},
{
"cell_type": "markdown",
- "id": "8ed4129c",
- "metadata": {},
- "source": [
- "## MPI implementation\n",
- "\n",
- "We consider the implementation using MPI. The programming model of MPI is generally better suited for data-parallel algorithms like this one than the task-based model provided by Distributed.jl. In any case, one can also implement it using Distributed.jl, but it requires some extra effort to setup the remote channels right for the communication between neighbor processes.\n",
- " \n",
- "Take a look at the implementation below and try to understand it.\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "e15082fb",
- "metadata": {},
- "outputs": [],
- "source": [
- "] add MPI"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "a66cbf9a",
- "metadata": {},
- "outputs": [],
- "source": [
- "using MPI"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "d39c7bb2",
- "metadata": {
- "code_folding": []
- },
- "outputs": [],
- "source": [
- "code = quote\n",
- " using MPI\n",
- " MPI.Init()\n",
- " function jacobi_mpi(n,niters)\n",
- " comm = MPI.COMM_WORLD\n",
- " nranks = MPI.Comm_size(comm)\n",
- " rank = MPI.Comm_rank(comm)\n",
- " if mod(n,nranks) != 0\n",
- " println(\"n must be a multiple of nranks\")\n",
- " MPI.Abort(comm,1)\n",
- " end\n",
- " load = div(n,nranks)\n",
- " u = zeros(load+2)\n",
- " u[1] = -1\n",
- " u[end] = 1\n",
- " u_new = copy(u)\n",
- " for t in 1:niters\n",
- " # Communication\n",
- " if rank != 0\n",
- " neig_rank = rank-1\n",
- " s = 2\n",
- " r = 1\n",
- " MPI.Sendrecv!(view(u,s:s),view(u,r:r),comm;dest=neig_rank,source=neig_rank)\n",
- " end\n",
- " if rank != (nranks-1)\n",
- " neig_rank = rank+1\n",
- " s = load+1\n",
- " r = load+2\n",
- " MPI.Sendrecv!(view(u,s:s),view(u,r:r),comm;dest=neig_rank,source=neig_rank)\n",
- " end\n",
- " # Local computation\n",
- " for i in 2:(load+1)\n",
- " u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
- " end\n",
- " u, u_new = u_new, u\n",
- " end\n",
- " # Gather the results\n",
- " if rank !=0\n",
- " lb = 2\n",
- " ub = load+1\n",
- " MPI.Send(view(u,lb:ub),comm,dest=0)\n",
- " u_all = zeros(0) # This will nevel be used\n",
- " else\n",
- " u_all = zeros(n+2)\n",
- " # Set boundary\n",
- " u_all[1] = -1\n",
- " u_all[end] = 1\n",
- " # Set data for rank 0\n",
- " lb = 2\n",
- " ub = load+1\n",
- " u_all[lb:ub] = view(u,lb:ub)\n",
- " # Set data for other ranks\n",
- " for other_rank in 1:(nranks-1)\n",
- " lb += load\n",
- " ub += load\n",
- " MPI.Recv!(view(u_all,lb:ub),comm;source=other_rank)\n",
- " end\n",
- " end\n",
- " return u_all\n",
- " end\n",
- " function jacobi(n,niters)\n",
- " u = zeros(n+2)\n",
- " u[1] = -1\n",
- " u[end] = 1\n",
- " u_new = copy(u)\n",
- " for t in 1:niters\n",
- " for i in 2:(n+1)\n",
- " u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
- " end\n",
- " u, u_new = u_new, u\n",
- " end\n",
- " u\n",
- " end\n",
- " function testit(load)\n",
- " comm = MPI.COMM_WORLD\n",
- " nranks = MPI.Comm_size(comm)\n",
- " rank = MPI.Comm_rank(comm)\n",
- " n = load*nranks\n",
- " niters = 100\n",
- " u_par = jacobi_mpi(n,niters)\n",
- " if rank == 0\n",
- " # Compare agains serial\n",
- " u_seq = jacobi(n,niters)\n",
- " if u_par ≈ u_seq\n",
- " println(\"Test passed 🥳\")\n",
- " else\n",
- " println(\"Test failed\")\n",
- " end\n",
- " end\n",
- " end\n",
- " testit(3)\n",
- "end\n",
- "run(`$(mpiexec()) -np 4 julia --project=. -e $code`);"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "eff25246",
- "metadata": {},
- "source": [
- "\n",
- "Question: In function jacobi_mpi, how many messages per iteration are sent from a process away from the boundary (excluding the messages to collect the data on rank 0)?\n",
- "
\n",
- "\n",
- " a) 1\n",
- " b) 2\n",
- " c) 3\n",
- " d) 4\n",
- "\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "98bd9b5e",
- "metadata": {},
- "outputs": [],
- "source": [
- "answer = \"x\" # replace x with a, b, c or d\n",
- "jacobi_2_check(answer)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "c9aa2901",
- "metadata": {},
- "source": [
- "### Latency hiding\n",
- "\n",
- "Can our implementation above be improved? Note that we only need communications to update the values at the boundary of the portion owned by each process. The other values (the one in green in the figure below) can be updated without communications. This provides the opportunity of overlapping the computation of the interior values (green cells in the figure) with the communication of the ghost values. This technique is called latency hiding, since we are hiding communication latency by overlapping it with computation that we need to do anyway.\n",
- "\n",
- "The modification of the implementation above to include latency hiding is leaved as an exercise (see below).\n"
- ]
- },
- {
- "attachments": {
- "fig16.png": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6gAAADyCAYAAABAvOgkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d17nFxlYf/xz9lssrmQm4EQkhACRi7h1kD4YSIqEGuwEi4tSNEakWpqBWuLtEKrgtW2WkDbn1VJ0Cra3w8lKMKGCIJc1BBCxCgghGsSJIEk5ELut93pH89s9uxkZnez7M7z7O7n/XqdV84zc2bmm9nNZL97Lk82e/bsApIkSZIkRVYTO4AkSZIkSWBBlSRJkiQlorZkPD3LsuUxgkiSJEmSepdCoTAeuKdp3KKgZlm2fNasWc9WO5QkSZIkqfeZM2cOhULzZZE8xFeSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCTUxg4gSZKSMxB4b3F9PrA1YpZq619cdtO7/t6SlAQLqiRJKjUCuLW4/mbgxQ4+Tw3wFmAyMAkYXLz9CtItf/8IfBa4H5gWOYsk9ToWVEmS1BW+AXwAGFLmvqtJt6DG8F7gHOA54PrIWSQpKguqJEkqtRG4qri+voPPMZHmcrqy+JzHvsFc1fAH4BHg91V8zZOAWcAvqF5BPRQ4ObccCWTAbcCnq5RBkvZhQZUkSaU2A19+g89xE3Ad8BjwKvAh4Ltv8Dmr4abi0pNNBhZXuO+gagaRpFIWVEmS1BX+X+wA3ci3CBej2rKfjzsSOJFwjvBjHXjdDcCvi4/9IDCmA88hSZ3KaWYkSVKpQ4FCcTkicpZquwR4ALihiq/5CqEkPrOfj5tBuJjVrP183LOEi1+9CXg34Zzgdfv5HJLUJdyDKkmS1OwI4HSgsYqvOZFwfu5a4MEqvN6m4iJJyXEPqiRJUlx/RtgT+vnYQSQpNguqJEmSJCkJFlRJkiRJUhI8B1WSJLXXUGBChfuew/MaJUlvkAVVkiS11zuAOyvcdxZwTxWz9BZHUPkqvacU/zwV+FKFbW4HFnV2KEnqKhZUSZLUXhupPN+me0+7xjjg021sc2JxKWcZFlRJ3YgFVZIktdcvgcmxQ/QyLwFfrnDfKcCZwO+Auyts89uuCCVJXcWCKkmSlK4Xgasq3PcpQkFd1Mo2ktSteBVfSZIkSVIS3IMqSZK6womECyc1OSm3/klgW278dWBLNUJJktJmQZUkSV2htSvLfq5k/D0sqJIkLKiSJGlfW4A5xfWOXp336dxztGVb25tUzRZgLbAhdpAu9h1gbG58RPHP6cC9udsfAP61WqEkyYIqSZJKbQD+6g0+xy+LS3fz78Wlp5sCHFXm9tHFpcna6sSRpMCCKkmSFNePgKXsfxmsJ0xD82IHXvNvgcHt2O6lDjy3JHWYBVWSJCmup4rL/nq2uHREpXlTJSkqp5mRJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJDjNjJJXKBSy+vr6R2PnaM0NN9xw2LJlywYCbN68+bUNGzakNrH5gcBBxfXtwPJ4UcoaAIzPjZ8BGuNEKW/cuHFHUfyl3iWXXPLy5MmTN0eOVFGWZbfOmDHjumq8Vn19/XcLhcKx1Xitjpg7d+7Ihx56aATAzp07t65evTq1OR0HAeNy46djBWnFMbn1l4CtsYJUMI7wPgKsA9ZEzLKPYcOGjRgyZMhIgDFjxuy4+uqrl8XO1Jq6urrTp0+f3uVf47vuuusdDQ0NN3T163TUjh07aq688sqjmsarV69evnPnzu0xM5UxnvD/J4Q5dF+LF6Wsgwg/fwBsA1ZEzLKP/v37Dxw5cuRhTeOvfe1rT2dZFjNSWz5xzjnnPBI7RDVYUJW8z3/+89nJJ588OXaO1qxfv56XXtr7c+8g4LBWNo9tEM3/YaTqpNgBSuW+vjQ0NByV+H9ii6r1QoVC4dgsy5L997l9+/bSf5sjI8Zpj2Tfy6Jj2t4kqtLCH93GjRvZuHEjAP369RuUZdmIyJFaVVNTU5WfDRsaGoan/NmRZVmLz30g2V/EFQ2i5S96UzOI5l+UJ2HHjh2lX+PJKf/fnmXZ0NgZqsVDfCVJkiRJSXAPqrqjm7IseyF2iLzdu3dfQ/EwmzFjxqxYuXLlNyNHKvWXwFuK67uAz0XMUs5k4ILc+IvAlkhZyqqtrf3Snj17AFi/fv0DWZbdEzlSqZmFQmFizACFQuHumpqaB2NmKLV58+aPU9yjNmLEiO3r1q37fORIpd4B/ElufFWsIK34UvPqmIfh4CXxopTz249AY11x8BLwjZhpSo0dO/bSl19++UiAhobCnhdfHDY7dqa8gQN3jRw1atuFkWPszrLss5EztLBz584DgM80jUeNGnXbq6+++uuIkcr5Z6Bfcf154FsRs5TzMZr36u4Aro2WpIzRo0eftmrVqrObxnv27Pmnurq6hpiZShUKhX8D0t2t20UsqOp2CoXCD2bMmHF/7Bx5EyZMuIpiQR03btzKlStXfjlypFLvormg7gZSy/cRWhbU/wJWR8pSVp8+ffYW1GXLli06++yzk3oP582bNwWIWlCzLPtFau/L1KlTz6NYUIcPH75j3bp1SeUjnGudL6j/DhQiZakkV1An/gZ+9v14Ucrp+8FcQV1FYp9vRxxxxJnNBbVhzyc/edr3YmfKu/TSpyaef/6L0Qtqap8dF1988cHkCuopp5xyT319fWoF8DM0F9QVJPa9D8ygZUFNKt/JJ5+8K19Qf/CDH1w/d+7cXTEzlaqvr/9XemFB9RBfSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEmpjB1CXGQBMzI2XAI1tPKYGmJQbPwVs7+RckiRJklSWBbXnOgL4dW48kLbLZr+SxxwPPNnJuSRJkiSpLA/xlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFtecqlIyzdjzmgK4IIkmSJEntYUHtubaUjNtTPg/riiCSJEmS1B4W1J5rPS33ok5ox2Pe1UVZJEmSJKlNFtSeawuwLDc+q43tBwOXdV0cSZIkSWqdBbVnuy+3fjkwpsJ2dcDNwKFdnkiSJEmSKrCg9mxzgMbi+nDgl8AFNJ+PeiBwEfAocD7w22oHlCRJkqQmtbEDqEs9BnwN+GRxfDgwt7jeSMtfUKwilNfnq5au4y6rr68/N3aIvH/4h38Y0LT+/PPPHw78Z8Q45RyZW+9HevmOLRl/EdgWI0gle/bs2bs+YcKEd9XX1w+MGGcfhULh+NgZgLPr6+tHxQ6R981vfnN80/ratWsHkN73/kkl4/+IkqLdfvt2mHBQ7BQtNfbPDcaT2Nd46dKlRzet9+lT0/emmx64ImaeUn377nlT7AxAv/r6+qS+bps2bRp4yy237B0vWLDgz4EUPmfz6nLrR5LY9z7h584myX3+Lly4cFJ+/P73v/+GmTNnNlbaPpL2zMLR41hQe75PFf/8BC0LaX79V8CfA+uqFeqNyLLsT2NnKFVb2/xPae3atYcAfxMvTZv6knY+gI/EDlCqoaFh7/rQoUMnA5PjpUnW1OKSjIEDm3+P8Prrr/cn/e/9xPOtPTEsyRpFYu/hmjVr9q736VPTZ9SorRdFjJOqWhL7uvXr16/FeP369dOAaXHStMuhJPYelqgjsXyvvfZai3FNTc3lkaKohAW152sA/hb4NqGEngQcRLjK7wuEPaoPEvao9gHel3vsH6oZVJIkSVLvZkHtPZ4oLq1poPkQ4GRcc801hXnz5t0ZO0drRo8efXRDQ8MQgA0bNqx65ZVXXo6dqcRoYGxxfQvwVMQs5RwATMyNf03z+dNJOProo0+uqanpA9C3b9/ngA2RI1WUZdnjVXythwinCCRpyJAhYydOnDgaYPv27ZuWLVu2NHamEkOAo3PjR2MFacX/ya0vBTbFClLBUcDQ4vorJPbL1YMPPviQESNGHFpc3wr8PnKkVu3YsWN3NV4ny7JXgJT/b6+ZOHHi3iNlVqxY8dTWrVtL55iPbSLN1xV5mfQ+i8cSfv6A8LmR1OfvAQccMHjcuHHHNI0LhcJiWk7RmJSGhoY1bW/VM2SzZ8/e+4XIsuyoWbNmPRszkCRJkiSpd5gzZ86RhULhmaaxV/GVJEmSJCXBgipJkpS+fm1vIkndnwVVkiQpfZ8lXHX/wNhBJKkrWVAlSZLS9yvgZOBx4MzIWSSpy1hQJUmS0rcQeB04BLiLMH1c36iJJKkLWFAlSZLStwnYU1zvD7wfWAIcES2ROqp/7ABSyiyokiRJ3cP9ufX+hHkwFwJ/ESeOOqA/4XDt2XjhK6ksC6okSVL3cCewNTfOgJHAfxXvGxgjlPbLjYRzic8DDoqcRUqSBVWSJKl7eAjYUub2ocB04GnghKom0v74OPAhoIGw13tl3DhSmiyokiRJ3cNaYFeF+/oB4wiHAV9VtURqrynAV4vrnwbujZhFSpoFVZIkqft4rI373wR8Abi9ClnUPgcDtxJ+iXA78JW4caS01cYOIEmSpHa7A3gPUFdy+zbgeeA24DfA+irnUnm1wA+BscAzwCVAIWYgKXUWVEmSpO7jQWAjYa8chEN++xGuDnslHjqamn8H3glsBs4nTBckqRUe4itJktR9LCdcZAfCFX1/Cnyb8DPd94FD4sRSGRcBf0fYY/phwkWsJLXBgipJktS9PEUoPU8DfwZ8AniCsFf1Zvz5LgWTCb84APg34EcRs0jdih9gkiRJ3cudhL2oFxT/3E7YW7cV+GPgc/GiCRgP1AODgPn49ZD2iwVVkiSpe5kP/F9gRe62p4GPFdc/B5xX7VACYDjh6zOKcLGqi2g+JFtSO1hQJUmSupcXgE+Vuf1/CHNtZoTzUY+rZijRD5gLHAOsBM4FtkRNJHVDFlRJkqSe4++Bu4EDgB8Dw+LG6TUy4FvANMKVev8EeDlqIqmbsqBKkiT1HA3AXwDLgLcA/x9/3quGLwIfJEz7cz7weNw4UvflB5YkSVLPso5wDupW4D3A1+PG6fH+EvhHwpWVZwH3x40jdW8WVEmSpJ7nceASoJFw8aQroqbpud4L3Fhcv4YwzY+kN8CCKkmS1DPdRjgnFeA64MKIWXqiMwnvcS3w38AX4saRegYLqiRJUs/1FeA/CT/zfQ84LW6cHmMKcAfQH/gJ8Fdx40g9hwVVkiSpZ7uCcEXfpjLl9DNvzCnATwlXSp5PmOt0T9REUg9iQZUkSerZGglX9n0YGAHcCxwVNVH3dSrwM2Ao4WJIFxCu3Cupk1hQJUmSer7thAv6/AYYBfwceHPURN3PaYRyOgz4BXAu4X2V1IksqJIkSb3DRmAasAQYAzwAjI8ZqBt5B+Fw3iHAQ4SyvyVqIqmHsqBKkiT1HhsJc6MuBQ4F7sOS2pZzgLuBwcU/34PlVOoyFlRJkqTeZTXwLuB5wmG+vwSOiZooXR8lXGBqAOGqvefhYb1Sl7KgSpIk9T4rgbcDvwPGAguAt0ZNlJ5PA3OAPsB3CRdE2hkzkNQbWFAlSZJ6p1eBM4FFwHDCBYDOiJooDXXAd4AvFcf/DHwYp5KRqsKCKkmS1HutJxzuex/N51heGjVRXAcTpo+5hFBIPwZcEzOQ1NtYUCVJknq3LcDZwA+BfsC3gRsIh7b2JpOAR4GphOL+HmB21ERSL2RBlSRJ0k7gYuAqoBG4AriLMOdnbzAT+BUwDngOeBthr7KkKrOgSpIkCaAAfBl4H7AVmA4sBibHDNXFhgFzgZuBgcCdwCmEaXgkRWBBlSRJUt6PCFf4XQ5MIFzh91NAFjFTVzgd+A3h6ry7gL8jTCPzesRMUq9nQZUkSWr2l4R5Qb8aO0hkSwjnZN5KOC/1emA+cEjMUJ3kAOC/CBdDOhx4gXBI738Q9iJLisiCKkmS1Oww4DTghNhBErARuAj4KLANOAt4Gvg43fdnyLOAx4HLiuMbCUX819ESSWqhu364SJIkqTq+RTgP9RFgKPB1wgWFjo8Zaj+9hXB+6U8Je02XEabX+Wtgc8RckkpYUCVJkpp9CTgIOD92kMQ8TTgM9jLCOZpTCOdv3gSMjZirLQcRDk9+EphBONf0esIe8vsj5pJUgQVVkiSp2TbgNWBT7CAJagS+AUwkXPm2FvgIYVqW64ED40Xbx0jgOsKe0k8RzqO9i7DX9+8Jc79KSpAFVZIkqdkUwhygF8QOkrBVhKlopgAPAP0JJXAFMIe4h/5OKmZYBlwJDAIWEabMORt4Nl40Se1hQZUkSWr2HuAGwrmJat0jwJnAuwklcCDhgkqPE4rrTMI8o13toOLrPkI47PijxSyPAH8CvBX4WRVySOoEtbEDSJIkqVu7t7hMBf4G+FPCHKOnE875vBf4MfAQYUqXN6oGOBY4g3Cu8NuBPsX7dhLmcb2RMF2QpG7GgipJkqTO8HBxGU2YT/YCwsWI3ltcAF4FFhD2dL5QXFYA6wjnuOb1J1w1+AjgSMKVeCcRinB+z2wBWEyYs/VmYG3n/rUkVZMFVZIkSZ1pFfCF4nIUoaieRZiqZhTwZ8Wl1G6aL140mNZ/Tt0MLCRMG/Nj4KXOCC4pPguqJEmSusozwL8UlzpCSZ0KHAO8mbB3dAyQAX2B4SWPbySUz+cJVwt+irAH9nGgoevjS6o2C6okSZKqYSehXC4oub0PMAQYQDisNyNM97MVp/uReh0LqiRJkmJqADYUF0m9nNPMSJIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhKcZkaSJKnZbOAu4PXYQSSpN7KgSpIkNVtZXCRJEXiIryRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBaWbULdx+++3jY2doTX19/YEbNmzoD7Bq1apNixYt2hQ7U4nBwNDi+i5gTcQs5fQDRubGK4FCpCxlnXvuuaNrampqAE466aR1xx133PbYmSoZMGDAprPOOmt9NV7r7rvvPmT79u111XitjliwYMGQF154YQjAli1bdtx7772vxc5Uog44KDd+OVaQVozNra8FdsYKUsGBQP/i+mYSm790ypQpg0eNGjUUYPDgwbvOP//81D5/WzjvvPNWZFnW5Z+/t95664C+ffse3NWv01G7d+/ObrnlljFN40cffXTNypUrd8XMVMZIwv+fEL7vN0fMUs6Q4gLhc2NtxCz7OPzww+v+6I/+aO/n78yZM1P8/N1r2LBhr55xxhk7YueoBguqknfttdfW1NbWLoudozWLFy/miSeeiB1DXeiOO+7Yuz5x4kRqa9P9+Ny9e/fXgcur8Vq7du26s7a2dnI1Xqsjli9fzu233x47hnqxhQsX7l2fMGECF154YcQ0bbvvvvuGUYWS379//3dnWfaTrn6djtqzZ4+fHT3csmXLWLas+cfLSy65hOLvoZO0devWs4B7YueohnS/CpIkSZKkXsWCKkmSJElKQrrHqEmV/UWWZQvb3qx6duzYsYTieRaTJk16bMmSJe+LHKnUzcBpxfVtwPERs5TzPuDfcuNTgaTOFayrq3th585w6t3SpUtvnDJlynWRI5W6sVAo/HHkDNdlWXZj5AwtrFmz5jZgEsD48eNfX758+UmRI5X6KHDV3tFX+Xi8KBX8Hd/Yu/5Obuc87o2YZl9/z/XsYWAYjH4W/vGq1h9QXUceecdnn3323kkAu3fv3p5l2XGxM+U1NjYen8Chtsm9L3/4wx8OBBY1jU899dSrFy1adGvESOU8AU3f+ywAZkbMUs4PgaZTQDZR/CxOxdSpUy99+OGH/6lpvGnTpqOHDx++O2amUoVC4Tl64Q5FC6q6nUKh8MqMGTNejJ0jb8KECY1N6/37998JJJUPyJ9UXyC9fKVldAWwOkaQ9ti6devGs88+O6n3cN68edtiZwA2pPa+TJ06de8FfWpraxtJ73u/5cWszmQNNWldIKyFYWzhXYn928xobB7U7oLLVsYLs6+6uvv3fg8WCoXG1P6N3HnnnSNiZwAKqb0vF1988db8eOTIka+R3udH7nufHaSXL39BteQ+f0eMGLEuP77zzjuXzZ07N6kLYdXX18eOEEWva+SSJEmSpDRZUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJNTGDtDDZcCbgTFAf2AV8BTQ8AafdwIwChgMbAReBFa/weeUJEmSpKhS34M6Hnghtwxux2NqSh5zXFeFK/p47rX+J5fhE8BS4DngQeBu4HFCkfwCMHA/X+dQ4OvAy8Xn/CUwH3gYeAX4DfBhWv+aXp3L+p9tvN4/0fJ9/HAb29+W2/ZDbWwrSZIkSftIfQ9qX+CI3Li9hTr/mLrOi1PW8NzrLSeU6NuAd1fYfgTwGeBdwHRgUzte4zLgesJe2HIyYBLw34QieR6wvsx2T+ayXgj8LVCo8Jzn0/J9PAf4ToVtDwBmAP2K499W2E6SJEmSKkq9oHY3NYS9qO8mHMb7EPAYsBk4DDgXOLC47VsJpXNWG895LXBNbrwV+CmwBHgdGEkouqcW7387cC/wNmBHyXM9BOwhfN0PASYCvy/zmm8iFN6804E+lD88+e00l9O1hD3FkiRJkrRfLKid6zTCe/p74CL2LX9XAj8CziyOLwW+CLxU4flmAJ/LjX9I2Ju6rmS7awh7RL9LOHT4JOBfgE+VbLcJWAxMKY6nlckIcAbNe6t3EvZCDwNOBh4ts/203Pr9VN4rK0mSJEkVpX4OandTC6wkFLxyxW8j8H5gS3Hch1Asy+lLOOc0K45vBS5m33LaZC7wkdz444S9q6V+nlufVub+0ttv3M/t76+wjSRJkiS1yoLa+T5DOMy1ktXAvNx4coXtLiRcGAlCob2ctvdM3kLYQwrhfNWLy2yTL5DvpPxe9KbCuRn4MtBYcnvegcAJufHPy2wjSZIkSW2yoHau3YQ9nW15LLc+vsI25+TW76D10pv349z628vc/zCwvbg+FDil5P5DgSOL678gXCH4d8Xx24ABJdufSfP30XLCVXwlSZIkab9ZUDvXM8C2dmy3Jrc+tMI2+XK5cD8yLM2tH1Pm/p3Ar3Lj0r2i+fF9xT+b9or2B6a2sr17TyVJkiR1mBdJ6lzlpnYpZ1duvV+Z++uA0bnxFbQ9DymEKW7yc8W+qcJ2Pwf+uLg+jXChJnLj/HZNf16Zuz9fRM8ss70kSZIk7TcLaufa00nPM7xkfETZrdo2qMLt+fNQpxCu/Nu057epcL5KmDcV4JeEUt2PlgX2MGBCcb0APNDBnJIkSZJkQU1Un5Lx/VS+em9rtle4/TfABkIRriNMj/MzwryoTXtu89PFbAUeAd5BmGpmePHx+bL6e0KplSRJkqQO6YkFtdwhs91NaRn9GvCTTnz+BuBB4PzieBqhoLZ2uO7PCQW1D3A6cDuefypJkiSpE6V+kaQdJeP+7XhMubk/u5sdtNwbeUKlDd+AcvOhlrtAUqXtMzz/VJIkSVInSr2gbioZH9KOx5zaFUEiyJ/PeU7FrTouXygnEYr96cXxc8BLJds/SpgXFUJBPRYYVRzvAR7qgoySJEmSepHUC+rrwOrc+G3teMxHuyhLtf0ot34y8J5Ofv6lwMrieg3hKr3DiuNye0N3E+ZFBTgamJm779fs+8sESZIkSdovqRdUaDkH6Mdo/bzZD9M8fUp3dzvhwkNN/hs4fD8efxDNhbOS/NV8L8+tlx7e2yRfXC+vcLskSZIkdUh3KKjfz60fB9wMDCnZZiDwOeAmYEuVcnW1RkLhbjoPdxSwGLiUyheC6gOcAcwBVgBvbuM18sVyQO51K00XU2770tslSZIkqUO6w1V8fwIsoPnw3vcDZxdve52wp/CthDk/GwmHnv64+jG7xGLC3+d7hAtEjQC+DXyFsGf5ZUKBHUbYu3oicMB+PH+5YrkEWF9h+yeANbS8ENV2Wu7lliRJkqQO6Q4FtRG4CLgXOKZ42xD2PSdzG/DXwB3Vi1YVc4HlwHcJ85QCDAXOauNxrwAb29jmZeAZ4KjcbZUO74UwL+r9wJ/nblvAvldbliRJkqT91h0KKoSL+ZwCXAF8gJaFagOhxP0H8DRh+pO5Jfd3padzr/dkOx/zUu4xq1vbsGgxcDzwp8AFhMN4S6fT2UK48NEvgHsIe0cb2vHcXwPemRv/qNKGRd8nHErc3u0lSZIkqV26S0EF2Ap8obgMJhzau5F9D0ctAO+rYq4fs/+HFD9cXPZHI3BbcYFwSPMIwtew3PvQXl8vLu01v7hIkiRJUqfqTgU1bzPNc3L2VluLiyRJkiT1CN3hKr6SJEmSpF7AgipJkiRJSoIFVZIkSZKUhO56DmpHfRAY0AnP8yzwYCc8jzogy7Jb6+vrd8XOkXf11VcPaVpfvHjxZGBVxDjlvCm3PpD08pX+u/wd4cJgydi1q/lbbtKkSZdfdtllH4oYZx+FQmF47AzA1fX19Z+IHSLvO9/5zoim9WXLlg0lve/9QS1Gp3JTpBztM58LuIezY8doYU/+PVx5NPS7K16YfT31VDa0ab1fv34D6+vrNL98BAAAAwFJREFUU/se7Bs7ADAgtfdl27ZtNbfccsve8fz5868D/jleorLynx+nkd7n24jcenKfv/Pnz2/x+fuBD3xg+cyZM2PFqaRX7kzsbQX1OuDgTniem7GgxjSi7U2qK8uyvet79uzpBxwSL02bMtLOB53z77RTFQqFves1NTUHAAfES5OswcUlGTU1zf+3NzQ01JD69/4OUvhFQ2W7GcDuTvlFbxdpqIWGA2OnyGtoOeFbd/j8jSG59yX/2QHQ0NAwDBgWJ0271JHYe1giua9xQ+k/zixLKl9v1itbuSRJkiQpPb1tD+pRdE4pT+rw0p7u2muvbZw3b94HY+dozQknnHDs6NGjhwGsXLnyD08++eRLsTOVGAscVlzfDDweMUs5g4ETcuNHgIYK20Yxbdq0t9bW1vYBGDRo0NNZlnV07uEu19jY+Ey1XqtPnz6fLRQKSe2xyhs9evRh06dPHwuwadOmjQsXLvx97EwlhgLH5cYLYgVpxdty608Cr8cKUsGxNO/ZehlYETHLPo455pgx48aNGw8wbNiwzVmWpfb528Lw4cO3VeN1amtrH2tsbEz2//Ysy/pMnz79rU3jJUuWPL5mzZrUpjg8Hmg6xWgF4fs/JeOAQ4vrrxM+P5IxatSooSeeeOLez98syx7OsqzQ2mNiqq2tTfqzozNls2fP3vuFyLLsqFmzZj0bM5AkSZIkqXeYM2fOkYVCYe8v1z3EV5IkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSajNDwqFwvg5c+bEyiJJkiRJ6kUKhcL4/Li25P57CoVC9dJIkiRJklTkIb6SJEmSpCRYUCVJkiRJSfhf0SSRmgN3HHEAAAAASUVORK5CYII="
- }
- },
- "cell_type": "markdown",
- "id": "7d66b1a2",
- "metadata": {},
- "source": [
- "\n",
- "![](\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA6gAAADyCAYAAABAvOgkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d17nFxlYf/xz9lssrmQm4EQkhACRi7h1kD4YSIqEGuwEi4tSNEakWpqBWuLtEKrgtW2WkDbn1VJ0Cra3w8lKMKGCIJc1BBCxCgghGsSJIEk5ELut93pH89s9uxkZnez7M7z7O7n/XqdV84zc2bmm9nNZL97Lk82e/bsApIkSZIkRVYTO4AkSZIkSWBBlSRJkiQlorZkPD3LsuUxgkiSJEmSepdCoTAeuKdp3KKgZlm2fNasWc9WO5QkSZIkqfeZM2cOhULzZZE8xFeSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCTUxg4gSZKSMxB4b3F9PrA1YpZq619cdtO7/t6SlAQLqiRJKjUCuLW4/mbgxQ4+Tw3wFmAyMAkYXLz9CtItf/8IfBa4H5gWOYsk9ToWVEmS1BW+AXwAGFLmvqtJt6DG8F7gHOA54PrIWSQpKguqJEkqtRG4qri+voPPMZHmcrqy+JzHvsFc1fAH4BHg91V8zZOAWcAvqF5BPRQ4ObccCWTAbcCnq5RBkvZhQZUkSaU2A19+g89xE3Ad8BjwKvAh4Ltv8Dmr4abi0pNNBhZXuO+gagaRpFIWVEmS1BX+X+wA3ci3CBej2rKfjzsSOJFwjvBjHXjdDcCvi4/9IDCmA88hSZ3KaWYkSVKpQ4FCcTkicpZquwR4ALihiq/5CqEkPrOfj5tBuJjVrP183LOEi1+9CXg34Zzgdfv5HJLUJdyDKkmS1OwI4HSgsYqvOZFwfu5a4MEqvN6m4iJJyXEPqiRJUlx/RtgT+vnYQSQpNguqJEmSJCkJFlRJkiRJUhI8B1WSJLXXUGBChfuew/MaJUlvkAVVkiS11zuAOyvcdxZwTxWz9BZHUPkqvacU/zwV+FKFbW4HFnV2KEnqKhZUSZLUXhupPN+me0+7xjjg021sc2JxKWcZFlRJ3YgFVZIktdcvgcmxQ/QyLwFfrnDfKcCZwO+Auyts89uuCCVJXcWCKkmSlK4Xgasq3PcpQkFd1Mo2ktSteBVfSZIkSVIS3IMqSZK6womECyc1OSm3/klgW278dWBLNUJJktJmQZUkSV2htSvLfq5k/D0sqJIkLKiSJGlfW4A5xfWOXp336dxztGVb25tUzRZgLbAhdpAu9h1gbG58RPHP6cC9udsfAP61WqEkyYIqSZJKbQD+6g0+xy+LS3fz78Wlp5sCHFXm9tHFpcna6sSRpMCCKkmSFNePgKXsfxmsJ0xD82IHXvNvgcHt2O6lDjy3JHWYBVWSJCmup4rL/nq2uHREpXlTJSkqp5mRJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJDjNjJJXKBSy+vr6R2PnaM0NN9xw2LJlywYCbN68+bUNGzakNrH5gcBBxfXtwPJ4UcoaAIzPjZ8BGuNEKW/cuHFHUfyl3iWXXPLy5MmTN0eOVFGWZbfOmDHjumq8Vn19/XcLhcKx1Xitjpg7d+7Ihx56aATAzp07t65evTq1OR0HAeNy46djBWnFMbn1l4CtsYJUMI7wPgKsA9ZEzLKPYcOGjRgyZMhIgDFjxuy4+uqrl8XO1Jq6urrTp0+f3uVf47vuuusdDQ0NN3T163TUjh07aq688sqjmsarV69evnPnzu0xM5UxnvD/J4Q5dF+LF6Wsgwg/fwBsA1ZEzLKP/v37Dxw5cuRhTeOvfe1rT2dZFjNSWz5xzjnnPBI7RDVYUJW8z3/+89nJJ588OXaO1qxfv56XXtr7c+8g4LBWNo9tEM3/YaTqpNgBSuW+vjQ0NByV+H9ii6r1QoVC4dgsy5L997l9+/bSf5sjI8Zpj2Tfy6Jj2t4kqtLCH93GjRvZuHEjAP369RuUZdmIyJFaVVNTU5WfDRsaGoan/NmRZVmLz30g2V/EFQ2i5S96UzOI5l+UJ2HHjh2lX+PJKf/fnmXZ0NgZqsVDfCVJkiRJSXAPqrqjm7IseyF2iLzdu3dfQ/EwmzFjxqxYuXLlNyNHKvWXwFuK67uAz0XMUs5k4ILc+IvAlkhZyqqtrf3Snj17AFi/fv0DWZbdEzlSqZmFQmFizACFQuHumpqaB2NmKLV58+aPU9yjNmLEiO3r1q37fORIpd4B/ElufFWsIK34UvPqmIfh4CXxopTz249AY11x8BLwjZhpSo0dO/bSl19++UiAhobCnhdfHDY7dqa8gQN3jRw1atuFkWPszrLss5EztLBz584DgM80jUeNGnXbq6+++uuIkcr5Z6Bfcf154FsRs5TzMZr36u4Aro2WpIzRo0eftmrVqrObxnv27Pmnurq6hpiZShUKhX8D0t2t20UsqOp2CoXCD2bMmHF/7Bx5EyZMuIpiQR03btzKlStXfjlypFLvormg7gZSy/cRWhbU/wJWR8pSVp8+ffYW1GXLli06++yzk3oP582bNwWIWlCzLPtFau/L1KlTz6NYUIcPH75j3bp1SeUjnGudL6j/DhQiZakkV1An/gZ+9v14Ucrp+8FcQV1FYp9vRxxxxJnNBbVhzyc/edr3YmfKu/TSpyaef/6L0Qtqap8dF1988cHkCuopp5xyT319fWoF8DM0F9QVJPa9D8ygZUFNKt/JJ5+8K19Qf/CDH1w/d+7cXTEzlaqvr/9XemFB9RBfSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEmpjB1CXGQBMzI2XAI1tPKYGmJQbPwVs7+RckiRJklSWBbXnOgL4dW48kLbLZr+SxxwPPNnJuSRJkiSpLA/xlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFtecqlIyzdjzmgK4IIkmSJEntYUHtubaUjNtTPg/riiCSJEmS1B4W1J5rPS33ok5ox2Pe1UVZJEmSJKlNFtSeawuwLDc+q43tBwOXdV0cSZIkSWqdBbVnuy+3fjkwpsJ2dcDNwKFdnkiSJEmSKrCg9mxzgMbi+nDgl8AFNJ+PeiBwEfAocD7w22oHlCRJkqQmtbEDqEs9BnwN+GRxfDgwt7jeSMtfUKwilNfnq5au4y6rr68/N3aIvH/4h38Y0LT+/PPPHw78Z8Q45RyZW+9HevmOLRl/EdgWI0gle/bs2bs+YcKEd9XX1w+MGGcfhULh+NgZgLPr6+tHxQ6R981vfnN80/ratWsHkN73/kkl4/+IkqLdfvt2mHBQ7BQtNfbPDcaT2Nd46dKlRzet9+lT0/emmx64ImaeUn377nlT7AxAv/r6+qS+bps2bRp4yy237B0vWLDgz4EUPmfz6nLrR5LY9z7h584myX3+Lly4cFJ+/P73v/+GmTNnNlbaPpL2zMLR41hQe75PFf/8BC0LaX79V8CfA+uqFeqNyLLsT2NnKFVb2/xPae3atYcAfxMvTZv6knY+gI/EDlCqoaFh7/rQoUMnA5PjpUnW1OKSjIEDm3+P8Prrr/cn/e/9xPOtPTEsyRpFYu/hmjVr9q736VPTZ9SorRdFjJOqWhL7uvXr16/FeP369dOAaXHStMuhJPYelqgjsXyvvfZai3FNTc3lkaKohAW152sA/hb4NqGEngQcRLjK7wuEPaoPEvao9gHel3vsH6oZVJIkSVLvZkHtPZ4oLq1poPkQ4GRcc801hXnz5t0ZO0drRo8efXRDQ8MQgA0bNqx65ZVXXo6dqcRoYGxxfQvwVMQs5RwATMyNf03z+dNJOProo0+uqanpA9C3b9/ngA2RI1WUZdnjVXythwinCCRpyJAhYydOnDgaYPv27ZuWLVu2NHamEkOAo3PjR2MFacX/ya0vBTbFClLBUcDQ4vorJPbL1YMPPviQESNGHFpc3wr8PnKkVu3YsWN3NV4ny7JXgJT/b6+ZOHHi3iNlVqxY8dTWrVtL55iPbSLN1xV5mfQ+i8cSfv6A8LmR1OfvAQccMHjcuHHHNI0LhcJiWk7RmJSGhoY1bW/VM2SzZ8/e+4XIsuyoWbNmPRszkCRJkiSpd5gzZ86RhULhmaaxV/GVJEmSJCXBgipJkpS+fm1vIkndnwVVkiQpfZ8lXHX/wNhBJKkrWVAlSZLS9yvgZOBx4MzIWSSpy1hQJUmS0rcQeB04BLiLMH1c36iJJKkLWFAlSZLStwnYU1zvD7wfWAIcES2ROqp/7ABSyiyokiRJ3cP9ufX+hHkwFwJ/ESeOOqA/4XDt2XjhK6ksC6okSVL3cCewNTfOgJHAfxXvGxgjlPbLjYRzic8DDoqcRUqSBVWSJKl7eAjYUub2ocB04GnghKom0v74OPAhoIGw13tl3DhSmiyokiRJ3cNaYFeF+/oB4wiHAV9VtURqrynAV4vrnwbujZhFSpoFVZIkqft4rI373wR8Abi9ClnUPgcDtxJ+iXA78JW4caS01cYOIEmSpHa7A3gPUFdy+zbgeeA24DfA+irnUnm1wA+BscAzwCVAIWYgKXUWVEmSpO7jQWAjYa8chEN++xGuDnslHjqamn8H3glsBs4nTBckqRUe4itJktR9LCdcZAfCFX1/Cnyb8DPd94FD4sRSGRcBf0fYY/phwkWsJLXBgipJktS9PEUoPU8DfwZ8AniCsFf1Zvz5LgWTCb84APg34EcRs0jdih9gkiRJ3cudhL2oFxT/3E7YW7cV+GPgc/GiCRgP1AODgPn49ZD2iwVVkiSpe5kP/F9gRe62p4GPFdc/B5xX7VACYDjh6zOKcLGqi2g+JFtSO1hQJUmSupcXgE+Vuf1/CHNtZoTzUY+rZijRD5gLHAOsBM4FtkRNJHVDFlRJkqSe4++Bu4EDgB8Dw+LG6TUy4FvANMKVev8EeDlqIqmbsqBKkiT1HA3AXwDLgLcA/x9/3quGLwIfJEz7cz7weNw4UvflB5YkSVLPso5wDupW4D3A1+PG6fH+EvhHwpWVZwH3x40jdW8WVEmSpJ7nceASoJFw8aQroqbpud4L3Fhcv4YwzY+kN8CCKkmS1DPdRjgnFeA64MKIWXqiMwnvcS3w38AX4saRegYLqiRJUs/1FeA/CT/zfQ84LW6cHmMKcAfQH/gJ8Fdx40g9hwVVkiSpZ7uCcEXfpjLl9DNvzCnATwlXSp5PmOt0T9REUg9iQZUkSerZGglX9n0YGAHcCxwVNVH3dSrwM2Ao4WJIFxCu3Cupk1hQJUmSer7thAv6/AYYBfwceHPURN3PaYRyOgz4BXAu4X2V1IksqJIkSb3DRmAasAQYAzwAjI8ZqBt5B+Fw3iHAQ4SyvyVqIqmHsqBKkiT1HhsJc6MuBQ4F7sOS2pZzgLuBwcU/34PlVOoyFlRJkqTeZTXwLuB5wmG+vwSOiZooXR8lXGBqAOGqvefhYb1Sl7KgSpIk9T4rgbcDvwPGAguAt0ZNlJ5PA3OAPsB3CRdE2hkzkNQbWFAlSZJ6p1eBM4FFwHDCBYDOiJooDXXAd4AvFcf/DHwYp5KRqsKCKkmS1HutJxzuex/N51heGjVRXAcTpo+5hFBIPwZcEzOQ1NtYUCVJknq3LcDZwA+BfsC3gRsIh7b2JpOAR4GphOL+HmB21ERSL2RBlSRJ0k7gYuAqoBG4AriLMOdnbzAT+BUwDngOeBthr7KkKrOgSpIkCaAAfBl4H7AVmA4sBibHDNXFhgFzgZuBgcCdwCmEaXgkRWBBlSRJUt6PCFf4XQ5MIFzh91NAFjFTVzgd+A3h6ry7gL8jTCPzesRMUq9nQZUkSWr2l4R5Qb8aO0hkSwjnZN5KOC/1emA+cEjMUJ3kAOC/CBdDOhx4gXBI738Q9iJLisiCKkmS1Oww4DTghNhBErARuAj4KLANOAt4Gvg43fdnyLOAx4HLiuMbCUX819ESSWqhu364SJIkqTq+RTgP9RFgKPB1wgWFjo8Zaj+9hXB+6U8Je02XEabX+Wtgc8RckkpYUCVJkpp9CTgIOD92kMQ8TTgM9jLCOZpTCOdv3gSMjZirLQcRDk9+EphBONf0esIe8vsj5pJUgQVVkiSp2TbgNWBT7CAJagS+AUwkXPm2FvgIYVqW64ED40Xbx0jgOsKe0k8RzqO9i7DX9+8Jc79KSpAFVZIkqdkUwhygF8QOkrBVhKlopgAPAP0JJXAFMIe4h/5OKmZYBlwJDAIWEabMORt4Nl40Se1hQZUkSWr2HuAGwrmJat0jwJnAuwklcCDhgkqPE4rrTMI8o13toOLrPkI47PijxSyPAH8CvBX4WRVySOoEtbEDSJIkqVu7t7hMBf4G+FPCHKOnE875vBf4MfAQYUqXN6oGOBY4g3Cu8NuBPsX7dhLmcb2RMF2QpG7GgipJkqTO8HBxGU2YT/YCwsWI3ltcAF4FFhD2dL5QXFYA6wjnuOb1J1w1+AjgSMKVeCcRinB+z2wBWEyYs/VmYG3n/rUkVZMFVZIkSZ1pFfCF4nIUoaieRZiqZhTwZ8Wl1G6aL140mNZ/Tt0MLCRMG/Nj4KXOCC4pPguqJEmSusozwL8UlzpCSZ0KHAO8mbB3dAyQAX2B4SWPbySUz+cJVwt+irAH9nGgoevjS6o2C6okSZKqYSehXC4oub0PMAQYQDisNyNM97MVp/uReh0LqiRJkmJqADYUF0m9nNPMSJIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhKcZkaSJKnZbOAu4PXYQSSpN7KgSpIkNVtZXCRJEXiIryRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBaWbULdx+++3jY2doTX19/YEbNmzoD7Bq1apNixYt2hQ7U4nBwNDi+i5gTcQs5fQDRubGK4FCpCxlnXvuuaNrampqAE466aR1xx133PbYmSoZMGDAprPOOmt9NV7r7rvvPmT79u111XitjliwYMGQF154YQjAli1bdtx7772vxc5Uog44KDd+OVaQVozNra8FdsYKUsGBQP/i+mYSm790ypQpg0eNGjUUYPDgwbvOP//81D5/WzjvvPNWZFnW5Z+/t95664C+ffse3NWv01G7d+/ObrnlljFN40cffXTNypUrd8XMVMZIwv+fEL7vN0fMUs6Q4gLhc2NtxCz7OPzww+v+6I/+aO/n78yZM1P8/N1r2LBhr55xxhk7YueoBguqknfttdfW1NbWLoudozWLFy/miSeeiB1DXeiOO+7Yuz5x4kRqa9P9+Ny9e/fXgcur8Vq7du26s7a2dnI1Xqsjli9fzu233x47hnqxhQsX7l2fMGECF154YcQ0bbvvvvuGUYWS379//3dnWfaTrn6djtqzZ4+fHT3csmXLWLas+cfLSy65hOLvoZO0devWs4B7YueohnS/CpIkSZKkXsWCKkmSJElKQrrHqEmV/UWWZQvb3qx6duzYsYTieRaTJk16bMmSJe+LHKnUzcBpxfVtwPERs5TzPuDfcuNTgaTOFayrq3th585w6t3SpUtvnDJlynWRI5W6sVAo/HHkDNdlWXZj5AwtrFmz5jZgEsD48eNfX758+UmRI5X6KHDV3tFX+Xi8KBX8Hd/Yu/5Obuc87o2YZl9/z/XsYWAYjH4W/vGq1h9QXUceecdnn3323kkAu3fv3p5l2XGxM+U1NjYen8Chtsm9L3/4wx8OBBY1jU899dSrFy1adGvESOU8AU3f+ywAZkbMUs4PgaZTQDZR/CxOxdSpUy99+OGH/6lpvGnTpqOHDx++O2amUoVC4Tl64Q5FC6q6nUKh8MqMGTNejJ0jb8KECY1N6/37998JJJUPyJ9UXyC9fKVldAWwOkaQ9ti6devGs88+O6n3cN68edtiZwA2pPa+TJ06de8FfWpraxtJ73u/5cWszmQNNWldIKyFYWzhXYn928xobB7U7oLLVsYLs6+6uvv3fg8WCoXG1P6N3HnnnSNiZwAKqb0vF1988db8eOTIka+R3udH7nufHaSXL39BteQ+f0eMGLEuP77zzjuXzZ07N6kLYdXX18eOEEWva+SSJEmSpDRZUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJNTGDtDDZcCbgTFAf2AV8BTQ8AafdwIwChgMbAReBFa/weeUJEmSpKhS34M6Hnghtwxux2NqSh5zXFeFK/p47rX+J5fhE8BS4DngQeBu4HFCkfwCMHA/X+dQ4OvAy8Xn/CUwH3gYeAX4DfBhWv+aXp3L+p9tvN4/0fJ9/HAb29+W2/ZDbWwrSZIkSftIfQ9qX+CI3Li9hTr/mLrOi1PW8NzrLSeU6NuAd1fYfgTwGeBdwHRgUzte4zLgesJe2HIyYBLw34QieR6wvsx2T+ayXgj8LVCo8Jzn0/J9PAf4ToVtDwBmAP2K499W2E6SJEmSKkq9oHY3NYS9qO8mHMb7EPAYsBk4DDgXOLC47VsJpXNWG895LXBNbrwV+CmwBHgdGEkouqcW7387cC/wNmBHyXM9BOwhfN0PASYCvy/zmm8iFN6804E+lD88+e00l9O1hD3FkiRJkrRfLKid6zTCe/p74CL2LX9XAj8CziyOLwW+CLxU4flmAJ/LjX9I2Ju6rmS7awh7RL9LOHT4JOBfgE+VbLcJWAxMKY6nlckIcAbNe6t3EvZCDwNOBh4ts/203Pr9VN4rK0mSJEkVpX4OandTC6wkFLxyxW8j8H5gS3Hch1Asy+lLOOc0K45vBS5m33LaZC7wkdz444S9q6V+nlufVub+0ttv3M/t76+wjSRJkiS1yoLa+T5DOMy1ktXAvNx4coXtLiRcGAlCob2ctvdM3kLYQwrhfNWLy2yTL5DvpPxe9KbCuRn4MtBYcnvegcAJufHPy2wjSZIkSW2yoHau3YQ9nW15LLc+vsI25+TW76D10pv349z628vc/zCwvbg+FDil5P5DgSOL678gXCH4d8Xx24ABJdufSfP30XLCVXwlSZIkab9ZUDvXM8C2dmy3Jrc+tMI2+XK5cD8yLM2tH1Pm/p3Ar3Lj0r2i+fF9xT+b9or2B6a2sr17TyVJkiR1mBdJ6lzlpnYpZ1duvV+Z++uA0bnxFbQ9DymEKW7yc8W+qcJ2Pwf+uLg+jXChJnLj/HZNf16Zuz9fRM8ss70kSZIk7TcLaufa00nPM7xkfETZrdo2qMLt+fNQpxCu/Nu057epcL5KmDcV4JeEUt2PlgX2MGBCcb0APNDBnJIkSZJkQU1Un5Lx/VS+em9rtle4/TfABkIRriNMj/MzwryoTXtu89PFbAUeAd5BmGpmePHx+bL6e0KplSRJkqQO6YkFtdwhs91NaRn9GvCTTnz+BuBB4PzieBqhoLZ2uO7PCQW1D3A6cDuefypJkiSpE6V+kaQdJeP+7XhMubk/u5sdtNwbeUKlDd+AcvOhlrtAUqXtMzz/VJIkSVInSr2gbioZH9KOx5zaFUEiyJ/PeU7FrTouXygnEYr96cXxc8BLJds/SpgXFUJBPRYYVRzvAR7qgoySJEmSepHUC+rrwOrc+G3teMxHuyhLtf0ot34y8J5Ofv6lwMrieg3hKr3DiuNye0N3E+ZFBTgamJm779fs+8sESZIkSdovqRdUaDkH6Mdo/bzZD9M8fUp3dzvhwkNN/hs4fD8efxDNhbOS/NV8L8+tlx7e2yRfXC+vcLskSZIkdUh3KKjfz60fB9wMDCnZZiDwOeAmYEuVcnW1RkLhbjoPdxSwGLiUyheC6gOcAcwBVgBvbuM18sVyQO51K00XU2770tslSZIkqUO6w1V8fwIsoPnw3vcDZxdve52wp/CthDk/GwmHnv64+jG7xGLC3+d7hAtEjQC+DXyFsGf5ZUKBHUbYu3oicMB+PH+5YrkEWF9h+yeANbS8ENV2Wu7lliRJkqQO6Q4FtRG4CLgXOKZ42xD2PSdzG/DXwB3Vi1YVc4HlwHcJ85QCDAXOauNxrwAb29jmZeAZ4KjcbZUO74UwL+r9wJ/nblvAvldbliRJkqT91h0KKoSL+ZwCXAF8gJaFagOhxP0H8DRh+pO5Jfd3padzr/dkOx/zUu4xq1vbsGgxcDzwp8AFhMN4S6fT2UK48NEvgHsIe0cb2vHcXwPemRv/qNKGRd8nHErc3u0lSZIkqV26S0EF2Ap8obgMJhzau5F9D0ctAO+rYq4fs/+HFD9cXPZHI3BbcYFwSPMIwtew3PvQXl8vLu01v7hIkiRJUqfqTgU1bzPNc3L2VluLiyRJkiT1CN3hKr6SJEmSpF7AgipJkiRJSoIFVZIkSZKUhO56DmpHfRAY0AnP8yzwYCc8jzogy7Jb6+vrd8XOkXf11VcPaVpfvHjxZGBVxDjlvCm3PpD08pX+u/wd4cJgydi1q/lbbtKkSZdfdtllH4oYZx+FQmF47AzA1fX19Z+IHSLvO9/5zoim9WXLlg0lve/9QS1Gp3JTpBztM58LuIezY8doYU/+PVx5NPS7K16YfT31VDa0ab1fv34D6+vrNL98BAAAAwFJREFUU/se7Bs7ADAgtfdl27ZtNbfccsve8fz5868D/jleorLynx+nkd7n24jcenKfv/Pnz2/x+fuBD3xg+cyZM2PFqaRX7kzsbQX1OuDgTniem7GgxjSi7U2qK8uyvet79uzpBxwSL02bMtLOB53z77RTFQqFves1NTUHAAfES5OswcUlGTU1zf+3NzQ01JD69/4OUvhFQ2W7GcDuTvlFbxdpqIWGA2OnyGtoOeFbd/j8jSG59yX/2QHQ0NAwDBgWJ0271JHYe1giua9xQ+k/zixLKl9v1itbuSRJkiQpPb1tD+pRdE4pT+rw0p7u2muvbZw3b94HY+dozQknnHDs6NGjhwGsXLnyD08++eRLsTOVGAscVlzfDDweMUs5g4ETcuNHgIYK20Yxbdq0t9bW1vYBGDRo0NNZlnV07uEu19jY+Ey1XqtPnz6fLRQKSe2xyhs9evRh06dPHwuwadOmjQsXLvx97EwlhgLH5cYLYgVpxdty608Cr8cKUsGxNO/ZehlYETHLPo455pgx48aNGw8wbNiwzVmWpfb528Lw4cO3VeN1amtrH2tsbEz2//Ysy/pMnz79rU3jJUuWPL5mzZrUpjg8Hmg6xWgF4fs/JeOAQ4vrrxM+P5IxatSooSeeeOLez98syx7OsqzQ2mNiqq2tTfqzozNls2fP3vuFyLLsqFmzZj0bM5AkSZIkqXeYM2fOkYVCYe8v1z3EV5IkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSajNDwqFwvg5c+bEyiJJkiRJ6kUKhcL4/Li25P57CoVC9dJIkiRJklTkIb6SJEmSpCRYUCVJkiRJSfhf0SSRmgN3HHEAAAAASUVORK5CYII=\")
\n",
- "
\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "9d4de5a9",
+ "id": "75f735a2",
"metadata": {},
"source": [
"## Extension to 2D\n",
"\n",
"\n",
- "Now, let us study the method for a 2D example."
+ "The Jacobi method studied so far was for a one dimensional Laplace equation. In real-world applications however, one solve equations in multiple dimensions. Typically 2D and 3D. The 2D and 3D cases are conceptually equivalent, but we will discuss the 2D case here for simplicity.\n",
+ "\n",
+ "Now the goal is to find the interior points of a 2D grid given the values at the boundary.\n",
+ "\n"
]
},
{
@@ -727,7 +485,7 @@
}
},
"cell_type": "markdown",
- "id": "f14142ea",
+ "id": "18659ed7",
"metadata": {},
"source": [
"\n",
@@ -737,7 +495,7 @@
},
{
"cell_type": "markdown",
- "id": "b1e1a12b",
+ "id": "ca411d89",
"metadata": {},
"source": [
"For the Laplace equation in 2D, the interior values in the computational grid (represented by a matrix $u$) are computed with this iterative scheme. The entry $(i,j)$ of matrix $u$ at iteration $t+1$ is computed as:\n",
@@ -750,7 +508,7 @@
},
{
"cell_type": "markdown",
- "id": "6f5d2895",
+ "id": "8c916627",
"metadata": {},
"source": [
"### Serial implementation\n",
@@ -761,7 +519,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "4ab59b2f",
+ "id": "1e843565",
"metadata": {},
"outputs": [],
"source": [
@@ -785,19 +543,49 @@
"end"
]
},
+ {
+ "cell_type": "markdown",
+ "id": "f4dc50b1",
+ "metadata": {},
+ "source": [
+ "If you run the function for zero iterations you will see the initial condition."
+ ]
+ },
{
"cell_type": "code",
"execution_count": null,
- "id": "6da0aa54",
+ "id": "4acf219e",
"metadata": {},
"outputs": [],
"source": [
- "u = jacobi_2d(10,0)"
+ "n = 10\n",
+ "niters = 0\n",
+ "u = jacobi_2d(n,niters)"
]
},
{
"cell_type": "markdown",
- "id": "45d786dd",
+ "id": "efd4df6a",
+ "metadata": {},
+ "source": [
+ "If you run the problem for enough iterations, you should converge to the exact solution. In this case, the exact solution is a matrix with all values equal to one."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "5c120b8b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "n = 10\n",
+ "niters = 300\n",
+ "u = jacobi_2d(n,niters)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "85895681",
"metadata": {},
"source": [
"### Where can we exploit parallelism?\n",
@@ -823,7 +611,7 @@
},
{
"cell_type": "markdown",
- "id": "f93e2024",
+ "id": "65e4f745",
"metadata": {},
"source": [
"### Parallelization strategies\n",
@@ -844,7 +632,7 @@
}
},
"cell_type": "markdown",
- "id": "267ecd2a",
+ "id": "11e834b5",
"metadata": {},
"source": [
"
@@ -8387,14 +8128,43 @@ d) 4
-
-
\n",
@@ -854,7 +642,7 @@
},
{
"cell_type": "markdown",
- "id": "e0cee28b",
+ "id": "7384ddfe",
"metadata": {},
"source": [
"Which of the thee alternatives is more efficient? To answer this question we need to quantify how much data is processed and communicated in each case. The following analysis assumes that the grid is of $N$ by $N$ cells and that the number of processes is $P$."
@@ -862,7 +650,7 @@
},
{
"cell_type": "markdown",
- "id": "f6ea16f5",
+ "id": "f733d88f",
"metadata": {},
"source": [
"### 1D block partition\n",
@@ -877,7 +665,7 @@
}
},
"cell_type": "markdown",
- "id": "0cfeca62",
+ "id": "6b983ffd",
"metadata": {},
"source": [
"
\n",
@@ -887,7 +675,7 @@
},
{
"cell_type": "markdown",
- "id": "4600c94a",
+ "id": "4f1e0942",
"metadata": {},
"source": [
"- We update $N^2/P$ items per iteration\n",
@@ -898,7 +686,7 @@
},
{
"cell_type": "markdown",
- "id": "969c42ee",
+ "id": "a9bed431",
"metadata": {},
"source": [
"### 2D block partition"
@@ -911,7 +699,7 @@
}
},
"cell_type": "markdown",
- "id": "e5142fa1",
+ "id": "6f7b5b36",
"metadata": {},
"source": [
"
-
@@ -8379,7 +8109,18 @@ d) 4
-\n",
@@ -921,7 +709,7 @@
},
{
"cell_type": "markdown",
- "id": "091b0310",
+ "id": "abb6520c",
"metadata": {},
"source": [
"- We update $N^2/P$ items per iteration\n",
@@ -932,7 +720,7 @@
},
{
"cell_type": "markdown",
- "id": "a8fb49a3",
+ "id": "9aaf1ca6",
"metadata": {},
"source": [
"### 2D cyclic partition"
@@ -945,7 +733,7 @@
}
},
"cell_type": "markdown",
- "id": "1bc30b74",
+ "id": "77701278",
"metadata": {},
"source": [
"
+\n",
@@ -955,7 +743,7 @@
},
{
"cell_type": "markdown",
- "id": "c45f4e44",
+ "id": "9cd32923",
"metadata": {},
"source": [
"- We update $N^2/P$ items\n",
@@ -966,7 +754,7 @@
},
{
"cell_type": "markdown",
- "id": "3d0693a7",
+ "id": "191fa03b",
"metadata": {},
"source": [
"### Summary\n",
@@ -980,7 +768,7 @@
},
{
"cell_type": "markdown",
- "id": "850b1848",
+ "id": "c0c6d216",
"metadata": {},
"source": [
"### Which partition is the best one?\n",
@@ -995,6 +783,935 @@
"\n"
]
},
+ {
+ "cell_type": "markdown",
+ "id": "8caf9b30",
+ "metadata": {},
+ "source": [
+ "## The Gauss-Seidel method \n",
+ "\n",
+ "Now let us study a slightly more challenging method. The implementation of Jacobi used an auxiliary array `u_new`. The usage of `u_new` seems a bit unnecessary at first sight, right? If we remove it, we get another method called Gauss-Seidel.\n",
+ "\n",
+ "\n",
+ "The following cell contains the sequential implementation of Gauss-Seidel. It is obtained taking the sequential implementation of Jacobi, removing `u_new`, and only using `u`. This method is more efficient in terms of memory consumed (only one vector required instead of two). However it is harder to parallelize."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "efd792c3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "function gauss_seidel(n,niters)\n",
+ " u = zeros(n+2)\n",
+ " u[1] = -1\n",
+ " u[end] = 1\n",
+ " for t in 1:niters\n",
+ " for i in 2:(n+1)\n",
+ " u[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ " end\n",
+ " u\n",
+ "end"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5a38cef6",
+ "metadata": {},
+ "source": [
+ "Note that the final solution is nearly the same as for Jacobi for a large enough number of iterations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c7efa254",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "n = 5\n",
+ "niters = 1000\n",
+ "gauss_seidel(n,niters)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "37b7c288",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "jacobi(n,niters)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "97d9ad59",
+ "metadata": {},
+ "source": [
+ "Jacobi method · XM_40017
@@ -7733,19 +7762,20 @@ a.anchor-link {
\n",
+ "Question: Which of the two loops in the Gauss-Seidel method are trivially parallelizable?\n",
+ "
\n",
+ "\n",
+ "```julia\n",
+ "for t in 1:niters\n",
+ " for i in 2:(n+1)\n",
+ " u[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ "end\n",
+ "```\n",
+ "\n",
+ " a) Both of them\n",
+ " b) The outer, but not the inner\n",
+ " c) None of them\n",
+ " d) The inner, but not the outer\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "0b383598",
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "answer = \"x\" # replace x with a, b, c or d\n",
+ "gauss_seidel_1_check(answer)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6b06f709",
+ "metadata": {},
+ "source": [
+ "If you look into the algorithm closely, you will realize that the steps in loop over `i` are not independent. To update `u[i]` we first need to update `u[i-1]`, which requires to update `u[i-2]` etc. This happens for Gauss-Seidel but not for Jacobi. For Jacobi the updates are written into the temporary array `u_new` instead of updating `u` directly. In other words, in Jacobi, updating `u_new[i]` does not require any entry updated entry of `u_new`. It only requires entries in vector `u`, which is not modified in the loop over `i`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "099634d9",
+ "metadata": {},
+ "source": [
+ "### Backwards Gauss-Seidel\n",
+ "\n",
+ "In addition, the the result of the Gauss-Seidel method depends on the order of the steps in the loop over `i`. This is another symptom that tells you that this loop is hard to parallelize. For instance, if you do the iterations over `i` by reversing the loop order, you get another method called *backward* Gauss-Seidel."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ff03c246",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "function backward_gauss_seidel(n,niters)\n",
+ " u = zeros(n+2)\n",
+ " u[1] = -1\n",
+ " u[end] = 1\n",
+ " for t in 1:niters\n",
+ " for i in (n+1):-1:2\n",
+ " u[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ " end\n",
+ " u\n",
+ "end"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "63c4ce1f",
+ "metadata": {},
+ "source": [
+ "Both Jacobi and *forward* and *backward* Gauss-Seidel converge to the same result, but they lead to slightly different values during the iterations. Check it with the following cells. First, run it with one `niters=1` and then with `niters=100`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "abe31423",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "n = 5\n",
+ "niters = 1\n",
+ "u_forward = gauss_seidel(n,niters)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "23430f70",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "u_backward = backward_gauss_seidel(n,niters)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "0b2b2437",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "u_jacobi = jacobi(n,niters)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0deb5549",
+ "metadata": {},
+ "source": [
+ "### Red-black Gauss-Seidel\n",
+ "\n",
+ "There is another version called *red-black* Gauss-Seidel. This uses a very clever order for the steps in the loop over `i`. It does this loop in two phases. First, one updates the entries with even index, and then the entries with odd index."
+ ]
+ },
+ {
+ "attachments": {
+ "g28304.png": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkUAAAJhCAYAAABPQ8ThAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N15mFxVnf/xd1V1eklI0lkgkARIAgQIIGERBJVFBcddEYHZHEfGCMjMiI6D4PgbZxydkRl1xj2Ny4zjNqKCuCMii+wgCRIggZgg2SBbZ+u9q35/nFup6qZ6r657q/r9ep566lbVrVvfrqru++lzzj03tXz58hySJEkTXDruAiRJkpLAUCRJkgTU9bv96lQqtT6OQiRJkiopl8stAH6Rv90nFKVSqfXLli1bU+miJEmSKq2lpYVcrjC02u4zSZIkDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBBiKJEmSAEORJEkSYCiSJEkCDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBBiKJEmSAEORJEkSYCiSJEkCDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBBiKJEmSAEORJEkSYCiSJEkCDEWSJEmAoUiSJAmAurgLkDQu5gJ/Hi3/AlgRYy2jNQv4q2j5DuC+GGuRNAEYiiQ4AbgyWr4W2B4t/wcwFfgx8KMY6hqLQ4F/i5ZbeWEomgP8c7T8n8AT0fLfA0cADwBfGecah3IghZ/hWgxFksaZoUiCw4Fl0fLHKYSitxN2zJupvlA0lOkUfubvUQhFbwLOBKYQfyiSpIpyTJEkSRKGIkmSJMBQJEmSBDimSBqtOuCcaPlZYDWQIozHOZ8w0DkH/CvwdL/nNkTPfTnhKLEUsIFwhNVtQHaYNbwceC0wjzCY+gngO8DOkf84ZXU8cDDh5/9VdN8RwNuAhUA9sAb4AeF9G4k08ArgVYSfezfwW+CGaHkohwPnRtfzgUnAVuBB4GfAnmFsIxPV8NJoG42EcWjbgMcJn+FQn8Ek4KzoMj/a5ibCd+BXQM8w6pBUZoYiaXQmA7+Mlj8H/DvwLcKOsti36RuK3hate3iJbf4D8DvgHYQd/UAOAv6XEL76uw64HHhq0OrH1zXAnxB27E2EYPh+Qvgr9i/AZ4G/Y3ghYCHwf8CLSzz2ceAiQqgo5ZXApwlHGg5kJ/BBoGWQdU4BvgEcM8g6vYRB+t8a4PE3RLUcUeKxa4AngUuBewZ5DUnjwFAkjd1UQkBaTGgx+C3QRdgBF3dRf5Cw804RWoPuAVYRdqKnAKdHz7mdsBN/cJDXelF0+zlCC8cO4GhCUPo6IYgkwccIoacV+CnhSL4FhBauJuBvgWZCEBzMgcCvCWFyNfAIoSXqjGh7BxGOEDwBeKbE85dEj/USpidYRWgVmkX43E4CZgDLCS15ny2xjUMJrTjTo9urCJ/Rtuj1D41eY3bROv1dCfwX4XuRA+4nBOEu4ERCqD4GuBV4DQOHPEnjwFAkwcOEVgYIXSl5f0XYQT4+xPP/lLCTu5rQAtAd3Z+Kng/wOgqB6FHgj0ts948ILSHTCN1gS4DOfuv8K4VAdAPwl8C+osdfRAhJHxyi5s0UfuZHi+7/MCEo/GGI5w9HBvgAIeS9jRAe8hYCPwGOBf4iWr5hkG39NdAGvAW4qej+OuCTwN8QAuPVwBUlnr+JEMC+1a+OvFMJUxMcTmht+z/g+X7r/C0h7OQILUHfKLGdDKF7rqPEY+cQ5oRKE4LdJbxw/qhzgO8DM6Naj2F4XXqSysCB1lIICDdEl7ai+2+O7ls1xPPrCC0i11EIRBB2nh2E37NPEQLRc8B5lA5aPyd0fQEsInRBFZtDYW6hJ4A/o28gghBwLuKFXVX97aHwMz9XdP9t0X33D/H84UgRQuYFvDCIrAPeTGghAfh/Q2yrjhCebup3fw+hJSof4t48wPO/D3ymRB15DwEXRsuNhKDb3ynR9eOUDkQQWqJuBX5T4rFPEkJTK2FMVKlZxm+n0Go2lxB6JVWIoUgauzbC7NcDOZfQRQOhJal/C0Sx7xAGXUOYSLHYhYQBuhACWBel3U0IN0nweQYedLyG0CIDYXD2iwZYD0LY6x+I8ropDOg+hNB9NRoPURiL1X9sGBRafyYTws1InAacHC1/nsJnXMqPKAxA7/8dkDSODEXS2N3P4Ec+nVO0/LMhtpWlMJbolH6PvWQE2/npEI9XykjqPGOQ9YYaW1Pc3TdriHVnEo4e+xNCy1vxJd/CdliJ5/0uus4P+D5uiNcpdk7R8lDvCRRa6vp/BySNI8cUSWO3fojHjy5a/gx9u9hKWRJd92/xWBBdt9K3y6uUJ4d4vFKGqqP48VJH5OUN1roGfVujGgdY50zC+d7OZeh/CEsNlP4CYZzZDOCt0WUdYQD4bRQGvJdS/B34BNA+xOvn159OmMJgoFZBSWVkKJLGrm2Ix2cULZ89gu02Enbe+XmLpkXXw5mPZ9cIXme85IC9Q6xTXGfzIOv1jrGWPwX+h0K311ZgLSFMFdd4DuFIt1LdY+sJXWCfJxwZliK0Gi0E3hnV+GvCYPX+J68t/g6U6pobTBOGIqkiDEXS2OWGeLx4h5afv2c02863LjSUWrGfgVpLKilFaOXofwRdseI6h2o9Ga2DgC8Sgs6zhMHLvxpg3XsIoWgg6wlHEs4D3kjohjubQpB6FSFYXUIY3J1X/B14J0OHxWJDhW5JZWIoksZfcVdX8WDekcofOTWL0HowWIg4dJSvUW7zCS0yAymuc6gustF6M+FwfQiH6w8UiCAM1B6OjYSg9UVCa95JwFWEFqk6QmvSzRS6Sou/AysI8yxJShgHWkvjr3hm4lePYTv5Wa7rCEczDebMMbxOOQ1VR/Hjg83iPRZHFS2XOlQ+bz6FcVsjkSXMdfVnwI3RfXMIR9TlFX8H/mgUryGpAgxF0vj7MYVWnfcSDukejeIjtd41yHozCJMlJsFfDfJYA2HuIQiDx+8epxqK52warGvsb8rwWg8XLRd/zr+gMBbsSgrjwyQliKFIGn9bCEcuQTjf1beBA4Z4zsvoewg+hMG7+cP1/5jSExWmgS8x+KDlSjqLEAJKuY5Cy8xXGL+xM8VHuL17gHVeT+j+GswyBj59B4Q5pF4TLfcS5mHKayVM4AlhUsbvDbEtCK2BZw2xjqQyckyRVBkfIszDcyZhgO6TwJcJrSM7CDvUQwmnm3gj4fQOy3jhUUyXE7pi6gkzT3+BMKB3e/ScqwhHN91POJdanPLnGfsM4SSuXycExAWEsT2vjdZ7hnCo/Hj5AWFyzenA+whdW98gTKA4jzAD+DsIY7Za6Xv4fLF/Jky++WPCIfiPEo5eayZMzvluCt2B36DvKWMgzHr+MsJg7PMIEzReT+jS20b4ezyfMDfRGwjdb+8H7hzdjy1ppAxFUmV0Ek7W+iXC2JN5wD8Osv5Ah7M/DFxMOC9WE6HLp3+3z63AR4n/ZKI5QuC4lXCusLeXWOcPhPdlONMMjNYOQjfd/xG67P4suhR7nnBetX9l4FAEoUvsIgrnjSvlp5RuHeshhJ3PApcSwtk/DFG75z2TKshQJI1OF9ASLQ93LMw+4M8JZ0n/K8IkgodROCz9WcJ51n4M/JCBTwVxE+GM6tcSumvmEMYsPQb8N+FM74cV1ffEMOsbD78ntHx8gHCakgWELr71hJByHQPPqdRK4WcY6mit3xWtu73E4z8ktJxdQ2ipmUUIqs9Ej32K0IqVP8VG/1YeCIfenxtdjiccqZaff2gjoXXufwhHnQ2kgzAe7LOElsBXEN6TpujxTYTP8WeEz3n9INuSVGap5cuX758HJZVKHb1s2bI1gz1BUtk1MPhcPkOpY2RzH423b1KYj2lSv8dShFA01skYx2oSQ88sPhzlmm16rN8BSaPQ0tKyOJfL5c816EBrKQHGujNMUiAaSo74AxGUJxBB+WaaNhBJCWAokiRJwlAkSZIEGIokSZIAjz6TVH73EwYyJ2HskCQNm6FIUrl9JrpIUlWx+0ySJAlDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEgB1cRcg1Yh6YH60/AzQG2MtkqRRMBRJ5XE08Gi0PBvYXsHXTgOLgWMJv9O9wA8q+PqSVBMMRSrlt8B04HLgljJsrwGYHC3vBbrLsE3Bh4FXAicDU4vub8NQJEkjZihSKQuAGcCUUT7/SOA84FTgFOA4Ct+184Bbx1hfEu2l8HNVKvT9JbCw6HYH0Fih15akmmMo0nh4D/DeuIuosHWEwDdSc4GbgV2EVp+R+DmwGXg4urwL+OgoapAkYShSaTPH+Pws8DiFnfUe4CtjLapG1RNa00YzBumKMtciSROaoUjj4e+A9xfdPiGuQiroSOA70fIrCS0/kqQqYihSKf8INAH/C6waxfNz5S2nKjQRWnzA3ytJqkr+8VYpf0sYaH0/owtFkiRVHWe0liRJwlAkSZIEGIqkStlOGGvV/7IuenzWAI/nKM8EmpKkIRiKJEmScKC1hu8g4JsDPPZl4P8qWEs1OgJIlbj/MGAFsINwWH8pnhZFkirAUKThagReNcBjtXjajnJrHeD+6dF1DthZoVokSSUYijRczzPwaSyermQhkiSNB0ORhqsDW4QkSTXMUCRVr7nAIf1uQziA4pR+6z5ckYokqYoZiqTqdTnwDyXubwQeKrqdBTIVqUiSqpihSOPhZPp2tRXvkG+k79FUfwz8ohJFjbMu4PfRcu8IntcTPW+ggdiD2Vn0moPJjmLbkjThGIo0HuoI504r5YB+tyeNcy2Vsppw2P1IbRjl8wA+FV0kSWVgKFIpJxPGpTw3yuevZPg7+i2jfA1JksrKUKRS1o/x+Z0Mr1tHkqTE8DQfkiRJGIokSZIAQ5EkSRJgKJIkSQKqeKD1F7/4xYPS6fRVcdcxmK9//eunt7W1NQJs3rx59ZYtWzbHXVM/RwHzouVtwKoYayllJnBCtJwF7oyxloGcU7S8ktHNNzSejgdmRcsbSN556uYRvocA7cADMdZSSiNwetHt+wl1JsmLgSnR8tPAxhhrKWURcGi0vAN4NMZaSmkGlhbdvpNwgubEOPHEE1+eTqczAOecc87vjjnmmO1x1zSQVCr1wLJly26Mu47RqtpQlMlkDszlch+Mu47BPPnkk2zfvv+7e0actQzTG+IuYAgvjbuAIVTDZ5x0r4i7gCEk/TNOen0Ar4u7gCEk7j1cuXLl/uUzzjjjJTGWMqRsNns9YZLeqmT3mSRJElXcUlTCb3K5XFfcRRTL5XLnAimAQ5uauqfX1XXEXFIfT+3bN6Uzm00DNDc3s2jRokSdDmLz5s2pzZs3pyC8icdNnbon5pJe4LE9e6bml4866qjc1KlTE9Xs/uijj6Z7enoAmDVpUu8hjY1tMZfUx7Pt7U27enrqABobG1myZEmivoN79+5NrVmzJpW/ffTkyW2TMpmRnMZl3K3as2f/l27+/PkcdNBBiXoPV69end63bx8AUzKZ3MLJk/fGXFIf27u66jd3djbkb5900knZVCo12FMqbsWKFelsNnysHR0d63O5XKLmgUulUscDB8VdRznUUii66LLLLkvUmJ3m5uZuovf4wvnz91w0Y8YzMZfUx0UrVx77bGdnI8AJJ5yQ+8QnPtETd03Frr/++szXvva1DEAmleL6Y49N2ngYznjggZPyy+95z3uyp512WqJ2mK997Wsntba2pgBOnz6988OLFiXqPfy7NWsW3N3aOgPgwAMP5HOf+1yivoMrVqxIX3HFFfv/Tl67aNGmxZMnJyqcv/zBB5f25HIpgAsuuCB7ySWXJOo9vPTSSyc98cQTKYBFTU29LQn7Pf7apk0HtmzYMD9/+zOf+UxPJpOs8ye/4hWvqO/oCP9TP/XUU9/+6le/em3MJfWxfPny7wJvi7uOcrD7TJIkCUORJEkSYCiSJEkCDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBNTW5I2j0QCcCJwSXU4FDose+xjwyZjqkiSpHOqA4+i7nzuScKKAbwB/E19pyTPRQ9EFwLcGeKypkoVIkjQOTgHuG+CxyZUspBrYfQa9wOPA/wLvBTrjLUeSpLJbC3wXuBp4NuZaEmuitxT9FGgGik9Q+PGYapEkqdweA2YCO4vue3tMtSTeRA9Fu+IuQJKkcbQvumgY7D6TJEnCUCRJkgQYiiRJkgBDkSRJEmAokiRJAmr36LMPAVNL3L+KMB+RJElSH7Uait4DHFLi/hsxFI2Lm2++Of3LX/4y/f73v793wYIFubjrqUbbt29PffjDH86cffbZuYsuuqg3lUrFXZIkTSi1Goo+BhxQ4v41lS5korjtttvSDz/8cPr222/PveMd7+iNu55qdMcdd6RWrFiR7urqyl188cW+h5JUYbUaij4fdwETzTnnnJN94IEH0nfccUfaUDQ6v/71r9MAZ511VjbuWiRpInKgtcrirLPOyqXTaVavXp3auHGj/T4jtGfPHlauXGkokqQYGYpUFjNnzsydcMIJOYA77rjD79UI3XHHHemenh4WLVqUc0yWJMXDnRc8ADxUdGmM7r+83/2Xx1JdFTn77LOzYCgajdtvvz0NhfdQksroJvruzxZF97+p3/0fjaW6BKnVMUUjcQqlw+Hc6JL3s8qUU73OPffc7Gc/+9nMqlWrUlu3bk0deOCBtngMQ1tbGw899FAawtisuOuRVHOOB44ocf/s6JL3ZGXKSS5DEVwMDGcMzBPjXUi1mzNnTu7oo4/OPfnkk6m77rordcEFFxiKhuHuu+9Od3V1MW/evNxRRx3leyap3P6a0kdk9/eH8S4k6QxF8L24C6glZ599dvbJJ5/M3HHHHekLLrjAVo9huPPOO+06kzSe7OkYJsd+qKzOPffcLMAjjzyS3rVrV9zlJF53dzf33XefXWeSlACGIpXVYYcdllu4cGGup6eH3/zmN36/hnDvvfem9+3bx+zZs3PHHXecXWeSFCN3Wio7j0IbvuKuM0/rIUnxcqelsst3Az3wwAPpffv2xV1OYvX29nL33Xfnu85sJZKkmBmKVHaLFy/OzZ07N9fV1bV/vIxe6OGHH07v2rWL6dOns3TpUscTSVLM3GFpXORbi+xCG1h+wsazzjorm8lk4i5HkiY8d1gaF/lQlJ+DR31ls1nuuuuuFHiuM0lKilqap+jclpaWnXEXUezqq6/eP3J23b59k34N0+Ksp7/2bHZ/KN6xY0fqnnvuKVtIzuVyNDc309rayte//vXMkiVLRjxmpvjEsjng1zt3Jur96+/xxx9P9fT0DOs9XLduXWr79u2phoYGent7Ked7X6ynp2f/e/h8V1cmae/hjq6uSfnl9vb2cXsfRmv9+vV9bv929+7JGzs7EzUivvgXa/369WX9PS6HvXv37l/e09ubStp3cF17e1Px7XvvvTedTifqLaS3t3f/8qxZsxa1tLS8JsZyXiCbzR5cKweKpJYvX77/dyqVSh29bNmyNXEWNFwtLS3H5XK5x+KuYzDXXnst27dvj7sMSVKNuOKKKzjxxBPjLmNAuVzu+ssuu2xZ3HUMV0tLy+JcLrc6fztZcViSJCkmVdt9ls1mc6lUqifuOgaTCVIA2Wy2N5fLJW3sSIZCMM4BvYOsO1r571gvfVv6hyNFqDGvuywVldekouXh/ozFP9d4f4czFM7tl2V8PuOxSFN4L8brOzhWxX8nexj593i81dH3M07a35k0ff/OJO3vdooXfsaJkk6ni+sbzd/Sikn6fnkoVRuKLrvsssfpu0NSMn0ZuBRYDrwn5lqS4qPAPwA3AhfEXIukhMtmCzn3C1/4QoyV1D67zzTeboyuL8DvW95bo+sfxFqFJKkPd1Iab7cCu4GDgZfEXEsSLAaOJXQF/iTmWiRJRQxFGm+dFHb+b4mzkIR4W3T9KyBRU0hI0kRnKFIl5LuJLqQwIHSiygdDu84kKWEMRaqEnwJtwAJgabylxGoBcDLh6KAfxVuKJKk/Q5EqoQ24JVqeyF1oFxBayu4CtsRciySpH0ORKiXfXTSRD0HPB8IbB11LkhQLQ5Eq5UdAF3AccEzMtcRhDnAGYdI1Q5EkJZChSJXSCvw6Wp6IrUVvIczc/CDwh5hrkSSVYChSJeW70CbiuKJ8ELSVSJISylCkSrqJcN6eU4GFMddSSc3A2dGyoUiSEspQpEp6Hrg7Wn5znIVU2JuAeuB3wOqYa5EkDcBQpErLt5RMpC60fNeZEzZKUoIZilRp3yMcgfVS4JCYa6mEA4DzomW7ziQpwQxFqrQNwEOE794bY66lEl4LNAHrgJUx1yJJGoShSHHIt5hMhEPz892EN8RahSRpSIYixeF70fW5wMw4CxlnDYSWIrDrTJISz1CkODwFrAImAa+PuZbxdD4wDdgI3B9zLZKkIRiKFJfvR9e13IVWfK6zXJyFSJKGZihSXPLdSa8GpsZZyDipA94QLXsoviRVAUOR4rICWAs0An8Ucy3j4RxgNrAduCveUiRJw2EoUpxqeSLH/M90E9ATZyGSpOExFClO+W6l1xNajGpFmsJpTOw6k6QqYShSnO4jTOY4FXhVzLWU00uAucAe4LaYa5EkDZOhSHHKAT+MlmupCy1/RN2PgI44C5EkDZ+hSHHLdy+9iXDEVi0oPhRfklQlDEWK2x3AVmAWcHbMtZTDScAioB34Wcy1SJJGwFCkuPUSupmgNrrQ8l1nPwf2xVmIJGlkDEVKgnwX2lup/u9kPhTZdSZJVabad0CqDbcCu4GDgdNjrmUsFgNLgG7gxzHXIkkaIUORkqAT+Em0XM3nQrswur4N2BlnIZKkkTMUKSny3U0XDrpWsuUDnRM2SlIVMhQpKX4CtAELgKXxljIqhwMnA1ng5phrkSSNgqFISdEG3BItV2MX2gVACvgNsCXmWiRJo2AoUpLku9CqMRTlpxOw60ySqpShSElyM9AFHAccE3MtIzEHOJNw2hIPxZekKmUoUpK0ArdHy9U0keNbgAzwEPCHmGuRJI2SoUhJk+9+qqYuNM91Jkk1wFCkpLmJcOqPU4GFZdzuhcA1wCvKuE2AZuCcaNlQJElVzFCkpHkOuCdaflMZt/sXwMeB15VxmwBvBOqBx4Any7xtSVIFGYqURNXUheaEjZJUIwxFSqLvEY7keilwSJm2+WngHcC3y7Q9gMnAedGyXWeSVOUMRUqiDcDDhO/nG8u0zduA/yEcIVYuryMEo3XAijJuV5IUA0ORkirfHVWuQ/PnAUcDB5Zpe1DoOvteGbcpSYqJoUhJlQ8arwBmlmF7XyIMhP5gGbYF0AC8Nlq260ySaoChSEn1FLAKmAS8PuZaSjkPmAZsBO6PuRZJUhkYipRk5e5CK6fiCRuzcRYiSSoPQ5GSLB+K/giYGmch/WSAN0TLdp1JUo0wFCnJVgBrgUbg1THXUuwcwoDt7cCd8ZYiSSoXQ5GSLt8Sk6SJHPNdZz8EeuIsRJJUPoYiJV0+FL2e0GIUtxSF0484i7Uk1RBDkZLuXsJkjlOBV8ZcC8AZwHxgD/CrmGuRJJWRoUhJlyN0U0EyutDyXWc/BjriLESSVF6GIlWDfBfam4C6OAuhEIrsOpOkGmMoUjW4HdgKzALOirGOpcARhBaiX8RYhyRpHBiKVA16gR9Fy3F2oeVf++eEMUWSpBpiKFK1yHehvYX4vrf5UOSEjZJUg+IenyEN1y+B3cBc4HTCUWkj8R3CZJB3j/L1jwKOA7oJg6wlSTXGUKRq0Qn8FLiE0Fo00lD0zTG+/oXR9W3AjjFuS5KUQHafqZrkj/i6cNC1xoddZ5JU4wxFqiY/AdqAhYQjwSplPnAKkAVuruDrSpIqyFCkatJGGFsEhfmCKuFCwuk97gY2V/B1JUkVZChStcl3oVXy0Pz8azlhoyTVsKodaP2Rj3wkPXPmzAPirmMwd9111+TOzs40wNq1aztWrVrVHXdN/TQCk6LlHpJ32ooM0BQt54C9hIHOXcDxhC6tNeNcw0HAmdHyrYRzsBUrvt1OmFMpSZoI7yOE960zxlpKmUThRL9ZQmtgkqSAKUW39xK+i0kymcJn3Ek4QjJJGuj7d6Y9xlpKyRDew7y9cRUykDe+8Y0HZLPZFMBRRx3VvnDhwp64axrI1KlTu/7yL/8yafuSYavaUDR37txjc7ncY3HXMZjbbruN7du3x11GLXuowq/3uwq/niRx882FoYxXXHEFDQ0NMVYzuM7OzuuBZXHXMVp2n0mSJGEokiRJAqq4+6y/VCr12t7e3kT1VWWz2XuI+vovOfTQ3WcfcECijlz6yNq1C57r6moAWLp0KVdeeWWixiL88Ic/zPzoRz9KA2RSKT53zDGr8499Yt26w9Z3dDS96aCDnv+jWbN2jsfr37Vz57RvbdlyyJz6+s6PHHHE+lLrXP7EE0fnl9/3vvdllyxZkqgxRVddddWkPXvCadrOnD6948/nzn0m5pL6+MKGDXN/t2fPVIA5c+bwsY99LFHfwaeeeir1iU98Yv/fyX9YtOjZeQ0NiRr3dOWTTy7uzeVSABdffHHuvPPOS9R4k4997GN169atSwEc0dTU+3cLFjwdd03Ffr5t24wfbt16UP52S0tLdzqdrPaCK664YlJXVxcAzz777H+fcMIJX4y5pD7S6fTHgVfGXUc51EwoyuVyKy6//PJEhY7m5ub9AzIPaWzsWTp1aqL+mNanUvvrmzp1au64445L1ADSe+65Z389KaD4/Ttv1qwd12/cOG9NW9vkDy5YsHE8Xv9rmzYdDHDOzJk7h/PZHXbYYYl7DzOZTI7w9tE8aVI2ad/BaZnM/h14fX09SXv/uru7U8W3j5o8uXPx5MmJeg+LCzz44IMT9x1sbGzcvzw5k8kl7Tv4yJ49xQPpWbJkSS6TyQy0eiyKQ1pra+vmyy+//IEYy3mB5cuX18ws/8mKw9IwvXr27NYU8OTevQc819U1acgnjFB7b296RdSC8apZs1rLvX1JUvIYilSVBBlQNwAAIABJREFU5jU0dC1samrLAb/avn168WM7enrq8t0Jw9Xa09PnX8Pbdu6c1pXNpmfX13cdP2VK0g4hliSNA0ORqtbLZsxoBbirtXV/KHq+u7vuTx999Jjf7d3bNPAz+3q+u7vurStXHvdMR0d9/r7bd+xoBnhZc7OtRJI0QRiKVLXOnzmzFeB3e/dObe3pyaxpa2u89LHHjm7PZjNNmUx2uNtpTKdzvblc6vInnli8cs+eyV25XOrh3bunAbwieg1JUu2rmYHWmhjae3vT+cBzxOTJnfMbGjo2dHY2vn/NmkXPtrc37untrZtWV9czOZ0ediianE5nG9Lp7M7u7kkfWLPmiEObmjras9nM9Lq67lOnTduXX68jl0s3pFLZEfXLSZKqhqFIVWN3b2/mTY88clxjOp2dXlfXk4XUtu7uSQCP7927/5QvKciNpKWoLpXKpaJTN+zp7a3Lb6sjm81ctHLlsXXpdG53T0/d7p6eum+ccMLjhzc2dpX7Z5Mkxc/uM1WNaZlM7wcXLnymJ5dLPdPR0fRsR0djRzb7gmNnc5AaSUsR9D2sOa8zm01v6OxsXN/e3tSRzabfPnfuJgORJNUuW4oKjiCcYHRhdPsu4J74ylEpr541a9cje/bs+MW2bbNKBaK8kbQUAaRSA3eK1aVSuaOnTNn3rnnznh/JNiUpYeYT9nOLCY0ijwI/i7WihJnooejFwMcJX5IZ/R77MIaiRLp6wYKNT+zdO+Xp9vbJ2RKH3qdgZMfjA+lBznw+a9Kkrk8eddS6ERcqSfFbCHwWOBWY0++xr2Ao6mOid58dCbyKQiB6DhhRC4MqLwX81zHHrG2uqyt5SojBWn0GkhngOdPq6no+ffTRa0fa8iRJCXEQ8DoKgWgn0BFfOck20UPReuCjwJuBQ4GD8ctSFZrr6no/sXjx7w8oOk1E3mi+1JmiU57kHZDJ9Lzv8MP/sLCpqXNURUpS/HYA1wEXExoCZgJrY60owSZ699m90UVV6PgpU9ovnT9/0/UbNsxr6+3dP74oNUhX2EDqU6k+LUGN6XT2lTNn7nj1rFm7ylGrJMXkKeDquIuoFhO9pUhV7pI5c7afNHXq7rqilp6BusIG01B0tFoKcvMaGjo+sGDBpjKVKUmqAoYiVb2PH3XUM3Pq68fUxVU80Lp50qSe/zr22LWlutQkSbXLUKSqV59K5f7rmGPWTslkegGmlBhnNJTpkyb1ADRlMr3/euSRv59VVzfibUiSqpuhSDVhXkND14cXLVrflE73No7iSLHJmUx2UiqVe+fcuZtOnDq1bTxqlCQlm6FINePsGTN2v/7AA7c2j6alqK6u57Tp01v/7JBDto1HbZKk5KvVo8+mUzrwdQN7K1yLKuiqww/f3NbbO+Kw/9eHHrq5Pp12DJEkTWC1GoqeAA4pcf+NwAUVrkUVlAKmjLL7bBzKkSRVkVoNRbuAxhL376t0IZIkqTrUaig6Nu4CJElSdXGgtSRJEoYiSZIkoHa7z0biQsL43Lz8ObSOA95WdP8TwGOVKkqSpDJ5DXBA0e1p0fVC+u7n/gDcX6mikshQBP9H6RazS6JL3r9gKJIkVZ/PAkeUuP8V0SXvmxiKJryHGV43oicHlSRVo8eA1mGst268C0k6QxGcFncBkiSNozfHXUC1cKC1JEkShiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJgNTy5ctz+2+kUkcvW7ZsTZwFDVdLS8txuVyu+LQbq4CeuOop5ZprrnnRjh07UgCz6+t7p6TTXXHXVGxTZ2dDdy6XBpgyZQqHHHJIbqjnVNKOHTvIv38p4LDGxvaYS3qBZzo6mvLL8+bNyzU1NQ22esWtW7cu1dvbC8DUurrszLq6zphL6mNrd3d9W29vBmDSpEkcfvjhifoOtre3s3Hjxv3nRpzf0NCZSaWycdbU3x86Opryb9qBBx7I9OnTE/UebtiwIdXR0QFAYzqdm1Nf3xFzSX3s7ump29nTMyl/+8gjj0zU+wfw+9//PpXNhq/du971rudOPfXULTGX1N/hQDNALpe7/rLLLlsWcz3D1tLSsjiXy63O366lGa2Pi7uA/lKpwnlmt3V1ZbZBsvaYRfbt28fTTz+dGnrNeOToG0CSqHjnmUR7enrSe3p6Evsednd3J/o7CLChs7Mh7hoGs3XrVrZu3ZrY97Ajm00l/fc46d/BSZMmzQHmxF1HrbL7TJIkiSpuKerq6tpSX1//gbjrGMwJJ5xweltbWyPAxo0bV2/cuHFz3DX1cxQwL1reRuiCTJKZwAnRci9wV4y1DOScouWVDO+ki5V0PDArWn4WWBtjLaXMBRZHy23AgzHWUkojcHrR7fuBpHXjvhiYEi0/DWyMsZZSFgKHRcs7gEdjrKWUZmBp0e07CY3TifHiF7/45alUKg3Q0NDwWCqV2h53TQPJ5XIr465hLKo2FF155ZXbgf+Iuw5NeJ+MuwBJte3BBx/cv6974IEH4iyl5tl9JkmShKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJANTFXYBUpTLA0mj5eeDZMWzrYGBetLwK6BjDtsbT4cDsaPm3QC7GWsbTYcCB0fIjQDbGWiRVkC1FUtjRvyq6NBbdf1Z036ISz5kCPBRd/n6Mr/8XRds6fIzbGk8folBn/Si3cTThPT2j6L5pFN7/6WMpsEyupvBzTo65FkkVZCiS4CXAL6PLnKL7vxfd9/Y4iqpRVxLe068U3beYwvt/TBxFSRIYiiRJkgBDkSRJEmAokiRJAjz6TCq3ycDJhLFJe4GHgW1l3P4BhMHYh0XLO4HfAc+NYltp4ETCkW8N0TaeBZ4pS6XJMIXC57Gb8HlsH+W2DgOOi7a5kTAQu3uYz80AC4C5hKMN24H1hKMNR3IU3xzgBGAG0EX4WZ4Dnh7BduYQfo5ZwD5gNbB2BDVINctQJJXHJOCfgSsIR1Pl9QDfAd4HbB3ltqcB7wZeB7yU0r+39wEfBm4dxvZmE46w+kvCjrG/Jwk1/9Moam0EvgG8Nbr9NULtww0PozWbwvv7KeAa4F+Ay4CpRet1A98C3s/ww9GRwBeBVwKpovu3A9cCLYM89wLgQuDVwMwSjz8H/Bfw74TvykBeAlwHvKxfDXlbgF8A7xhkG+cD/0g48q//NlYB/wDcNMjzpZpn95k0dmngB8AHCTvgfGtLlhBg/gy4GzhklNs/irBDPDvaXjvwB/ru1F9C2Cm+a4htLSXMvfN3FALRLuD30TWEI8DeN4o6ZxFCWT4Q/RNwKeMfiPrLADcDHyC06vwhumQJ4fUvgN8ABw1jW0sJrUuvim7vpBBeZgHLgb8e5Pn/BvwxIRD1EMLLBqA3enwO8HHgxwz8T+r5wB3AywlhpjPaxrpoGULr058M8PxU9Bq/AM6MbndEz98ZrXMccCMhNEkTlqFICgGmJbrsLbr/69F9Dw/x/EuA1wM/AY4gdLMsiC7fj9Y5CvhfSv+XPxyPAe8lzPOT70KbTZjX51JCF10a+Gz0WqXMBX4OzI9u/x/wIqA5qruZMCfT/2Pkk1EuIgS/lxJ2/pcCH+GFXTp3E97T7xbdt5XC+//8CF+3lD8ntMz8MKrr8OiyMLoPQvD772Fs67uEAPEOQovdzOj6ckL3FYTgU6oVCELQ/DQh0EwjBONDCd2sLyW8H0T1vr/E81PAFwjzQu0jTA/RHG1jEaFlbn50/x0D1HAVoeUM4FHgjwjhfREh2L0UWBk9/hEKoVaaeJYvX57LX1paWhbHXY9UJaYRdvj5y52EVoj+MoT/0PPrvb7EOlcXPX50icfrGTpMLSXspHPAZwZY59tFr/NvQ2yv1D9MLUXPbyi6/8WEFpAcsIew06202fT9PH5F6ZaXuuix/Hrnl1jn80WP72bguZOuKVpvoBa6hgHuL378gWgbmwjfl2JHFr3GNYzcYYTWpBxhDNSUAdZrJrQW5oCnStQh1aSWlpbFxTnIliKpPN5P6W6iXsJ/6nmXjmLb+bAzmBXAz6LlV5Z4fB7wtmj5ccL4kcEM99QWbwB+TegG2kzo4vv5MJ87nt5H6TE6PfRtkRnq8/hPwhirUv6naPm0AdbpHOD+4sc/HS0fQujGKlYcYjYPsa1SrqQw+/gVhNamUloJY7AgBLEzR/FaUtUzFEljtw54cJDHHyd0W0A4dchou9CKTSWMI1lUdMl3PR3NC3+3X0nhv/+vMPig3uG6jDAOZQrwBGFH+tsybHesVlPoDiplBYWgc/YQ2/rhII9tJozvguGPF2sgdLUVf25tRY8f22/9pykE1PcTugBHIt9q93tCi9RgbilaNhRpQvLoM2nsVgxznRcRdogHM/L/+tPARcCfAqdTOGFpKRnCWKOdRfe9qGj53hG+dikfpzAY+y7gzcCOMmy3HIb7eRxDaOGaxcBHog02PUGOcPTYAsI4r4GcSAiQ+fPoDfbP6Ix+t/cRjgi7ADgeWENombuF0GX7MIVB2/3VU2h5ageW9Xt8Gn27yYrrmjtIjVLNMhRJYzeceYiKBxDPZGShaDphx3hOicfaKHTRNFE4oW3/E7YWH3o/mjmN+ssHop3Am+gbwOI2ms9joFDUMcR2BgokedcAH+WFY3R6COOviB7LT+NQagzSXxE+29cQ/mafF10g/Kw/IbT+3dXveTMpBJ3jCEfKDde0oVeRao+hSKqM4i6zkUzWB/AlCoHoJsKcOY8SgkjxmJVPAH8/wDZG+ppD+V/CUV4zCF1ob6Cwk68GY/k8huuNhBY1CN1gHyccIbaVvu/VSQze7bgTeC2hhfAthMH6Swg/w2zCFAN/AXwO+BsKP0/x3/fVDHx0WinlaE2Uqo6hSBq72cNYp7i7ayStKocQus0gHN5/4SDr9u96KVbcEnIwYYzJWCwjHG5+JWFczi2ElozWMW63HMbz8xiJ90bXrYTD3geabmCwz63Y/dHlg4T6X0Fhcsg04bN4gBBYoe9nvpYwiaakQTjQWhq7k4axzsnR9XbC4evDdQqF39NvDbHu0kEeKx5nU45BtDnCpIXXRbdfQjjUfTiBZLwN5/PIr7OF0Z/2YyinRte/ZPD5lwb73AaylTDP1MWEVrq8S4qW2wljkCB8Pv4TLA3BUCSN3QJC18ZAjiMMkoXQhTGS7priAbyDPW8JhZ1wKbdRmDLgnZRvB3k1YcI/CMHv14SWqDgtZvBgdBKF+aBuH6caUgw8J1D/9f5sjK/1Uwqhq/8A6Z9G1zMpTMkgaQCGIqk8/oOBJ2/8dNHtL49wuxuKlgeaFLGBMIh2sEP9NxMmb4Rw2PfHhnjdkfxt+CfCWKYcIfzdQZhxOU6fZODJGz9ZdHukn8dw5SdjhNB1NtDRae9l8AA3hzD79WBmUZhRe1O/x/6Twszb/0mYg2goCyjPtBFS1TEUSWO3jXCizpvpu9NZAHyPwpFCt1CYYHG4HqBwpNq7CEczNRU9fhqh2+plDN0t9wEKO82/B24gtO4U7wCPIJzm47ER1vnvhO60HKGl5g5GPqdOuWwDziUMAF9UdP/C6L5zo9s/Jrx34yU/x9FcwntdXMscQpD+JIN/bucQztv2KcJn3Njv8dOBH1EIgN/p9/gzFCarPIjwfXovL2zNm0cYu/Y9nNFaE5h9zNLYfZdw/qk3EnYomwjhYC6FwPEk4fxUI9VFOKLouxRO7PlRYCOh9SHfQvBdwiSSVw+yrecJ59j6CeH0DxdGl32EcTUzKbRo7B5FrZ8njGNpIQSQOwlz86wexbbG4huEYPZ64HWEUNn/83iM0I04nj5KmK5gPqGVby1hLFBnVEua8F15NyHYDGQWYVb0qwiH8j9H+F70b0X6JuFn7+9z0XofJwzq/nR02UE4198MwmSg0oRnS5E0OjnCUUs7CaHircC/Eo7Imkv4zztF2Hl9jdCFMtD8QB1F2yo17833CDvXtdHtDCHUzCS0MlxNOBN7W9F2BjpNx2OE7pp/pzCfz5Roe/lAtIKwQ++vePsD+SrhUP3t0XZvIrQ+VVKWMJnkdYRD34s/j07gesIJWrcO8Pzin3Oo8V+7ovVKTUfwHOFz/0nRdg4khKRuQpA9GVhf9Hr950V6EPgvwhQMWcI/svMIoTMfiB4nzGX09kHqvY7QqvgDCrNwzyR87vlA9BzhvXktQ8+/JNWk1PLly/f/EqVSqaOXLVu2ZrAnSBpUIyF0HEw4FHsF5TvkO00Ys7OQsNPaSNhZjnYHliYMAj+UMB5qC6G7ZSRHxyXFbAoh51MUuoyaCJ/HHMLn8QjxTBtwCGFm68mE8LFqFHU0Ez6ruYTJFXcQji57doTbaSTMcH4Q4f3ZTghm6xn+Oe+kmtDS0rI4l8vtb822+0wqrw7Gb+K7LCEEPTrUiiPY3u+iS61qB+6JuwhCF95oTuharDW6jPXz6mDo86BJE5LdZ5IkSRiKJEmSAEORJEkSYCiSJEkCHGgtqTZsw1mYJY2RLUWSJEkYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiQA6uIuQFJFNQJNQA+wJ+ZaJClRbCmSJpb3ATuAn8Xw2hngeOCtwNuAl8RQgyQNyJYiVaulwJej5dOAbIy1aGCzgQ8DpwAnAZOLHvsOcF8cRUlSKYYiVasDCDtagNQot/FiQmvFKcCpwDGE1ox2+u68a8k64FZgVYVe72Dgb/rd10t4nyUpUQxFmsi+DRwRdxEV9u3oMlJvBt4D3AL8+wie1w58H3i46PIT4PRR1CBJ48pQpGp1HzAzWu4d5TZ6gJUUdtanAO8ce2k16VDgVcCGET5vLXBh+cuRpPIzFKla9QA7x7iNE4DuotsfGuP2qsEfE4LfKuC9MdciSYliKFK1mg/8abR8HZAbxTa6h16l5iwktPg0xV2IJCWNoUjVagHwb9HyfzD6LjRJkgDnKZIkSQIMRZIkSYChSFLBSwljs0pdPhOt845B1uk/H5EkVRVDkSRJEg60Vu15NXDpAI+9j5HPszORPMzAk1m+HfhHwkSMfz/AOtvHoyhJqhRDkWrNkYSTjZbykQrWUY06gN8P8Fg+8OwZZB1JqmqGItWaXwAXDfCYrUSSpAEZilRrno4ukiSNiKFI0nhbQt8ZtKdE1zMJ55vLayWcK02SYmEokjTevgWcWOL+86NL3k3AWypSkSSVYCjSRHYVcGXR7eboupG+LRY5wgDuWtBOOJHunhE+ryN6XtsoXnM3wzt5775RbFuSysZQpIlsBrCoxP2pfveP5mSzSfXp6DJS10eX0ThrlM+TpIoyFKlarQBOjZazo9zGl4AflqccSVK1MxSpWu0lTDY4FpuiiyRJnuZDkiQJDEWSJEmAoUiSJAkwFEmSJAFVPNB6+fLls3O53GVx1zGYb37zm6e3t7c3AmzatGn1pk2bNsddUz9HAfOi5W3AqhhrKWUmcEK0nAXujLGWgZxTtLySMCtzkhwPzIqWN5C8U6DMBRZHy23AgzHWUkojcHrR7fsJcz0lyYspzBL+NLAxxlpKWQQcGi3vJPyeJEkzsLTo9p0kbBqOk08++eXpdDoD8LKXveyxY445ZlvcNQ3it5dddtlP4y5itKo2FKVSqTnAR+OuYzCPP/4427fnTy7Oq+KspUacP/QqsfIzHrs3xl3AEJL+GSe9PoC3xV3AEBL3Hv72t7/dv3zaaae9MpVKxVjN4HK53PVA1YYiu88kSZKo4paiEn4LdMddRLFcLncaYXZk5jQ09EzLZDpjLqmPZzo6mrqy2TTA1KlTOfTQQxPVZLx161a2bt2agvAmHjl5cuJOA/FUW1u+24LDDz88N2XKlMFWr7jVq1enent7AWiuq8seWF+fqK6fzZ2djXt7ezMA9fX1HHnkkYn6Dra1tbF+/fr9/5Yf3tjYUZ9O98ZZU39Pt7VNyb9pBx98MDNnzkzUe7hu3bpUe3v42jVlMrn5DQ2jOVXMuNnZ3T1pW3d3ff72kiVLEvX+ATz55JOpbDbMUdvV1bUJeDbeil7gKMJwh6pXS6Ho9e9+97sTNWanubm5m+g9/pPDDtt90YwZz8RcUh8XrVx57LOdnY0AS5cuzV133XWJCpXXX3995qtf/WoGIJNK8fXjj18Td039nfHAAyfll6+66qrs6aefnqgd5mte85pJra2tKYAzm5s7PrxoUaLew79bs2bB3a2tMwDmzJnDV77ylUR9B1esWJG+/PLL9/+d/Ocjj9ywePLkkZ43bly9/MEHl/bkcimAiy++OHvJJZf0xF1TsXe+852TnnjiiRTAkU1NvS1LliTqO/i1TZsObNmwYX7+dktLS3cmk4mzpBc499xz6zs6OgBYvXr1/3z5y1++NuaS+li+fPl3SX636LDYfSZJkoShSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQJqa/LG0cgAxwCnRJdTgYOjxz4FfD6muiRJKocUcASFfdwpwOHRYzcAH4yprkSa6KHoIuBbAzw2o5KFSJI0Dk4D7hvgsdmVLKQa2H0WPAP8APgQ0BVzLZIkldsW4CfAPwObYq4lsSZ6S9EtwIHAtqL7PgTUl15dkqSqsgqYD2wsuu+twNx4ykm2iR6KtsddgCRJ42hvdNEw2H0mSZKEoUiSJAkwFEmSJAGGIkmSJMBQJEmSBNTu0WcfAqaWuH8V8L8VrkWSJFWBWg1F7wEOKXH/jRiKJqxdPT2Z72zZMvuptram/1i8eH3c9VSrn2zb1nxva+u01x944I6XTJ/uob6SakathqLPM3BLkSaoFPCNzZsP6cnlUqvb2rYcPXlyR9w1VaOfb9s286Hdu6cfWF/fbSiSVEtqNRR9LO4ClDzT6up6T5g6dc8ju3dPu3XHjuajJ0/eEndN1WZfb2965Z49UwHOnzWrNe56JKmcHGitCeWsGTNaAe7euXN63LVUo1/t2DG9O5dLH1xf33nslCntcdcjSeVkKNKEct6sWbsyqVRuXXv75Gfa2xvirqfa3BGFyTOjcClJtaRWu89G4hbCcJO8/I7yHcDZRfd/E/jvypSk8TKrrq7n6ClT9j2+d+8Bt+zYMf1d8+Y9H3dN1aIrl0v9dvfuaQCvmDlzV9z1SBq2/6HvCWAXRNevAX5ZdP9twL9WqKZEMhTBKyndYnZEdMm7rzLlaLy9vLm59fG9ew+4e+fOZkPR8N2+c+e0jmw2M2PSpO6Tp07dF3c9kobtpfTdn+XNpW9Yeq4y5SSXoQgupm9L0UCeGO9CVBmvnjWrtWXDhvlr2tqmbOnqmnRwfX133DVVgzt27GgGOLO5uXU4vzCSEuOvgQOGsd4fxruQpDMUwffiLkCVdUhDQ/eRTU37nmpvn/LL7dun//khh2yLu6ak68nlUg/u2jUN4JUzZzqeSKouP4u7gGrhQGtNSC+dMWMXwF2trc1x11IN7m5tPWBPb2/d1Eym58XTptl1JqkmGYo0IZ0XzbHz+N69B2zv6bHFdAi379zZDHD69Om76lKpXNz1SNJ4MBRpQlrU1NR5WGNje28ul7pt+/ZpcdeTZDngvtbW6QDn2nUmqYYZijRhndncvAvgzqgVRKU9tHv3lNaenklNmUzvy2bM2BN3PZI0XgxFmrDyXWgr9uyZtru3NxN3PUn1q+ios1OnTdtdb9eZpBpmKNKEtWTKlPa5DQ2dPblc6vYdO+xCG8C9UdfZOc5iLanGGYo0ob2kubkVCqevUF8r9+yZ/HxXV0N9Op09Z8aM3XHXI0njqZaOujn9S1/60o64iyh2zTXX7J/jbkN7e9196fRwJs+qmM5cbn99u3btSj3yyCOJmpNvy5Yt++vJAfft2lX2929ufX0XwEO7d0+/c+fOqfXp9Ki7h9auXZuqr69P1HvY09Ozv54d3d2Zkb6HNz3//CyAIyZPbnt0797J5a5vV2/v/r9BnZ2dJO07uHbt2j63V+3b17ijuztRXYjFxWzYsCFxv8dtbW37l/f29qbG4/d4LDZ2dvY5B+IjjzySzmQyifqMs9ns/uUZM2Yc+qUvfemsGMspZXbcBZRLavny5fs//FQqdfSyZcvWxFnQcLW0tByXy+Uei7uOwVx77bVs37497jIkSTXiiiuu4MQTT4y7jAHlcrnrL7vssmVx1zFcLS0ti3O53Or8bbvPJEmSMBRJkiQBVTym6F3vetfjN9xwQ8PQa8bnrW99a117e3sK4N577+15+umns0M9p8Iy0QUgC/TGWEspKQrf0RwwXiduzQDPAAcBrwVuHcFz64uWe+g7xCMJMhT++ellZJ/x/cBJwOXAV8pcV16avp9xzzi9zlhMKlruiq2KgdXR9zNO2t+ZNH3/ziTtM07R9zNO3Amizz///EnTp09PARxzzDHdM2bMSNpQxwu4AAARf0lEQVTfmWJJ24+MSNWGolSYLyWJf6CKJb2+xP3yl9BZgdfoBm4G/gp4AyM7eWI1vIejsQBYStiJ3cT4/pyV+IzHwt/j2pfoz/iWW27ZX98NN9wQZyk1z+4zKfhBdH0Bhf9qJ7ILCP9B3wVsibkWSaoIQ5EU3ArsBOYAZ8RcSxJcEF3/YNC1JKmGGIqkoBv4abT8ljgLSYB8MMwRus4kaUIwFEkFxV1oiZoAr8IuIPxteBD4Q8y1SFLFGIqkgp8D+wiDjE+Kt5RY5VvK7DqTNKEYiqSCNuAX0fIFg61Yw5qBs6Nlu84kTSiGIqmvG6Prt8VaRXzeTJh76XfA6iHWlaSaYiiS+voRYc6SxcCxMdcSB7vOJE1YhiKpr13Ar6LlidaFdgBwXrRsKJI04RiKpBfKd6FNtFD0WqAJWAc8GnMtklRxhiLphW4knJ/pZGBRzLVUUj4Eeh4BSROSoUh6oW3A3dHym+MspIIagNdEyzcOtqIk1SpDkVRa8USOE8H5wDRgI3B/zLVIUiwMRVJp3yec5uIMYG7MtVRC8bnOcnEWIklxMRRJpW0knOYiDbwp5lrGWx3w+mjZrjNJE5ahSBpYvgut1k8Qew4wG9gO3BVvKZIUH0ORNLDvR9fnEkJDrcqHvpsIR91J0oRkKJIG9jThdBfF3Uu1Jk3hCDsnbJQ0oRmKpMHVehdafiD5HuC2mGuRpFgZiqTB5UPRqwmHrNeafNj7EdARZyGSFDdDkTS4R4E19J3csJZ4AlhJihiKpKHdFF3XWhfaSYTTmLQDP4+5FkmKnaFIGlq+FeV1QGOchZRZfsLGnwP74ixEkpLAUCQN7QHgWeAA4LyYaymnfChywkZJwlAkDUeOQhdarZwLbTGwBOgGfhxzLZKUCIYiaXjyrSlvBibFWUiZXBhd/wrYGWchkpQUhiJpeO4EngeagbNjrqUc7DqTpH4MRdLw9AI3R8vV3oV2OHAykKXwM0nShGcokoYv36pyAdX9u3MBkAJ+A2yJuRbp/7d3t7FxVfkdx793ZvwU25PEBCUkPO8mKaSBjXgoYUVIRLdturQhu2SD1CK1QFAQSYUqpKLICazEm6LQGqFWSUpV+qIVJd3CtstGS5UIUliUdhOw4ogmLFg8GJc8b9ZxHjwzty9mxh47Tsg4Y9879vcjXfmMM77+6+aO/fM5Z86RYqOaf7BLY+0/gePAdPLbY1SrYk+XCzZKUglDkXTx+oCfFtrVupBjMdCFOJ9IkgYxFEnlKfaufJ/8EFS1WQ4kgV8An0VciyTFiqFIKs828qs/X0t+m4xq415nknQehiKpPL3Am4V2tQ2hTQEWF9oOnUnSEIYiqXzFXpZqe2v+MqAW6AD2R1yLJMWOoUgq338AZ8lvk3FDxLWUw6EzSboAQ5FUvl8BOwrtauktagJ+p9A2FEnSMAxF0sgUg0Wl5xX9WeGYVeHzLgUagE6gvcLnlqRxwVAkjcxrQAa4Bbi+gud9oXB8s4LnhIEerX+t8HkladwwFEkjcxh4t9C+L8pCLkId8PuFtkNnknQehiJp5Ipva6/kENrjheOjCp7zO0Aa6AJ2VfC8kjSuGIqkkfsR+e0y7gSuqNA5/7ZwfFmh88HA0Nlr5OuVJA3DUCSN3BfA/5B/HS2r0DmvLBy1FTpfEviDQtsFGyXpAgxF0qUpBo1KvTX/88KxsELnWwxMA44AOyt0TkkalwxF0qX5UeHjYqAlwjrOpzjf6XXy75aTJJ2HoUi6NB+R3zajhoFhqrgIGBjWc+hMkr6GoUi6dMXeoritbr2Q/PykXwPbI65FkmLPUCRdumIvzO8CzVEWMkRx6OwnwOkoC5GkamAoki5dO3CA/CKJSyOupZQbwEpSGQxFUmX8uPCx0nuhjdS3gG8Ap4BtEdciSVXBUCRVRrE35l6gPspCCorzm34GnIyyEEmqFoYiqTJ2kV9fqIn8thpRK4Yih84k6SIZiqTKCInPENocYB7QR36StSTpIhiKpMop9sr8IZCKsI77Cx93AMcirEOSqoqhSKqcncAh4DLg7gjrKPZUuWCjJJXBUCRVThb490I7qoUcrwRuAXIltUiSLoKhSKqs4hDa9xjZ6+uvC8fnI/z+95Pf3uMdoHuE55CkCSnKeQ/SeLQd+BUwA7gD+HmZX//nl/j9iz1UDp1JUpnsKZIq6wzwRqE91kNo04E7C+0fX+iJkqRzGYqkyiv20nyf/FDWWLkPSAK/ADrH8PtK0rhgKJIq76fkV5G+lvx2G2PFBRsl6RIYiqTK6wXeLLTHaghtCrC40HY+kSSNQFVPtH7mmWdiH+r27dsXAGzdujUXdS3DCBgY3gmjLOQCSuuLY42l92Bpfa+TXy/oe8DTY1DHMqAW2AccGFJXtVzDONYGg4dAfR2PTLXcgxDD+lasWBEcOnQoAFi8eHEc78F+Tz/9dBgEQeyu4cWq2lC0ZcuWeWEYdkRdx4WsW7eOI0eORF2GonUj+fWLxsq8Mf5+kkbZ1q1b+9s33ngjN998c4TVXNjmzZv/Dng06jpGKvY9LZIkSWPBUCRJkkQVD58NFYbhimQyeTTqOkrlcrk3yb9FmvtnzjxxR3PzVxGXNMhznZ3XHDx7thbgpptu4pFHHslEXVOpbdu2JbZt25YASAYBfzlnzkdR1zTUk/v3zy62H3/88dzcuXP7x/t7enpobW1N5XI5NmzYkJ02bdqojLO//PLLyT179gT33HNPbtmyZYPmG6xbty7V09MDwO3p9JkfXHHFZ6NRw0j9Q1fXFft6epoALr/8ctavXx+re/CTTz6hra2t/+fkk9dc0zWjvr43ypqG+osDB76ZDcMAYPny5eGSJUtiNXy6cePG5GeffRYAXNfQkH386qs/ibqmUtuPHp267dChacXHbW1tmUQiXv0FTz75ZOrs2bMAdHV1/fOCBQv+PuKSBsnlchuIdr/Hihk3oSgIgndXrVoVq20NpkyZ0v9L8KrGxsy3J0/uibKeoeqCoP8X6OTJk8PbbrstVhP4Pvjgg/4JrgEQt+s31OzZs8+5hvPnzw/b29uDY8eOsXTp0opf376+Pvbv358CWLlyZXb+/PmDglcqlQopTHKdVlubjds1fO2rr/qK7fr6euJ2D9bU1Az67Ti/ufnUnEmTYnUNS2eBX3311bF7HTc2NiaL7aZkMozbPXjg5MmG0se33nprLplMnu/pkSgNaUePHv101apVOyIs5xybN29eHXUNlRKvOCyNM8V3irz99tuj8lrbtWtX4uTJk7S0tITz5s2r2nd8SFIcGIqkUbRkyZJcEAR0dHQEhw8fHrS6dXFY62L19fXR19c36HPFsLV48eJc3Lr8Jana+FNUGkXTp08P586dG+ZyOXbu3Nkfinbv3h088MADNeWc64033kisXbs2lcvlR0dyuRzvvPNOAuDuu++2l0iSLpGhSBplxSG0t956KwGwffv2RGtra00Ylpdj6urq2LdvX2Lt2rWp3t5edu/enTh+/DjpdJpbbrklVvNIJKkajZuJ1lJcZLNZgiDonxy5ZMmS3KZNm5J79uxJrF69OtXZ2Zk4ceIEM2bMKOu89fX1pFIp9u7dm1i5cmXNrFmzQoBFixYNmhiayWRIpXxpS1K5/MkpVdi7776b2LBhQ2rmzJlhOp2mt7eXZDJJNpulvb29v3e23DlA9fX1JJNJTp8+zeHDh4MjR44EAO+9915i1apVNdlslkOHDtHY2Mgrr7zS93XnkyQNZiiSKmzRokW522+/Pbdr165EZ2fneZ9XW1tb1nkbGhrC0h6h4vDbkSNHKAakdDrN888/byCSpBFwTtGAGcB3ye/Z8iiwINpyVM2effbZzPTp0y84aai+vr6sc9bX11+wd6m5uZnVq1dn5syZ46RrScOZCnyHgd9z42LBxUqa6KFoAfndzD8HuoGfAJsLx3cjrEtVrra2lra2tszUqVPPG1BGEoqCIBj232pra1m4cGFu+fLlTriWVOoq4F+Aj4GjwJsM/J57MMK6Ymmih6LfAJYBVxYe/xrwr2xVxMyZM8PW1tZsc3PzsP8+adKksu61hoYGgiA452uCICh+r1htkSEpFmYCPwCuLzw+BZyNrpx4m+ih6Avgr4A/Ih+QppC/YaSKuPPOO3P33ntvrrGx8ZwwM2nSpLLO1dDQMGyImjp1atjW1papqSlr2SNJE8Mx4G+Ah4CbgTQQu30k42KiT7T+r8IhjZo1a9ZkOjo6aj788EMymYHOnMbGxrLO09DQQFjY+LMonU6Hra2t2a+bvyRpwjoArIm6iGox0XuKpFGXSCTYuHFjX0tLy6DgMlzv0YXU1NRQXM0a8j1NK1asyC1cuNB5RJJUAYYiaQyk02k2btyYaWpq6v9cuT1FQH9PUxAEzJ07N/fwww9nK1akJE1whiJpjMyePTtcs2ZNpvius3Q6XfaQV3FydktLS/jcc89lzvduNElS+QxF0hhatmxZ7q677sqlUikaGhpGdI7Jkyfz4osvDup1kiRdOkORNMbWr1+fmTVrVljuu88gvynsE088kbnuuuucWC1JFTbR330mjbmamhpeeumlvrq6urK/dtOmTZnLLrvMQCRJo2C89hR9SX4RxqHHv0VZlFTU1NTESNYVMhBJ0ugZr6FIkiSpLON1+Gxm1AVIkqTqYk+RJEkShiJJkiRg/A6fleMeoHQFvGTh4zeA3y75fCfw8VgVJUlShXwbKF0YrbjI2SwG/577P6BjrIqKI0MRvMnwPWZ/UjiKngXWj0E9kiRV0j+S/0N/qN8rHEX/BPzxmFQUU4Yi2M3FDSN+OdqFSJI0CjqA4xfxvM7RLiTuDEVwe9QFSJI0iu6LuoBq4URrSZIkDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBBiKJEmSAEORJEkSMI4WbwyC4PUtW7b0RV1Hqaeeeqq4jxqvfP55eltX1+wo6xnqYF9fbbHd3t4ePProozVR1jPUwYMH+9vZMORPOzpidf2GeuGFFxJNTU2x+kOjp6enf1+/nx8/Xh+3a9h15kx9sX3w4EHidg/29vYOevzDjz+eVRsE2YjKGVY2DPv/j1999dXEjh07YnUNP/300/76Pj51Khm3e/BoJjPoej322GOxun4AZ8+e7W/PmTPnwQcffHBRhOWcIwzDuVHXUCnjJhSFYRi7lamDYGCf2e7Tp1PdA5vwxc6JEyfYu3dv8PXPjEYI/G9vb2yvH0BnZ2dsrx/A8UwmcTyTie01PHPmTKzvQYBPTp1q+PpnRae7u5vu7u7YXsPebDaI++s47vdgXV3dlWEYXhl1HeNVrP6qlSRJikrV9hSlUqmDfX19P4y6jgu54YYbfuv06dP1AF1dXfu7u7u7o65piNnArEL7MLAvwlqG0wLML7RzwM4IazmfxSXtdi5u08Wx9JvAZYX2F8AvI6xlOLPI34cAp4D/jrCW4dQBd5Q83kW+zji5DWgstH8JdEVYy3CuB64qtI+Rf53EyRTgWyWP346qkPNZsGDBXUEQJAHq6ur2AkciLulCdkddwKWo2lD00EMPHQKeiboOTXixDuaSqt/777/f396zZ0+ElYx/Dp9JkiRhKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCYBU6YMwDK/dsmVLVLVIkiSNmTAMry19nBry7z8Lw3DsqpEkSYoJh88kSZIwFEmSJAHw/8Ve17voNSEvAAAAAElFTkSuQmCC"
+ }
+ },
+ "cell_type": "markdown",
+ "id": "dda0a371",
+ "metadata": {},
+ "source": [
+ "\n",
+ "![](\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAkUAAAJhCAYAAABPQ8ThAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N15mFxVnf/xd1V1eklI0lkgkARIAgQIIGERBJVFBcddEYHZHEfGCMjMiI6D4PgbZxydkRl1xj2Ny4zjNqKCuCMii+wgCRIggZgg2SBbZ+u9q35/nFup6qZ6r657q/r9ep566lbVrVvfrqru++lzzj03tXz58hySJEkTXDruAiRJkpLAUCRJkgTU9bv96lQqtT6OQiRJkiopl8stAH6Rv90nFKVSqfXLli1bU+miJEmSKq2lpYVcrjC02u4zSZIkDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBBiKJEmSAEORJEkSYCiSJEkCDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBBiKJEmSAEORJEkSYCiSJEkCDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBBiKJEmSAEORJEkSYCiSJEkCDEWSJEmAoUiSJAmAurgLkDQu5gJ/Hi3/AlgRYy2jNQv4q2j5DuC+GGuRNAEYiiQ4AbgyWr4W2B4t/wcwFfgx8KMY6hqLQ4F/i5ZbeWEomgP8c7T8n8AT0fLfA0cADwBfGecah3IghZ/hWgxFksaZoUiCw4Fl0fLHKYSitxN2zJupvlA0lOkUfubvUQhFbwLOBKYQfyiSpIpyTJEkSRKGIkmSJMBQJEmSBDimSBqtOuCcaPlZYDWQIozHOZ8w0DkH/CvwdL/nNkTPfTnhKLEUsIFwhNVtQHaYNbwceC0wjzCY+gngO8DOkf84ZXU8cDDh5/9VdN8RwNuAhUA9sAb4AeF9G4k08ArgVYSfezfwW+CGaHkohwPnRtfzgUnAVuBB4GfAnmFsIxPV8NJoG42EcWjbgMcJn+FQn8Ek4KzoMj/a5ibCd+BXQM8w6pBUZoYiaXQmA7+Mlj8H/DvwLcKOsti36RuK3hate3iJbf4D8DvgHYQd/UAOAv6XEL76uw64HHhq0OrH1zXAnxB27E2EYPh+Qvgr9i/AZ4G/Y3ghYCHwf8CLSzz2ceAiQqgo5ZXApwlHGg5kJ/BBoGWQdU4BvgEcM8g6vYRB+t8a4PE3RLUcUeKxa4AngUuBewZ5DUnjwFAkjd1UQkBaTGgx+C3QRdgBF3dRf5Cw804RWoPuAVYRdqKnAKdHz7mdsBN/cJDXelF0+zlCC8cO4GhCUPo6IYgkwccIoacV+CnhSL4FhBauJuBvgWZCEBzMgcCvCWFyNfAIoSXqjGh7BxGOEDwBeKbE85dEj/USpidYRWgVmkX43E4CZgDLCS15ny2xjUMJrTjTo9urCJ/Rtuj1D41eY3bROv1dCfwX4XuRA+4nBOEu4ERCqD4GuBV4DQOHPEnjwFAkwcOEVgYIXSl5f0XYQT4+xPP/lLCTu5rQAtAd3Z+Kng/wOgqB6FHgj0ts948ILSHTCN1gS4DOfuv8K4VAdAPwl8C+osdfRAhJHxyi5s0UfuZHi+7/MCEo/GGI5w9HBvgAIeS9jRAe8hYCPwGOBf4iWr5hkG39NdAGvAW4qej+OuCTwN8QAuPVwBUlnr+JEMC+1a+OvFMJUxMcTmht+z/g+X7r/C0h7OQILUHfKLGdDKF7rqPEY+cQ5oRKE4LdJbxw/qhzgO8DM6Naj2F4XXqSysCB1lIICDdEl7ai+2+O7ls1xPPrCC0i11EIRBB2nh2E37NPEQLRc8B5lA5aPyd0fQEsInRBFZtDYW6hJ4A/o28gghBwLuKFXVX97aHwMz9XdP9t0X33D/H84UgRQuYFvDCIrAPeTGghAfh/Q2yrjhCebup3fw+hJSof4t48wPO/D3ymRB15DwEXRsuNhKDb3ynR9eOUDkQQWqJuBX5T4rFPEkJTK2FMVKlZxm+n0Go2lxB6JVWIoUgauzbC7NcDOZfQRQOhJal/C0Sx7xAGXUOYSLHYhYQBuhACWBel3U0IN0nweQYedLyG0CIDYXD2iwZYD0LY6x+I8ropDOg+hNB9NRoPURiL1X9sGBRafyYTws1InAacHC1/nsJnXMqPKAxA7/8dkDSODEXS2N3P4Ec+nVO0/LMhtpWlMJbolH6PvWQE2/npEI9XykjqPGOQ9YYaW1Pc3TdriHVnEo4e+xNCy1vxJd/CdliJ5/0uus4P+D5uiNcpdk7R8lDvCRRa6vp/BySNI8cUSWO3fojHjy5a/gx9u9hKWRJd92/xWBBdt9K3y6uUJ4d4vFKGqqP48VJH5OUN1roGfVujGgdY50zC+d7OZeh/CEsNlP4CYZzZDOCt0WUdYQD4bRQGvJdS/B34BNA+xOvn159OmMJgoFZBSWVkKJLGrm2Ix2cULZ89gu02Enbe+XmLpkXXw5mPZ9cIXme85IC9Q6xTXGfzIOv1jrGWPwX+h0K311ZgLSFMFdd4DuFIt1LdY+sJXWCfJxwZliK0Gi0E3hnV+GvCYPX+J68t/g6U6pobTBOGIqkiDEXS2OWGeLx4h5afv2c02863LjSUWrGfgVpLKilFaOXofwRdseI6h2o9Ga2DgC8Sgs6zhMHLvxpg3XsIoWgg6wlHEs4D3kjohjubQpB6FSFYXUIY3J1X/B14J0OHxWJDhW5JZWIoksZfcVdX8WDekcofOTWL0HowWIg4dJSvUW7zCS0yAymuc6gustF6M+FwfQiH6w8UiCAM1B6OjYSg9UVCa95JwFWEFqk6QmvSzRS6Sou/AysI8yxJShgHWkvjr3hm4lePYTv5Wa7rCEczDebMMbxOOQ1VR/Hjg83iPRZHFS2XOlQ+bz6FcVsjkSXMdfVnwI3RfXMIR9TlFX8H/mgUryGpAgxF0vj7MYVWnfcSDukejeIjtd41yHozCJMlJsFfDfJYA2HuIQiDx+8epxqK52warGvsb8rwWg8XLRd/zr+gMBbsSgrjwyQliKFIGn9bCEcuQTjf1beBA4Z4zsvoewg+hMG7+cP1/5jSExWmgS8x+KDlSjqLEAJKuY5Cy8xXGL+xM8VHuL17gHVeT+j+GswyBj59B4Q5pF4TLfcS5mHKayVM4AlhUsbvDbEtCK2BZw2xjqQyckyRVBkfIszDcyZhgO6TwJcJrSM7CDvUQwmnm3gj4fQOy3jhUUyXE7pi6gkzT3+BMKB3e/ScqwhHN91POJdanPLnGfsM4SSuXycExAWEsT2vjdZ7hnCo/Hj5AWFyzenA+whdW98gTKA4jzAD+DsIY7Za6Xv4fLF/Jky++WPCIfiPEo5eayZMzvluCt2B36DvKWMgzHr+MsJg7PMIEzReT+jS20b4ezyfMDfRGwjdb+8H7hzdjy1ppAxFUmV0Ek7W+iXC2JN5wD8Osv5Ah7M/DFxMOC9WE6HLp3+3z63AR4n/ZKI5QuC4lXCusLeXWOcPhPdlONMMjNYOQjfd/xG67P4suhR7nnBetX9l4FAEoUvsIgrnjSvlp5RuHeshhJ3PApcSwtk/DFG75z2TKshQJI1OF9ASLQ93LMw+4M8JZ0n/K8IkgodROCz9WcJ51n4M/JCBTwVxE+GM6tcSumvmEMYsPQb8N+FM74cV1ffEMOsbD78ntHx8gHCakgWELr71hJByHQPPqdRK4WcY6mit3xWtu73E4z8ktJxdQ2ipmUUIqs9Ej32K0IqVP8VG/1YeCIfenxtdjiccqZaff2gjoXXufwhHnQ2kgzAe7LOElsBXEN6TpujxTYTP8WeEz3n9INuSVGap5cuX758HJZVKHb1s2bI1gz1BUtk1MPhcPkOpY2RzH423b1KYj2lSv8dShFA01skYx2oSQ88sPhzlmm16rN8BSaPQ0tKyOJfL5c816EBrKQHGujNMUiAaSo74AxGUJxBB+WaaNhBJCWAokiRJwlAkSZIEGIokSZIAjz6TVH73EwYyJ2HskCQNm6FIUrl9JrpIUlWx+0ySJAlDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEgB1cRcg1Yh6YH60/AzQG2MtkqRRMBRJ5XE08Gi0PBvYXsHXTgOLgWMJv9O9wA8q+PqSVBMMRSrlt8B04HLgljJsrwGYHC3vBbrLsE3Bh4FXAicDU4vub8NQJEkjZihSKQuAGcCUUT7/SOA84FTgFOA4Ct+184Bbx1hfEu2l8HNVKvT9JbCw6HYH0Fih15akmmMo0nh4D/DeuIuosHWEwDdSc4GbgV2EVp+R+DmwGXg4urwL+OgoapAkYShSaTPH+Pws8DiFnfUe4CtjLapG1RNa00YzBumKMtciSROaoUjj4e+A9xfdPiGuQiroSOA70fIrCS0/kqQqYihSKf8INAH/C6waxfNz5S2nKjQRWnzA3ytJqkr+8VYpf0sYaH0/owtFkiRVHWe0liRJwlAkSZIEGIqkStlOGGvV/7IuenzWAI/nKM8EmpKkIRiKJEmScKC1hu8g4JsDPPZl4P8qWEs1OgJIlbj/MGAFsINwWH8pnhZFkirAUKThagReNcBjtXjajnJrHeD+6dF1DthZoVokSSUYijRczzPwaSyermQhkiSNB0ORhqsDW4QkSTXMUCRVr7nAIf1uQziA4pR+6z5ckYokqYoZiqTqdTnwDyXubwQeKrqdBTIVqUiSqpihSOPhZPp2tRXvkG+k79FUfwz8ohJFjbMu4PfRcu8IntcTPW+ggdiD2Vn0moPJjmLbkjThGIo0HuoI504r5YB+tyeNcy2Vsppw2P1IbRjl8wA+FV0kSWVgKFIpJxPGpTw3yuevZPg7+i2jfA1JksrKUKRS1o/x+Z0Mr1tHkqTE8DQfkiRJGIokSZIAQ5EkSRJgKJIkSQKqeKD1F7/4xYPS6fRVcdcxmK9//eunt7W1NQJs3rx59ZYtWzbHXVM/RwHzouVtwKoYayllJnBCtJwF7oyxloGcU7S8ktHNNzSejgdmRcsbSN556uYRvocA7cADMdZSSiNwetHt+wl1JsmLgSnR8tPAxhhrKWURcGi0vAN4NMZaSmkGlhbdvpNwgubEOPHEE1+eTqczAOecc87vjjnmmO1x1zSQVCr1wLJly26Mu47RqtpQlMlkDszlch+Mu47BPPnkk2zfvv+7e0actQzTG+IuYAgvjbuAIVTDZ5x0r4i7gCEk/TNOen0Ar4u7gCEk7j1cuXLl/uUzzjjjJTGWMqRsNns9YZLeqmT3mSRJElXcUlTCb3K5XFfcRRTL5XLnAimAQ5uauqfX1XXEXFIfT+3bN6Uzm00DNDc3s2jRokSdDmLz5s2pzZs3pyC8icdNnbon5pJe4LE9e6bml4866qjc1KlTE9Xs/uijj6Z7enoAmDVpUu8hjY1tMZfUx7Pt7U27enrqABobG1myZEmivoN79+5NrVmzJpW/ffTkyW2TMpmRnMZl3K3as2f/l27+/PkcdNBBiXoPV69end63bx8AUzKZ3MLJk/fGXFIf27u66jd3djbkb5900knZVCo12FMqbsWKFelsNnysHR0d63O5XKLmgUulUscDB8VdRznUUii66LLLLkvUmJ3m5uZuovf4wvnz91w0Y8YzMZfUx0UrVx77bGdnI8AJJ5yQ+8QnPtETd03Frr/++szXvva1DEAmleL6Y49N2ngYznjggZPyy+95z3uyp512WqJ2mK997Wsntba2pgBOnz6988OLFiXqPfy7NWsW3N3aOgPgwAMP5HOf+1yivoMrVqxIX3HFFfv/Tl67aNGmxZMnJyqcv/zBB5f25HIpgAsuuCB7ySWXJOo9vPTSSyc98cQTKYBFTU29LQn7Pf7apk0HtmzYMD9/+zOf+UxPJpOs8ye/4hWvqO/oCP9TP/XUU9/+6le/em3MJfWxfPny7wJvi7uOcrD7TJIkCUORJEkSYCiSJEkCDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBNTW5I2j0QCcCJwSXU4FDose+xjwyZjqkiSpHOqA4+i7nzuScKKAbwB/E19pyTPRQ9EFwLcGeKypkoVIkjQOTgHuG+CxyZUspBrYfQa9wOPA/wLvBTrjLUeSpLJbC3wXuBp4NuZaEmuitxT9FGgGik9Q+PGYapEkqdweA2YCO4vue3tMtSTeRA9Fu+IuQJKkcbQvumgY7D6TJEnCUCRJkgQYiiRJkgBDkSRJEmAokiRJAmr36LMPAVNL3L+KMB+RJElSH7Uait4DHFLi/hsxFI2Lm2++Of3LX/4y/f73v793wYIFubjrqUbbt29PffjDH86cffbZuYsuuqg3lUrFXZIkTSi1Goo+BhxQ4v41lS5korjtttvSDz/8cPr222/PveMd7+iNu55qdMcdd6RWrFiR7urqyl188cW+h5JUYbUaij4fdwETzTnnnJN94IEH0nfccUfaUDQ6v/71r9MAZ511VjbuWiRpInKgtcrirLPOyqXTaVavXp3auHGj/T4jtGfPHlauXGkokqQYGYpUFjNnzsydcMIJOYA77rjD79UI3XHHHemenh4WLVqUc0yWJMXDnRc8ADxUdGmM7r+83/2Xx1JdFTn77LOzYCgajdtvvz0NhfdQksroJvruzxZF97+p3/0fjaW6BKnVMUUjcQqlw+Hc6JL3s8qUU73OPffc7Gc/+9nMqlWrUlu3bk0deOCBtngMQ1tbGw899FAawtisuOuRVHOOB44ocf/s6JL3ZGXKSS5DEVwMDGcMzBPjXUi1mzNnTu7oo4/OPfnkk6m77rordcEFFxiKhuHuu+9Od3V1MW/evNxRRx3leyap3P6a0kdk9/eH8S4k6QxF8L24C6glZ599dvbJJ5/M3HHHHekLLrjAVo9huPPOO+06kzSe7OkYJsd+qKzOPffcLMAjjzyS3rVrV9zlJF53dzf33XefXWeSlACGIpXVYYcdllu4cGGup6eH3/zmN36/hnDvvfem9+3bx+zZs3PHHXecXWeSFCN3Wio7j0IbvuKuM0/rIUnxcqelsst3Az3wwAPpffv2xV1OYvX29nL33Xfnu85sJZKkmBmKVHaLFy/OzZ07N9fV1bV/vIxe6OGHH07v2rWL6dOns3TpUscTSVLM3GFpXORbi+xCG1h+wsazzjorm8lk4i5HkiY8d1gaF/lQlJ+DR31ls1nuuuuuFHiuM0lKilqap+jclpaWnXEXUezqq6/eP3J23b59k34N0+Ksp7/2bHZ/KN6xY0fqnnvuKVtIzuVyNDc309rayte//vXMkiVLRjxmpvjEsjng1zt3Jur96+/xxx9P9fT0DOs9XLduXWr79u2phoYGent7Ked7X6ynp2f/e/h8V1cmae/hjq6uSfnl9vb2cXsfRmv9+vV9bv929+7JGzs7EzUivvgXa/369WX9PS6HvXv37l/e09ubStp3cF17e1Px7XvvvTedTifqLaS3t3f/8qxZsxa1tLS8JsZyXiCbzR5cKweKpJYvX77/dyqVSh29bNmyNXEWNFwtLS3H5XK5x+KuYzDXXnst27dvj7sMSVKNuOKKKzjxxBPjLmNAuVzu+ssuu2xZ3HUMV0tLy+JcLrc6fztZcViSJCkmVdt9ls1mc6lUqifuOgaTCVIA2Wy2N5fLJW3sSIZCMM4BvYOsO1r571gvfVv6hyNFqDGvuywVldekouXh/ozFP9d4f4czFM7tl2V8PuOxSFN4L8brOzhWxX8nexj593i81dH3M07a35k0ff/OJO3vdooXfsaJkk6ni+sbzd/Sikn6fnkoVRuKLrvsssfpu0NSMn0ZuBRYDrwn5lqS4qPAPwA3AhfEXIukhMtmCzn3C1/4QoyV1D67zzTeboyuL8DvW95bo+sfxFqFJKkPd1Iab7cCu4GDgZfEXEsSLAaOJXQF/iTmWiRJRQxFGm+dFHb+b4mzkIR4W3T9KyBRU0hI0kRnKFIl5LuJLqQwIHSiygdDu84kKWEMRaqEnwJtwAJgabylxGoBcDLh6KAfxVuKJKk/Q5EqoQ24JVqeyF1oFxBayu4CtsRciySpH0ORKiXfXTSRD0HPB8IbB11LkhQLQ5Eq5UdAF3AccEzMtcRhDnAGYdI1Q5EkJZChSJXSCvw6Wp6IrUVvIczc/CDwh5hrkSSVYChSJeW70CbiuKJ8ELSVSJISylCkSrqJcN6eU4GFMddSSc3A2dGyoUiSEspQpEp6Hrg7Wn5znIVU2JuAeuB3wOqYa5EkDcBQpErLt5RMpC60fNeZEzZKUoIZilRp3yMcgfVS4JCYa6mEA4DzomW7ziQpwQxFqrQNwEOE794bY66lEl4LNAHrgJUx1yJJGoShSHHIt5hMhEPz892EN8RahSRpSIYixeF70fW5wMw4CxlnDYSWIrDrTJISz1CkODwFrAImAa+PuZbxdD4wDdgI3B9zLZKkIRiKFJfvR9e13IVWfK6zXJyFSJKGZihSXPLdSa8GpsZZyDipA94QLXsoviRVAUOR4rICWAs0An8Ucy3j4RxgNrAduCveUiRJw2EoUpxqeSLH/M90E9ATZyGSpOExFClO+W6l1xNajGpFmsJpTOw6k6QqYShSnO4jTOY4FXhVzLWU00uAucAe4LaYa5EkDZOhSHHKAT+MlmupCy1/RN2PgI44C5EkDZ+hSHHLdy+9iXDEVi0oPhRfklQlDEWK2x3AVmAWcHbMtZTDScAioB34Wcy1SJJGwFCkuPUSupmgNrrQ8l1nPwf2xVmIJGlkDEVKgnwX2lup/u9kPhTZdSZJVabad0CqDbcCu4GDgdNjrmUsFgNLgG7gxzHXIkkaIUORkqAT+Em0XM3nQrswur4N2BlnIZKkkTMUKSny3U0XDrpWsuUDnRM2SlIVMhQpKX4CtAELgKXxljIqhwMnA1ng5phrkSSNgqFISdEG3BItV2MX2gVACvgNsCXmWiRJo2AoUpLku9CqMRTlpxOw60ySqpShSElyM9AFHAccE3MtIzEHOJNw2hIPxZekKmUoUpK0ArdHy9U0keNbgAzwEPCHmGuRJI2SoUhJk+9+qqYuNM91Jkk1wFCkpLmJcOqPU4GFZdzuhcA1wCvKuE2AZuCcaNlQJElVzFCkpHkOuCdaflMZt/sXwMeB15VxmwBvBOqBx4Any7xtSVIFGYqURNXUheaEjZJUIwxFSqLvEY7keilwSJm2+WngHcC3y7Q9gMnAedGyXWeSVOUMRUqiDcDDhO/nG8u0zduA/yEcIVYuryMEo3XAijJuV5IUA0ORkirfHVWuQ/PnAUcDB5Zpe1DoOvteGbcpSYqJoUhJlQ8arwBmlmF7XyIMhP5gGbYF0AC8Nlq260ySaoChSEn1FLAKmAS8PuZaSjkPmAZsBO6PuRZJUhkYipRk5e5CK6fiCRuzcRYiSSoPQ5GSLB+K/giYGmch/WSAN0TLdp1JUo0wFCnJVgBrgUbg1THXUuwcwoDt7cCd8ZYiSSoXQ5GSLt8Sk6SJHPNdZz8EeuIsRJJUPoYiJV0+FL2e0GIUtxSF0484i7Uk1RBDkZLuXsJkjlOBV8ZcC8AZwHxgD/CrmGuRJJWRoUhJlyN0U0EyutDyXWc/BjriLESSVF6GIlWDfBfam4C6OAuhEIrsOpOkGmMoUjW4HdgKzALOirGOpcARhBaiX8RYhyRpHBiKVA16gR9Fy3F2oeVf++eEMUWSpBpiKFK1yHehvYX4vrf5UOSEjZJUg+IenyEN1y+B3cBc4HTCUWkj8R3CZJB3j/L1jwKOA7oJg6wlSTXGUKRq0Qn8FLiE0Fo00lD0zTG+/oXR9W3AjjFuS5KUQHafqZrkj/i6cNC1xoddZ5JU4wxFqiY/AdqAhYQjwSplPnAKkAVuruDrSpIqyFCkatJGGFsEhfmCKuFCwuk97gY2V/B1JUkVZChStcl3oVXy0Pz8azlhoyTVsKodaP2Rj3wkPXPmzAPirmMwd9111+TOzs40wNq1aztWrVrVHXdN/TQCk6LlHpJ32ooM0BQt54C9hIHOXcDxhC6tNeNcw0HAmdHyrYRzsBUrvt1OmFMpSZoI7yOE960zxlpKmUThRL9ZQmtgkqSAKUW39xK+i0kymcJn3Ek4QjJJGuj7d6Y9xlpKyRDew7y9cRUykDe+8Y0HZLPZFMBRRx3VvnDhwp64axrI1KlTu/7yL/8yafuSYavaUDR37txjc7ncY3HXMZjbbruN7du3x11GLXuowq/3uwq/niRx882FoYxXXHEFDQ0NMVYzuM7OzuuBZXHXMVp2n0mSJGEokiRJAqq4+6y/VCr12t7e3kT1VWWz2XuI+vovOfTQ3WcfcECijlz6yNq1C57r6moAWLp0KVdeeWWixiL88Ic/zPzoRz9KA2RSKT53zDGr8499Yt26w9Z3dDS96aCDnv+jWbN2jsfr37Vz57RvbdlyyJz6+s6PHHHE+lLrXP7EE0fnl9/3vvdllyxZkqgxRVddddWkPXvCadrOnD6948/nzn0m5pL6+MKGDXN/t2fPVIA5c+bwsY99LFHfwaeeeir1iU98Yv/fyX9YtOjZeQ0NiRr3dOWTTy7uzeVSABdffHHuvPPOS9R4k4997GN169atSwEc0dTU+3cLFjwdd03Ffr5t24wfbt16UP52S0tLdzqdrPaCK664YlJXVxcAzz777H+fcMIJX4y5pD7S6fTHgVfGXUc51EwoyuVyKy6//PJEhY7m5ub9AzIPaWzsWTp1aqL+mNanUvvrmzp1au64445L1ADSe+65Z389KaD4/Ttv1qwd12/cOG9NW9vkDy5YsHE8Xv9rmzYdDHDOzJk7h/PZHXbYYYl7DzOZTI7w9tE8aVI2ad/BaZnM/h14fX09SXv/uru7U8W3j5o8uXPx5MmJeg+LCzz44IMT9x1sbGzcvzw5k8kl7Tv4yJ49xQPpWbJkSS6TyQy0eiyKQ1pra+vmyy+//IEYy3mB5cuX18ws/8mKw9IwvXr27NYU8OTevQc819U1acgnjFB7b296RdSC8apZs1rLvX1JUvIYilSVBBlQNwAAIABJREFU5jU0dC1samrLAb/avn168WM7enrq8t0Jw9Xa09PnX8Pbdu6c1pXNpmfX13cdP2VK0g4hliSNA0ORqtbLZsxoBbirtXV/KHq+u7vuTx999Jjf7d3bNPAz+3q+u7vurStXHvdMR0d9/r7bd+xoBnhZc7OtRJI0QRiKVLXOnzmzFeB3e/dObe3pyaxpa2u89LHHjm7PZjNNmUx2uNtpTKdzvblc6vInnli8cs+eyV25XOrh3bunAbwieg1JUu2rmYHWmhjae3vT+cBzxOTJnfMbGjo2dHY2vn/NmkXPtrc37untrZtWV9czOZ0ediianE5nG9Lp7M7u7kkfWLPmiEObmjras9nM9Lq67lOnTduXX68jl0s3pFLZEfXLSZKqhqFIVWN3b2/mTY88clxjOp2dXlfXk4XUtu7uSQCP7927/5QvKciNpKWoLpXKpaJTN+zp7a3Lb6sjm81ctHLlsXXpdG53T0/d7p6eum+ccMLjhzc2dpX7Z5Mkxc/uM1WNaZlM7wcXLnymJ5dLPdPR0fRsR0djRzb7gmNnc5AaSUsR9D2sOa8zm01v6OxsXN/e3tSRzabfPnfuJgORJNUuW4oKjiCcYHRhdPsu4J74ylEpr541a9cje/bs+MW2bbNKBaK8kbQUAaRSA3eK1aVSuaOnTNn3rnnznh/JNiUpYeYT9nOLCY0ijwI/i7WihJnooejFwMcJX5IZ/R77MIaiRLp6wYKNT+zdO+Xp9vbJ2RKH3qdgZMfjA+lBznw+a9Kkrk8eddS6ERcqSfFbCHwWOBWY0++xr2Ao6mOid58dCbyKQiB6DhhRC4MqLwX81zHHrG2uqyt5SojBWn0GkhngOdPq6no+ffTRa0fa8iRJCXEQ8DoKgWgn0BFfOck20UPReuCjwJuBQ4GD8ctSFZrr6no/sXjx7w8oOk1E3mi+1JmiU57kHZDJ9Lzv8MP/sLCpqXNURUpS/HYA1wEXExoCZgJrY60owSZ699m90UVV6PgpU9ovnT9/0/UbNsxr6+3dP74oNUhX2EDqU6k+LUGN6XT2lTNn7nj1rFm7ylGrJMXkKeDquIuoFhO9pUhV7pI5c7afNHXq7rqilp6BusIG01B0tFoKcvMaGjo+sGDBpjKVKUmqAoYiVb2PH3XUM3Pq68fUxVU80Lp50qSe/zr22LWlutQkSbXLUKSqV59K5f7rmGPWTslkegGmlBhnNJTpkyb1ADRlMr3/euSRv59VVzfibUiSqpuhSDVhXkND14cXLVrflE73No7iSLHJmUx2UiqVe+fcuZtOnDq1bTxqlCQlm6FINePsGTN2v/7AA7c2j6alqK6u57Tp01v/7JBDto1HbZKk5KvVo8+mUzrwdQN7K1yLKuiqww/f3NbbO+Kw/9eHHrq5Pp12DJEkTWC1GoqeAA4pcf+NwAUVrkUVlAKmjLL7bBzKkSRVkVoNRbuAxhL376t0IZIkqTrUaig6Nu4CJElSdXGgtSRJEoYiSZIkoHa7z0biQsL43Lz8ObSOA95WdP8TwGOVKkqSpDJ5DXBA0e1p0fVC+u7n/gDcX6mikshQBP9H6RazS6JL3r9gKJIkVZ/PAkeUuP8V0SXvmxiKJryHGV43oicHlSRVo8eA1mGst268C0k6QxGcFncBkiSNozfHXUC1cKC1JEkShiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJgNTy5ctz+2+kUkcvW7ZsTZwFDVdLS8txuVyu+LQbq4CeuOop5ZprrnnRjh07UgCz6+t7p6TTXXHXVGxTZ2dDdy6XBpgyZQqHHHJIbqjnVNKOHTvIv38p4LDGxvaYS3qBZzo6mvLL8+bNyzU1NQ22esWtW7cu1dvbC8DUurrszLq6zphL6mNrd3d9W29vBmDSpEkcfvjhifoOtre3s3Hjxv3nRpzf0NCZSaWycdbU3x86Opryb9qBBx7I9OnTE/UebtiwIdXR0QFAYzqdm1Nf3xFzSX3s7ump29nTMyl/+8gjj0zU+wfw+9//PpXNhq/du971rudOPfXULTGX1N/hQDNALpe7/rLLLlsWcz3D1tLSsjiXy63O366lGa2Pi7uA/lKpwnlmt3V1ZbZBsvaYRfbt28fTTz+dGnrNeOToG0CSqHjnmUR7enrSe3p6Evsednd3J/o7CLChs7Mh7hoGs3XrVrZu3ZrY97Ajm00l/fc46d/BSZMmzQHmxF1HrbL7TJIkiSpuKerq6tpSX1//gbjrGMwJJ5xweltbWyPAxo0bV2/cuHFz3DX1cxQwL1reRuiCTJKZwAnRci9wV4y1DOScouWVDO+ki5V0PDArWn4WWBtjLaXMBRZHy23AgzHWUkojcHrR7fuBpHXjvhiYEi0/DWyMsZZSFgKHRcs7gEdjrKWUZmBp0e07CY3TifHiF7/45alUKg3Q0NDwWCqV2h53TQPJ5XIr465hLKo2FF155ZXbgf+Iuw5NeJ+MuwBJte3BBx/cv6974IEH4iyl5tl9JkmShKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJANTFXYBUpTLA0mj5eeDZMWzrYGBetLwK6BjDtsbT4cDsaPm3QC7GWsbTYcCB0fIjQDbGWiRVkC1FUtjRvyq6NBbdf1Z036ISz5kCPBRd/n6Mr/8XRds6fIzbGk8folBn/Si3cTThPT2j6L5pFN7/6WMpsEyupvBzTo65FkkVZCiS4CXAL6PLnKL7vxfd9/Y4iqpRVxLe068U3beYwvt/TBxFSRIYiiRJkgBDkSRJEmAokiRJAjz6TCq3ycDJhLFJe4GHgW1l3P4BhMHYh0XLO4HfAc+NYltp4ETCkW8N0TaeBZ4pS6XJMIXC57Gb8HlsH+W2DgOOi7a5kTAQu3uYz80AC4C5hKMN24H1hKMNR3IU3xzgBGAG0EX4WZ4Dnh7BduYQfo5ZwD5gNbB2BDVINctQJJXHJOCfgSsIR1Pl9QDfAd4HbB3ltqcB7wZeB7yU0r+39wEfBm4dxvZmE46w+kvCjrG/Jwk1/9Moam0EvgG8Nbr9NULtww0PozWbwvv7KeAa4F+Ay4CpRet1A98C3s/ww9GRwBeBVwKpovu3A9cCLYM89wLgQuDVwMwSjz8H/Bfw74TvykBeAlwHvKxfDXlbgF8A7xhkG+cD/0g48q//NlYB/wDcNMjzpZpn95k0dmngB8AHCTvgfGtLlhBg/gy4GzhklNs/irBDPDvaXjvwB/ru1F9C2Cm+a4htLSXMvfN3FALRLuD30TWEI8DeN4o6ZxFCWT4Q/RNwKeMfiPrLADcDHyC06vwhumQJ4fUvgN8ABw1jW0sJrUuvim7vpBBeZgHLgb8e5Pn/BvwxIRD1EMLLBqA3enwO8HHgxwz8T+r5wB3AywlhpjPaxrpoGULr058M8PxU9Bq/AM6MbndEz98ZrXMccCMhNEkTlqFICgGmJbrsLbr/69F9Dw/x/EuA1wM/AY4gdLMsiC7fj9Y5CvhfSv+XPxyPAe8lzPOT70KbTZjX51JCF10a+Gz0WqXMBX4OzI9u/x/wIqA5qruZMCfT/2Pkk1EuIgS/lxJ2/pcCH+GFXTp3E97T7xbdt5XC+//8CF+3lD8ntMz8MKrr8OiyMLoPQvD772Fs67uEAPEOQovdzOj6ckL3FYTgU6oVCELQ/DQh0EwjBONDCd2sLyW8H0T1vr/E81PAFwjzQu0jTA/RHG1jEaFlbn50/x0D1HAVoeUM4FHgjwjhfREh2L0UWBk9/hEKoVaaeJYvX57LX1paWhbHXY9UJaYRdvj5y52EVoj+MoT/0PPrvb7EOlcXPX50icfrGTpMLSXspHPAZwZY59tFr/NvQ2yv1D9MLUXPbyi6/8WEFpAcsIew06202fT9PH5F6ZaXuuix/Hrnl1jn80WP72bguZOuKVpvoBa6hgHuL378gWgbmwjfl2JHFr3GNYzcYYTWpBxhDNSUAdZrJrQW5oCnStQh1aSWlpbFxTnIliKpPN5P6W6iXsJ/6nmXjmLb+bAzmBXAz6LlV5Z4fB7wtmj5ccL4kcEM99QWbwB+TegG2kzo4vv5MJ87nt5H6TE6PfRtkRnq8/hPwhirUv6naPm0AdbpHOD+4sc/HS0fQujGKlYcYjYPsa1SrqQw+/gVhNamUloJY7AgBLEzR/FaUtUzFEljtw54cJDHHyd0W0A4dchou9CKTSWMI1lUdMl3PR3NC3+3X0nhv/+vMPig3uG6jDAOZQrwBGFH+tsybHesVlPoDiplBYWgc/YQ2/rhII9tJozvguGPF2sgdLUVf25tRY8f22/9pykE1PcTugBHIt9q93tCi9RgbilaNhRpQvLoM2nsVgxznRcRdogHM/L/+tPARcCfAqdTOGFpKRnCWKOdRfe9qGj53hG+dikfpzAY+y7gzcCOMmy3HIb7eRxDaOGaxcBHog02PUGOcPTYAsI4r4GcSAiQ+fPoDfbP6Ix+t/cRjgi7ADgeWENombuF0GX7MIVB2/3VU2h5ageW9Xt8Gn27yYrrmjtIjVLNMhRJYzeceYiKBxDPZGShaDphx3hOicfaKHTRNFE4oW3/E7YWH3o/mjmN+ssHop3Am+gbwOI2ms9joFDUMcR2BgokedcAH+WFY3R6COOviB7LT+NQagzSXxE+29cQ/mafF10g/Kw/IbT+3dXveTMpBJ3jCEfKDde0oVeRao+hSKqM4i6zkUzWB/AlCoHoJsKcOY8SgkjxmJVPAH8/wDZG+ppD+V/CUV4zCF1ob6Cwk68GY/k8huuNhBY1CN1gHyccIbaVvu/VSQze7bgTeC2hhfAthMH6Swg/w2zCFAN/AXwO+BsKP0/x3/fVDHx0WinlaE2Uqo6hSBq72cNYp7i7ayStKocQus0gHN5/4SDr9u96KVbcEnIwYYzJWCwjHG5+JWFczi2ElozWMW63HMbz8xiJ90bXrYTD3geabmCwz63Y/dHlg4T6X0Fhcsg04bN4gBBYoe9nvpYwiaakQTjQWhq7k4axzsnR9XbC4evDdQqF39NvDbHu0kEeKx5nU45BtDnCpIXXRbdfQjjUfTiBZLwN5/PIr7OF0Z/2YyinRte/ZPD5lwb73AaylTDP1MWEVrq8S4qW2wljkCB8Pv4TLA3BUCSN3QJC18ZAjiMMkoXQhTGS7priAbyDPW8JhZ1wKbdRmDLgnZRvB3k1YcI/CMHv14SWqDgtZvBgdBKF+aBuH6caUgw8J1D/9f5sjK/1Uwqhq/8A6Z9G1zMpTMkgaQCGIqk8/oOBJ2/8dNHtL49wuxuKlgeaFLGBMIh2sEP9NxMmb4Rw2PfHhnjdkfxt+CfCWKYcIfzdQZhxOU6fZODJGz9ZdHukn8dw5SdjhNB1NtDRae9l8AA3hzD79WBmUZhRe1O/x/6Twszb/0mYg2goCyjPtBFS1TEUSWO3jXCizpvpu9NZAHyPwpFCt1CYYHG4HqBwpNq7CEczNRU9fhqh2+plDN0t9wEKO82/B24gtO4U7wCPIJzm47ER1vnvhO60HKGl5g5GPqdOuWwDziUMAF9UdP/C6L5zo9s/Jrx34yU/x9FcwntdXMscQpD+JIN/bucQztv2KcJn3Njv8dOBH1EIgN/p9/gzFCarPIjwfXovL2zNm0cYu/Y9nNFaE5h9zNLYfZdw/qk3EnYomwjhYC6FwPEk4fxUI9VFOKLouxRO7PlRYCOh9SHfQvBdwiSSVw+yrecJ59j6CeH0DxdGl32EcTUzKbRo7B5FrZ8njGNpIQSQOwlz86wexbbG4huEYPZ64HWEUNn/83iM0I04nj5KmK5gPqGVby1hLFBnVEua8F15NyHYDGQWYVb0qwiH8j9H+F70b0X6JuFn7+9z0XofJwzq/nR02UE4198MwmSg0oRnS5E0OjnCUUs7CaHircC/Eo7Imkv4zztF2Hl9jdCFMtD8QB1F2yo17833CDvXtdHtDCHUzCS0MlxNOBN7W9F2BjpNx2OE7pp/pzCfz5Roe/lAtIKwQ++vePsD+SrhUP3t0XZvIrQ+VVKWMJnkdYRD34s/j07gesIJWrcO8Pzin3Oo8V+7ovVKTUfwHOFz/0nRdg4khKRuQpA9GVhf9Hr950V6EPgvwhQMWcI/svMIoTMfiB4nzGX09kHqvY7QqvgDCrNwzyR87vlA9BzhvXktQ8+/JNWk1PLly/f/EqVSqaOXLVu2ZrAnSBpUIyF0HEw4FHsF5TvkO00Ys7OQsNPaSNhZjnYHliYMAj+UMB5qC6G7ZSRHxyXFbAoh51MUuoyaCJ/HHMLn8QjxTBtwCGFm68mE8LFqFHU0Ez6ruYTJFXcQji57doTbaSTMcH4Q4f3ZTghm6xn+Oe+kmtDS0rI4l8vtb822+0wqrw7Gb+K7LCEEPTrUiiPY3u+iS61qB+6JuwhCF95oTuharDW6jPXz6mDo86BJE5LdZ5IkSRiKJEmSAEORJEkSYCiSJEkCHGgtqTZsw1mYJY2RLUWSJEkYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiTAUCRJkgQYiiRJkgBDkSRJEmAokiRJAgxFkiRJgKFIkiQJMBRJkiQBhiJJkiQA6uIuQFJFNQJNQA+wJ+ZaJClRbCmSJpb3ATuAn8Xw2hngeOCtwNuAl8RQgyQNyJYiVaulwJej5dOAbIy1aGCzgQ8DpwAnAZOLHvsOcF8cRUlSKYYiVasDCDtagNQot/FiQmvFKcCpwDGE1ox2+u68a8k64FZgVYVe72Dgb/rd10t4nyUpUQxFmsi+DRwRdxEV9u3oMlJvBt4D3AL8+wie1w58H3i46PIT4PRR1CBJ48pQpGp1HzAzWu4d5TZ6gJUUdtanAO8ce2k16VDgVcCGET5vLXBh+cuRpPIzFKla9QA7x7iNE4DuotsfGuP2qsEfE4LfKuC9MdciSYliKFK1mg/8abR8HZAbxTa6h16l5iwktPg0xV2IJCWNoUjVagHwb9HyfzD6LjRJkgDnKZIkSQIMRZIkSYChSFLBSwljs0pdPhOt845B1uk/H5EkVRVDkSRJEg60Vu15NXDpAI+9j5HPszORPMzAk1m+HfhHwkSMfz/AOtvHoyhJqhRDkWrNkYSTjZbykQrWUY06gN8P8Fg+8OwZZB1JqmqGItWaXwAXDfCYrUSSpAEZilRrno4ukiSNiKFI0nhbQt8ZtKdE1zMJ55vLayWcK02SYmEokjTevgWcWOL+86NL3k3AWypSkSSVYCjSRHYVcGXR7eboupG+LRY5wgDuWtBOOJHunhE+ryN6XtsoXnM3wzt5775RbFuSysZQpIlsBrCoxP2pfveP5mSzSfXp6DJS10eX0ThrlM+TpIoyFKlarQBOjZazo9zGl4AflqccSVK1MxSpWu0lTDY4FpuiiyRJnuZDkiQJDEWSJEmAoUiSJAkwFEmSJAFVPNB6+fLls3O53GVx1zGYb37zm6e3t7c3AmzatGn1pk2bNsddUz9HAfOi5W3AqhhrKWUmcEK0nAXujLGWgZxTtLySMCtzkhwPzIqWN5C8U6DMBRZHy23AgzHWUkojcHrR7fsJcz0lyYspzBL+NLAxxlpKWQQcGi3vJPyeJEkzsLTo9p0kbBqOk08++eXpdDoD8LKXveyxY445ZlvcNQ3it5dddtlP4y5itKo2FKVSqTnAR+OuYzCPP/4427fnTy7Oq+KspUacP/QqsfIzHrs3xl3AEJL+GSe9PoC3xV3AEBL3Hv72t7/dv3zaaae9MpVKxVjN4HK53PVA1YYiu88kSZKo4paiEn4LdMddRLFcLncaYXZk5jQ09EzLZDpjLqmPZzo6mrqy2TTA1KlTOfTQQxPVZLx161a2bt2agvAmHjl5cuJOA/FUW1u+24LDDz88N2XKlMFWr7jVq1enent7AWiuq8seWF+fqK6fzZ2djXt7ezMA9fX1HHnkkYn6Dra1tbF+/fr9/5Yf3tjYUZ9O98ZZU39Pt7VNyb9pBx98MDNnzkzUe7hu3bpUe3v42jVlMrn5DQ2jOVXMuNnZ3T1pW3d3ff72kiVLEvX+ATz55JOpbDbMUdvV1bUJeDbeil7gKMJwh6pXS6Ho9e9+97sTNWanubm5m+g9/pPDDtt90YwZz8RcUh8XrVx57LOdnY0AS5cuzV133XWJCpXXX3995qtf/WoGIJNK8fXjj18Td039nfHAAyfll6+66qrs6aefnqgd5mte85pJra2tKYAzm5s7PrxoUaLew79bs2bB3a2tMwDmzJnDV77ylUR9B1esWJG+/PLL9/+d/Ocjj9ywePLkkZ43bly9/MEHl/bkcimAiy++OHvJJZf0xF1TsXe+852TnnjiiRTAkU1NvS1LliTqO/i1TZsObNmwYX7+dktLS3cmk4mzpBc499xz6zs6OgBYvXr1/3z5y1++NuaS+li+fPl3SX636LDYfSZJkoShSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQJqa/LG0cgAxwCnRJdTgYOjxz4FfD6muiRJKocUcASFfdwpwOHRYzcAH4yprkSa6KHoIuBbAzw2o5KFSJI0Dk4D7hvgsdmVLKQa2H0WPAP8APgQ0BVzLZIkldsW4CfAPwObYq4lsSZ6S9EtwIHAtqL7PgTUl15dkqSqsgqYD2wsuu+twNx4ykm2iR6KtsddgCRJ42hvdNEw2H0mSZKEoUiSJAkwFEmSJAGGIkmSJMBQJEmSBNTu0WcfAqaWuH8V8L8VrkWSJFWBWg1F7wEOKXH/jRiKJqxdPT2Z72zZMvuptram/1i8eH3c9VSrn2zb1nxva+u01x944I6XTJ/uob6SakathqLPM3BLkSaoFPCNzZsP6cnlUqvb2rYcPXlyR9w1VaOfb9s286Hdu6cfWF/fbSiSVEtqNRR9LO4ClDzT6up6T5g6dc8ju3dPu3XHjuajJ0/eEndN1WZfb2965Z49UwHOnzWrNe56JKmcHGitCeWsGTNaAe7euXN63LVUo1/t2DG9O5dLH1xf33nslCntcdcjSeVkKNKEct6sWbsyqVRuXXv75Gfa2xvirqfa3BGFyTOjcClJtaRWu89G4hbCcJO8/I7yHcDZRfd/E/jvypSk8TKrrq7n6ClT9j2+d+8Bt+zYMf1d8+Y9H3dN1aIrl0v9dvfuaQCvmDlzV9z1SBq2/6HvCWAXRNevAX5ZdP9twL9WqKZEMhTBKyndYnZEdMm7rzLlaLy9vLm59fG9ew+4e+fOZkPR8N2+c+e0jmw2M2PSpO6Tp07dF3c9kobtpfTdn+XNpW9Yeq4y5SSXoQgupm9L0UCeGO9CVBmvnjWrtWXDhvlr2tqmbOnqmnRwfX133DVVgzt27GgGOLO5uXU4vzCSEuOvgQOGsd4fxruQpDMUwffiLkCVdUhDQ/eRTU37nmpvn/LL7dun//khh2yLu6ak68nlUg/u2jUN4JUzZzqeSKouP4u7gGrhQGtNSC+dMWMXwF2trc1x11IN7m5tPWBPb2/d1Eym58XTptl1JqkmGYo0IZ0XzbHz+N69B2zv6bHFdAi379zZDHD69Om76lKpXNz1SNJ4MBRpQlrU1NR5WGNje28ul7pt+/ZpcdeTZDngvtbW6QDn2nUmqYYZijRhndncvAvgzqgVRKU9tHv3lNaenklNmUzvy2bM2BN3PZI0XgxFmrDyXWgr9uyZtru3NxN3PUn1q+ios1OnTdtdb9eZpBpmKNKEtWTKlPa5DQ2dPblc6vYdO+xCG8C9UdfZOc5iLanGGYo0ob2kubkVCqevUF8r9+yZ/HxXV0N9Op09Z8aM3XHXI0njqZaOujn9S1/60o64iyh2zTXX7J/jbkN7e9196fRwJs+qmM5cbn99u3btSj3yyCOJmpNvy5Yt++vJAfft2lX2929ufX0XwEO7d0+/c+fOqfXp9Ki7h9auXZuqr69P1HvY09Ozv54d3d2Zkb6HNz3//CyAIyZPbnt0797J5a5vV2/v/r9BnZ2dJO07uHbt2j63V+3b17ijuztRXYjFxWzYsCFxv8dtbW37l/f29qbG4/d4LDZ2dvY5B+IjjzySzmQyifqMs9ns/uUZM2Yc+qUvfemsGMspZXbcBZRLavny5fs//FQqdfSyZcvWxFnQcLW0tByXy+Uei7uOwVx77bVs37497jIkSTXiiiuu4MQTT4y7jAHlcrnrL7vssmVx1zFcLS0ti3O53Or8bbvPJEmSMBRJkiQBVTym6F3vetfjN9xwQ8PQa8bnrW99a117e3sK4N577+15+umns0M9p8Iy0QUgC/TGWEspKQrf0RwwXiduzQDPAAcBrwVuHcFz64uWe+g7xCMJMhT++ellZJ/x/cBJwOXAV8pcV16avp9xzzi9zlhMKlruiq2KgdXR9zNO2t+ZNH3/ziTtM07R9zNO3Amizz///EnTp09PARxzzDHdM2bMSNpQxwu4AAARf0lEQVTfmWJJ24+MSNWGolSYLyWJf6CKJb2+xP3yl9BZgdfoBm4G/gp4AyM7eWI1vIejsQBYStiJ3cT4/pyV+IzHwt/j2pfoz/iWW27ZX98NN9wQZyk1z+4zKfhBdH0Bhf9qJ7ILCP9B3wVsibkWSaoIQ5EU3ArsBOYAZ8RcSxJcEF3/YNC1JKmGGIqkoBv4abT8ljgLSYB8MMwRus4kaUIwFEkFxV1oiZoAr8IuIPxteBD4Q8y1SFLFGIqkgp8D+wiDjE+Kt5RY5VvK7DqTNKEYiqSCNuAX0fIFg61Yw5qBs6Nlu84kTSiGIqmvG6Prt8VaRXzeTJh76XfA6iHWlaSaYiiS+voRYc6SxcCxMdcSB7vOJE1YhiKpr13Ar6LlidaFdgBwXrRsKJI04RiKpBfKd6FNtFD0WqAJWAc8GnMtklRxhiLphW4knJ/pZGBRzLVUUj4Eeh4BSROSoUh6oW3A3dHym+MspIIagNdEyzcOtqIk1SpDkVRa8USOE8H5wDRgI3B/zLVIUiwMRVJp3yec5uIMYG7MtVRC8bnOcnEWIklxMRRJpW0knOYiDbwp5lrGWx3w+mjZrjNJE5ahSBpYvgut1k8Qew4wG9gO3BVvKZIUH0ORNLDvR9fnEkJDrcqHvpsIR91J0oRkKJIG9jThdBfF3Uu1Jk3hCDsnbJQ0oRmKpMHVehdafiD5HuC2mGuRpFgZiqTB5UPRqwmHrNeafNj7EdARZyGSFDdDkTS4R4E19J3csJZ4AlhJihiKpKHdFF3XWhfaSYTTmLQDP4+5FkmKnaFIGlq+FeV1QGOchZRZfsLGnwP74ixEkpLAUCQN7QHgWeAA4LyYaymnfChywkZJwlAkDUeOQhdarZwLbTGwBOgGfhxzLZKUCIYiaXjyrSlvBibFWUiZXBhd/wrYGWchkpQUhiJpeO4EngeagbNjrqUc7DqTpH4MRdLw9AI3R8vV3oV2OHAykKXwM0nShGcokoYv36pyAdX9u3MBkAJ+A2yJuRbp/7d3t7FxVfkdx793ZvwU25PEBCUkPO8mKaSBjXgoYUVIRLdturQhu2SD1CK1QFAQSYUqpKLICazEm6LQGqFWSUpV+qIVJd3CtstGS5UIUliUdhOw4ogmLFg8GJc8b9ZxHjwzty9mxh47Tsg4Y9879vcjXfmMM77+6+aO/fM5Z86RYqOaf7BLY+0/gePAdPLbY1SrYk+XCzZKUglDkXTx+oCfFtrVupBjMdCFOJ9IkgYxFEnlKfaufJ/8EFS1WQ4kgV8An0VciyTFiqFIKs828qs/X0t+m4xq415nknQehiKpPL3Am4V2tQ2hTQEWF9oOnUnSEIYiqXzFXpZqe2v+MqAW6AD2R1yLJMWOoUgq338AZ8lvk3FDxLWUw6EzSboAQ5FUvl8BOwrtauktagJ+p9A2FEnSMAxF0sgUg0Wl5xX9WeGYVeHzLgUagE6gvcLnlqRxwVAkjcxrQAa4Bbi+gud9oXB8s4LnhIEerX+t8HkladwwFEkjcxh4t9C+L8pCLkId8PuFtkNnknQehiJp5Ipva6/kENrjheOjCp7zO0Aa6AJ2VfC8kjSuGIqkkfsR+e0y7gSuqNA5/7ZwfFmh88HA0Nlr5OuVJA3DUCSN3BfA/5B/HS2r0DmvLBy1FTpfEviDQtsFGyXpAgxF0qUpBo1KvTX/88KxsELnWwxMA44AOyt0TkkalwxF0qX5UeHjYqAlwjrOpzjf6XXy75aTJJ2HoUi6NB+R3zajhoFhqrgIGBjWc+hMkr6GoUi6dMXeoritbr2Q/PykXwPbI65FkmLPUCRdumIvzO8CzVEWMkRx6OwnwOkoC5GkamAoki5dO3CA/CKJSyOupZQbwEpSGQxFUmX8uPCx0nuhjdS3gG8Ap4BtEdciSVXBUCRVRrE35l6gPspCCorzm34GnIyyEEmqFoYiqTJ2kV9fqIn8thpRK4Yih84k6SIZiqTKCInPENocYB7QR36StSTpIhiKpMop9sr8IZCKsI77Cx93AMcirEOSqoqhSKqcncAh4DLg7gjrKPZUuWCjJJXBUCRVThb490I7qoUcrwRuAXIltUiSLoKhSKqs4hDa9xjZ6+uvC8fnI/z+95Pf3uMdoHuE55CkCSnKeQ/SeLQd+BUwA7gD+HmZX//nl/j9iz1UDp1JUpnsKZIq6wzwRqE91kNo04E7C+0fX+iJkqRzGYqkyiv20nyf/FDWWLkPSAK/ADrH8PtK0rhgKJIq76fkV5G+lvx2G2PFBRsl6RIYiqTK6wXeLLTHaghtCrC40HY+kSSNQFVPtH7mmWdiH+r27dsXAGzdujUXdS3DCBgY3gmjLOQCSuuLY42l92Bpfa+TXy/oe8DTY1DHMqAW2AccGFJXtVzDONYGg4dAfR2PTLXcgxDD+lasWBEcOnQoAFi8eHEc78F+Tz/9dBgEQeyu4cWq2lC0ZcuWeWEYdkRdx4WsW7eOI0eORF2GonUj+fWLxsq8Mf5+kkbZ1q1b+9s33ngjN998c4TVXNjmzZv/Dng06jpGKvY9LZIkSWPBUCRJkkQVD58NFYbhimQyeTTqOkrlcrk3yb9FmvtnzjxxR3PzVxGXNMhznZ3XHDx7thbgpptu4pFHHslEXVOpbdu2JbZt25YASAYBfzlnzkdR1zTUk/v3zy62H3/88dzcuXP7x/t7enpobW1N5XI5NmzYkJ02bdqojLO//PLLyT179gT33HNPbtmyZYPmG6xbty7V09MDwO3p9JkfXHHFZ6NRw0j9Q1fXFft6epoALr/8ctavXx+re/CTTz6hra2t/+fkk9dc0zWjvr43ypqG+osDB76ZDcMAYPny5eGSJUtiNXy6cePG5GeffRYAXNfQkH386qs/ibqmUtuPHp267dChacXHbW1tmUQiXv0FTz75ZOrs2bMAdHV1/fOCBQv+PuKSBsnlchuIdr/Hihk3oSgIgndXrVoVq20NpkyZ0v9L8KrGxsy3J0/uibKeoeqCoP8X6OTJk8PbbrstVhP4Pvjgg/4JrgEQt+s31OzZs8+5hvPnzw/b29uDY8eOsXTp0opf376+Pvbv358CWLlyZXb+/PmDglcqlQopTHKdVlubjds1fO2rr/qK7fr6euJ2D9bU1Az67Ti/ufnUnEmTYnUNS2eBX3311bF7HTc2NiaL7aZkMozbPXjg5MmG0se33nprLplMnu/pkSgNaUePHv101apVOyIs5xybN29eHXUNlRKvOCyNM8V3irz99tuj8lrbtWtX4uTJk7S0tITz5s2r2nd8SFIcGIqkUbRkyZJcEAR0dHQEhw8fHrS6dXFY62L19fXR19c36HPFsLV48eJc3Lr8Jana+FNUGkXTp08P586dG+ZyOXbu3Nkfinbv3h088MADNeWc64033kisXbs2lcvlR0dyuRzvvPNOAuDuu++2l0iSLpGhSBplxSG0t956KwGwffv2RGtra00Ylpdj6urq2LdvX2Lt2rWp3t5edu/enTh+/DjpdJpbbrklVvNIJKkajZuJ1lJcZLNZgiDonxy5ZMmS3KZNm5J79uxJrF69OtXZ2Zk4ceIEM2bMKOu89fX1pFIp9u7dm1i5cmXNrFmzQoBFixYNmhiayWRIpXxpS1K5/MkpVdi7776b2LBhQ2rmzJlhOp2mt7eXZDJJNpulvb29v3e23DlA9fX1JJNJTp8+zeHDh4MjR44EAO+9915i1apVNdlslkOHDtHY2Mgrr7zS93XnkyQNZiiSKmzRokW522+/Pbdr165EZ2fneZ9XW1tb1nkbGhrC0h6h4vDbkSNHKAakdDrN888/byCSpBFwTtGAGcB3ye/Z8iiwINpyVM2effbZzPTp0y84aai+vr6sc9bX11+wd6m5uZnVq1dn5syZ46RrScOZCnyHgd9z42LBxUqa6KFoAfndzD8HuoGfAJsLx3cjrEtVrra2lra2tszUqVPPG1BGEoqCIBj232pra1m4cGFu+fLlTriWVOoq4F+Aj4GjwJsM/J57MMK6Ymmih6LfAJYBVxYe/xrwr2xVxMyZM8PW1tZsc3PzsP8+adKksu61hoYGgiA452uCICh+r1htkSEpFmYCPwCuLzw+BZyNrpx4m+ih6Avgr4A/Ih+QppC/YaSKuPPOO3P33ntvrrGx8ZwwM2nSpLLO1dDQMGyImjp1atjW1papqSlr2SNJE8Mx4G+Ah4CbgTQQu30k42KiT7T+r8IhjZo1a9ZkOjo6aj788EMymYHOnMbGxrLO09DQQFjY+LMonU6Hra2t2a+bvyRpwjoArIm6iGox0XuKpFGXSCTYuHFjX0tLy6DgMlzv0YXU1NRQXM0a8j1NK1asyC1cuNB5RJJUAYYiaQyk02k2btyYaWpq6v9cuT1FQH9PUxAEzJ07N/fwww9nK1akJE1whiJpjMyePTtcs2ZNpvius3Q6XfaQV3FydktLS/jcc89lzvduNElS+QxF0hhatmxZ7q677sqlUikaGhpGdI7Jkyfz4osvDup1kiRdOkORNMbWr1+fmTVrVljuu88gvynsE088kbnuuuucWC1JFTbR330mjbmamhpeeumlvrq6urK/dtOmTZnLLrvMQCRJo2C89hR9SX4RxqHHv0VZlFTU1NTESNYVMhBJ0ugZr6FIkiSpLON1+Gxm1AVIkqTqYk+RJEkShiJJkiRg/A6fleMeoHQFvGTh4zeA3y75fCfw8VgVJUlShXwbKF0YrbjI2SwG/577P6BjrIqKI0MRvMnwPWZ/UjiKngXWj0E9kiRV0j+S/0N/qN8rHEX/BPzxmFQUU4Yi2M3FDSN+OdqFSJI0CjqA4xfxvM7RLiTuDEVwe9QFSJI0iu6LuoBq4URrSZIkDEWSJEmAoUiSJAkwFEmSJAGGIkmSJMBQJEmSBBiKJEmSAEORJEkSMI4WbwyC4PUtW7b0RV1Hqaeeeqq4jxqvfP55eltX1+wo6xnqYF9fbbHd3t4ePProozVR1jPUwYMH+9vZMORPOzpidf2GeuGFFxJNTU2x+kOjp6enf1+/nx8/Xh+3a9h15kx9sX3w4EHidg/29vYOevzDjz+eVRsE2YjKGVY2DPv/j1999dXEjh07YnUNP/300/76Pj51Khm3e/BoJjPoej322GOxun4AZ8+e7W/PmTPnwQcffHBRhOWcIwzDuVHXUCnjJhSFYRi7lamDYGCf2e7Tp1PdA5vwxc6JEyfYu3dv8PXPjEYI/G9vb2yvH0BnZ2dsrx/A8UwmcTyTie01PHPmTKzvQYBPTp1q+PpnRae7u5vu7u7YXsPebDaI++s47vdgXV3dlWEYXhl1HeNVrP6qlSRJikrV9hSlUqmDfX19P4y6jgu54YYbfuv06dP1AF1dXfu7u7u7o65piNnArEL7MLAvwlqG0wLML7RzwM4IazmfxSXtdi5u08Wx9JvAZYX2F8AvI6xlOLPI34cAp4D/jrCW4dQBd5Q83kW+zji5DWgstH8JdEVYy3CuB64qtI+Rf53EyRTgWyWP346qkPNZsGDBXUEQJAHq6ur2AkciLulCdkddwKWo2lD00EMPHQKeiboOTXixDuaSqt/777/f396zZ0+ElYx/Dp9JkiRhKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCTAUSZIkAYYiSZIkwFAkSZIEGIokSZIAQ5EkSRJgKJIkSQIMRZIkSYChSJIkCYBU6YMwDK/dsmVLVLVIkiSNmTAMry19nBry7z8Lw3DsqpEkSYoJh88kSZIwFEmSJAHw/8Ve17voNSEvAAAAAElFTkSuQmCC\")
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9707f13b",
+ "metadata": {},
+ "source": [
+ "The implementation is given in the following cell. Note that we introduced an extra loop that represent\n",
+ "each one of the two phases."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c0cbe2d8",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "function red_black_gauss_seidel(n,niters)\n",
+ " u = zeros(n+2)\n",
+ " u[1] = -1\n",
+ " u[end] = 1\n",
+ " for t in 1:niters\n",
+ " for color in (0,1)\n",
+ " for i in (n+1):-1:2\n",
+ " if color == mod(i,2)\n",
+ " u[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ " end\n",
+ " end\n",
+ " end\n",
+ " u\n",
+ "end"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "13ce270f",
+ "metadata": {},
+ "source": [
+ "Run the method for several values of `niters`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "59c1e7f8",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "n = 5\n",
+ "niters = 1000\n",
+ "red_black_gauss_seidel(n,niters)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d1bacc26",
+ "metadata": {},
+ "source": [
+ "\n",
+ "Question: Which of the loops in the red-black Gauss-Seidel method are trivially parallelizable?\n",
+ "
\n",
+ "\n",
+ "```julia\n",
+ "for t in 1:niters\n",
+ " for color in (0,1)\n",
+ " for i in (n+1):-1:2\n",
+ " if color == mod(i,2)\n",
+ " u[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ " end\n",
+ " end\n",
+ "end\n",
+ "```\n",
+ "\n",
+ " a) All loops\n",
+ " b) Loop over t only\n",
+ " c) Loop over color only\n",
+ " d) Loop over i only\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d511cc02",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "answer = \"x\" # replace x with a, b, c or d\n",
+ "gauss_seidel_2_check(answer)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "8946b83f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "gauss_seidel_2_why()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8ed4129c",
+ "metadata": {},
+ "source": [
+ "## MPI implementation\n",
+ "\n",
+ "We consider the implementation of the Jacobi method using MPI. We will consider the 1D version for simplicity.\n",
+ "\n",
+ "\n",
+ "\n",
+ "Note: The programming model of MPI is generally better suited for data-parallel algorithms like this one than the task-based model provided by Distributed.jl. In any case, one can also implement it using Distributed.jl, but it requires some extra effort to setup the remote channels right for the communication between neighbor processes. \n",
+ "
\n",
+ "\n",
+ "\n",
+ "We are going to implement the method in a function with the following signature\n",
+ "\n",
+ "```julia\n",
+ " u = jacobi_mpi(n,niters,comm)\n",
+ "```\n",
+ "\n",
+ "The signature will be very similar to the sequential function `jacobi`, but there are some important differences:\n",
+ "\n",
+ "1. The parallel one takes an MPI communicator object. This allows the user to decide which communicator to use. For instance, `MPI.COMM_WORLD` directly or a duplicate of this one (which is the recommended approach).\n",
+ "2. Function `jacobi_mpi` will be called on multiple MPI ranks. Thus, the values of `n` and `niters` should be the same in all ranks. The caller of this function has to make sure that this is true. Otherwise, the parallel code will not work.\n",
+ "3. The result `u` is NOT the same as the result of the sequential function `jacobi`. It will contained only the local portion of `u` stored at each rank. To recover the same vector as in the sequential case, one needs to gather these pieces in a single rank. We will do this later.\n",
+ "\n",
+ "The implementation of the parallel function is given in the following code snipped:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d31c1e41",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "code1 = quote\n",
+ " function jacobi_mpi(n,niters,comm)\n",
+ " # Initialization\n",
+ " u, u_new = init(n,comm)\n",
+ " for t in 1:niters\n",
+ " # Communication\n",
+ " ghost_exchange!(u,comm)\n",
+ " # Local computation\n",
+ " local_update!(u,u_new)\n",
+ " u, u_new = u_new, u\n",
+ " end\n",
+ " return u\n",
+ " end\n",
+ "end;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "04a38c37",
+ "metadata": {},
+ "source": [
+ "In the following cells, we discuss the implementation of the helper functions `init`, `ghost_exchange`, and `local_update`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "96b8d40e",
+ "metadata": {},
+ "source": [
+ "### Initialization\n",
+ "\n",
+ "Let us start with function `init`. This is its implementation:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b98a9348",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "code2 = quote\n",
+ " function init(n,comm)\n",
+ " nranks = MPI.Comm_size(comm)\n",
+ " rank = MPI.Comm_rank(comm)\n",
+ " if mod(n,nranks) != 0\n",
+ " println(\"n must be a multiple of nranks\")\n",
+ " MPI.Abort(comm,1)\n",
+ " end\n",
+ " load = div(n,nranks)\n",
+ " u = zeros(load+2)\n",
+ " if rank == 0\n",
+ " u[1] = -1\n",
+ " end\n",
+ " if rank == nranks-1\n",
+ " u[end] = 1\n",
+ " end\n",
+ " u_new = copy(u)\n",
+ " u, u_new\n",
+ " end\n",
+ "end;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1b9e75d8",
+ "metadata": {},
+ "source": [
+ "This function crates and initializes the vector `u` and the auxiliary vector `u_new` and fills in the boundary values. Note that we are not creating the full arrays like in the sequential case. We are only creating the parts to be managed by the current rank. To this end, we start by computing the number of entries to be updated in this rank, i.e., variable `load`. We have assumed that `n` is a multiple of the number of ranks for simplicity. If this is not the case, we stop the computation with `MPI.Abort`. Note that we are allocating two extra elements in `u` (and `u_new`) for the ghost cells or boundary conditions. The following figure displays the arrays created for `n==9` and `nranks==3` (thus `load == 3`). Note that the first and last elements of the arrays are displayed with gray edges denoting that they are the extra elements allocated for ghost cells or boundary conditions."
+ ]
+ },
+ {
+ "attachments": {
+ "fig.png": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6gAAAFzCAYAAAAzJyTtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d15mBxVvf/xd08mmewkISEEIoTFsCMSBWRT2d0uCIr7DoGrXnC/qCigV38oCoKKTBCv4Mp1AwHZjIIiyBIBZQmIAUNICISELJBJMjP9++N0p2tmumeyzPQ5PfV+PU8/nOruTH/o7uqqb9Wpcwqtra1FJEmSJEmKrCl2AEmSJEmSwAJVkiRJkpSI5m7LRxcKhSdiBJEkSZIk5UuxWJwG3Fhe7lKgFgqFJ2bOnPlovUNJkiRJkvJn1qxZFIuVYZHs4itJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSkJz7ACS6ueyyy4b097e/ufYOXpz8cUXb7948eIWgFWrVi1ZtWrV0tiZupkITCi1XwQWRMxSzUhgamb50VhBejE9015AeB9TMpXwPgIsBZZEzFLNBML3EKANmB8xSzXDge0yy48BnZGy1LIzlYP0C4FVEbNUsw0wutR+HngmYpZqxgFbldprgSfiRempubl52MSJE6eVlz/+8Y/PGzduXHvESL0qFotfOPXUU6+JnUNKhQWqlCNNTU1DgJfFztGbZcuW8fTTT5cXX1K6pWoEsGXsEH1I+vMGXho7QB+2Ld1SNQIYHztEH/aKHaAPO8UO0IcRwJTYIXoxgsR+Z9rb27PbEYrF4h4R4/SpUChM6PtZUn7YxVeSJEmSlATPoEo5ViwWL2xqakqqi2pnZ+dXgaEAU6ZMeWTRokXfjxypu/+i0n3xReCsiFmqeS3w+szyp2MFqaEJ+Fpm+Vrg1khZavkS4awQwL+B70TMUs1MKmee1wKfj5ilmv2At2aWvwysiJSllvMy7T8Cv4sVpIYvAmNK7UXA+RGzVPMeYO9SuwM4I2KWHiZMmLDr0qVLP1ReXrly5UUTJkx4Mmam7orF4heAsbFzSCmyQJXy7cczZ868J3aIrClTppxDqUCdOnXqvEWLFn0jcqTu3kKlQG0DUss3hK4Famr5mulaoN4CfDNOlJo+R6VAXUh67+HRVArUdaSX74N0LVC/RyiyUvJ1oFBq30V67+EnqRSoS0gv3350LVCTyjdjxozX33zzzesL1Dlz5vz4K1/5yt0xM3XX2tr6cSxQpars4itJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokjZXM7AX8AHCfJF/Bu4p3T5Z5ywFwjygrcADwEqgE3gauAb4jzrnqWYL4GTgt8ACYA2wGngE+DawQ7xo6+0BnAvcASwFisAywmf7n8CweNGAMA3RccCPgccI89GuA54Efgq8Kl609aYAnwJuBhYD7cALwH2EeVYnxou23kGEdfZ+wjylReAZ4HrgRCrTsMQyGngf8GtgPmFapzbCZ34JsEu8aOtNJ3yetxGmgykCy4HbgY9Tma4oliHAG4AfEn5jXiB8F58CfkH4vYxtEnA64Xv3NGFdfhH4B+F3aOtIuYYC+xJ+ry8hfKblbdsHImWScsF5UCVtjv8kzH83ssbjf65jlh2AK4FXVnlsMvDG0u3HwPsJc/fV2/uB84HxVR6bXrp9AHgXcHX9Yq3XAvwEOJ6exck44ODS7YPA6wg75PX2KuBHwE5VHpsKvAN4O3A2oXCI4RzgvwnvZ9ZI4GWl28mEwuFv9Y0GwDaEdeXgKo9NAo4p3d5JKFTX1i/aem8FvlvK091Opdv7gA8RDkrUW4HwPXwHPQ/2jyV8T19FyHcUYT7detuH8N7sVuWxbQhzOr+FMA/xp+qYK+tThHV1VLf7m4E9S7eTCAekbqtjrjNLt+7rcNmUOmaRcscCVdLmmEzP4vR5QjFTb7tRKU4fBq4lnGkZCxxGKKgA3l26/5x6BwTeRChO1wG/B24hnKHcHngvsB1hR+3nhB2zf9U530jghFL7aUKR/I/S8isIhfPQUvtHVN7TejqISnF6J3Aj4czpJMJO7H6E4uEcwvfgFxEynkjYsV0F/I5wJno1lQMQ4wlnhX4L7Fp6Xj3tQKU4/Rfhc36U8N07BDiW8B4eSziD9Yk654NQIE8iHEj6Q+m2hHAQ4l3AzsBw4HLgQcJZ4HoqlHJQynV1KUMH4QDE+wjfgT0I38GDCWdX6+mVVIrTewnfxSeALQkHRw4pPfZJYC7w/Trng/B7M4qwflwP/IWwPuxIOBA2iZD3KsK6Uq+DYlPoWpwWCb0MtqjT60u5ZoEqaXM8Q+h+N6d0uwd4DfDLSHn+AXwGuKHb/d8APkLozgihO9lXCF3d6qkIXAr8D6HLYtZXCUXrgYQd748QpzB4hnDm4HJ6njlrJRTVLYQCYk9CV+p6mw2cQfi+ZZ0LXAR8tLT8SeIUqGsIn/EFhAMQWV8H7iIcjNiWcLY3RmHwKGFduYbQDb7sfELh9ePS8qnAFwhdQ+vtx4Sza90P1HyVkPsown7M6YRipt6WAmcBlxEKrKzvEoqt0YR1+sDScr39hbCudD/7+DXCb+DnSsufJM73sJ3w+3we4bcn61zgr4Su3FsSeqB8o065FhJ6GZS3bX8jnC2/uE6vL+Wa16BK2hwXE46Af5VwJuu5iFn+CsygZ3Fa9l3g36X2eMIR+nr7CDCTnsUphB3cszLLM+qSqKsXCGdcLqV6t86/Eg5IlO1bj1DdXAEcQc/iFMIBgM9RKbheTpzrKI8mFHXdi1MI16N+M7Mc43N+kHCW72q6FqdlP6FyRnIE1buIDrTPAe+hei+CtYSDKGUx3sMi4X35Dj2LU4C/UynyIc66cjVwKLW7xp5NuNYTwtnJ7t1s6+EtwKfpWZxC6I3z/zLL9fycv0I4eHQe4ez983V8bSn3LFAlDRZLCV1ne/NUPYL0YnEfj2fzxSis1lK9qMqKnbHajmzWSkJXvLIYGfv6nBdk2jHyPU8YbKg3sT/n1NeV8oBSvYmd8RmqH4AoW0fX/wc/Z0lJsECVlBfbUDkC/yzweMQstbwp074zWorahgGvzyzfFStILw6hcg30PfS+gx5LdjTpFD/niVSuUV0FPBQxSy2prysFwqBsZSlm3Idw/TuEs+r1vhZ6Q6T+OUsaABaokvKghdDdrjzoxdfp+2xrvb0C+GKpvYLK9bIp+Tawe6n9a8IgRCmZRLgesOyrsYL04q2EAbEgdF+9MmKWaoYC/0cYXAzgW8S5/rQ3u1Pp+tlGuG42NWcC+5fas0mvuJpAuD67fFbyKxGz1HI08OFSeyFhqhxJOWCBKmmwawF+RWW+v6tIb4d2X8I1vKMIo4B+gMr1sqn4EuH6WQgjfp4SMUs1kwhzjr60tPw14Lp4cao6lnCgpEA4CPE2KtcApqCZMDpzeV35PfDleHGq2o2Qazyhm+2ppHeg5JNUpjj6N2FE35RsQVg3di4tXwL8LF6cqg4nHARrJlzj+3bCXMyScsBRfCUNZkMJOznlbqk3EkZiTKnb58uo7HB3EOZN/HWv/6L+ziYM+gPhrN9RxJkDtZYJhLNUe5WWLwY+Gy9OVW8inLEaSuhK+SbC6KCpGEIont9WWr4NeDNx5kCtZTrwR8L0VkXC6L2XR03U08epjDS7gDCgV+xr37NGE34Hy2d3f0wYvC0lryWM0jyCMCL2W6jvnNqSIrNAlTSYtVIpTq8HjqfvwWHqaRvCqMPjCUXz++k68mcKTqYyuvBjhJ3HBbWfXnfNhJ3ZcnH6HeA06j/nZG9eSejKO5TQXfZ11B5ZNZZvUClO/0SYJzOlaxInADcRilOA/yKMzJ2St1EZoXkBYcqtes9l3JsC4SBJuTi9gjA9T0oH7HYnjD5cLk6PJRTUknLELr6SBquDCV1lAe4mHIVPqTiFMFfm1qX2p0ivOB1H5WzQEsI1YSkVpxB2sA8sta8kveIUwrW7IwiFwFtIrzjdm/C+QRgs5z9IqziFcAa/PKDPl0ivOB1O+JzL3bePIa3iFML1z8eU2tcTemt0xItT1QXAmFL7fVicSrnkGVRJWV8ijNRazSLgwjpm2VzHZtqfJa1r/cqOK/13Hmm+t4dRGSznQkLO1GQ/50+QXnE6hcoZqxuoPU9vTP9B5YD1WcDyiFlqKX/OzxIO7KTmIMJ10BDmEX4wYpZajsu0Pwm0xwpSwxjCtacAt5PeAGKS6sQCVVLWJ6g9Wfv9pFlE1fKSTDu1QVQgvM/jS+1HSKubXdnUTDvF9xAqGZ8njPSZmkZ6DyHNKWUKVDI+RnojcEP67yFUMnYSBjpLzdaEa6Eh3XVFUh1YoErKupfQFbGaf9YzSD9oybTXREtR2/BMO7Wux2WNlDHFzxga4z3MrispZmwiXL8LaeaDxvqc15JeTwNojPdQUh1YoErKOiR2gAGS4s5YVur5GkEjvIeNkDF1voebrxHew0bIKGmAWKBKGqxOJVxnBaH7Z2qeB3YqtV+IGaQXs4BfltpPxwzSiyMIZ9dSG+yl7G4qn3OK30OAz1CZ7/TJmEFq6KDyHq6OGaQXPyPMwwvwTMwgvTiecBY11eLvESqf84qYQSTFZYEqabBaHDtAHzpIc9ChrOdJt6gqS7Ggymoj/c/52dItZam/hytIv6hKaT7WataS/ucsqQ6cZkbSYHUh4YzGzXQdMCkV21HJd0HkLLXMpJLx6MhZavkNId9NsYPUcAiV9/BjkbPU8hUqGXeNnKWaLankuzRyllreRSXj8ZGz1PJjKhlrjTUQ075U8p0ZOYukiDyDKmlzbEGlC2jZVpn28cCe3R4/h/rMA/kKKvNj1hqZOKZRhO6p0HVwkJS8lErGn8YM0ovXEOZrTXEUZAjrQ/k9TPXs0D5UMm4RM0gNLVTyPRAzSC92pJLxuphBenEQMK3UTnH/bwKV93BpzCCEOXe/3+2+7EjNHwBe2+3xj5Pu91NqKCn+QElqHMOo7FBUs13plvW9gYsjSdJmG03v27adS7escQMXR8oXC1RJm2Mt8PuN/Df1GkBkJbCs1E5xAJ0OKvlWxgzSi9VUMqY6jcvzhEFfUj2DupbKe/hizCC9WEUlY3vMIDV0UsmX6nWe2XUl1SlSllPJmOJASeuo5Is9cNwqNn7blvr1+lLDsECVtDmWA0fGDlHDMbED9OFRQpe2lH2xdEvZDrED9OEa0v+c3xY7QB+eJv338BulW8r2iR2gD7eSzuf8b9LdtkmDnoMkSZIkSZKSYIEqSZIkSUqCBaokSZIkKQkWqJIkSZKkJFigSpIkSZKSYIEqSZIkSUqC08xIOVYoFE5vbW1dHDtH1pe//OWh5fbjjz++K+lN3bBdpj2S9PLt1205tXyFbstvBKbECNKLkZn29qT3Hr400x5Gevn27Lb8ReLPa9ld9nv4WtJ7D8dk2lNIL9/emfYQEsv317/+tcv0U/vvv/9pb37zm5Pa1gFjYweQUmWBKuXbu2MH6K6pqdKxY8mSJTsAn4yXpk/DSTsfpJ/vNaVbqrYh7fdwKGnnAzg1doA+7EfPAzspmUjan/EQEsu3cuXKLssjRoxIblsnqTa7+EqSJEmSkuAZVClHhgwZsq5YLN4YO0dvJk+evFtzc/NIgGXLlj21bNmyp2Nn6mZbYOtSewXwz4hZqhlL1y6gc2IF6cWMTPtRYGWtJ0YynUoXy6eBpyJmqWZrwvcQQtfZuRGzVDMK2DWzfB/QESlLLfsQzvwB/At4PmKWanYCxpXazwLzI2apZhKVyx3agAcjZumhpaVl+LbbbrtH5q5/FIvFtdEC9aGpqSm13xgpKgtUKUfe+973vgAcEzuHJEkDZc2aNcybN2/98hlnnBExjaSNZRdfSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSaoff3M1kP4ELAVeEzmHBhd/tyTVlT86klQfw4GfAJ+KHUSD1hhgPDAsdhANGk3Ar4CzI+eQlCMWqJI08LYBbgPeDnyptCz1t47Sf4dETaHB5FjgOOAs4H/x4IekOrBAlaSBtTdwBzADeA54HbAwaiINVp2l/7ptV3/5DXAKsA54P/BHYFLMQJIGPzdikjRwXgf8GdgO+CdwIHBr1EQazCxQNRBmAW8Anif8ht0B7BY1kaRBzY2YJA2M04FrgbHA74H9gEejJtJgZ4GqgXIzcDDwOLAT8BfgsKiJJA1absQkqX81A98FvkX4jf0+8HrC2QdpIFmgaiA9CLySMFr0eOBG4MNRE0kalNyISVL/ye60dQBnACcTrt+SBpoFqgbac8BRwI+pHIy7EL9zkvqRPyiS1D92pNLtbRVwPPC1qImUNxaoqoc1wHsJB+A6gdOoXM4gSZvNjZgkbb7swCFPAa8Gfhs1kfLIAlX1UiQcgHs7sJquA8JJ0mZxIyZJm+ftwGxgK+A+4ADgb1ETKa8sUFVvvyD0GllMmFLrr8AroiaS1PDciEnSpikAZwM/A4YDvwIOAhZEzKR86yj9d0jUFMqbclF6HzAFuAV4c8xAkhqbBaokbbzhhEFCziotXwScCLwYLZHkGVTFswA4lHAt6ijCAbuzYwaS1LjciEnSxplCmGbhncBa4P2EOU87e/k3Uj1YoCqmlcBxhAN2BcIBvMuAYTFDSWo8bsQkacPtRRgM6ZXAUuBo4PKoiaQKC1TF1kE4YHcK0A58EPgDMClmKEmNxY2YJG2YY4DbgO2Bxwgj994SM5DUjQWqUjELeAOwnHBt/h3ArlETSWoYbsQkqW+nU5nnbzbhDOojURNJPVmgKiU3AQcDTwA7EeaJfm3MQJIagxsxSaqtGfgO8C3CyKiXEeb7ez5mKKkGC1Sl5gHCAb0/AxMIRet/Rk0kKXluxCSpujHA1cBHCJPSnwOcBKyLGUrqhQWqUrQEOBL4CeGg38XAhfg9lVSDPw6S1NOOhLn9Xg+8QJjT7+yYgaQNYIGqVK0B3kM40FcETgN+CYyMGUpSmtyISVJXryIM6LE7sBB4NeFMqpS6coE6JGoKqboi4UDfO4DVhAN/twMviZhJUoIsUCWp4m2EKRG2Au4HDgDmRE0kbbiO0n/dtitlVwKHA88ALyP0VpkRNZGkpLgRk6QwqfzZwM+A4cCvCdPIPBkxk7Sx7OKrRnEH4QDgQ8A2wK3AcVETSUqGGzFJedcC/Ag4i1CoXgS8FXgxZihpE1igqpE8TihSrwNGEQ4Mnh0zkKQ0uBGTlGcTgd8D7wLWAh8gzHna2ds/khJlgapGsxI4ljCdV4FwoPD7wNCYoSTF5UZMUl7tCdxDmEh+KXAM8MOYgaTNZIGqRtQB/BdwCtAOfIgwFsDEmKEkxeNGTFIeHQ3cBmwPPAYcBPwxaiJp81mgqpHNAt4IrCAcOLwD2CVqIklRuBGTlDczgWuBLQhF6quAuVETSf3DAlWN7kZCcfpvYGfCNDSviRlIUv25EZOUF0MIAyC1As3ADwhTHSyJGUrqRxaoGgz+QThweBcwgVC0vi9qIkl15UZMUh6MAa4mXOdUBM4hXOe0NmYoqZ9ZoGqwWAS8GvgpMIwwPsCF+N2WcsEVXdJgtwNhIvg3AC8Ax+NUBhqcOkr/dduuwaANeDfhgGIROA34P2BkzFCSBp4bMUmD2QGEgTZ2BxYSrmW6KmYgaQB5BlWDTZFwQPGdhIL1BOAvwNSImSQNMDdikgarEwlTFUwG/k64pumeqImkgWWBqsHq54QxA54B9iH0itk3aiJJA8aNmKTBpgD8N2GHZgRwPWFUyPkxQ0l1YIGqwex2woHGh4FtgVuBY6MmkjQg3IhJGkxagCuAcwmF6kWEefVWxgwl1YkFqga7ecD+hAOPo4FfEQ5IShpE3IhJGiwmAjcTBtVoBz4MnE5lp10a7CxQlQcrgf8AvkuYPuxc4FJgaMxQkvqPGzFJg8GewN3AIcAy4Gjge1ETSfVngaq8aAc+CnyMMHr1ScDvgHExQ0nqH27EJDW6o4DbgGnAv4ADCYMjSXljgaq8uZBwGccK4AjgLmB61ESSNpsbMUmNbCZwLbAFYeqBVwFzoyaS4ikXqEOippDq6wZC75n5wEsJgym9OmoiSZvFAlVSIxpCOHLeSrju6H+Bw4BnY4aSIvMMqvLq74R5r+8GtgRuBN4bNZGkTeZGTFKjGQ1cBZxGmMT9HOCDwNqYoaQEWKAqzxYBhwI/I4zofjnhQKbrg9RgXGklNZKpwJ8I1xy1Ae8Czo4ZSEqIBaryrrxdOKe0fBpwJTAyWiJJG82NmKRGcQBwD/Byuh4plxRYoEqhZ83ZVHrWvIUwcN7WETNJ2ghuxCQ1grcSdjAm0/VaI0kVFqhSRXZsgv2pHOCUlDg3YpJSVgD+G/g5MIKuozVK6soCVeoqO7r7toRLRN4UNZGkPrkRk5Sq8iAX5xJ+qy6iMt+dpJ4sUKWesvNjjwZ+QzjwKSlRbsQkpWhL4CbgPUA78FHgdKAjZigpcRaoUnXLgKOB7xGmKTsXmEWYpkxSYtyISUrNS4E7CIMgrQT+A/hu1ERSY7BAlWprBz4MfIywrpwMXAeMixlKUk9uxCSl5EjgLkKROo8wsMX1URNJjaPcw2BI1BRS2i4kXC6ykrDNuQ3YIWoiSV1YoEpKRfZo9u2EgS0ejppIaiyeQZU2zPXAwYQB9/YgjAp/aNREktZzIyYptu7XA/0cOBx4JmYoqQFZoEob7u+EA6H30HXcA0mRuRGTFFN2RMUicA7wTqAtZiipQVmgShtnIfAa4Cp6jhwvKRJXQEmxbAvcSpiTrg14N3A2oVCVtPEsUKWN9wJwPOEAafe5tyVF4EZMUgz7E7pV7QssIlz789OoiaTGZ4EqbZoi4QDph4C1wFsJ86ZOjphJyi03YpLq7S2EDf/WwD8I1wDdHTWRNDhYoEqb5weEMRCWAAcQDqTuEzWRlENuxCTV0+nAlcBI4EbCKIr/jppIGjwsUKXNdxvhwOlcYCrwZ8K0NJLqxI2YpHoYRhh84luE351ZhA3+ipihpEHGAlXqH48BBwF/JAzmdxVwWtREUo64EZM00MrD978X6AD+CzgFaI8ZShqEygXqkKgppMFhKXAM8EPCOnUh0Ao0R8wk5YIFqqSB9FLgduDVwErgWOA7URNJg8cBwLjMckfpv9ltewF4G6FbvaSNsxb4APAxwgGgmcB1wBYxQ0mDnQWqpIFyBHAXMB14nLAzfV3URNLg8g7gScL0GNvRtYvvWOBdhOvofkCleJW08S4kjOz7InAU4TrVaTEDSYOZBaqkTbUNcFaNx04Cfkc4u3MHoTh9qE65pLy4GFgDfJZwMOiS0v0HAI+WlqcDN5SeJ2nT/Ro4kHBQaE/C6POH1HjuZdgVWNpkFqiSNtX3gJO73VcgzCV3KTCUMGLv4cAzdU0m5cMjwLOEdW0yMKF0f0tpeTSwjHD2R9Lmu59wAGgOMBG4GXh3t+dMAt6PgypJm8wCVdKmmEEYhr+ZykTmo4HfEM6qFgndDt8BrI4RUMqJ79D7Oraa0B1RUv9YSBhX4WrCwaArgHMJB2gBjgTagE/jtarSJrFAlbQpfkA4SjyecJR4G+AWwiBIa4D3EM6kFuPEk3LjR8CqGo8VCdNjdNZ4XNKmeQF4M+FAbAH4b+DnwAhCYTqSMIL9ebECSo3MAlXSxnobYUAWCPObnkq4FmcGsIQwONJP4kSTcmcFcG+Nx5YQuuJL6n9FwoHYk4B1wImEdXFa6fGhwHHAThGySQ3NAlXSxmghHBHOTm0xgXAG9SHgldidUKq3bxCuNe1uBfBAnbNIeXMZ8DrCVGq70HX7OLH0uKSNYIEqaWN8jtBtqbsi4WyqoxZK9TebntehtgOXR8gi5dEIwpyp3RUII/6+tr5xpMZmgSppQ21F6M47sspjBWBn4HbCTvHL65hLyrtO4Bd0veZ7KeFacUkDYyjwJsJUapdT/eAtpftbcZ9b2mCuLJI21LepvQEum0S4DucqQkErqT4uBJZnlhcCT0XKIuXBqwiF6d5UpniqZWvglAFPJA0SFqiSNsTuhKHzh9R4fCVhrtNflZ43Dec+lerpcSrXoXYA342YRcqDPxGuMT0a+DWwmDC6bzVjCFOwja1PNKmxWaBK2hBXEqaUyVpHKELvAmYCLwHeQhgkyellpPr7GuFa1OcIXX4lDaxOwjbvBGAH4EOELr+LCdvIrK0I66ikPligSurLscC2meWlwDzgTMLgD/sT5n+rNkCEpPr5KWGn+AG6dveVNPBWEw7mHgjsC3wR+CdhmwlhrIbjge2jpJMaiAWqpN4MJQzu0Aw8DXyHUJDuBHwdeDZeNEndrARuAs6PHUTKuYXAucB0wtzgPy3dNwaYFTGX1BCcEkJSb7YHbgQuBf6CXXel1L01dgBJXdwLvIuwz3004VKYJkL3YElVWKBK6s1jwPtih5AkqcG1A9eVbpJ6YRdfSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJaE5dgDly6xZs/YpFovXx87Rm89//vOT1qxZUwBYvXr18vb29jWxM3WzBdBSaq8GVkbMUs0IYEyp3Q4sjZilh6ampiGjRo3asrx85plnLhk3blxnzEy9KRaLHzv11FOvrMdrtba2PkXCBy5bPs0SoAAAHehJREFUW1vH/fOf/xwGsHbt2hfXrFmzKnambkYCo0vtdcCyiFmqaQYmZJafBYqRstQykcp3cDmQ1O/viBEjtmhubm4B2G+//VafeOKJqf3+Zj1+yimnHFiPF2ptbf0M8PF6vNameOyxx4Zecskl48vLK1eufCZmnhomAYVS+3lgbcQs1YwDhpXaLwJJ/f4edthhI1//+tePBmhqamofNWpUUvse3Q0ZMmSPk046KemMMVmgqq46OjqGNTU1bR07R29WrVpFW1tbeXF8b89NwKjSLVXDgKQ+787OTlaurOxTFovFrSLG2RAj6/haU6jsICVn9erV2c9uNJViMEUtJPbdr2Jy7AB9SO73d/Xq1evb69atS/3394V6vVChUBhTLBaT/b53/90n/XVzQt9PiSq539/m5mbGjCkfG09v36O7jo6OZA8Gp8A3R5IkSZKUBM+gKqpCoTATaOvzifV1Rbmx3XbbXTd//vy6dK/cCBdSObOwAPhcxCzVfATYv9RuBz4YMUsPW2211V7PPPPMp8vLS5cuPXvChAnzYmbqrlgsfov4R9C/XigUHoicoYt169ZdTOmo/dSpU+ctWLDg7KiBevoUsHepvRY4KWKWag4ETs0sfwJYEilLLZdTOYt/A/DTiFl6mDx58oWLFy8eD/DCCy88VSgUPhs7U1ZnZ+dhhULh/ZFjzC0UCl+NnKGLZ5999ijg3eXloUOHnrRu3brUutBekWn/Crg6VpAavkvl8p3HgbMiZulh/PjxHwdeDrB8+fK148aNS+r3t1gsbgf8T+wcjcICVVG1tbVdedppp62InSOrpaVl/Q7S9ttv/9D8+fN/FDlSd+dSKVCXA6nleyOVArWTxPLttddex8yePXt9gXrffffd8PWvf/3OmJm6a21t/SrxC9TZM2fOvClyhi6mTZt2AaUCdeLEiUsWLFiQ1HcLeAeVArWdxL77hOtNswXqVYQdzZRcnmk/TGLv4RZbbPH/ygXqmjVrls+cOTOpfJdccsko4P2RYzyT2vty+OGHjydToB5zzDE/u+aaa16MGKmabIF6P4l994HzqRSoz5FYvjFjxpxIqUBta2vrSO07WBqDxQJ1A9nFV5IkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJcFBkjSYbAnsS5hMGmAx8Kd4cSRJkiRtDAtUNbKphFH5ZgCvAKZ1e/z3wJF1ziRJkqTG1gKMLLXXAKmN+jyoWaCqke0P/L/YISRJktSwmoFdCCc8srfhpce/D5wcJ1o+WaBqMGgjzBk2B1gEfDluHEmSJDWI04FvxA6hCgtUNbI7gH2AB4H20n3TsUCVJEnSxusE/gk8CxwcOUtuWaCqkS0s3SRJkqRNcT/wKeAe4F5gBXAUcGPMUHlmgSpJkiQpr35fuikRzoMqSZIkSUqCBaokSZIkKQl28Y3vDcCepfYcNqyLwUFULtyeB/xiAHJJkiRJUl1ZoMZ3IvDeUvvbbFiBegRwdqn9OyxQJUmSJA0CdvGVJEmSJCXBM6hKyRh6/04uq1cQSZIkSfVngaqU3AQcUOOxDvy+SpIkSYOaXXwlSZIkSUnwjJRSchqwRY3HivUMIkmSJKn+LFCVkrtjB5AkSZIUj118JUmSJElJsECVJEmSJCXBLr5qdIcCwzLLUzPtCcAR3Z7/L+DxgQ4lSZIkaeNZoDamQuwACbkS2LrGY/sCN3e77yzgSwOaSJIkSdImsYtvfB2Z9oYeMKg10q0kSZKkDTecMFtE9nZj5vGTqjz+3jpnzBXPoMa3MtPe0MJz14EI0qCOBoZuxPMXDlQQSZIkSZvHAjW+pzPtPTfg+ROAQwYoSyP6e+wAkiRJaljtwBkb+W/+NhBBFFigxndvpr0XsAvwSC/PPxMYNaCJJEmSpHxoB74WO4QqvAY1vluB50vtAvA9oKXK8wrAx0o3SZIkSRp0LFDjW00oSsteC8wBTiV05X0t8FHgDuCC0vOvrnNGSZIkSRpwdvFNw5eBI4FXlJb3oGvRWrYWeF/p8WPrE02SJEmS6sMzqGlYDRwOXAF01njOA4Qzqr+sVyhJkiRJqifPoKZjBeHs6BeBo4DtCfMyPQncCfw189yvAReV2uvqmFGSJEmSBowFanr+DVzax3PaSjdJkiRJGjQsUBVVS0vLrNbW1qTOAp9++umFcvv+++9/PTAlYpxqxmfa2wI/ihWkhv0z7WYSy3f77bdvk10+6KCDzjrhhBOei5WnhgmxAwCfaW1tfU/sEFnf/OY3R5fbjz/++E4k9t0CXpZpDyO9fDt0Wz4fWBUjSC8KmfYxwKRYQapZuHDh+t/f0aNHb9va2praZ/zS2AGAXVN7X+6///5dZs+evX75uuuu+z7QES9Rn04ApscO0c2YTHsHEvt9e+aZZ15ebm+xxRbDUvsOFovF8X0/S2UWqIrtbbED9GbFihV7EAalStU44N2xQ/SiicTyrV69ustyS0vL6yJFSd3hsQN0N2TIkPXt5cuXb0li361umkk7H8BxsQP0YbfSLRmrVlXq+WHDhm1B+p9xDFuR2PsyYsSILsudnZ3viBRlQ72Mrge8UpPc729bW6Vj4fDhw4eQWD5tHAdJkiRJkiQlwTOoqqvm5uZni8XiFbFz9Gb69OkHtLe3DwFYuHDh3Oeffz617p+7Eo5eAjwN/CtilmqmADuW2quBv0XM0sPw4cNH7LjjjvuWlwuFwj2FQmFNzEy9aWpqeqxer1UsFi9vampK9sDllClT9igUCuMAli1b9tSiRYueiBqop22BaaX2KuD+eFGqGk3XszJ3Au2RstSyP5V9k0eAJRGz9LDddtvtOnr06C0Bxo8f/3ShUEjt93e9YrG4uF6v1dnZeW9TU1Oy2/bm5uaxu++++17l5blz597e2dlZjJmpigOpdHF/CFgWMUs1u1O5xOgp4Il4UXpatWrVtg8//PA0gGHDhr2w88473xc5Uq/WrVvnWDK9KLS2tq5fQQuFwi4zZ858NGYgSZIkSVI+zJo1a3qxWHykvJzskXJJkiRJUr5YoEqSJEmSkmCBKkmSJElKggWqJEmSJCkJFqiSJEmSpCRYoEqSJEmSkmCBKkmSJElKggWqJEmSJCkJFqiSJEmSpCRYoEqSJEmSkmCBKuXbZcBvgO1jB5E0oFqA/wOuBoZHziJp4B1MWOe/EjuItLGaYweQFNUbga2AL8QOImlAFYC3ltpDgbaIWSQNvG0J6/wfYweRNpZnUKV8ay/914NV0uDWkWm7vkuDX3mdHxI1hbQJLFClfHMDJuVDe6bt+i4Nfh6AVsOyQJXyzQJVyodi6QbusEp54PZdDcsCVco3j7BK+VFe391hlQa/coHq9l0NxwJVyjePsEr54fou5YcHpNSwLFClfPMMqpQfru9SfnhASg3LAlXKNzdgUn64vkv54QEpNSwLVCnf3GGV8sMdVik/3L6rYVmgSvnmDquUH+6wSvnh9l0NywJVyjd3WKX8cFRPKT/cvqthWaBK+eYRVik/HNVTyg8PSKlhWaBK+eYRVik/XN+l/PCAlBqWBaqUb+6wSvlhjwkpP9y+q2FZoEr55g6rlB/usEr54fZdDcsCVco3d1il/PCaNCk/3L6rYVmgSvnmDquUH16TJuWH23c1LAtUKd/cYZXywzMqUn64fVfDskCV8s0dVik/vCZNyg+372pYFqhSvrnDKuWHO6xSfrh9V8OyQJXyzR1WKT/cYZXyoyPTdhuvhmKBKuWbgyhI+eEBKSk/2jNt13k1FAtUKd8cREHKDw9ISfmRPYPqOq+GYoEq5ZtnVKT88ICUlB928VXDskCV8s1r0qT88ICUlB928VXDskCV8s0dVik/PCAl5YddfNWwLFClfLNAlfLD9V3Kj06gWGq7zquhWKBK+eYZFSk/HCRJyhfXeTUkC1Qp3zyjIuWHgyRJ+eI6r4ZkgSrlmxsvKT88ICXli+u8GpIFqpRvtbr/FOodRNKAq9Wl3/VdGpxc59WQLFClfLkGOA0YWlrufnR1BPB5YF6dc0nqf3cA76Syre++vo8DLgTurHMuSf3vZcBfgL0z93Vf53cFbgc+V8dc0kbzomkpXx4GzgPOAO6h8hvwCuBGwgZuPPDbKOkk9afFwPcJ6/xtwNTS/UcCrwd2ASaUHpfU2B4E9gRmA88BN1E5GH0m8HJgIjAKODlGQGlDWaBK+fId4H3AFOBNmfv3y7SXlp4nqbGdBxwMbAOcmLn/8Ey7XMRKamztwM8JxedEwgGosndl2vMIxayULLv4SvkyH3iqj+esBv5chyySBtZfgBf6eM5zwON1yCJp4J0PLOnl8Q7gR3XKIm0yC1Qpf84HVtZ4rAhcS5jgW1Lju5zKQCndrQUuq2MWSQPrEUIvqFqeA35YnyjSprNAlfLnl4SzpNUsAb5XxyySBtbF1N5hXQb8pI5ZJA28i6i9jX8GeKJ+UaRNY4Eq5U8bcDPhbGl3LwL31zeOpAH0NLW78D5JuAZV0uBxBbCiyv1rgUvrnEXaJBaoUj5dQOjqk9UB/CxCFkkD6zx67rC+CHw3QhZJA2sVYZT+7uwxoYZhgSrl0xx67rA+h9ejSYPR1YSCNGslobu/pMHnPHoehH68yn1SkixQpfxqpevgKc8Bj0XKImngtAPX0bVb/wOEMy2SBp8/0fU61DacPk4NxAJVyq/LqJxV6QR+EDGLpIF1AZX1fS1hIBVJg1ORcC1q+aBUG3BVvDjSxrFAlfLrOeChUns5zo0mDWYPAs+W2iuBGyJmkTTwLgaeL7Xvo+85kaVkWKBK+fYlQje/eTiapzTYfRNYB/yRcBZV0uD1FPAvQs+Jb0TOIm0UC1Qp324iXJ92cewgkgbcjwhd/rwWTcqH8wiX8NwUO4i0MZpjB5AUVQdwInB77CCSBtxy4K3An2MHkVQXVxF6Sa2LHUTaGBaokm6OHUBS3fw2dgBJdbMW+F3sENLGsouvJEmSJCkJFqiSJEmSpCRYoEqSJEmSkmCBKkmSJElKggWqJEmSJCkJFqiSJEmSpCQ4zYzq6uyzz26aMmXK8Ng5enPDDTeMaG9vLwA8+uijax555JGO2Jm6GUZl3V1HevObNRMyQpggvC1ilh4mT55c2G+//UaUlw899NDVY8eOLcbM1Jvx48evOfHEE+vyHTz//PNHjBo1qlCP19oUt956a8vKlSuHACxcuHDdnDlz/O5vnAIwIrP8YqwgvRhByAmwhjBXczJe85rXDBszZkwzwJQpU9bNmDEjte/gesOGDev8wAc+UJfvYGtr61BgaD1ea1MsWrSoac6cOev3Pa655prUv/tthN+QlLQAQ0rt5PY9dt555+bddtttGMDw4cM7jzjiiNR+f7s45ZRTUvwOJsMCVXU1efLkVwB3xs7Rm+uvv562tqR/17QZFi9ezDXXXLN++aCDDoqYpm9Lly79IPC/9XitUaNGvUBlByk59957Lw8//HDsGMqxW265ZX370EMPZcaMGfHC9GHt2rX/Anaux2sVCoUvFovFM+vxWpti5cqVXX73NfjsuOOOvPGNb4wdY4O1trZOOuWUU5bEzpEqu/hKkiRJkpJggSpJkiRJSoJdfBVVoVA4ksSugyoWi7dR6ua46667/mTu3LkXR47U3dXAxFL7ceDdEbNU8yXg8FK7HXh1xCw97LjjjgfMmzfvm+XlhQsXnjxhwoSHYmbqrlgs/gbYKnKMjxcKhbsiZ+hi7dq1vwO2ANh5550feuyxx06OHKm7bwIHlNptVNaDVBwNfDGz/BZgUaQstaz//QV+Dnw7YpYepk6detWCBQsmAaxYseKJQqHwrtiZsorF4nHApyPHuK9QKHwkcoYunnzyybcCHysvDx8+/LC2trY1ESNV85dM+1Lgh5Fy1LL+9xeYC3woYpYeJk2a9HXgIIClS5eu2XLLLQ+LHKmLYrH4UtL7TJNlgaqo2tra7jrttNNWxM6R1dLSsr49adKkhXPnzr09Ypxq1mbaLwKp5Xsu0+4ksXw77LDD2Hnz5q1fnjt37j8uuuiipK6Lbm1tXdv3swbcQzNnzkzqs5s2bVp7uT169OhVJPbdApZl2sl994Eduy3/jXCQK1WLSOw9HDly5Pp1s729/cXU1pFLLrlk70Ih+mXkK1J7Xw4//PBXZJePPPLIOxMdKKnsSRL77tN1UKTkfn9HjBix/vd33bp1nal9B2fNmvVisZjseIzJsYuvJEmSJCkJFqiSJEmSpCRYoEqSJEmSkmCBKkmSJElKgoMkabAYDuwNzADGlu57ArgyViBJkiRJG8cCNdgD2AuYAiwH/gncQZgiY1PtVfq7k4AisBi4E5i/WUmVtQvwKUJRuicwtNvjv8cCVZIkSRuuAOxM5YTHEuDf8eLkT3918b2UUIQVgUs28N98NvNvbu6nHL15PvN6x5TuezVwN/AA8DPgfOAy4E/AU8DGzrE3Cvg8sAD4e+lvXkSYx+3/CF/uuwhz0dXSAryQyXpAL8/dgjDsd/m5LxLOJNZyeOa5zwFD+vofStyewEnAy+lZnEqSJEl9mQy8HTgP+COhZngUuKd0OzNetHzK8xnUjwLfonaRthUwC9ieDfti7gFcC0zr43mvBG4Avg6cQSgWs9YQJio/qrR8OPDXGn/r1XT9DEcABwJ/qPH87KTxtwAdfWRtBJ2EM95zSrfncCJkSZIkbZh3A9+IHUIVeS1Q3wB8mHAG+U7gKkLX26HAq4D3Eoo9gM8BNxHOqtayB/BnYHxpuRO4Grix9HebgN2A95eeC/AZwpnSL1X5e7OpFKhHAF+p8bqHV7nvCDasQJ1d4zmN5PfAOGBl5r7pkbJIkiSpsS0mnPBYBZwYOUtu5bVA/SjhTOVJwI+7PXY58F3gVkLBWQA+Te0CdQThOsdycfoEcDxwb7fnXQdcQDhC87HSfV8knHX9W7fnZgvMVwEjCd13uysXnKsJZ4KHUb1ohVDIzajxGo1qeewAkiRJami3AMcRCtMFpfuOwgI1mjxPM3MaPYvTsn8A52SWjyEUidWcTOWs6ErCF7p7cVrWAXyCUKxCKCo/U+V59wJLS+0W4KAqz9ka2L3Uvo1wJhhCETquyvNfTaU781PA3BoZJUmSpLyYQ+j5uKCvJ6o+8lqgziUM7NSbn2bazYQpTLorAKdnls8nXA/ZmyLwhczym+lZ/HYQjuaUHVHl7xxWen0IXV3LXXaHAK+p8vzB1r1XkiRJ0iCT1wL11/QcnKi7Z4GFmeWpVZ6zC7BjZvlHG/j692b+9jDCwEndZbvgVuu2273gnF3jsWr3DYbuvZIkSZIGmbwWqA9s4POey7THVnk82/V2GfCvjcgwL9Peqcrj2YLz5cCEbo+XC86lhIL3TsIF3dnHyqYQBmmq9rclSZIkKQl5LVA3dHCddZl2tXk2s2dVxxKKxXYqc432djs482+7F58QuiE/VWo3Aa/NPLYTYfobCPM1dZaylgdy2g3YNvP8w6l0B34E+9hLkiRJSlBeC9TOfvo72cJyCGEk31rzqvZmRI37a3XbzbZ/X+P5h9V4vmdPJUmSJCUpr9PM9JeOTPspao8K3Je/1Lj/D4Q5WaF2gTq7RvsIKtfEZovVlK8/3RUYVeOxIj2n45EkSZI0iAxEgVro+ykADB+A1663JZn2WuCMfv772YJzOvASQvfccnff+XQdNfjvhMGdJlEpSl8KbFdqdxK6BKfqf4EDajzWgQdUJEmSpEGtv7r4vphp1+qu2t02/fTaMT2WaW8PTOznv7+AcM1o2eHAywgFKPTsrlukcoZ0KuGMZPZsa3Z+VUmSJElKSn+dkcoOOrRdzWd1dXDfT0neHwlFYYFQ7B8HfL+fX+MPhOlsIHTbzV73Wu160tnA20rtw+k6J2rK3XsB3gK0xA4hSZIkKY7+KlAfyrRfQbiO8IVenn8E4exeo3sWuBk4qrT8WeCndD2jvLlmA/9Zah9GpUAtUrtALTuSrgcCUh8g6am+nyJJkiRpsOqvLr5/oTIy7ijgo708d2tgVj+9bgrOIRSLADsSBiba0OtrW4BX9/Gc8jQyEOYzPbLUfhB4usrz5wGPl9pvALYstdcCt21gLkmSJEmqu/4qUJ+k63Qn/wOcBgzL3DeU0IXzr8AOVC+uGtHtwNcyy8cT/h/fRPUz1EMJAwGdBzwBfKGPv7+UcO1oWflv9nY2tPxY9vXvoPez2pIkSZIUVX+OivpJYA6hKG0GLiQUqo+WlncExpSeewPhbN7/9OPrx3QmYYCkk0rLLwN+SygIHyQUmcMIgxvtQtfC/cEN+Pt/AGZ0u6+vAvWkbvelfv3ppno3MDKzPDnT3haY2e359+B0NZIkSVKS+rNAfQA4Afg5lbksx9CzsPopcArwX/342rF1ACcTzlL+D6ErLoT3Yb9e/t0K4NYN+PuzgU9nltv7+HfZwZuyf2MwOo/Qbbya3YDWbvedhQWqJEmSggKhd2dWdt9yLOFEW9YzwKqBDJVn/T2v5LWEM4SfAI4BphHOFi4C7gR+AFxfeu4dVLrGPsbA+xaVa0PnbeC/+SFhECSA+zbg+T8gFOBvI1wruh/hCz4GWEM4k/pI6W/NLt1Wb8DfvY2u3YifIRS3tSwGPgeMKy0Xgbs24HUkSZKkPGkB/tXL4yeWblnvA64YsEQ5198FKoSRWD9ZuvXmltKtXs7ehH/z7U34N23A5aVbf3kBOGMj/825/fj6KduVjbuWekMOCEiSJEmKYCAKVKmelvf9FEmSJKmqdfQ8Q9oXeyYOIAtUSZIkSXnVAfwidghV9Nc0M5IkSZIkbRYLVEmSJElSElLs4rsvG98PvJbzCSPeSpIkSZISl2KBuhfw3/30t67AAlWSJEmSGkKKBeoaYFk//a2Ofvo7kiRJkqQBlmKB+vPSTZIkSZKUIykWqMqR4cOH39za2prUme7TTz+9UG7ffffd7wIOjhinmomZ9g7A7bGC1DA90x5KYvn+9Kc/bZFdPuSQQy494YQTVsXKU8NWsQMAF7S2tiY1z/AFF1yw/rObO3fu7iT23QJ2zbSHk16+id2Wf0notZSSQqb9duCAWEGqmT9//qRye+zYsTu0tram9hlPjh0A2Ce19+XBBx+cMnv27PXL11577R+AzniJ+nQy8LrYIboZl2nvSmK/b4sWLVr/+zthwoSW1L6DxWJxZOwMjcQCVVEVi8X9YmfoTVtb2zbANrFz9GIk8KrYIXpRILF869at67I8dOjQvSJFSd3usQN0VyhUape2trbRJPbd6qaJtPNBGJQwZVNKt2S0tbWtbzc3N48g/c84hrEk9r4MHTq0y3KxWNw/UpQN9ZLSLVXJ/f5mt+1Dhw5thN9f9cJpZiRJkiRJSfAMqupq2LBhCzo6Os6KnaM3++yzz8Hr1q1rBpg/f/4Dzz777JLYmbrZAyh3M3sK+GfELNVsQ6Wb74vAXRGz9DBq1KiRu+666/oz901NTXcUCoXUujmuVywW/1bHlzurkD1NmZhp06btPXLkyAkAzz333PwnnnhiXuxM3bwE2KnUXgnMiZilmjHAjMzybUB7pCy1HES4NADgQeDZiFl6mD59+u5jxozZCmCrrbZ6qlAopPb7u16xWFxar9fq6OiYPWTIkHV9PzOOoUOHjpsxY8Y+5eV777331s7OzmLMTFUcSuXE0d+Bun1+G2hvYEKpPR9I6vd3xYoVL5kzZ85OAC0tLav22muve2Jn6s2qVateiJ0hZYXW1tb1K2ihUNhl5syZj8YMJEmSJEnKh1mzZk0vFouPlJft4itJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkNGcXisXitFmzZsXKIkmSJEnKkWKxOC273Nzt8RuLxWL90kiSJEmSVGIXX0mSJElSEixQJUmSJElJ+P/lhrmotJ793gAAAABJRU5ErkJggg=="
+ }
+ },
+ "cell_type": "markdown",
+ "id": "756b8d01",
+ "metadata": {},
+ "source": [
+ "\n",
+ "![](\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA6gAAAFzCAYAAAAzJyTtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d15mBxVvf/xd08mmewkISEEIoTFsCMSBWRT2d0uCIr7DoGrXnC/qCigV38oCoKKTBCv4Mp1AwHZjIIiyBIBZQmIAUNICISELJBJMjP9++N0p2tmumeyzPQ5PfV+PU8/nOruTH/o7uqqb9Wpcwqtra1FJEmSJEmKrCl2AEmSJEmSwAJVkiRJkpSI5m7LRxcKhSdiBJEkSZIk5UuxWJwG3Fhe7lKgFgqFJ2bOnPlovUNJkiRJkvJn1qxZFIuVYZHs4itJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSkJz7ACS6ueyyy4b097e/ufYOXpz8cUXb7948eIWgFWrVi1ZtWrV0tiZupkITCi1XwQWRMxSzUhgamb50VhBejE9015AeB9TMpXwPgIsBZZEzFLNBML3EKANmB8xSzXDge0yy48BnZGy1LIzlYP0C4FVEbNUsw0wutR+HngmYpZqxgFbldprgSfiRempubl52MSJE6eVlz/+8Y/PGzduXHvESL0qFotfOPXUU6+JnUNKhQWqlCNNTU1DgJfFztGbZcuW8fTTT5cXX1K6pWoEsGXsEH1I+vMGXho7QB+2Ld1SNQIYHztEH/aKHaAPO8UO0IcRwJTYIXoxgsR+Z9rb27PbEYrF4h4R4/SpUChM6PtZUn7YxVeSJEmSlATPoEo5ViwWL2xqakqqi2pnZ+dXgaEAU6ZMeWTRokXfjxypu/+i0n3xReCsiFmqeS3w+szyp2MFqaEJ+Fpm+Vrg1khZavkS4awQwL+B70TMUs1MKmee1wKfj5ilmv2At2aWvwysiJSllvMy7T8Cv4sVpIYvAmNK7UXA+RGzVPMeYO9SuwM4I2KWHiZMmLDr0qVLP1ReXrly5UUTJkx4Mmam7orF4heAsbFzSCmyQJXy7cczZ868J3aIrClTppxDqUCdOnXqvEWLFn0jcqTu3kKlQG0DUss3hK4Famr5mulaoN4CfDNOlJo+R6VAXUh67+HRVArUdaSX74N0LVC/RyiyUvJ1oFBq30V67+EnqRSoS0gv3350LVCTyjdjxozX33zzzesL1Dlz5vz4K1/5yt0xM3XX2tr6cSxQpars4itJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokjZXM7AX8AHCfJF/Bu4p3T5Z5ywFwjygrcADwEqgE3gauAb4jzrnqWYL4GTgt8ACYA2wGngE+DawQ7xo6+0BnAvcASwFisAywmf7n8CweNGAMA3RccCPgccI89GuA54Efgq8Kl609aYAnwJuBhYD7cALwH2EeVYnxou23kGEdfZ+wjylReAZ4HrgRCrTsMQyGngf8GtgPmFapzbCZ34JsEu8aOtNJ3yetxGmgykCy4HbgY9Tma4oliHAG4AfEn5jXiB8F58CfkH4vYxtEnA64Xv3NGFdfhH4B+F3aOtIuYYC+xJ+ry8hfKblbdsHImWScsF5UCVtjv8kzH83ssbjf65jlh2AK4FXVnlsMvDG0u3HwPsJc/fV2/uB84HxVR6bXrp9AHgXcHX9Yq3XAvwEOJ6exck44ODS7YPA6wg75PX2KuBHwE5VHpsKvAN4O3A2oXCI4RzgvwnvZ9ZI4GWl28mEwuFv9Y0GwDaEdeXgKo9NAo4p3d5JKFTX1i/aem8FvlvK091Opdv7gA8RDkrUW4HwPXwHPQ/2jyV8T19FyHcUYT7detuH8N7sVuWxbQhzOr+FMA/xp+qYK+tThHV1VLf7m4E9S7eTCAekbqtjrjNLt+7rcNmUOmaRcscCVdLmmEzP4vR5QjFTb7tRKU4fBq4lnGkZCxxGKKgA3l26/5x6BwTeRChO1wG/B24hnKHcHngvsB1hR+3nhB2zf9U530jghFL7aUKR/I/S8isIhfPQUvtHVN7TejqISnF6J3Aj4czpJMJO7H6E4uEcwvfgFxEynkjYsV0F/I5wJno1lQMQ4wlnhX4L7Fp6Xj3tQKU4/Rfhc36U8N07BDiW8B4eSziD9Yk654NQIE8iHEj6Q+m2hHAQ4l3AzsBw4HLgQcJZ4HoqlHJQynV1KUMH4QDE+wjfgT0I38GDCWdX6+mVVIrTewnfxSeALQkHRw4pPfZJYC7w/Trng/B7M4qwflwP/IWwPuxIOBA2iZD3KsK6Uq+DYlPoWpwWCb0MtqjT60u5ZoEqaXM8Q+h+N6d0uwd4DfDLSHn+AXwGuKHb/d8APkLozgihO9lXCF3d6qkIXAr8D6HLYtZXCUXrgYQd748QpzB4hnDm4HJ6njlrJRTVLYQCYk9CV+p6mw2cQfi+ZZ0LXAR8tLT8SeIUqGsIn/EFhAMQWV8H7iIcjNiWcLY3RmHwKGFduYbQDb7sfELh9ePS8qnAFwhdQ+vtx4Sza90P1HyVkPsown7M6YRipt6WAmcBlxEKrKzvEoqt0YR1+sDScr39hbCudD/7+DXCb+DnSsufJM73sJ3w+3we4bcn61zgr4Su3FsSeqB8o065FhJ6GZS3bX8jnC2/uE6vL+Wa16BK2hwXE46Af5VwJuu5iFn+CsygZ3Fa9l3g36X2eMIR+nr7CDCTnsUphB3cszLLM+qSqKsXCGdcLqV6t86/Eg5IlO1bj1DdXAEcQc/iFMIBgM9RKbheTpzrKI8mFHXdi1MI16N+M7Mc43N+kHCW72q6FqdlP6FyRnIE1buIDrTPAe+hei+CtYSDKGUx3sMi4X35Dj2LU4C/UynyIc66cjVwKLW7xp5NuNYTwtnJ7t1s6+EtwKfpWZxC6I3z/zLL9fycv0I4eHQe4ez983V8bSn3LFAlDRZLCV1ne/NUPYL0YnEfj2fzxSis1lK9qMqKnbHajmzWSkJXvLIYGfv6nBdk2jHyPU8YbKg3sT/n1NeV8oBSvYmd8RmqH4AoW0fX/wc/Z0lJsECVlBfbUDkC/yzweMQstbwp074zWorahgGvzyzfFStILw6hcg30PfS+gx5LdjTpFD/niVSuUV0FPBQxSy2prysFwqBsZSlm3Idw/TuEs+r1vhZ6Q6T+OUsaABaokvKghdDdrjzoxdfp+2xrvb0C+GKpvYLK9bIp+Tawe6n9a8IgRCmZRLgesOyrsYL04q2EAbEgdF+9MmKWaoYC/0cYXAzgW8S5/rQ3u1Pp+tlGuG42NWcC+5fas0mvuJpAuD67fFbyKxGz1HI08OFSeyFhqhxJOWCBKmmwawF+RWW+v6tIb4d2X8I1vKMIo4B+gMr1sqn4EuH6WQgjfp4SMUs1kwhzjr60tPw14Lp4cao6lnCgpEA4CPE2KtcApqCZMDpzeV35PfDleHGq2o2Qazyhm+2ppHeg5JNUpjj6N2FE35RsQVg3di4tXwL8LF6cqg4nHARrJlzj+3bCXMyScsBRfCUNZkMJOznlbqk3EkZiTKnb58uo7HB3EOZN/HWv/6L+ziYM+gPhrN9RxJkDtZYJhLNUe5WWLwY+Gy9OVW8inLEaSuhK+SbC6KCpGEIont9WWr4NeDNx5kCtZTrwR8L0VkXC6L2XR03U08epjDS7gDCgV+xr37NGE34Hy2d3f0wYvC0lryWM0jyCMCL2W6jvnNqSIrNAlTSYtVIpTq8HjqfvwWHqaRvCqMPjCUXz++k68mcKTqYyuvBjhJ3HBbWfXnfNhJ3ZcnH6HeA06j/nZG9eSejKO5TQXfZ11B5ZNZZvUClO/0SYJzOlaxInADcRilOA/yKMzJ2St1EZoXkBYcqtes9l3JsC4SBJuTi9gjA9T0oH7HYnjD5cLk6PJRTUknLELr6SBquDCV1lAe4mHIVPqTiFMFfm1qX2p0ivOB1H5WzQEsI1YSkVpxB2sA8sta8kveIUwrW7IwiFwFtIrzjdm/C+QRgs5z9IqziFcAa/PKDPl0ivOB1O+JzL3bePIa3iFML1z8eU2tcTemt0xItT1QXAmFL7fVicSrnkGVRJWV8ijNRazSLgwjpm2VzHZtqfJa1r/cqOK/13Hmm+t4dRGSznQkLO1GQ/50+QXnE6hcoZqxuoPU9vTP9B5YD1WcDyiFlqKX/OzxIO7KTmIMJ10BDmEX4wYpZajsu0Pwm0xwpSwxjCtacAt5PeAGKS6sQCVVLWJ6g9Wfv9pFlE1fKSTDu1QVQgvM/jS+1HSKubXdnUTDvF9xAqGZ8njPSZmkZ6DyHNKWUKVDI+RnojcEP67yFUMnYSBjpLzdaEa6Eh3XVFUh1YoErKupfQFbGaf9YzSD9oybTXREtR2/BMO7Wux2WNlDHFzxga4z3MrispZmwiXL8LaeaDxvqc15JeTwNojPdQUh1YoErKOiR2gAGS4s5YVur5GkEjvIeNkDF1voebrxHew0bIKGmAWKBKGqxOJVxnBaH7Z2qeB3YqtV+IGaQXs4BfltpPxwzSiyMIZ9dSG+yl7G4qn3OK30OAz1CZ7/TJmEFq6KDyHq6OGaQXPyPMwwvwTMwgvTiecBY11eLvESqf84qYQSTFZYEqabBaHDtAHzpIc9ChrOdJt6gqS7Ggymoj/c/52dItZam/hytIv6hKaT7WataS/ucsqQ6cZkbSYHUh4YzGzXQdMCkV21HJd0HkLLXMpJLx6MhZavkNId9NsYPUcAiV9/BjkbPU8hUqGXeNnKWaLankuzRyllreRSXj8ZGz1PJjKhlrjTUQ075U8p0ZOYukiDyDKmlzbEGlC2jZVpn28cCe3R4/h/rMA/kKKvNj1hqZOKZRhO6p0HVwkJS8lErGn8YM0ovXEOZrTXEUZAjrQ/k9TPXs0D5UMm4RM0gNLVTyPRAzSC92pJLxuphBenEQMK3UTnH/bwKV93BpzCCEOXe/3+2+7EjNHwBe2+3xj5Pu91NqKCn+QElqHMOo7FBUs13plvW9gYsjSdJmG03v27adS7escQMXR8oXC1RJm2Mt8PuN/Df1GkBkJbCs1E5xAJ0OKvlWxgzSi9VUMqY6jcvzhEFfUj2DupbKe/hizCC9WEUlY3vMIDV0UsmX6nWe2XUl1SlSllPJmOJASeuo5Is9cNwqNn7blvr1+lLDsECVtDmWA0fGDlHDMbED9OFRQpe2lH2xdEvZDrED9OEa0v+c3xY7QB+eJv338BulW8r2iR2gD7eSzuf8b9LdtkmDnoMkSZIkSZKSYIEqSZIkSUqCBaokSZIkKQkWqJIkSZKkJFigSpIkSZKSYIEqSZIkSUqC08xIOVYoFE5vbW1dHDtH1pe//OWh5fbjjz++K+lN3bBdpj2S9PLt1205tXyFbstvBKbECNKLkZn29qT3Hr400x5Gevn27Lb8ReLPa9ld9nv4WtJ7D8dk2lNIL9/emfYQEsv317/+tcv0U/vvv/9pb37zm5Pa1gFjYweQUmWBKuXbu2MH6K6pqdKxY8mSJTsAn4yXpk/DSTsfpJ/vNaVbqrYh7fdwKGnnAzg1doA+7EfPAzspmUjan/EQEsu3cuXKLssjRoxIblsnqTa7+EqSJEmSkuAZVClHhgwZsq5YLN4YO0dvJk+evFtzc/NIgGXLlj21bNmyp2Nn6mZbYOtSewXwz4hZqhlL1y6gc2IF6cWMTPtRYGWtJ0YynUoXy6eBpyJmqWZrwvcQQtfZuRGzVDMK2DWzfB/QESlLLfsQzvwB/At4PmKWanYCxpXazwLzI2apZhKVyx3agAcjZumhpaVl+LbbbrtH5q5/FIvFtdEC9aGpqSm13xgpKgtUKUfe+973vgAcEzuHJEkDZc2aNcybN2/98hlnnBExjaSNZRdfSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSaoff3M1kP4ELAVeEzmHBhd/tyTVlT86klQfw4GfAJ+KHUSD1hhgPDAsdhANGk3Ar4CzI+eQlCMWqJI08LYBbgPeDnyptCz1t47Sf4dETaHB5FjgOOAs4H/x4IekOrBAlaSBtTdwBzADeA54HbAwaiINVp2l/7ptV3/5DXAKsA54P/BHYFLMQJIGPzdikjRwXgf8GdgO+CdwIHBr1EQazCxQNRBmAW8Anif8ht0B7BY1kaRBzY2YJA2M04FrgbHA74H9gEejJtJgZ4GqgXIzcDDwOLAT8BfgsKiJJA1absQkqX81A98FvkX4jf0+8HrC2QdpIFmgaiA9CLySMFr0eOBG4MNRE0kalNyISVL/ye60dQBnACcTrt+SBpoFqgbac8BRwI+pHIy7EL9zkvqRPyiS1D92pNLtbRVwPPC1qImUNxaoqoc1wHsJB+A6gdOoXM4gSZvNjZgkbb7swCFPAa8Gfhs1kfLIAlX1UiQcgHs7sJquA8JJ0mZxIyZJm+ftwGxgK+A+4ADgb1ETKa8sUFVvvyD0GllMmFLrr8AroiaS1PDciEnSpikAZwM/A4YDvwIOAhZEzKR86yj9d0jUFMqbclF6HzAFuAV4c8xAkhqbBaokbbzhhEFCziotXwScCLwYLZHkGVTFswA4lHAt6ijCAbuzYwaS1LjciEnSxplCmGbhncBa4P2EOU87e/k3Uj1YoCqmlcBxhAN2BcIBvMuAYTFDSWo8bsQkacPtRRgM6ZXAUuBo4PKoiaQKC1TF1kE4YHcK0A58EPgDMClmKEmNxY2YJG2YY4DbgO2Bxwgj994SM5DUjQWqUjELeAOwnHBt/h3ArlETSWoYbsQkqW+nU5nnbzbhDOojURNJPVmgKiU3AQcDTwA7EeaJfm3MQJIagxsxSaqtGfgO8C3CyKiXEeb7ez5mKKkGC1Sl5gHCAb0/AxMIRet/Rk0kKXluxCSpujHA1cBHCJPSnwOcBKyLGUrqhQWqUrQEOBL4CeGg38XAhfg9lVSDPw6S1NOOhLn9Xg+8QJjT7+yYgaQNYIGqVK0B3kM40FcETgN+CYyMGUpSmtyISVJXryIM6LE7sBB4NeFMqpS6coE6JGoKqboi4UDfO4DVhAN/twMviZhJUoIsUCWp4m2EKRG2Au4HDgDmRE0kbbiO0n/dtitlVwKHA88ALyP0VpkRNZGkpLgRk6QwqfzZwM+A4cCvCdPIPBkxk7Sx7OKrRnEH4QDgQ8A2wK3AcVETSUqGGzFJedcC/Ag4i1CoXgS8FXgxZihpE1igqpE8TihSrwNGEQ4Mnh0zkKQ0uBGTlGcTgd8D7wLWAh8gzHna2ds/khJlgapGsxI4ljCdV4FwoPD7wNCYoSTF5UZMUl7tCdxDmEh+KXAM8MOYgaTNZIGqRtQB/BdwCtAOfIgwFsDEmKEkxeNGTFIeHQ3cBmwPPAYcBPwxaiJp81mgqpHNAt4IrCAcOLwD2CVqIklRuBGTlDczgWuBLQhF6quAuVETSf3DAlWN7kZCcfpvYGfCNDSviRlIUv25EZOUF0MIAyC1As3ADwhTHSyJGUrqRxaoGgz+QThweBcwgVC0vi9qIkl15UZMUh6MAa4mXOdUBM4hXOe0NmYoqZ9ZoGqwWAS8GvgpMIwwPsCF+N2WcsEVXdJgtwNhIvg3AC8Ax+NUBhqcOkr/dduuwaANeDfhgGIROA34P2BkzFCSBp4bMUmD2QGEgTZ2BxYSrmW6KmYgaQB5BlWDTZFwQPGdhIL1BOAvwNSImSQNMDdikgarEwlTFUwG/k64pumeqImkgWWBqsHq54QxA54B9iH0itk3aiJJA8aNmKTBpgD8N2GHZgRwPWFUyPkxQ0l1YIGqwex2woHGh4FtgVuBY6MmkjQg3IhJGkxagCuAcwmF6kWEefVWxgwl1YkFqga7ecD+hAOPo4FfEQ5IShpE3IhJGiwmAjcTBtVoBz4MnE5lp10a7CxQlQcrgf8AvkuYPuxc4FJgaMxQkvqPGzFJg8GewN3AIcAy4Gjge1ETSfVngaq8aAc+CnyMMHr1ScDvgHExQ0nqH27EJDW6o4DbgGnAv4ADCYMjSXljgaq8uZBwGccK4AjgLmB61ESSNpsbMUmNbCZwLbAFYeqBVwFzoyaS4ikXqEOippDq6wZC75n5wEsJgym9OmoiSZvFAlVSIxpCOHLeSrju6H+Bw4BnY4aSIvMMqvLq74R5r+8GtgRuBN4bNZGkTeZGTFKjGQ1cBZxGmMT9HOCDwNqYoaQEWKAqzxYBhwI/I4zofjnhQKbrg9RgXGklNZKpwJ8I1xy1Ae8Czo4ZSEqIBaryrrxdOKe0fBpwJTAyWiJJG82NmKRGcQBwD/Byuh4plxRYoEqhZ83ZVHrWvIUwcN7WETNJ2ghuxCQ1grcSdjAm0/VaI0kVFqhSRXZsgv2pHOCUlDg3YpJSVgD+G/g5MIKuozVK6soCVeoqO7r7toRLRN4UNZGkPrkRk5Sq8iAX5xJ+qy6iMt+dpJ4sUKWesvNjjwZ+QzjwKSlRbsQkpWhL4CbgPUA78FHgdKAjZigpcRaoUnXLgKOB7xGmKTsXmEWYpkxSYtyISUrNS4E7CIMgrQT+A/hu1ERSY7BAlWprBz4MfIywrpwMXAeMixlKUk9uxCSl5EjgLkKROo8wsMX1URNJjaPcw2BI1BRS2i4kXC6ykrDNuQ3YIWoiSV1YoEpKRfZo9u2EgS0ejppIaiyeQZU2zPXAwYQB9/YgjAp/aNREktZzIyYptu7XA/0cOBx4JmYoqQFZoEob7u+EA6H30HXcA0mRuRGTFFN2RMUicA7wTqAtZiipQVmgShtnIfAa4Cp6jhwvKRJXQEmxbAvcSpiTrg14N3A2oVCVtPEsUKWN9wJwPOEAafe5tyVF4EZMUgz7E7pV7QssIlz789OoiaTGZ4EqbZoi4QDph4C1wFsJ86ZOjphJyi03YpLq7S2EDf/WwD8I1wDdHTWRNDhYoEqb5weEMRCWAAcQDqTuEzWRlENuxCTV0+nAlcBI4EbCKIr/jppIGjwsUKXNdxvhwOlcYCrwZ8K0NJLqxI2YpHoYRhh84luE351ZhA3+ipihpEHGAlXqH48BBwF/JAzmdxVwWtREUo64EZM00MrD978X6AD+CzgFaI8ZShqEygXqkKgppMFhKXAM8EPCOnUh0Ao0R8wk5YIFqqSB9FLgduDVwErgWOA7URNJg8cBwLjMckfpv9ltewF4G6FbvaSNsxb4APAxwgGgmcB1wBYxQ0mDnQWqpIFyBHAXMB14nLAzfV3URNLg8g7gScL0GNvRtYvvWOBdhOvofkCleJW08S4kjOz7InAU4TrVaTEDSYOZBaqkTbUNcFaNx04Cfkc4u3MHoTh9qE65pLy4GFgDfJZwMOiS0v0HAI+WlqcDN5SeJ2nT/Ro4kHBQaE/C6POH1HjuZdgVWNpkFqiSNtX3gJO73VcgzCV3KTCUMGLv4cAzdU0m5cMjwLOEdW0yMKF0f0tpeTSwjHD2R9Lmu59wAGgOMBG4GXh3t+dMAt6PgypJm8wCVdKmmEEYhr+ZykTmo4HfEM6qFgndDt8BrI4RUMqJ79D7Oraa0B1RUv9YSBhX4WrCwaArgHMJB2gBjgTagE/jtarSJrFAlbQpfkA4SjyecJR4G+AWwiBIa4D3EM6kFuPEk3LjR8CqGo8VCdNjdNZ4XNKmeQF4M+FAbAH4b+DnwAhCYTqSMIL9ebECSo3MAlXSxnobYUAWCPObnkq4FmcGsIQwONJP4kSTcmcFcG+Nx5YQuuJL6n9FwoHYk4B1wImEdXFa6fGhwHHAThGySQ3NAlXSxmghHBHOTm0xgXAG9SHgldidUKq3bxCuNe1uBfBAnbNIeXMZ8DrCVGq70HX7OLH0uKSNYIEqaWN8jtBtqbsi4WyqoxZK9TebntehtgOXR8gi5dEIwpyp3RUII/6+tr5xpMZmgSppQ21F6M47sspjBWBn4HbCTvHL65hLyrtO4Bd0veZ7KeFacUkDYyjwJsJUapdT/eAtpftbcZ9b2mCuLJI21LepvQEum0S4DucqQkErqT4uBJZnlhcCT0XKIuXBqwiF6d5UpniqZWvglAFPJA0SFqiSNsTuhKHzh9R4fCVhrtNflZ43Dec+lerpcSrXoXYA342YRcqDPxGuMT0a+DWwmDC6bzVjCFOwja1PNKmxWaBK2hBXEqaUyVpHKELvAmYCLwHeQhgkyellpPr7GuFa1OcIXX4lDaxOwjbvBGAH4EOELr+LCdvIrK0I66ikPligSurLscC2meWlwDzgTMLgD/sT5n+rNkCEpPr5KWGn+AG6dveVNPBWEw7mHgjsC3wR+CdhmwlhrIbjge2jpJMaiAWqpN4MJQzu0Aw8DXyHUJDuBHwdeDZeNEndrARuAs6PHUTKuYXAucB0wtzgPy3dNwaYFTGX1BCcEkJSb7YHbgQuBf6CXXel1L01dgBJXdwLvIuwz3004VKYJkL3YElVWKBK6s1jwPtih5AkqcG1A9eVbpJ6YRdfSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJcECVZIkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJaE5dgDly6xZs/YpFovXx87Rm89//vOT1qxZUwBYvXr18vb29jWxM3WzBdBSaq8GVkbMUs0IYEyp3Q4sjZilh6ampiGjRo3asrx85plnLhk3blxnzEy9KRaLHzv11FOvrMdrtba2PkXCBy5bPs0SoAAAHehJREFUW1vH/fOf/xwGsHbt2hfXrFmzKnambkYCo0vtdcCyiFmqaQYmZJafBYqRstQykcp3cDmQ1O/viBEjtmhubm4B2G+//VafeOKJqf3+Zj1+yimnHFiPF2ptbf0M8PF6vNameOyxx4Zecskl48vLK1eufCZmnhomAYVS+3lgbcQs1YwDhpXaLwJJ/f4edthhI1//+tePBmhqamofNWpUUvse3Q0ZMmSPk046KemMMVmgqq46OjqGNTU1bR07R29WrVpFW1tbeXF8b89NwKjSLVXDgKQ+787OTlaurOxTFovFrSLG2RAj6/haU6jsICVn9erV2c9uNJViMEUtJPbdr2Jy7AB9SO73d/Xq1evb69atS/3394V6vVChUBhTLBaT/b53/90n/XVzQt9PiSq539/m5mbGjCkfG09v36O7jo6OZA8Gp8A3R5IkSZKUBM+gKqpCoTATaOvzifV1Rbmx3XbbXTd//vy6dK/cCBdSObOwAPhcxCzVfATYv9RuBz4YMUsPW2211V7PPPPMp8vLS5cuPXvChAnzYmbqrlgsfov4R9C/XigUHoicoYt169ZdTOmo/dSpU+ctWLDg7KiBevoUsHepvRY4KWKWag4ETs0sfwJYEilLLZdTOYt/A/DTiFl6mDx58oWLFy8eD/DCCy88VSgUPhs7U1ZnZ+dhhULh/ZFjzC0UCl+NnKGLZ5999ijg3eXloUOHnrRu3brUutBekWn/Crg6VpAavkvl8p3HgbMiZulh/PjxHwdeDrB8+fK148aNS+r3t1gsbgf8T+wcjcICVVG1tbVdedppp62InSOrpaVl/Q7S9ttv/9D8+fN/FDlSd+dSKVCXA6nleyOVArWTxPLttddex8yePXt9gXrffffd8PWvf/3OmJm6a21t/SrxC9TZM2fOvClyhi6mTZt2AaUCdeLEiUsWLFiQ1HcLeAeVArWdxL77hOtNswXqVYQdzZRcnmk/TGLv4RZbbPH/ygXqmjVrls+cOTOpfJdccsko4P2RYzyT2vty+OGHjydToB5zzDE/u+aaa16MGKmabIF6P4l994HzqRSoz5FYvjFjxpxIqUBta2vrSO07WBqDxQJ1A9nFV5IkSZKUBAtUSZIkSVISLFAlSZIkSUmwQJUkSZIkJcFBkjSYbAnsS5hMGmAx8Kd4cSRJkiRtDAtUNbKphFH5ZgCvAKZ1e/z3wJF1ziRJkqTG1gKMLLXXAKmN+jyoWaCqke0P/L/YISRJktSwmoFdCCc8srfhpce/D5wcJ1o+WaBqMGgjzBk2B1gEfDluHEmSJDWI04FvxA6hCgtUNbI7gH2AB4H20n3TsUCVJEnSxusE/gk8CxwcOUtuWaCqkS0s3SRJkqRNcT/wKeAe4F5gBXAUcGPMUHlmgSpJkiQpr35fuikRzoMqSZIkSUqCBaokSZIkKQl28Y3vDcCepfYcNqyLwUFULtyeB/xiAHJJkiRJUl1ZoMZ3IvDeUvvbbFiBegRwdqn9OyxQJUmSJA0CdvGVJEmSJCXBM6hKyRh6/04uq1cQSZIkSfVngaqU3AQcUOOxDvy+SpIkSYOaXXwlSZIkSUnwjJRSchqwRY3HivUMIkmSJKn+LFCVkrtjB5AkSZIUj118JUmSJElJsECVJEmSJCXBLr5qdIcCwzLLUzPtCcAR3Z7/L+DxgQ4lSZIkaeNZoDamQuwACbkS2LrGY/sCN3e77yzgSwOaSJIkSdImsYtvfB2Z9oYeMKg10q0kSZKkDTecMFtE9nZj5vGTqjz+3jpnzBXPoMa3MtPe0MJz14EI0qCOBoZuxPMXDlQQSZIkSZvHAjW+pzPtPTfg+ROAQwYoSyP6e+wAkiRJaljtwBkb+W/+NhBBFFigxndvpr0XsAvwSC/PPxMYNaCJJEmSpHxoB74WO4QqvAY1vluB50vtAvA9oKXK8wrAx0o3SZIkSRp0LFDjW00oSsteC8wBTiV05X0t8FHgDuCC0vOvrnNGSZIkSRpwdvFNw5eBI4FXlJb3oGvRWrYWeF/p8WPrE02SJEmS6sMzqGlYDRwOXAF01njOA4Qzqr+sVyhJkiRJqifPoKZjBeHs6BeBo4DtCfMyPQncCfw189yvAReV2uvqmFGSJEmSBowFanr+DVzax3PaSjdJkiRJGjQsUBVVS0vLrNbW1qTOAp9++umFcvv+++9/PTAlYpxqxmfa2wI/ihWkhv0z7WYSy3f77bdvk10+6KCDzjrhhBOei5WnhgmxAwCfaW1tfU/sEFnf/OY3R5fbjz/++E4k9t0CXpZpDyO9fDt0Wz4fWBUjSC8KmfYxwKRYQapZuHDh+t/f0aNHb9va2praZ/zS2AGAXVN7X+6///5dZs+evX75uuuu+z7QES9Rn04ApscO0c2YTHsHEvt9e+aZZ15ebm+xxRbDUvsOFovF8X0/S2UWqIrtbbED9GbFihV7EAalStU44N2xQ/SiicTyrV69ustyS0vL6yJFSd3hsQN0N2TIkPXt5cuXb0li361umkk7H8BxsQP0YbfSLRmrVlXq+WHDhm1B+p9xDFuR2PsyYsSILsudnZ3viBRlQ72Mrge8UpPc729bW6Vj4fDhw4eQWD5tHAdJkiRJkiQlwTOoqqvm5uZni8XiFbFz9Gb69OkHtLe3DwFYuHDh3Oeffz617p+7Eo5eAjwN/CtilmqmADuW2quBv0XM0sPw4cNH7LjjjvuWlwuFwj2FQmFNzEy9aWpqeqxer1UsFi9vampK9sDllClT9igUCuMAli1b9tSiRYueiBqop22BaaX2KuD+eFGqGk3XszJ3Au2RstSyP5V9k0eAJRGz9LDddtvtOnr06C0Bxo8f/3ShUEjt93e9YrG4uF6v1dnZeW9TU1Oy2/bm5uaxu++++17l5blz597e2dlZjJmpigOpdHF/CFgWMUs1u1O5xOgp4Il4UXpatWrVtg8//PA0gGHDhr2w88473xc5Uq/WrVvnWDK9KLS2tq5fQQuFwi4zZ858NGYgSZIkSVI+zJo1a3qxWHykvJzskXJJkiRJUr5YoEqSJEmSkmCBKkmSJElKggWqJEmSJCkJFqiSJEmSpCRYoEqSJEmSkmCBKkmSJElKggWqJEmSJCkJFqiSJEmSpCRYoEqSJEmSkmCBKuXbZcBvgO1jB5E0oFqA/wOuBoZHziJp4B1MWOe/EjuItLGaYweQFNUbga2AL8QOImlAFYC3ltpDgbaIWSQNvG0J6/wfYweRNpZnUKV8ay/914NV0uDWkWm7vkuDX3mdHxI1hbQJLFClfHMDJuVDe6bt+i4Nfh6AVsOyQJXyzQJVyodi6QbusEp54PZdDcsCVco3j7BK+VFe391hlQa/coHq9l0NxwJVyjePsEr54fou5YcHpNSwLFClfPMMqpQfru9SfnhASg3LAlXKNzdgUn64vkv54QEpNSwLVCnf3GGV8sMdVik/3L6rYVmgSvnmDquUH+6wSvnh9l0NywJVyjd3WKX8cFRPKT/cvqthWaBK+eYRVik/HNVTyg8PSKlhWaBK+eYRVik/XN+l/PCAlBqWBaqUb+6wSvlhjwkpP9y+q2FZoEr55g6rlB/usEr54fZdDcsCVco3d1il/PCaNCk/3L6rYVmgSvnmDquUH16TJuWH23c1LAtUKd/cYZXywzMqUn64fVfDskCV8s0dVik/vCZNyg+372pYFqhSvrnDKuWHO6xSfrh9V8OyQJXyzR1WKT/cYZXyoyPTdhuvhmKBKuWbgyhI+eEBKSk/2jNt13k1FAtUKd8cREHKDw9ISfmRPYPqOq+GYoEq5ZtnVKT88ICUlB928VXDskCV8s1r0qT88ICUlB928VXDskCV8s0dVik/PCAl5YddfNWwLFClfLNAlfLD9V3Kj06gWGq7zquhWKBK+eYZFSk/HCRJyhfXeTUkC1Qp3zyjIuWHgyRJ+eI6r4ZkgSrlmxsvKT88ICXli+u8GpIFqpRvtbr/FOodRNKAq9Wl3/VdGpxc59WQLFClfLkGOA0YWlrufnR1BPB5YF6dc0nqf3cA76Syre++vo8DLgTurHMuSf3vZcBfgL0z93Vf53cFbgc+V8dc0kbzomkpXx4GzgPOAO6h8hvwCuBGwgZuPPDbKOkk9afFwPcJ6/xtwNTS/UcCrwd2ASaUHpfU2B4E9gRmA88BN1E5GH0m8HJgIjAKODlGQGlDWaBK+fId4H3AFOBNmfv3y7SXlp4nqbGdBxwMbAOcmLn/8Ey7XMRKamztwM8JxedEwgGosndl2vMIxayULLv4SvkyH3iqj+esBv5chyySBtZfgBf6eM5zwON1yCJp4J0PLOnl8Q7gR3XKIm0yC1Qpf84HVtZ4rAhcS5jgW1Lju5zKQCndrQUuq2MWSQPrEUIvqFqeA35YnyjSprNAlfLnl4SzpNUsAb5XxyySBtbF1N5hXQb8pI5ZJA28i6i9jX8GeKJ+UaRNY4Eq5U8bcDPhbGl3LwL31zeOpAH0NLW78D5JuAZV0uBxBbCiyv1rgUvrnEXaJBaoUj5dQOjqk9UB/CxCFkkD6zx67rC+CHw3QhZJA2sVYZT+7uwxoYZhgSrl0xx67rA+h9ejSYPR1YSCNGslobu/pMHnPHoehH68yn1SkixQpfxqpevgKc8Bj0XKImngtAPX0bVb/wOEMy2SBp8/0fU61DacPk4NxAJVyq/LqJxV6QR+EDGLpIF1AZX1fS1hIBVJg1ORcC1q+aBUG3BVvDjSxrFAlfLrOeChUns5zo0mDWYPAs+W2iuBGyJmkTTwLgaeL7Xvo+85kaVkWKBK+fYlQje/eTiapzTYfRNYB/yRcBZV0uD1FPAvQs+Jb0TOIm0UC1Qp324iXJ92cewgkgbcjwhd/rwWTcqH8wiX8NwUO4i0MZpjB5AUVQdwInB77CCSBtxy4K3An2MHkVQXVxF6Sa2LHUTaGBaokm6OHUBS3fw2dgBJdbMW+F3sENLGsouvJEmSJCkJFqiSJEmSpCRYoEqSJEmSkmCBKkmSJElKggWqJEmSJCkJFqiSJEmSpCQ4zYzq6uyzz26aMmXK8Ng5enPDDTeMaG9vLwA8+uijax555JGO2Jm6GUZl3V1HevObNRMyQpggvC1ilh4mT55c2G+//UaUlw899NDVY8eOLcbM1Jvx48evOfHEE+vyHTz//PNHjBo1qlCP19oUt956a8vKlSuHACxcuHDdnDlz/O5vnAIwIrP8YqwgvRhByAmwhjBXczJe85rXDBszZkwzwJQpU9bNmDEjte/gesOGDev8wAc+UJfvYGtr61BgaD1ea1MsWrSoac6cOev3Pa655prUv/tthN+QlLQAQ0rt5PY9dt555+bddtttGMDw4cM7jzjiiNR+f7s45ZRTUvwOJsMCVXU1efLkVwB3xs7Rm+uvv562tqR/17QZFi9ezDXXXLN++aCDDoqYpm9Lly79IPC/9XitUaNGvUBlByk59957Lw8//HDsGMqxW265ZX370EMPZcaMGfHC9GHt2rX/Anaux2sVCoUvFovFM+vxWpti5cqVXX73NfjsuOOOvPGNb4wdY4O1trZOOuWUU5bEzpEqu/hKkiRJkpJggSpJkiRJSoJdfBVVoVA4ksSugyoWi7dR6ua46667/mTu3LkXR47U3dXAxFL7ceDdEbNU8yXg8FK7HXh1xCw97LjjjgfMmzfvm+XlhQsXnjxhwoSHYmbqrlgs/gbYKnKMjxcKhbsiZ+hi7dq1vwO2ANh5550feuyxx06OHKm7bwIHlNptVNaDVBwNfDGz/BZgUaQstaz//QV+Dnw7YpYepk6detWCBQsmAaxYseKJQqHwrtiZsorF4nHApyPHuK9QKHwkcoYunnzyybcCHysvDx8+/LC2trY1ESNV85dM+1Lgh5Fy1LL+9xeYC3woYpYeJk2a9HXgIIClS5eu2XLLLQ+LHKmLYrH4UtL7TJNlgaqo2tra7jrttNNWxM6R1dLSsr49adKkhXPnzr09Ypxq1mbaLwKp5Xsu0+4ksXw77LDD2Hnz5q1fnjt37j8uuuiipK6Lbm1tXdv3swbcQzNnzkzqs5s2bVp7uT169OhVJPbdApZl2sl994Eduy3/jXCQK1WLSOw9HDly5Pp1s729/cXU1pFLLrlk70Ih+mXkK1J7Xw4//PBXZJePPPLIOxMdKKnsSRL77tN1UKTkfn9HjBix/vd33bp1nal9B2fNmvVisZjseIzJsYuvJEmSJCkJFqiSJEmSpCRYoEqSJEmSkmCBKkmSJElKgoMkabAYDuwNzADGlu57ArgyViBJkiRJG8cCNdgD2AuYAiwH/gncQZgiY1PtVfq7k4AisBi4E5i/WUmVtQvwKUJRuicwtNvjv8cCVZIkSRuuAOxM5YTHEuDf8eLkT3918b2UUIQVgUs28N98NvNvbu6nHL15PvN6x5TuezVwN/AA8DPgfOAy4E/AU8DGzrE3Cvg8sAD4e+lvXkSYx+3/CF/uuwhz0dXSAryQyXpAL8/dgjDsd/m5LxLOJNZyeOa5zwFD+vofStyewEnAy+lZnEqSJEl9mQy8HTgP+COhZngUuKd0OzNetHzK8xnUjwLfonaRthUwC9ieDfti7gFcC0zr43mvBG4Avg6cQSgWs9YQJio/qrR8OPDXGn/r1XT9DEcABwJ/qPH87KTxtwAdfWRtBJ2EM95zSrfncCJkSZIkbZh3A9+IHUIVeS1Q3wB8mHAG+U7gKkLX26HAq4D3Eoo9gM8BNxHOqtayB/BnYHxpuRO4Grix9HebgN2A95eeC/AZwpnSL1X5e7OpFKhHAF+p8bqHV7nvCDasQJ1d4zmN5PfAOGBl5r7pkbJIkiSpsS0mnPBYBZwYOUtu5bVA/SjhTOVJwI+7PXY58F3gVkLBWQA+Te0CdQThOsdycfoEcDxwb7fnXQdcQDhC87HSfV8knHX9W7fnZgvMVwEjCd13uysXnKsJZ4KHUb1ohVDIzajxGo1qeewAkiRJami3AMcRCtMFpfuOwgI1mjxPM3MaPYvTsn8A52SWjyEUidWcTOWs6ErCF7p7cVrWAXyCUKxCKCo/U+V59wJLS+0W4KAqz9ka2L3Uvo1wJhhCETquyvNfTaU781PA3BoZJUmSpLyYQ+j5uKCvJ6o+8lqgziUM7NSbn2bazYQpTLorAKdnls8nXA/ZmyLwhczym+lZ/HYQjuaUHVHl7xxWen0IXV3LXXaHAK+p8vzB1r1XkiRJ0iCT1wL11/QcnKi7Z4GFmeWpVZ6zC7BjZvlHG/j692b+9jDCwEndZbvgVuu2273gnF3jsWr3DYbuvZIkSZIGmbwWqA9s4POey7THVnk82/V2GfCvjcgwL9Peqcrj2YLz5cCEbo+XC86lhIL3TsIF3dnHyqYQBmmq9rclSZIkKQl5LVA3dHCddZl2tXk2s2dVxxKKxXYqc432djs482+7F58QuiE/VWo3Aa/NPLYTYfobCPM1dZaylgdy2g3YNvP8w6l0B34E+9hLkiRJSlBeC9TOfvo72cJyCGEk31rzqvZmRI37a3XbzbZ/X+P5h9V4vmdPJUmSJCUpr9PM9JeOTPspao8K3Je/1Lj/D4Q5WaF2gTq7RvsIKtfEZovVlK8/3RUYVeOxIj2n45EkSZI0iAxEgVro+ykADB+A1663JZn2WuCMfv772YJzOvASQvfccnff+XQdNfjvhMGdJlEpSl8KbFdqdxK6BKfqf4EDajzWgQdUJEmSpEGtv7r4vphp1+qu2t02/fTaMT2WaW8PTOznv7+AcM1o2eHAywgFKPTsrlukcoZ0KuGMZPZsa3Z+VUmSJElKSn+dkcoOOrRdzWd1dXDfT0neHwlFYYFQ7B8HfL+fX+MPhOlsIHTbzV73Wu160tnA20rtw+k6J2rK3XsB3gK0xA4hSZIkKY7+KlAfyrRfQbiO8IVenn8E4exeo3sWuBk4qrT8WeCndD2jvLlmA/9Zah9GpUAtUrtALTuSrgcCUh8g6am+nyJJkiRpsOqvLr5/oTIy7ijgo708d2tgVj+9bgrOIRSLADsSBiba0OtrW4BX9/Gc8jQyEOYzPbLUfhB4usrz5wGPl9pvALYstdcCt21gLkmSJEmqu/4qUJ+k63Qn/wOcBgzL3DeU0IXzr8AOVC+uGtHtwNcyy8cT/h/fRPUz1EMJAwGdBzwBfKGPv7+UcO1oWflv9nY2tPxY9vXvoPez2pIkSZIUVX+OivpJYA6hKG0GLiQUqo+WlncExpSeewPhbN7/9OPrx3QmYYCkk0rLLwN+SygIHyQUmcMIgxvtQtfC/cEN+Pt/AGZ0u6+vAvWkbvelfv3ppno3MDKzPDnT3haY2e359+B0NZIkSVKS+rNAfQA4Afg5lbksx9CzsPopcArwX/342rF1ACcTzlL+D6ErLoT3Yb9e/t0K4NYN+PuzgU9nltv7+HfZwZuyf2MwOo/Qbbya3YDWbvedhQWqJEmSggKhd2dWdt9yLOFEW9YzwKqBDJVn/T2v5LWEM4SfAI4BphHOFi4C7gR+AFxfeu4dVLrGPsbA+xaVa0PnbeC/+SFhECSA+zbg+T8gFOBvI1wruh/hCz4GWEM4k/pI6W/NLt1Wb8DfvY2u3YifIRS3tSwGPgeMKy0Xgbs24HUkSZKkPGkB/tXL4yeWblnvA64YsEQ5198FKoSRWD9ZuvXmltKtXs7ehH/z7U34N23A5aVbf3kBOGMj/825/fj6KduVjbuWekMOCEiSJEmKYCAKVKmelvf9FEmSJKmqdfQ8Q9oXeyYOIAtUSZIkSXnVAfwidghV9Nc0M5IkSZIkbRYLVEmSJElSElLs4rsvG98PvJbzCSPeSpIkSZISl2KBuhfw3/30t67AAlWSJEmSGkKKBeoaYFk//a2Ofvo7kiRJkqQBlmKB+vPSTZIkSZKUIykWqMqR4cOH39za2prUme7TTz+9UG7ffffd7wIOjhinmomZ9g7A7bGC1DA90x5KYvn+9Kc/bZFdPuSQQy494YQTVsXKU8NWsQMAF7S2tiY1z/AFF1yw/rObO3fu7iT23QJ2zbSHk16+id2Wf0notZSSQqb9duCAWEGqmT9//qRye+zYsTu0tram9hlPjh0A2Ce19+XBBx+cMnv27PXL11577R+AzniJ+nQy8LrYIboZl2nvSmK/b4sWLVr/+zthwoSW1L6DxWJxZOwMjcQCVVEVi8X9YmfoTVtb2zbANrFz9GIk8KrYIXpRILF869at67I8dOjQvSJFSd3usQN0VyhUape2trbRJPbd6qaJtPNBGJQwZVNKt2S0tbWtbzc3N48g/c84hrEk9r4MHTq0y3KxWNw/UpQN9ZLSLVXJ/f5mt+1Dhw5thN9f9cJpZiRJkiRJSfAMqupq2LBhCzo6Os6KnaM3++yzz8Hr1q1rBpg/f/4Dzz777JLYmbrZAyh3M3sK+GfELNVsQ6Wb74vAXRGz9DBq1KiRu+666/oz901NTXcUCoXUujmuVywW/1bHlzurkD1NmZhp06btPXLkyAkAzz333PwnnnhiXuxM3bwE2KnUXgnMiZilmjHAjMzybUB7pCy1HES4NADgQeDZiFl6mD59+u5jxozZCmCrrbZ6qlAopPb7u16xWFxar9fq6OiYPWTIkHV9PzOOoUOHjpsxY8Y+5eV777331s7OzmLMTFUcSuXE0d+Bun1+G2hvYEKpPR9I6vd3xYoVL5kzZ85OAC0tLav22muve2Jn6s2qVateiJ0hZYXW1tb1K2ihUNhl5syZj8YMJEmSJEnKh1mzZk0vFouPlJft4itJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkWKBKkiRJkpJggSpJkiRJSoIFqiRJkiQpCRaokiRJkqQkNGcXisXitFmzZsXKIkmSJEnKkWKxOC273Nzt8RuLxWL90kiSJEmSVGIXX0mSJElSEixQJUmSJElJ+P/lhrmotJ793gAAAABJRU5ErkJggg==\")
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3f0e49e4",
+ "metadata": {},
+ "source": [
+ "### Communication\n",
+ "\n",
+ "Once the working arrays have been created, we can start with the iterations of the Jacobi method. Remember that this is implemented in parallel by updating the ghost cells with data from neighboring processes and then performing the sequential Jacobi update. See figure:"
+ ]
+ },
+ {
+ "attachments": {
+ "fig15.png": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6gAAAFACAYAAACxyVHuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N15fFT19f/x12cmIQFkE1wANxBFrTu4b1itimQC2mKrtfrtRtXazbbWrmJba+1m7S5fv9Wqrbb5VWBugFatpe4buGvdFVcU2cISSOZ+fn+cO2QyTCAsmXszeT8fj/vIvXPv3Dm5M3PnnvvZ3DXXXOMRERERERERiVkq7gBEREREREREQAmqiIiIiIiIJERV0fLJzrlX4whEREREREREehbv/W7AP/PL7RJU59yrU6ZMeb7cQYmIiIiIiEjPM23aNLxv6xZJVXxFREREREQkEZSgioiIiIiISCIoQRUREREREZFEUIIqIiIiIiIiiaAEVURERERERBJBCaqIiIiIiIgkghJUERERERERSQQlqCIiIiIiIpIISlBFREREREQkEZSgioiIiIiISCIoQRUREREREZFEUIIqIiIiIiIiiaAEVURERERERBKhKu4ARDbGe+8aGxsvizuODbnuuusOWLhw4SCAhQsXvvbiiy++EndMRXYBRkbzy4H5McZSSn/g4ILlu4FcTLGUdPjhhx+dTqerAE455ZSnDjzwwEVxx9QR59yDdXV1s8rxWtls9nPOueHleK3NMXv27BFPPPHErgArV65c8thjjz0ed0xFBgEHFCzPjSmODRlXMP84sCSmODpyAHYcARYAL8cYy3pGjBixy7Bhw0YCbLvttsunTJmStPNvO+l0+vJTTz11TVe/ThAEewFndfXrbK7m5ub0L3/5y2Pyy88+++z8xYsXL48zphIOxn4/wT73C2KMpZQRwK7R/FLgsRhjWc/gwYMH7rXXXgfmly+55JK5MYazUWEY3jBx4sQX446jHJSgSuJddtllbsyYMd+NO44NeeGFF3jyySfzi8dsaNuEmBB3ABuRuGP4wAMPrJsfN25c4uIr5L3/LVCWBBX4DDC2TK+1yd577z3uvffewofq44qlkxL92SL58SXOK6+8wiuv2D3LUaNGQcLPv9XV1T8HujxB9d6Pds4l+re96NyR9M9+0uMDyMQdQKH333+/3XschuExqVRyK5em0+l7gR6RoCb3XRAREREREZEeRSWo0h09jlUVSYwwDI8i+j717j1w5erVPBdzSEVW7AGt/Ww+HUK/RFWzgTVDYPUubcsDnwRaYgunBOeWHey9B2DNmjULgKRV494XGBxzDK+QsCpma9euPYioClxNTZ/cmjW9ElbFt3lHaB7WtjwwgdU/lxZUv+/7JlQvjC+WUpYdAD5t871WQp9EnX/79HGjVq1a0h/Ae58D7ok5pGL9aN/EIg4h1rQjMcIw7AUckV+uqdl2wZo1YcKadiw7EHxU2FTTBL1fiDeeYk17Qm4bm0/noF+izr+1tentm5vf3ym/7L2/C/AxhlTKcXEHEAclqNLteO8vqq+vvzPuOAqNGjVqCTAQYOTII196+ulZ58ccUpHBv4bFh9p87RpYkrD4jq+Hud9uW/7zxXDq4vjiWV91de8H165tBuCpp576y89+9rNvxhxSO42NjTO89xNjDuN/M5nMFTHH0M6RRx55P3A4wHbbjVj1xhtPJeyzP+YTMP/CtuWkfTcB3INt80dOh9tujC+WUqpvh9aoHd6Or8BriTqGu+/+4auffPLWwwFaWlqaM5nMuJhDaiebzR7inHso5jASd1zOPPPMHYB38sujR3/xxieeuDQbY0glVM2FXG+b3+m/8OKFG9y87AZNg6VRG/u+q5N2fttrr0vOfOyxK7+cX/7zn//8oYaGhrVxxlQsCIIcPbDGa4/7h0VERERERCSZlKCKiIiIiIhIIihBFRERERERkURQgioiIiIiIiKJoARVREREREREEkEJqoiIiIiIiCSChpmRnqwa2A/YPVp+Bng6vnBERERERHo2JajS0+wBXASMAfYHagrWfR+4NI6gRERERKTbGhlNAK8Dz8UYS7enBFV6mn2B8wqWffTXxRCLiIiIiHQ/w4AvYAUeY4FBBeuuBr4cR1CVQgmq9DSLgZuBedE0H3iV9icWEREREZGO7AVcEncQlUoJqvQ0/4kmEREREZHNsRKYQ1uBxzzgLmC3GGOqGEpQRUREREREOu9B4NS4g6hUGmZGREREREREEkEJqoiIiIiIiCSCElQRERERERFJBCWoIiIiIiIikghKUEVERERERCQRlKBKpZiHjXFaPP0tzqBERERERKTzNMxM5doD+EfB8geA5o08pwZ4pmB5PPD8Vo6rqwwABpV4fJtyByIiIiIiIptHCWrl6gWMLFh2nXiOK3pOr60aUdcaQ+kaAS3lDkRERERERDaPElSpFMviDkBERERERLaM2qCKiIiIiIhIIqgEVXqig2h/cyYd/R2GVRXOewd4s1xBiYiIiIj0dEpQpSe6B+hT4vHPRFPelcAlZYlIRERERLqTXWkr5IC2vGoA7ft0WQ4sKldQlUAJqvRErwC9O7Hd4q4ORERERES6pQeAHUs8/j/RlPcH4PwyxFMxlKBKT7Rv3AGIiIiISLe2DBuicWNWdXUglUYJqoiIiIiIyKbZK+4AKpV68RUREREREZFEUIIqIiIiIiIiiaAEVQp1ph69iIiIiIhIl1CCWrlWFy13ptfaUj2RiYiIiIiIlIUS1Mq1rGh5eCeec0RXBCIiIiIiItIZSlAr1/u0HxT46I1s74DPdV04IiIiIiIiG6YEtbLdVzD/eaB6A9teBBzeteGIiIiIiIh0TAlqZftTwfwHgFuAIUXbDAGuBn4KLC1TXCIiIiIiIuupijsA6VIzgLuAY6Pl04EJwCNYG9XtgQOxz0ELcBYwu/xhbhrn3KFBENTGHUehr33ta+u+S8uXv90PPnZknPGsb+3AtvlcKnnxvT6q/fLvDoEbmuKJpbQwDNfNb7/99iODIDg1xnDW473fIe4YgNFJOy5XXXXVus9+c3NTOnmf/UW7tl9OWnzFXt81eTH6gmuZVYk7/y5b9vqg/HwqlUon7TsC7BF3AEDijsvChQsH3nzzzeuWFy26f1TSPlvgCwqaVg5IXnxr+7fNtybu/Lto0ZIRhcuTJ08+5ZxzzmmNKx5powS1soXAGcA/gQOix2qAo4q2WwKcC/y7fKFtkSviDqBYr1691s2//vqjI+DRq2IMZyOaa+CvCY4PYNb3446gWGvBT9bOO+98BvbdkvbOjabE2HbbbdfNL1q0oA8sSPhnP+nfzf9mbEqqRbsm7RguWNA2X1VVVQvMii2Y5KohYcdlwIAB7ZbfeuufHwU+Gk80nfHOnkn77Le3qnfS4nvjjfbLNTU1M+OJRIopQa18C4HDsDaoZwIHY1W7Q2AB8Dfgt9F8NTCt4LmLyxqpiIiIiIj0aEpQe4Y1wC+iKQUMBJqwar2FWkhgT76XXnqpb2xsXBN3HBuSTqerq6urUwC5XC4XhmHSqoikafu+h6z/3sfNAb0KlhP3fldXV/fC4gQ7fuEGNo+V975s728qlVrjvU/c+5WXSqWqqqur0wDe+7C1tTVpn/0U7TuwS+KxrCmYT+Jnv5q2PjVyQKLOv6lUKp1Op6sAqqqqknj+baelpcWX6aVyJPPzvk51dfW6z35ra+ta7325jk1nFX72W7FjmiSJvvZwzqWqqqqqC5YT/Xkkee9vl3HXXHPNui+bc270lClTno8zIBEREREREekZpk2btqf3/rn8snrxFRERERERkURQgioiIiIiIiKJoARVRCqZ2/gmIiLr6JwhIhIzJagiUomqgG8Ac9B5TrrGsLgDkK3uKOBRYP+4A5GKNAjoHXcQIt2BLtxEpBLtBnwXOBn4WryhSAUaDTwP3AD0jTkW2XouwcYMvwmojTkWqSwO+1w9jg39JyIboARVRCrRi8AXo/kfAofHGItUlhrgZiwx3QlYHW84shV9Cngb2A/4acyxSGX5CnAqsDM6Z4hslBJUEalUfwRuwcaJuwVVyZSt4yfAQcBC4CySNyaobL73gHOw9/TzwP/EGo1UijHAFdH8F4EnYoxFpFtQgioilew84FlgV2A2MCDecKSbm4JdYHrgM8A78YYjXeAO4AdYlcxpwPh4w5FubjgwHegF/B3433jDEekelKCKSCVbhl1gvoW1LbsVq6IpsqkmAr+L5r8LNMYYi3Sty4DrsNoXfwMOiTcc6aa2BW7DqvU+AXw63nBEug8lqCJS6V7DOktaCnwQK0ntF2tE0t0cg7U7TQN/AC6PNxzpYh74LDAT2AZLMo6ONSLpbvpin599gAVY+9NlsUYk0o0oQRWRnuApYBKwAktSb8PubotszAnYTY3eWKnpF+INR8okB5wN3A0MBP6JqvtK5wyi7abGMiADvBlrRCLdjBJUEekp/oMlp+9jvfreBewea0SSdGdiY+lugyWnZwCtsUYk5bQCq33RCPTBSsQ+GWtEknTDsN+WI4FFwIdQp0gim0wJqoj0JA9j1TVfBz4AzAc+EmtEkkQO+DY2bmF19Pd0NDxET7Qaq32Rb5P6R2z82z5xBiWJNBa4H9gXeAM4FvvNEZFNpARVRHqaZ4EjgHuB/lgnKFdivSyKbAsE2Pi5KeBq4FygJc6gJFY5rIOb70TznwDuAfaIMyhJlPOwz8QuwH+x6r3PxhqRSDemBFVEeqI3gXFYYgpwMVaaenhcAUkiHAM8AkzASs4+CXwZjXUq1nHS5Vib5HewsXAfB6ZiJavSM20L/Bn4PdZD/N+Bw7DO+URkMylBFZGeqhW4BPgwsBCr8nsv8Es0XmpPMwArKZ0LjABexG5WXB9fSJJQ/8Gqct6GdZx1KfBA9Jj0LBmsfelZtP2eTAaWxxmUSCVQgioiPd10YC9gGtb28EvAy8A3gNoY45Ku57COkJ4Bvhgt/wEYgzo2kY69iXWedAbWEc7BwENYc4FRMcYl5TEa6zgrCwwHngSOwmrk+BjjEqkYSlBFRGyM1M8BJwFPY9W2foy1Ifo0VnVLKsuJWHXev2A9b74UPXY+KgGRzmnAal78CUtMJmPnj98Au8UXlnSR7bCaFk9hzQBasKT0EOwGhYhsJUpQRUTa3AHsj5WMvIpdZF6LDbQ+FY2d2t05rOTrTuB2rORrGfAtrOfNO+MLTbqpd4H/AfbDEtZewOexGx4B1h5RurddgJ8Dr2A1LaqAGdh7fgmwJr7QRCqTElQRkfZC7EJzH+Cr2JA022NtzV7DEtYjYotONkcN1vPqo8A/gOOxi8qrsLFwrwCaY4tOKsEz2I2tcVj71BRQh7VPvRtLYvvGFJtsnoOAG7E26Rdh7999WA+9pwHPxReaSGVTgioiUtpq4BdYAnMO1mPnNliV3/uwqnwXY53qSDJ9AHsP38DGrjwAaMIS0z2xi873Y4tOKtF/sFL6A7Hxc1uwhOY64C2srfsHgXRcAcoGDcCaezyE9ex+NtZL8x3AKVhb03tji06kh1CCKiKyYS3YXfQDsWFIrgdWYiWsV2IdKj0CfDN6TOK1G/A14EGsrdhXgCFYSfglWHW9i7Bq2yJd5XGs1H5X7HP3PDbu8meBfwFvY8nqyagztrj1wXpz/wt2E+EPWLvSNdgQMgcDHwL+GVeAIj1NVdwBiIh0I/dE0xex6nwfw6r0jYmmH2FtV+dE039QhztdzWE3D8Zj1e4Kh/tYi7UDvBardqnxTKXc3sZuZP0Eu8H1cexzuh2WrH4WWIW1f56NtY1+MZZIe5btsZsDGeBU2le/fgo7Z9yEaliIxEIJqojIpmsC/i+ahgCTsDvw47ASvPOjKYcNV3I3bcnt22WPtvLshF3sn4glpkML1uWw8Uz/H/B34L1yBydSggfuiqYLsHPFR7B2qjtFf+uibd+m/TnjCexzLZuvD9Zh1TjsnDGG9rUIX8bOGQ1YjRgRiZESVBGRLbMIu9t+LXYRNA67ADoJa+d4UDR9Mdr+Jeyi827gYaxzldayRty9OKzq9NHRdAxWbbLQCqyN2Gysd00lpZJkOaya77+wG1n7Y+0bxwOHYzdczogmsFoY92FtH+/D2kYuLW/I3c4QrL3oMdHfMVhb0jwPzMM6TZsezYtIQihBFRHZelZhSdLsaHkH4FDsAulorF3T7tF0brRNC/ACdoGUn+ZH++qJdqAtqT8cO3aDi7ZpxY7T3dgF5t1YdV6R7uiJaPoJdl12AHa+OArrUGkwlsCeUvCct2l/zniEnls7owYb8uVgrIr/UcDe2M2tQguwc8UdWBOMhWWMUUQ2gRJUEZGusxBrAxlEy/2AI2lLWA8CBmIlhPtgnaqAJWDPYsOiPIl1uPIElXdBNYy29rv7YL3ulupoaiXwGFbyfC9WTXJZmWIUKaf8zZd5wNVYNdR9gWOx88ahwEislLWwWjBYBz+PYueK/DnjBSqrhsY2WJvz/PliDJaU1pTY9mXsfHEPlpS+XKYYRWQLKUEVESmfJqwnyMLeIEdid/4PKvi7A1YisF/R89+l7cIzPz2NlcImWW/sYvIA7H/aP5rftsS2OazH00exUqF7ovlKusgW6ayQtu/6b6LHBtJWyyB/3hiN3fAZBkwoeH4zdo7I7+NJ7GZP0jv/cdgQXvsXTAdg58tSI1C8i50n5gP3Y4np4rJEKiJbnRJUEZF4vUxbBx15w7CLzvxF2f7AHljPkx+Kpry12AXoY9H0KJbExtV78DDsovkA2sdeatzHfOzzsbjzsa8sS6Qi3dNS4N/RlNcH+64dSNv3bl9saJt8LYVCC2h/zngM64E8DvnYD8Di3x+7kdWvg+0X0Ha+yJ873uj6MEWkXJSgiogkz1vR1FjwWL4UMl+asB92MbctbSUpeR5Leh/FBpy/D6sy2LwVY0xhnUAdGE0HRX+372D7hbRVPXwymn8GtR0V2RpWAQ9EU15hKWRhzYXdsfGAdwHqC7ZfgiWq87ESyPuBd7ZynENoO1/lzx17UvoGVnHpb/78kfTSXxHZQkpQRUS6h9VYldfiIRDy7TgL22TtTVtnTB+JtmvFLu7uxZLVu9i0EpPtgCOwdnBHYheYfUtsl28/+xjt28JVWvtZ6YSJEyfuAwxLpVJPTJ8+/d244ylWV1c3Lp1OV82cOfOOuGPpAvkbVS9jvVvnbYNVCc6fL8ZgNTYGAcdH01ejbd+mre33POyGV2dvKvWK9ps/bxyGDalTylvYOeNxKrf9rIh0khJUEZHuLV/aGhQ8Noi2UoojsIRyKOtX9XsF69XydqwTkcLSkpG09SR6NKV7xVwBPIeVhKoHYllPLpf7qnPuU7lcbjLtq7EngnNuRhiGA7AaAT7ueMpkBW3f1xuix6qxm1wHYh0xHYmVug4FJkcTWDv6B7Ahcu7AammE0boBWE/l+fPGUVjNj2LqgVhENkgJqohI5VkC3BlNebthF4z5hHV/rPrfCOAc7OL8Laz951DWb/8VYtXt8mMxPoiVcISIJFB9ff3F3vsrvfdTGxsbL4s7noRroa308k/RY/1oGybriGgaQPt28MuxG1t9geEl9vsubZ0W3Y+Vkq7okv9ARCqGElQRkZ7h1Wj6c7TcH/gkcCZWalJD+wvMfML6b+AvWFKqoV1kk4Rh+H3n3O9I6BAfuVxunHMuTc8pPd0UTVhJ6b+i5RQ2LutnsWrA22Hnkf4Fz1mB1aL4K3Ab8GK5ghWRyqEEVUSk53BY9buPA5Ow4Wzy1mIXky1Y9d5+WMJ6NnZRejNwI1bCItIps2fPfg14Le44OjJ79uzH4o6hGxgOnAWcgTURKKzq/wZWSrodsDPWvvVY4HDgH9g5o5Gt20GbiFQ4JagiIpVvD6wa79lYVd+8xdjF40xsbNb88C7VwHHAaVjbs2FYpylfxdqM/Qa4hW520Tlx4sR9wjC8ELuA3h7rDfQVYGZra+v1c+bMWVOwuauvr/+o9/6TWPvbKu/9K865W1pbW6cVbUtdXd2hzrmvOefu8t7/EbgEuwmwHfAS8LMgCGYATJgwYe90Ov1t7/3hQB/n3APe+28FQfDfwn3W19eP9t7/wHv/xLBhw658++23LwI+ilXBft17/5vGxsYbACZNmrRba2vrt5xzx2LVMB8Lw/A7s2bNmle4zwkTJuybSqW+BzwUBMHPio9RJpM5GLjEOXdvNpu9uuDxk4DPOOeyNTU1M1avXv0t51wmOo4LvPfXNzY2/o6iksj6+vrzvffHA78IgqCwh9n8+uO9958FxmLje74LPOW9/2tjY+P0/Hbjx4/fqaqq6jSsaulI7ObK+8DD3vtfNzY2PlS03997748EcM5NzmQyH8ivc841ZLPZhuj/uh7oEwTBR0vEvkMYhhc558YDO2LVWR/03l9d/HrR9lO99/uk0+lvhmHY13v/LSxRqwYe9d5f3tjYeH/x8xKqL9bB2iew0tL82KM5rLruDOy8UVgyPgr7zH8Uez/ro2kx8H/A77Hvm4jIBpUa7FhERLq/FDAemIN1ZPQdLDldClwLnIBd5J8L3Er7sUdbsA5QPo+VntQDf8MS0jHAddhYhJfTvhQ2sTKZzGfCMHwcOB/rROoJrGfkY4E/1NTUDM1vO3Xq1FQmk7nJe38zdpzecs696Jw7GPhVVVXVv8ePH9+/6CWGA5O998di1aK/iZU0rcba8N2ayWTOzGQyR6dSqQe99xnn3FKg1nt/GnB3fX39sMIdOucGA5Odc8e//fbbWex493bOLQfGOuf+VF9f/6UJEybsm8vlHnbOnYV1UJUGTkmlUnMnTJiwd9E+d4j2eWSp4+ScGxr9H4cVrRoVPX5Mc3Pzfc65r0f/3zLgYOfcb+rr668s3l8YhmOj5xX33urq6up+7b2/E6tmHnrvH8c+t6c756YVblxVVfV14FdYDYA1WHvoPsDZzrl7M5nMmYXbe+/3o63H2HxP1/mp8DhPoq0DoHUmTpy4n/f+CefcxVhy+iT2vTjLOXd/XV3decXP8d4fB0xubW093Xv/ADDeObcIS1AnOOf+PXHixJLHPUFGAD8FXgeuxz7/AP/BqvbuiN28uor1q22/CPwM6yhpH+BHWOn5tsDXsTbrM6Lni4h0SAmqiEhl6QWcB/wXmA2cgpUMNWJV9IZiF5p30rkhHFqwHoI/il2cfhlry7od8C3sAvQa2l/0J0p9ff2xwB+wJOhTQRDsFATBiUEQHNzU1DTEe3+uc64pv/28efMuxKo0vgUcEATB4dls9tjq6uqRzrl5wBHpdPqqDl7udMCnUqmRQRDsFwTBSOfcFwDnnPsJcJNz7g9NTU2Ds9ns2Nra2uHOuX8CQ7z3X+tgn8cBuzjnPhAEwd7ZbHa0c+6jAN77y1Kp1M3AzNra2u2CIDh46NChw7Eq2dukUqnvbPkRbOdTwJtVVVU7BkGwbxAEezrn6rFje1Fxkt2RTCbzJefchcC7zrnjgiDYq7Gx8UNBEOxTVVU11Dn37aKn3OW9PzEIgsFBEIwJgmBcEAS7Ouc+gSXKfyi8aRAEwdHOuSsBvPdXB0Gwe34qLBkuZcqUKdVhGDZgpcPXtra27hwEwQlBEOztnDsTCJ1zv45Km9fjnPuR9/47Y8aMGZTNZsc2NTUNw24K1YRheHlnjk8MDsWSxxeBr2E3cV4Avo0lreOw/2FRJ/f3bPTckVipd0P0+ERgLjZ0TWarRC4iFUcJqohIZeiFVeN9FqtKtwdWJXEaNt5hBrtI3JJqucuAq7HStNOwjpNqgCnYhe0vsGqaieK9vwwrVby0sbHxOgqqcs6dO7e5sbHxhunTp78PVnoalQ7ivT8vCIKn89veeuutbwMfA3LOuXM6SMZawzA8a+bMma/nH8hms78FXvXe7+ScW5DNZi+eO3duK0BDQ8Nq59yl0aYdla6lnHPnZrPZ5wr22YCNSTkASDc1NZ3X0NCwGmDatGkt6XT6WxvZ5+ZaWVVVdW7+eEWxzMJuYqS994dubAfjxo2rxUr0SaVSH89ms3cVrp8+ffr72Wy2XQlqEAR/b2xs/Bftq+H6bDZ7U1Slun9VVdUpm/9vtXnrrbcmYuOEPjd06NALCqtzZ7PZW7Aq7lVYIldKY2Nj4y+mTp0aAsydO7c1nU5/HfvuHc76wzXF6XDsvXsASx4dVnviDKxq+4+w2hKbKyzY3z7A77DjcBSQxcZjPnwL9i8iFUgJqohI9+awTo9ewoaHGIlVz7sQKy39HFaaujXlsNKWo4BjsNLY3sBXoji+hCWEsZs8efIALEbCMLxmY9s//PDDo6PqqEvGjh07q3h9Npt9ERsuoyqq0lls/qxZs4qrPnrsxgHe+1uLn+C9fyaa3aWDsF7JZrOPlHg8/7xsPuHNmzFjxqtYdd/hkydP3prvxdzp06e/W+LxJ6K/u21sB/379z8CGAy8OHPmzDs25cXHjRtXW19fP6q+vv7YiRMnnjhx4sQTnXPNAM65fTZlXx1xzp0Yzf512rRpLcXrwzC8KZo9oXhd5G/FD8yYMWMpVtug9vTTT99xa8S5hfbEalXcD9RhtSn+COxFW4lnbiu/5vNYs4FdgSux6u/HYDe6/oZ1siQiogRVRKQbOwgrgbgJa2+3AKuCuyfwWyxB6Wr3YBfqJwFPYe3NfomNk1qyCmQ5rVy5cgSWLL87a9asJRvbPpVK7RbNvpovASvhpejviOIV3vuOeqxd2tH6bDbbhCUDgzp4bskSLO/9MgDn3IZesxrrWXVr6aiTmyVRLBstQffe7x7NPt/ZFz355JO3zWQyf+zXr99i7/0L3vv/hGF4exiGtwNfiPZb3C54szjndov+lhwap0+fPvmhU7afPHlyqWO7wWOUy+U6ep/LoS8wFbuhMAGrwn8jVsvi02zCe7IF3sU6Edsd6zzJY+2An8LOX7o2FenhdBIQEel+emNVbR/BOo1ZgV3w7RE9HkfvurdjCfPnsHZqY7Ak9cdYkhSLdDpdCxB1SLRR3vvaaHZDY77m19UWr3DOrVfiVqQz7X6Lldync84DhGG4OfvcLJ34/zqzj/xx69R7Mnny5HSvXr3mYOP2PoGV0E9yzh0XhuFY59wP87vehyroNQAAIABJREFU0tgAvPc1AGEYLu9gkxVEpYvNzc2lPgNlez82UR3WrvRSrGp+I1bt9pzo8XJ7G/gM1qnSvdh4qlcBd2PJq4j0UEpQRUS6l0OA+cAXsXN4I1b6cSU2lmmcWrE2r/tgVfaqgG9gPYCOiiMg7/370d9dpk6d2pnfvHwpa3Gvs+s45/JVERdvYXhll0+evPcd3TTo8tK9/HuCVfXcqObm5uOwTnzmDx069JggCH4VBMHMbDZ7VzSMzlatiuqcy5cGl/wMrF27dhhWKp9ramrqVJIdsz7YjassVu3/BSxZzWBtx+M2H6vqOwVowtpNz8MSZxHpgZSgioh0Dw7rWOY+rJ3Ym8DJ2EXmlnRi0hXew3r9PQNL4o7ALjjryh3I2LFjX8JKdGvnz5+/0SrH0VAnIbDraaedNrh4/dSpU1Pe+zHR4qNbNdgycM69Gc121Nvu/l0dQxiGD+Zfq4Mqsu147/eMZu8q1Sa04P0ofjzfNnWTSvC9949FsyU/L977Q6L9Plnc9jeBDsWGyPkiVpX2p8B+wHrtq2Pmgf/FPn//wTr/+hPWQ3hsNTBEJB5KUEVEkq8v1mnJD7BSyQbgAOC2OIPqhHycd2PV96Zj1TPLZurUqWHUyyve+5+NHz++poPtUgBRO9V/ANUtLS3fLd5u/vz5n8E6M1rQ1NR0b9dF3jVaWlpex0raD5g0aVK7apTjx4/fyXu/3vieW1vUidSdQL/Vq1dfUWqbwtLuVCr1VjS7XidI0RBC40vtwzn3RvR3t02JL5fL3YIlTB8rHkd23Lhxtd77bwOEYfiXTdlvDD6KDekyEuug6QTgYmwc2aR6FfggNkRNDitV/QfWqZaI9BBVcQcgIiIbtAswEzgQa1t6Hlay0F28AZyIddr0GawDpb2BC7CSyi6XSqV+6L3PAMdVVVXdV19ff1XUc+5AYF/n3Ccfe+yx07CLY1Kp1NfDMBznnPtSJpPZJgzDP1VVVa0NwzDjvb8YS16+1A1Kz9YzZ86cNZlM5m/A2blc7h/19fWXee/fwkquvoKVxn+gq+Nwzl3gvX/QOXdhJpPZ1Xv/v1Hp7g7AIfPmzZtEVILZ0tJyf1VV1SrgpEwm81vv/U2pVKrZe39ylCy+iA0L005ra+u8dDqd895PrqurW+Wcy3dudW8QBPd0FNvs2bOfz2QyVwNfTqVSd9bV1X3POTfPOTfUe//NKK5ne/fu/Zute1S2mhRwOVa93mE9bp+DVZ/tDkJseJvHgb9gCeuDWI2RlzbwPBGpECpBFRFJrg9gF2YHAu8Ax9O9ktO8tcBnsQ6UWqK/f6ZMN0mz2WxTa2vrcd77LHCw9/5GrMrxv4CrvffDwzBcnd9+5syZzzjnTgJeBj6dSqXuCsPwAaxUZznw8SAIZpQj9i7yFayDrVHRsfgX1jnNv4nGJ+1q2Wz2uVQqdQzW/jDjnMti78ls4LLCbefMmfOe9/5srHOiC5xz93nv5wNXOOf+BPys1GvMnj37Ne/9Z4ElzrnPYh12/bhgGJkO1dbWfs0591NgsHNuGjDPe9+IDa10Z3V19Qn5cWcTphq4Ges0DSxR/TDdJzktNAtrc/8c1mnSPZQoRReRyqMSVBGRZNofG+B+O+AxrK3pG7FGtOWmAW8B/w/4GFa6czale7b9OZYQbpUeiefMmfMeMPHUU0/ds6qq6mjv/XbAIu/9SytWrLinuDQ0m83eO2XKlL0WLlx4XBiG+2Cd4rxSW1t7R0NDw4ri/VdVVf27paVlbHV19fvF6yLfDsPw5wVDlBQ7NN8rb16vXr2eWLVq1VjnXMneZFOp1C9aW1tvSqVSJYeZcc6Nz+Vy1Vhit04QBIsmT558+KpVq050zn3AOdecSqXunjlz5pOTJ08esGrVqrGpVKpdB1Ctra0NqVTqwTAMF3bwWn/J5XJ3pdPptwsfD8Pw+86532HJfjszZ858EhibyWTGOufGhmG4DbDQOfdUEATzC7dtbGycnslkRgDHASOcc4tzudzcWbNmvZzJZIaEYfhoVVXVeuOzNjY2XgdcN2nSpIEtLS07ATWFMeZyuXHOuTRWKr5OQ0NDDrj49NNPv6q1tfUEYFgYhstTqdSD2Wy2ZNvjXC73Oedcv1Qq9Wyp9cAnwzDs29TUVHL4mq2gF3ALcBpWjfd/ouXu7HngWKyX8P2xquEnAE+X2PYUrHfimWWLTkS6hLvmmmvWnZSdc6OnTJlSjjGwRER6uv2AHwITS6w7CGtfOgQrVTqJbthj7AZMAv6KXVDfCJxL+wThbKykeB+s9EREzJ3Ap4iqoxfohX2nJmHJ6WQgKGtkXWsIdsPuAKw2yZG0H292EFbVeyZ2fESkG5k2bdqe3vt1v/eq4isiEo/RwKlYclZoFFZaMAS4HystqKTkFKxN3Iexqr+fAAo7IzoWq266gk4OQyLSg4zBOh8rrAHngJuw5HQV1lt2JSWnYD1xfxDrOXtH7P8bEK3bBqumPgCNnypSEZSgiojEYzR2kfkz7MISrBSgEeuxMt8pyLJYout6jVhbVICpWKnp14G/Y8n5NsBucQQmklD9gNVYJ2PTsVJTsN69J2Mlp3VYSWMlWozd1Hsda5//VyxhfxTYF6uGPzy26ERkq1EbVBGReOSHrxgCXAvUY8NBjMaGhJhI9+zYpDO2BXoD/8Sq+H4CuAG7+O4TbZOiDL3JinQju2KlpX2xmhVPYN+bb0XrP4t1dFWJqrAezZcCX8Da1p6M1bjoXbBdbflDE5GtTQmqiEg8RhXMD8aq+qawsf9uBcZhJSSPAU+WO7gudj5W6vMuVurhsQvvPkXb7VnmuESSbFfakrHe2M2sH2LfnYejv2dg36eGOALsQrtgCXkr1nFaOnq8d9F2VViSulU6VxOReChBFRGJx/ZFy/kmF2lsGJC12HiAx5UzqDK5HLuI/DxWrbkju5QnHJFuYSRW9b2Qi/4eAvwRO2dcS+UlqC8Dh2G1LjZWjXdn4IUuj0hEuozaoIqIxGNjVdEWA4cDD5Uhljh8FytFXbqBbfqVKRaR7mBf2hLSUlZgQzldUJ5wyu5p4BisCUTYwTa1qHM1kW5PCaqISPltQ8fnX491AnIM8HjZIorHVVhHSYs6WN+Ltqp8Ij3dHhtYtxi4FLiwTLHE5RXsxt1LlB4/uR8woqwRichWpwRVRKT88p2dFAuxqmxHYGP69QR/A84C3utg/dAyxiKSZDt18Pj7wJeAq8sYS5zeAQ4FnsGaQhRKYeMni0g3pgRVRKT8Cjs7yWsB/ou1s3qz7BHF63ZseIyFRY/XoHaoInnFnYiB3dj5H2wc1J5kKXYj7xGs9+9Co8sfjohsTUpQRUTKb3fad3ayGuut9zCsNKQnegj4IPBGwWP9UXsyEbDq7r2KHnsXOB0bU7gnWgUcD8ylfa+9o0puLSLdhhJUEZHyO4i2Kr5rsAuso7BOTnqyZ7C2t29Fy6quJ2J2ov1NrXeADwH3xBNOYqwFJgA3Y0N0wfo9HYtIN6MEVUSk/PKlgi3YmKd10bzAq8DB2DAROWCvWKMRSYZdaWtv+S5wNDYuqFjHcp8CfgoswYZQVOdqIt2YElQRkfLbGVgG/BbrIKijIRN6qoVYJyj/RVV8RcC+B9tgN272x3qxlfa+iSWpQ1DnaiLdWlXcAYiI9EADgB9Hk5S2FEtSPxR3ICIJsAfwLFZyuizmWJLsCuBJdIxEujUlqCIi5XcYVpVVNmwVMDPuIEQS4NfAD2jfGZCU1lM7jRKpGEpQRUTK79W4AxCRbuWduAMQESkXJaiSeN5719jY+ELccWzIFVdcscMLL7xQA7B69erlK1asWBp3TEX6AwOj+bUk72KnBtihYPl1rOOLxBgyZMjOzjkHcP755793yCGHFI+9lyQ3ZjKZy8rxQtls9u/Ouf3L8Vqb469//evA2267bQBAS0tL85IlS4rHWo1bLe0/+6/FFcgGFLYDXkjySvF2wI4jWNXORJ1/+/btO6Bv374DAXbZZZc1l156adLOv+2sXr36oDPOOKPLexQPguAE4A9d/Tqba82aNe6CCy5YNw7z0qVL31m7du2aOGMqYUfs9xPsc5+0qs0DsSYtYOeNRJ1/a2pqagcMGLDu/Hvttde+Fv3MJ9UnM5lMj+i5WwmqJN5ll13mxowZs3vccWxIU1MT7733Xn5xcDQlVTU2DmeSjYw7gGKLFi1aN5/L5YbFGEpnbFfG19qFBI872NzczLvvvptfrAb6xRhOZyT2WEZ2ijuAjRgSTYmxcuVKVq5cCUD//v2rSfh7PGjQoLL0gOu938Y5l9hj4b0vPHeAdW6XZNtR3nP/pkrc+XfNmjXt3mPv/agkJ6jOub5xx1Au6sVXREREREREEkElqNId/cI591zcQRRqaWm5CugDsMMOe725cOF2N8QcUpF5H4NVI2y+Zi0c+vN44yn2ygHwxqlty4f9CnqtjC+e9VVVPfDN1lYbqnTBgpX39+8/eG68EbU3YsTSD/fundszzhiWLq35z5tv9rk/zhiKLVmy+hPAcIBBg4Y1L1ky8pcxh1TkxcPgnePblo9OYM/O91zSNr/bv2GnB+OLpZT7vgxhVMV30FvwgUSdf3fccelH33nnqREAuVzY8vTTgxJ1/u3fv2XozjuvODfeKFzL008PTNRxaW5e2hf4Qn55yJDD5yxaVPV4jCGVcN9FEPay+cGvwt63xBrOeuadDaujWhe1a2DsVfHG097226895N13Hzohv9za2vr56urqXJwxFXPO/R5IbrFuF1GCKt2O935WJpO5M+44Co0aNepKogR1yJCR7y9cOGtGzCEVGXxCW4JalYO7Ehbf8WH7BPV7c+DUxfHFs75Uqvc3wRLUV19d9fyNNx6RqGN40023HxV3grp4cc3Tl1xy1PQ4Yyh22GFrJhAlqH37DmpZsuTuRMUHY7Zpn6AmLT4AV5Cg7vE03JawGKsvaEtQ+y1O2jHcbrsPjytIUHNJ+4586lPP7BN3guo9iTsuO+10/7YUJKjDhp0yf9GiS7MxhlRC1Zfa5ge+l7TPPgwa35ag9mpJWnzDhl1SW5ig3nLLLdc2NDSsjTOmYkEQ/I4emKCqiq+IiIiIiIgkghJUERERERERSQQlqCIiIiIiIpIISlBFREREREQkEZSgioiIiIiISCIoQRUREREREZFE0DAzItAX2ANIA6uBZ+INR0REREQSLAWMBsYCY4C9sOvIZ4AvbeB50glKUKWnupC2k8re2EkF4Elg/7iCEhEREZFE6wMsBLYpsa5vmWOpSEpQpaf6ddwBiIiIiEi3k6ItOV0AzAP2wUpUZStQG1TpqZ4ArsNKUo8Abo43HBERERHpBtYApwI7ALsCpwOPxxpRhVEJamX6GtamEqABuKMTzzkXODKa/w/wly6IK0kOKFr+bCxRiIiIiEh30gLMiTuISqYEtTJlgGOj+efoXIJ6HPDJaL6Vyk9QRUREREQkYVTFV0RERERERBJBCaqIiIiIiIgkghJUERERERERSQQlqCIiIiIiIpIISlBFREREREQkEdSLr1SSbwGf7mDdPti4VSIiIiIiklBKUKWSbAuM7GCdK2cgIiIiIiKy6ZSgSiW5AvhdB+tUeioiIiIiknBKUKWSvB9NIiIiIiLSDamTJMlLxx2AiIiIiIj0bCpBrUytBfPVnXzOtl0RSIL1BXoVLNdEf9PAoILHc8DycgUlIiIiIol3AXBAwfLY6O9I4JqCxxcAl5crqEqhBLUyFSZUnU089+uKQBJsGnBWicf3ARYXLD8PjC5LRCIiIiLSHZwM1Jd4fAdgSsHyfJSgbjIlqJXp9YL5gzqx/eHArl0Ui4iIiIhIJbkBuK8T2y3s6kAqkRLUyvRIwfw4YGfaJ62FqoCfd3VACfTxaBIRERER2RR/jzuASqZOkipTI9AczVcDf8LaXBbbFmgAjgR8eUITEREREREpTSWolWkx8HvgK9Hy8cB/gb8ALwK1WMPuDwMDgeeAZ4DTyh6piIiIiIhIRAlq5foucBhWOgqwE3Bxie1eATLAN8sUl4iIiIiISEmq4lu5VgInAT8DVnew/lrgYOAFYBWwJJpWlSlGERERERGRdVSCWtlWAl8HvoeVpA4HHNZh0iO0H47mwmgSERERERGJhRLUnmE18K+4gxAREREREdkQVfEVERERERGRRFCCKiIiIiIiIomgBFVEREREREQSQW1Qpdtxzn0zCIJPxR1HoW984xt98vOvvz5/Zxh+WZzxrK9p97b5Nb2SF9/Kndovf/rrkFobTyyltba2rJvff/+Bx3z603fsEGM46+nTp2WvuGMYOnTliddff8fIuOMo1NDQd+f8/OLFb9Qm77O/fET75aTFV+yRD8HwUXFH0V6ud9v8op2Sdgxfe82vO15VVenq66+/I1HxVVfnBsQdA/heSTsuq1ev6HX++W3LL730fxNh2pj4Iiol7NU2/87IpH32YcUubfOrE3f+ffHF1G6Fy2efffb155xzThhTOB1xcQcQByWo0h2dGHcAxdLp9Lr55cvfGQScEl80G9OahrcSHB/AOx+MO4JiYcFP1sCB1SMHD25OVCKWBL175/bs3Tu3Z9xxFOrdu3rd/KpVy6phWcI/+0n/bi4ZbVNSrRoIqxJ1DJcX9JefSrn04MHNiYovCZwjlbTj0tzc2m555crX9wX2jSeazlg5GFYm6hi211KVtPPbihXrPXRmDGFICariKyIiIiIiIomgElRJvEsvvdQ3NjZeF3ccG7LLLrvsV1VVNQDg/ffff33BggWvxR1TkZ2BXaP5JuDxGGMppR9wQMHy/UAuplhKOuCAA45IpVJpgJqammeB92MOaUPuL9cLpVKprPf+yXK93qYaNGjQbgcddNBOAKtWrVr63HPPPRV3TEUG0r5U5p64AtmAowvmnwKWxhVIB/bFjiPAG8Cr8YWyvuHDh++0/fbb7waw/fbbrwAeizeiDVuyZElZmlek0+nXwjBM8m97+qCDDjoiv/Diiy8+3tTU1BRnQCUcgP1+AryGjXOfJLti1x8Ay4BE/VYMHDhwwIgRI/bLL6dSqSSef9cJw/CNuGMoF3fNNdf4dQvOjZ4yZcrzcQYkIiIiIiIiPcO0adP29N4/l19WFV8RERERERFJBCWoIiIiIiIikghKUEVERERERCQRlKCKiIiIiIhIIihBFRERERERkURQgioiIiIiIiKJoARVREREREREEkEJqoiIiIiIiCSCElQRERERERFJBCWoIiIiIiIikghKUEVERERERCQRlKCKyJb4CfAI8JG4AxGRLrMf9j2fEXcgItKtjMHOHQ1xByLdS1XcAYhIt7Yr9gO0Y9yBiEiX6Y19zwfHHYiIdCt9sHPHNnEHIt2LSlBFZEvkor/pWKMQka7UGv3VTW0R2RQ6d8hmUYIqIlsin6Dqx0ekculGlIhsDp07ZLMoQRWRLZG/O6ofH5HKpRtRIrI5dO6QzaIEVUS2hO6OilQ+3YgSkc2hc4dsFiWoIrIl1L5EpPLpRpSIbA6dO2SzKEEVkS2hHx+RyqcbUSKyOXTukM2iBFVEtoSq74hUPt2IEpHNoXOHbBYlqCKyJdQBgkjlUymIiGwOnTtksyhBFZEtobujIpVP33MR2Rw6d8hmUYIqIltCd0dFKl/+ItOh6wYR6TzVspLNoh8aEdkSujsqUvlaC+b1XReRzlI/FbJZlKCKyJbQj49I5csVzKskREQ6S7UvZLPoh0ZEtoSq74hUPpWgSrdXV1d3aDqd7h+G4X1BEKyKO54eovjcEcYViHQvupshIltCVXxFKp9KUKXbS6VSvwvD8Hbv/fC4Y+lBdO6QzaIPi4hsCXWSJFL5VIIqW0UmkxkCvAe8GgTBiK213/r6+tO99393zt2QzWbP3Vr7lS2mc4dsFl1UisiWUAmqSOULAY+1I9N3XbqlXC73CaBPGIYL4o6lByksQdW5QzpNCaqIbAklqCI9Qw67ZtB1g3RLs2bNejbuGHogVfGVzaIPi4hsCVXxFekZ8gmqbkZtwNSpU1OPPPLIZOfc2cD+QG/gHeAR59yN2Wz234Xbn3zyydv26tXrIqAOGA6sBh4GfhsEwZ3F+89kMl8DDvXeX55Op8MwDL8DHAHgvb+3qqrqOzNmzHgJoK6u7uPOufOAPYAVwP9ramqaOnfu3OaifX4GOMk595tcLvdeKpX6DnAUUAM84b3/SWNj479KxPJnwAVBcFapY5HJZP7mnFudr3KbyWTO8N5/3DkHsF0mk/lbweaLgiC4INpuiPf+NOfcKcBoYAdgJfCY935aY2Pj7MLXqa+vnxqG4Yecc3jvjyva791BEPw62u+PgZHV1dVfuvXWW98u3Mf48eP7V1dXf8l7PwnY2Tm3xnv/mHPuD9lsdlbx/1ZXV3eec+6DqVTqly0tLW+m0+mpwHHANs65/3rvfx4EwcxSx6WHyaHaF7IZ1EmSiGwJlaCK9Ay6GbURkydP7j1v3rzAOXcLcCqwCHjSOVcDnOO9/37h9pMmTdq9V69ejwLfBnYGnnDOvQ+cDvwrk8l8u8TLHAFMBiaFYfggcBLwNpYYfSyXy91dX1+/QyaT+YVz7iYs6X0r2v83+vfvf2OJfR4U7XNiKpV6GDgeeNo59w6WuN6eyWQ+XeJ5pwMf3tAhiRK+vGHOuQOi+RpgTMG0X8F2ZznnpgEnYp+7J5xza4CJzrlZxcfFe7+nc273aHFw0X7XtXN1zp0ITF67du02hc+vq6sbXl1d/XD0/uwOPAUsBCZ47xszmczPiv8x59zY6P87KZ1OzwcmO+cWA6u998cA0zOZTMnEvQdSb/+yyZSgisiW0EWrSM+gm1Eb0dzc/EssMX3aObdPEARjgiA4IZvNjvbe7wr8rmBzl8vlbgZ2Af4K7Bxte5BzbjxWkvqDurq6E0q9lnPuu8BlY8aMGRwEwWFr1qzZGfgXMNR7Px04BzghCIKRQRAcHIbhGGCp9/4jEyZMGFNqn977LwMNtbW1uwVBMCGbzR4EfAR7739TX18/akuOTxAEvwTGRotvBEGwe8F0TMH/9qRzbmJTU9PgIAgOLDiGxwPLgO+feuqpexbs9yzn3AXRc28t2u9FnQjteu/9nsDs1tbWXYIg+GA2mx0LHAssB76ayWRKJuLe++8Af2ptbR2czWbHBkGwK/BNrMTwx1OnTtV1ts4dshn0xRGRLaEfHpGeQaUgG5DJZEYAnwbWAJlsNvtc4frGxsY3gyC4uWD744FDgDeBTxWOy5nNZv8B/ApwzrmLO3jJ/wRBcOXUqVNDgNtuu21lGIaXReuOcM59q7CK8KxZs54CbgRIp9NHdLDPpWvWrLmwoaFhbf6BIAj+7r3/A1Drvf98Jw7FFstms//OZrPZuXPnFvYAS2Nj41zv/eVAKp1Of2RrvFYmkzk4KlldEobh2XPmzFmeXxcEwT3AFdHiNzrYxRO1tbVfnzNnzpr8A7W1tT8FXgN2fvjhh0dvjTi7OZ07ZJMpQRWRLaEfHpGeIZ8s6GZUCVF7yTRwexAEr3TiKSdEz5tRmJzmhWE4LZo9dvLkyb1KPP/vxQ+kUql1nQB5728tXu+9fyaa3aWDmG657bbbVpbY7y2FMZfL5MmTe02YMGHkhAkTjpk4ceKJEydOPNE5lwZwzu2zNV4jSk5xzs2eNWvWkuL1YRheE82OnTRp0sDi9d77WxsaGgo7AiJafgoglUrtvDXi7OZ07pBNpgRVRLaEfnhEegbVltgA7/3uAM655zv5lN0AwjB8udTKQw455FVgLVC7cuXKocXrnXOvFT/W1NS0NJpdGQTBohLPWRa95nqJVrS+ZCxAPuHeauOWbsi4ceNqM5nML5qbm99LpVIvpVKpu8IwvD0Mw9uJSjS99/23xmtFVa/x3pf836Ok9T3ArV27drfi9alU6tUOdr0EwDlXuzXi7OZ07pBNpgRVRLaEfnhEega1N9+AgkRk6QY3bFMTPW95qZVR1d0VhdsWyuVyrcWPFdjQug1pKvVgTU1NPsayJFv9+vW7AfgK8Ib3/mJgknPuuDAMxwLnRZu5rfRyNQDe+5L/e2Q5QHV19XrvA5t/rHsSnTtkk+nDIiJbQj88Ij2DbkZt2CKAMAx36+T2+RK2nUqtPOmkk/oC2wKk0+nFWyG+jfLeDy/1eHNzcz7G94tWtQK9p06dmsq3hc2bMGHCoM2JYdKkSbvlcrnJwFu1tbVHNjQ0LCtcn8lktkrV3gL5ar0l/3csER4OkMvlyvI+VCCdO2STqQRVRLaEfnhEegbdjNoA7/2DAM65I+hc6d6j0fNK9qjbq1ev46LZ10tV1+0K3vsjO1h1VPT30aLH3wTSDz300PbFTygYTqad2tra/BispdrV4r3P9xQ8rzg5jdaXPF5hGDZHf0vudwMejeIdW2plXV3d4VjJ8bKxY8e+tIn7FqNzh2wyJagisiX0wyPSM+hm1AYMHTr0DuBVYO9MJvOFUtsUDjninPs71uPv+Lq6una96o4bN67KOfedaPHPXRPx+pxzJ2YymcOLYqkFvhqtv6XoKS8BpFKpdkOwTJkypdo5dxklNDQ0rMCGihlSX1/fr8Qmb0V/9xo3bly735VTTz11T+dcqfFYSaVSb0Qxjiy1viOtra2NWNXmo+rr608pXDd16tSUc+570eItxaXE0mk6d8gm00WliGwJ/fCI9AzqsXsDpk2b1pLJZD4N/AP4ZSaTOdA59zfv/ftYFdHj5s2bNwrIAGSz2YWZTOYHwA+dc7Pq6+svC8PwfmCIc+4i4AhgQTqdvrKM/8ZLQLa+vv47wAPAMO/9t4G9gUeWL1/eLll2zt3ovT/VOffT+vr6/mEYPgTs8vbbb1/Ahtur3g+c4r2fU1dXd7tzrtk5tzybzf5+2bJlz/fr128BsEe/fv0a6uvrf9Xa2roklUodFSWLbwB7Fe9w+fLlz/fr128xcGhdXd12xFpHAAAVo0lEQVTNzrmnsBuoTwdB0NhRIHPmzFmeyWQuAX7rvW/IZDI/TKVSc8MwHDBv3rwLgVOAd3O53NROH0UppnOHbDKVoIrIltAPj0jPoB67NyIad/QU4GXgk977OcBDwHTgy0Sd7RRs/yPg29gYo790zj3onJuFDedyfzqdPm7GjBmd7XRpa7gCuMN7f433/vEo/qO99/c55+qKxyXNZrO3AL+O4v+Rc+4O59wfnXNVuVyurqMXCcPw8865u4EjnXNTgR977y8BmDt3bmsqlToDWAhM8t7fmU6nH3XO/Qa42zn3lVL7nDt3brNz7mPAi9HfHwI/Bj66sX86CILfAV+MFn8chuEDwD+xmwmPhmE4bvbs2e9sbD/SIZ07ZJMpQRWRTVGciHb0w7N9iW1FpPso/v6Wqi2RAnYsTzjdQxAEdzY1Ne3lnDsO+DzwFefcmc65PYIg+HjR5j4Igh9VVVXt7L3/mPf+q977851zhwRBcNSMGTNeLd5/Op2+OAzDsVFpaztz587NhWE4NpfLjSsV29q1a/8ZPffyUuu99y1BEJzlvT8QOM8592Xn3HFjx449JpvNLuzg//1iGIb7R3F/OZVKfaimpmbs7NmzX4tiOa74ObNmzXo5m80eW1tbW5tOp0eEYTg2lUqtS2hnzpz54Jo1a3b33k/AevP9TBiG+wVB8JGampp7wjAcWypRzWaztwdBsAfQF9g76vU3X0WXXC73iSimBSX+j1+n0+mdgY8AF2Hv3RFjxowZO2vWrGeLt8/lcj8Iw3DsmjVrbit1XIDvRf/XXR2sr2SdOXek0blDNsBdc801ft2Cc6OnTJnS2TG8RKTn+QlwLHA+1rnEOODfwDPAB4BBWHulr2LVwl6NI0gR2SLbAi8AlwH/C6wGHgbGAhOAOcAHgd8DdwMl2wVK95DJZH4LXOC9P7exsfGGuOORbm1H4GngO8B1QDPwBLAfcCJwJ/Ah7NzxT+CCeMKUpJk2bdqe3vvn8ssq4RCRTfFn7Afln9hF69vR4ztjP0LbYxe3r6Hk9P+3d+9Rctb1Hcffs5ck5LZgkAYwgDXREBOMiEhpvWCItVYkeOmRHoxaFeyBY1UsJV6jHEtVopYeL8mxogVKKZATslC1hFBPhMZCIiRcAkFNsoZLICHJJiHZS57+8ZvJPjM7M3thd57f7L5f58zZ32+eZ2e+2cw+O5/5/Z7fI9WrXUAb8A1gEeG8v1Py2/6REFqPApqBn9S+PEmRegbYASwBvgJsAwqXKbqG8B7hKEL++GkWBao+GFAlDcRDhFBauBTASfmvkwifkAJ0Aj+qcV2ShtY1hFGOqRRPxUtfPmQ78KtaFiUpekuA7wJ/lL8VzE212wjnZ0tleQ6qpIH6AeHyCJXswk9GpXp3K3CgyvYEaM1/VX37DXBLQ0PDlqwL0YhwE7C/yvYEuA2PHarCEVRJA/Vj4AqKPxlN206Y5iOpfh0EVgF/XWH788D3a1eOhktra+uPcNaLhs5+4F7gggrbnyOcJiBV5AiqpIHaDTxcYduL+KZVGim+Qwii5ewHNtawFkn14xpgZ4VtewgLK0oVGVAlDcY3gBfK3L+XMDVQUv17gJJrd+Z1A/9e41ok1Y/7gH1l7u8Crq9xLapDBlRJg3E3YbS01CbCp6OSRoal9D7nfCdOCZVU3XVAR8l9O/P3S1UZUCUNxmHCCEp36r524NpsypE0TP6VMK0/7Tng9xnUIql+/JDex45nCJetkqoyoEoarO9R/MdnH3BnRrVIGh47gcdS/S5CaJWkap4FfpvqdxJmZEh9MqBKGqwthD9ABb+m+uVnJNWnbxHeXEKY2n9jhrVIqh/fpPjYcXOGtaiOGFAlvRTXEFbz3IPTe6WR6ueE6xsDbAZ2ZFiLpPpxBz0LKj5Kz3FEqsqAKuml+A/C1N4DwC8zrkXS8DhMGPnoxA+iJPVfF7CCcOz454xrUR0xoEp6KV4EVgP/RXgTK2lkupawKNptWRciqa58hxBQb8+6ENWPpqwLkFT3/g7DqTTS/RZ4PeWvbShJlWwC3kD5S9NJZRlQJb1Uz2VdgKSa2JR1AZLqkscODYhTfCVJkiRJUTCgSpIkSZKiYECVJEmSJEXBgCpJkiRJioIBVZIkSZIUBQOqJEmSJCkKXmZGdWHFihXTsq6hmhUrVkxpb28fC7Bjx472NWvWtGddU4mJwOR8+xCwM8NaymkGXp7qPw0kGdVS1nvf+96puVyuAWDu3Lm75syZczDrmipJkqT9ggsu2F2L51q+fPlxDQ0NY2vxXIOxZs2aSVu2bJkEsHfv3kN33XVXbK/9McCxqf5TWRVSxQmp9vNAR1aFVDAFKLwG2/O3aJx++ukTX/nKV04GmDRp0qEFCxbE9hoscv755/8hl8sN+/H3nnvuGbdnz56X971nNjo6OnI333zz8YX+/fff/9y2bds6s6ypjPRrfy/xXac46vce06ZNG3PmmWceOf5+6EMfivH4e0RLS8tz55xzTrTvPYaSAVXRW7x4cUNjY+O2rOuoZt26dWzcuDHrMjSMli9ffqQ9c+ZMGhsbM6ymT98DLqvFEzU1Nd2Zy+XOqMVzDcbWrVu57bbbsi5Do9j69etZv349ANOnT+d973tfxhVVt2rVqqOBPcP9PO3t7X/e2Ni4YrifZ7ByuZzHjhGura2Ntra2I/2FCxfS0BDv5NL9+/e/E/hF1nXUQrz/C5IkSZKkUcWAKkmSJEmKglN8VXeSJFmYy+XWZl1H2sGDBx8gf57F9Olv2fTkk+d/MeOSSnzty7DntNAefxCuuijbekpdPw8e/Nue/mc/BicO+xSzgWhu/vytnZ2HANi8efPSs846a0nGJZX6PnBuxjUsAZZmXEORHTt2/CcwF2Dq1Ffve+aZSz6SbUWlli2Ax1O/j0ven10tlVx+a0/7jTfAByOblnnFT6B7Ymif8ARc/vlMyykxY8bPvrh586q5AJ2dnS8Cr8u4pCK5XG5OkiRZz2WN7ueyZcuWY4H7Cv3p0z/6gyefnH13hiWV8fc3wOFxoT1tI3z6q9nWU2rxVdB+amhP2A9f+3C29RSbMePBd2/efP1HCv329vbXtrS0xHae8SZG4YCiAVX1aPt55523Oesi0qZPn3640B47dmIHfLat2v619/XUSfW5JL76WncV9+c9Be/aVX7fbORyXzjS3rt37wuxvQbvuOOO/UmS+bpSO2P7uZx99tlHXvtNTc0RvvZvLFnMKrb6AC5PtY/eHV+N/3C4p90U3fF33Lh7j7wGkyQ5HNvvyMqVK4/O5XJZl5HE9nO58MIL96b748efvCu21xZckTrojzkYX31XpRZUa4zu+DthwpVFx9/bb7/9yVtuuSWqReBaW1uzLiEToy6RS5IkSZLiZECVJEmSJEXBgCpJkiRJioIBVZIkSZIUBQOqJEmSJCkKBlRJkiRJUhS8zIxGo0bgNcAbgDPyX4/Pb/sEsDqjuiRJklR/JhFyVRfQnnEtdc+AqtHoKmBRhW3ja1mIJEmS6koDMJ/igY6T8tv+Fzg7o7pGDAOqRrOtwHqgDfhUxrVIkiQpfuOBn2ddxEhmQB2cBuCPgROBHPA08PgQPOargOOAo4AXgCeBPS/xcdXbUuDbwPP5/gwMqJIkSeqffcCDwLr87SPA27MsaCTJKqCeCXw/394HvK0f33MCsDLVPxfYPbRlFfkCcEG+fQewGBgDfA64GDi5ZP82YAnwPcL88/56NXAl8JeEcJrWBdwHXAO0VnmMqwlTDQBuAL5bZd9vAeek+ldQ/ZzLXwBT8u3PAGuq7FsvtmZdgCRJkurSfuBooDt137szqmVEyiqgTibM14b+jxCOTX0PDH/tJ6WebyNhEZ0VhHBdzjRCMDwH+ADQ2cfj54CvEIJwpX9LE/CW/O0WYCFwsMx+21O1dlE5oObyj5EOwudROaBOB96Rbx8GHq2wnyRJkjQaJBSHUw0xp/j2zzhgOSGcdgD3EELrAcL00POAifl9zyeMSn69j8dcBnw81d8N3EkIgfsJI8bvBmblt3+AMPX3PYRfjLR0wDyD8KlOudHlOfQepZ1Xpcb0tgeBnVX2lSRJkqSXxIDaP+8n/Kx+RRiB/H3J9hMJJ0vPzvevAL5DCLDlfJLicPovhJHU0mWpFwGXEc6XbCQE1svy+6c9CjxFCLWNwFuB28s8bzpwHiKMSs8GpgLP9LH/3RX+LZIkSZI0JBqyLqBONBFGTN9B73AKYYrthfSMbE4G3lXhsY4hnAda8G3CAj3lrpl0GLgW+FLqvkWEYFkqPYp6boXnTgfOpfmvOcqf1N1A8bmqBlRJkiRJw8qA2n+fBl6ssv1hYG2qf0aF/S6mZzpwG2HktC9LCCEYwrmw7yyzTzpAlpu220w4lxXCisM/7WP/1wHH5tsdhNFjSZIkSRo2TvHtn6eovtJtwQPAn+Tbp1TYZ0GqfSPlFz0q1UFYxfeT+f6b6T2FNx1QTyVMO96euu9MYFJq38I5pVMoH1DT960lnBcbs6OofKmYezFgS5IkSdEzoPbPb/q5345Uu6XM9vEUr0R83wBq2JRqzyqzvQ14gnDZGgjTdq9PbS89n/QwYbGn9xMumTMD2Jza5+0l+8duPPBPFbZ9FQOqJEmSFD0Dav+80M/9OlLtMWW2v4Iw1bbgm4RLzfTHlFT7ZRX2WU1PQJ1H+YDaTQimAKsIAbWwvRBQx9AzHRjqI6C+CHyjwrZ7a1mIJEmSpMExoPZP1xA9zjEl/ZmDfJwJFe6/m55pwOkR0wnAWfn2enoCd3ra8jzgh/n2m1LP0Q783yDrrKUDwJVZFyFJkiRp8AyotdVY0r+D6gsvVdJW4f57CFN3GwijtTMJU4P/jJ4R3fRo6GZgK2GK7zn57ztMcbhdA3QOokZJkiRJGpB6CqjlpszWm+dL+l+m/+e39sdOwuJHp+f78wgBtdr1TFcDHyVMIZ5LGGH1+qeSJEmSai6ry8ykV64d18/vOW44CqmxpyieLjxnGJ6jdNpu+utBep+Puapk/4mEKb4FIzGgngLclbr9JLXtqpJtb65xbZIkSdKoldUI6p5Ueyxh0Z9dfXzPm/rYXg/2AffTcyma84F/G+LnuBv4XL79NkKwn5vv30fvKcWrgQTIAecCj9CzkNNzwIYhri8GEwn/1nLmlvR/WHYvSZIkjVY3An+R6hfWbnkjxZlmA+H9uAYgq4D6O8JqsoVzMs8mnI9ZSQPwN8NdVI3cSnFAnQ08PISPv4awmvAYwqJMl9MzUr6qzP7PEELpbMK5qk+kthXC60jzFHBJP/ddP5yFSJIkqe5MpPfipxCyVfr+SbUpZ2TJKqDuJ3yi8Pp8/1KqB9QvAacOd1E1spSw2uzLCQH9ZsI00r5GkAumEa63eqjC9v3AWnouE3NZalul6bp3EwLqeIo/CFhdfve6twtYlnURkiRJqkuXAJ/px36V3q+riqzOQYXia3S+E7iGMN037RjgWmAxYXrsSLAf+DhhtVyAWYRRuguo/P8xhjCN4Abgt/T9aUw6iI7Pf30BWDeA/UvvlyRJkhRmIP6uH7ftWRVYz7JcxXcpYeT0Vfn+5cCHCedJ7iNcJuVNhNB6EPgEcFPtyxwWKwn/3iWEUHoysJwwMvpr4A+EAHs0MB04DThqAI+/GvhqyX3/Q5hWXc4vCYs3pV8PWwhhWJIkSZJqIsuAegBYAPw3cHz+vmOB95Tstwu4iHC5lJHku8DjhEV4TsrfdxxwXh/ft4W+pwv8mhDyJ6buqzYaupfixZv62l+SJEmShlzW10F9mDA6uAj4AOH8yoJnCSOm3wbaCOHtltT24Z7TvS71fPf383s2pb6nPwsf/QyYQQjgCwjnjbaU7LMHeJQwyvkzwiJIfS1c1EkYnZ1V8lzVLCWM3BaMlNFqSZIkSXUi64AK8DxhuuvlwBRCQNtJ8aVoIEx//asa1rWMgS+kszJ/G4gO4Mf5G8Bkws8hRxg93j3AxytYPMD9f5q/SZIkSVImYgioaTvzt9Fsb/4mSZIkSaNKlqv4SpIkSZJ0hAFVkiRJkhQFA6okSZIkKQqxnYM6GI3Ax4bosdYDDwzRY2mY5HK5m1tbWzuyriNt0aJFkwvtTZtWnQrj7syynt46J/e0D4yLr76uscX9BTdAQ1+rVddUZ2fPS27u3LmXXXrppQszLKeXJElelnUNwJWtra2XZV1E2nXXXXdsof30049PjPC1X3KN69jqK3XPR2HcB7OuolhX6pJq22fG9jN87DGOHH/HjBkzvrW1dXuW9ZTRnHUBwFGx/VwOHDjQcNNNPRc0eOSRqz8FV1+SYUlldI/raW+ZG9trHzpSV6ZonxBbfRs3Mi7dv+iii36/cGFUf9phlA4mjoSA2ky4RMpQ+BoG1HpwbN+71FYulzvS7u7uaCbCGnskOTgUcX0AnVOyrqBUkorLjY2NEym+zrCCyflbNBoaev62d3d35aAr8td+7L+bXRPCLVbdTdAd1c+wq6uomwNOyKaSqEX3c0kfOwC6uw9NAiZlU01/dDfH9tovFt97j+7uXndF9RoczUZlKpckSZIkxWckjKAeAoZqatvBIXocDaHFixcfXrlyZXRzLtJmzZo1e+rUqS0Azz77bNuGDRu2ZV1TiVcAJ+fb7cCGDGspZxJwWqq/Fuj92WaG5s2bd1ZDQ0MjwPjx4x9LkmRX1jVVsamGz/XlJEmi+lQ87fjjjz95/vz5rwBob2/fvXbt2keyrqlECzA71b83q0Kq+NNU+2F6X6c8a68Fjs63/wBszbCWXmbOnHnitGnTTgFoaWlpT5IktuNvkWOOOeZALZ6nqalpXVdXV7R/25MkaZw/f/5Zhf5DDz20YceOHe1Z1lTGafSM6m4D2jKspZyTgGn59h7C8SMaU6dObZkzZ86R42+SJPclSRLV6UVpzc3NUR87hlJu6dKlR/4jcrncay6++OInsixIkiRJkjQ6LFu27NVJkjxe6DvFV5IkSZIUBQOqJEmSJCkKBlRJkiRJUhQMqJIkSZKkKBhQJUmSJElRMKBKkiRJkqJgQJUkSZIkRcGAKkmSJEmKggFVkiRJkhQFA6okSZIkKQoGVEmSJElSFAyokiRJkqQoGFAlSZIkSVEwoEqSJEmSomBAlSRJkiRFwYAqSZIkSYqCAVWSJEmSFAUDqiRJkiQpCgZUSZIkSVIUDKiSJEmSpCgYUCVJkiRJUTCgSpIkSZKiYECVJEmSJEXBgCpJkiRJioIBVZIkSZIUBQOqJEmSJCkKBlRJkiRJUhQMqJIkSZKkKBhQJUmSJElRMKBKkiRJkqJgQJUkSZIkRcGAKkmSJEmKQlO6kyTJKcuWLcuqFkmSJEnSKJIkySnpflPJ9l8kSVK7aiRJkiRJynOKryRJkiQpCgZUSZIkSVIU/h9zhnJWRK6mqgAAAABJRU5ErkJggg=="
+ }
+ },
+ "cell_type": "markdown",
+ "id": "6b214777",
+ "metadata": {},
+ "source": [
+ "\n",
+ "![](\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA6gAAAFACAYAAACxyVHuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N15fFT19f/x12cmIQFkE1wANxBFrTu4b1itimQC2mKrtfrtRtXazbbWrmJba+1m7S5fv9Wqrbb5VWBugFatpe4buGvdFVcU2cISSOZ+fn+cO2QyTCAsmXszeT8fj/vIvXPv3Dm5M3PnnvvZ3DXXXOMRERERERERiVkq7gBEREREREREQAmqiIiIiIiIJERV0fLJzrlX4whEREREREREehbv/W7AP/PL7RJU59yrU6ZMeb7cQYmIiIiIiEjPM23aNLxv6xZJVXxFREREREQkEZSgioiIiIiISCIoQRUREREREZFEUIIqIiIiIiIiiaAEVURERERERBJBCaqIiIiIiIgkghJUERERERERSQQlqCIiIiIiIpIISlBFREREREQkEZSgioiIiIiISCIoQRUREREREZFEUIIqIiIiIiIiiaAEVURERERERBKhKu4ARDbGe+8aGxsvizuODbnuuusOWLhw4SCAhQsXvvbiiy++EndMRXYBRkbzy4H5McZSSn/g4ILlu4FcTLGUdPjhhx+dTqerAE455ZSnDjzwwEVxx9QR59yDdXV1s8rxWtls9nPOueHleK3NMXv27BFPPPHErgArV65c8thjjz0ed0xFBgEHFCzPjSmODRlXMP84sCSmODpyAHYcARYAL8cYy3pGjBixy7Bhw0YCbLvttsunTJmStPNvO+l0+vJTTz11TVe/ThAEewFndfXrbK7m5ub0L3/5y2Pyy88+++z8xYsXL48zphIOxn4/wT73C2KMpZQRwK7R/FLgsRhjWc/gwYMH7rXXXgfmly+55JK5MYazUWEY3jBx4sQX446jHJSgSuJddtllbsyYMd+NO44NeeGFF3jyySfzi8dsaNuEmBB3ABuRuGP4wAMPrJsfN25c4uIr5L3/LVCWBBX4DDC2TK+1yd577z3uvffewofq44qlkxL92SL58SXOK6+8wiuv2D3LUaNGQcLPv9XV1T8HujxB9d6Pds4l+re96NyR9M9+0uMDyMQdQKH333+/3XschuExqVRyK5em0+l7gR6RoCb3XRAREREREZEeRSWo0h09jlUVSYwwDI8i+j717j1w5erVPBdzSEVW7AGt/Ww+HUK/RFWzgTVDYPUubcsDnwRaYgunBOeWHey9B2DNmjULgKRV494XGBxzDK+QsCpma9euPYioClxNTZ/cmjW9ElbFt3lHaB7WtjwwgdU/lxZUv+/7JlQvjC+WUpYdAD5t871WQp9EnX/79HGjVq1a0h/Ae58D7ok5pGL9aN/EIg4h1rQjMcIw7AUckV+uqdl2wZo1YcKadiw7EHxU2FTTBL1fiDeeYk17Qm4bm0/noF+izr+1tentm5vf3ym/7L2/C/AxhlTKcXEHEAclqNLteO8vqq+vvzPuOAqNGjVqCTAQYOTII196+ulZ58ccUpHBv4bFh9p87RpYkrD4jq+Hud9uW/7zxXDq4vjiWV91de8H165tBuCpp576y89+9rNvxhxSO42NjTO89xNjDuN/M5nMFTHH0M6RRx55P3A4wHbbjVj1xhtPJeyzP+YTMP/CtuWkfTcB3INt80dOh9tujC+WUqpvh9aoHd6Or8BriTqGu+/+4auffPLWwwFaWlqaM5nMuJhDaiebzR7inHso5jASd1zOPPPMHYB38sujR3/xxieeuDQbY0glVM2FXG+b3+m/8OKFG9y87AZNg6VRG/u+q5N2fttrr0vOfOyxK7+cX/7zn//8oYaGhrVxxlQsCIIcPbDGa4/7h0VERERERCSZlKCKiIiIiIhIIihBFRERERERkURQgioiIiIiIiKJoARVREREREREEkEJqoiIiIiIiCSChpmRnqwa2A/YPVp+Bng6vnBERERERHo2JajS0+wBXASMAfYHagrWfR+4NI6gRERERKTbGhlNAK8Dz8UYS7enBFV6mn2B8wqWffTXxRCLiIiIiHQ/w4AvYAUeY4FBBeuuBr4cR1CVQgmq9DSLgZuBedE0H3iV9icWEREREZGO7AVcEncQlUoJqvQ0/4kmEREREZHNsRKYQ1uBxzzgLmC3GGOqGEpQRUREREREOu9B4NS4g6hUGmZGREREREREEkEJqoiIiIiIiCSCElQRERERERFJBCWoIiIiIiIikghKUEVERERERCQRlKBKpZiHjXFaPP0tzqBERERERKTzNMxM5doD+EfB8geA5o08pwZ4pmB5PPD8Vo6rqwwABpV4fJtyByIiIiIiIptHCWrl6gWMLFh2nXiOK3pOr60aUdcaQ+kaAS3lDkRERERERDaPElSpFMviDkBERERERLaM2qCKiIiIiIhIIqgEVXqig2h/cyYd/R2GVRXOewd4s1xBiYiIiIj0dEpQpSe6B+hT4vHPRFPelcAlZYlIRERERLqTXWkr5IC2vGoA7ft0WQ4sKldQlUAJqvRErwC9O7Hd4q4ORERERES6pQeAHUs8/j/RlPcH4PwyxFMxlKBKT7Rv3AGIiIiISLe2DBuicWNWdXUglUYJqoiIiIiIyKbZK+4AKpV68RUREREREZFEUIIqIiIiIiIiiaAEVQp1ph69iIiIiIhIl1CCWrlWFy13ptfaUj2RiYiIiIiIlIUS1Mq1rGh5eCeec0RXBCIiIiIiItIZSlAr1/u0HxT46I1s74DPdV04IiIiIiIiG6YEtbLdVzD/eaB6A9teBBzeteGIiIiIiIh0TAlqZftTwfwHgFuAIUXbDAGuBn4KLC1TXCIiIiIiIuupijsA6VIzgLuAY6Pl04EJwCNYG9XtgQOxz0ELcBYwu/xhbhrn3KFBENTGHUehr33ta+u+S8uXv90PPnZknPGsb+3AtvlcKnnxvT6q/fLvDoEbmuKJpbQwDNfNb7/99iODIDg1xnDW473fIe4YgNFJOy5XXXXVus9+c3NTOnmf/UW7tl9OWnzFXt81eTH6gmuZVYk7/y5b9vqg/HwqlUon7TsC7BF3AEDijsvChQsH3nzzzeuWFy26f1TSPlvgCwqaVg5IXnxr+7fNtybu/Lto0ZIRhcuTJ08+5ZxzzmmNKx5powS1soXAGcA/gQOix2qAo4q2WwKcC/y7fKFtkSviDqBYr1691s2//vqjI+DRq2IMZyOaa+CvCY4PYNb3446gWGvBT9bOO+98BvbdkvbOjabE2HbbbdfNL1q0oA8sSPhnP+nfzf9mbEqqRbsm7RguWNA2X1VVVQvMii2Y5KohYcdlwIAB7ZbfeuufHwU+Gk80nfHOnkn77Le3qnfS4nvjjfbLNTU1M+OJRIopQa18C4HDsDaoZwIHY1W7Q2AB8Dfgt9F8NTCt4LmLyxqpiIiIiIj0aEpQe4Y1wC+iKQUMBJqwar2FWkhgT76XXnqpb2xsXBN3HBuSTqerq6urUwC5XC4XhmHSqoikafu+h6z/3sfNAb0KlhP3fldXV/fC4gQ7fuEGNo+V975s728qlVrjvU/c+5WXSqWqqqur0wDe+7C1tTVpn/0U7TuwS+KxrCmYT+Jnv5q2PjVyQKLOv6lUKp1Op6sAqqqqknj+baelpcWX6aVyJPPzvk51dfW6z35ra+ta7325jk1nFX72W7FjmiSJvvZwzqWqqqqqC5YT/Xkkee9vl3HXXHPNui+bc270lClTno8zIBEREREREekZpk2btqf3/rn8snrxFRERERERkURQgioiIiIiIiKJoARVRCqZ2/gmIiLr6JwhIhIzJagiUomqgG8Ac9B5TrrGsLgDkK3uKOBRYP+4A5GKNAjoHXcQIt2BLtxEpBLtBnwXOBn4WryhSAUaDTwP3AD0jTkW2XouwcYMvwmojTkWqSwO+1w9jg39JyIboARVRCrRi8AXo/kfAofHGItUlhrgZiwx3QlYHW84shV9Cngb2A/4acyxSGX5CnAqsDM6Z4hslBJUEalUfwRuwcaJuwVVyZSt4yfAQcBC4CySNyaobL73gHOw9/TzwP/EGo1UijHAFdH8F4EnYoxFpFtQgioilew84FlgV2A2MCDecKSbm4JdYHrgM8A78YYjXeAO4AdYlcxpwPh4w5FubjgwHegF/B3433jDEekelKCKSCVbhl1gvoW1LbsVq6IpsqkmAr+L5r8LNMYYi3Sty4DrsNoXfwMOiTcc6aa2BW7DqvU+AXw63nBEug8lqCJS6V7DOktaCnwQK0ntF2tE0t0cg7U7TQN/AC6PNxzpYh74LDAT2AZLMo6ONSLpbvpin599gAVY+9NlsUYk0o0oQRWRnuApYBKwAktSb8PubotszAnYTY3eWKnpF+INR8okB5wN3A0MBP6JqvtK5wyi7abGMiADvBlrRCLdjBJUEekp/oMlp+9jvfreBewea0SSdGdiY+lugyWnZwCtsUYk5bQCq33RCPTBSsQ+GWtEknTDsN+WI4FFwIdQp0gim0wJqoj0JA9j1TVfBz4AzAc+EmtEkkQO+DY2bmF19Pd0NDxET7Qaq32Rb5P6R2z82z5xBiWJNBa4H9gXeAM4FvvNEZFNpARVRHqaZ4EjgHuB/lgnKFdivSyKbAsE2Pi5KeBq4FygJc6gJFY5rIOb70TznwDuAfaIMyhJlPOwz8QuwH+x6r3PxhqRSDemBFVEeqI3gXFYYgpwMVaaenhcAUkiHAM8AkzASs4+CXwZjXUq1nHS5Vib5HewsXAfB6ZiJavSM20L/Bn4PdZD/N+Bw7DO+URkMylBFZGeqhW4BPgwsBCr8nsv8Es0XmpPMwArKZ0LjABexG5WXB9fSJJQ/8Gqct6GdZx1KfBA9Jj0LBmsfelZtP2eTAaWxxmUSCVQgioiPd10YC9gGtb28EvAy8A3gNoY45Ku57COkJ4Bvhgt/wEYgzo2kY69iXWedAbWEc7BwENYc4FRMcYl5TEa6zgrCwwHngSOwmrk+BjjEqkYSlBFRGyM1M8BJwFPY9W2foy1Ifo0VnVLKsuJWHXev2A9b74UPXY+KgGRzmnAal78CUtMJmPnj98Au8UXlnSR7bCaFk9hzQBasKT0EOwGhYhsJUpQRUTa3AHsj5WMvIpdZF6LDbQ+FY2d2t05rOTrTuB2rORrGfAtrOfNO+MLTbqpd4H/AfbDEtZewOexGx4B1h5RurddgJ8Dr2A1LaqAGdh7fgmwJr7QRCqTElQRkfZC7EJzH+Cr2JA022NtzV7DEtYjYotONkcN1vPqo8A/gOOxi8qrsLFwrwCaY4tOKsEz2I2tcVj71BRQh7VPvRtLYvvGFJtsnoOAG7E26Rdh7999WA+9pwHPxReaSGVTgioiUtpq4BdYAnMO1mPnNliV3/uwqnwXY53qSDJ9AHsP38DGrjwAaMIS0z2xi873Y4tOKtF/sFL6A7Hxc1uwhOY64C2srfsHgXRcAcoGDcCaezyE9ex+NtZL8x3AKVhb03tji06kh1CCKiKyYS3YXfQDsWFIrgdWYiWsV2IdKj0CfDN6TOK1G/A14EGsrdhXgCFYSfglWHW9i7Bq2yJd5XGs1H5X7HP3PDbu8meBfwFvY8nqyagztrj1wXpz/wt2E+EPWLvSNdgQMgcDHwL+GVeAIj1NVdwBiIh0I/dE0xex6nwfw6r0jYmmH2FtV+dE039QhztdzWE3D8Zj1e4Kh/tYi7UDvBardqnxTKXc3sZuZP0Eu8H1cexzuh2WrH4WWIW1f56NtY1+MZZIe5btsZsDGeBU2le/fgo7Z9yEaliIxEIJqojIpmsC/i+ahgCTsDvw47ASvPOjKYcNV3I3bcnt22WPtvLshF3sn4glpkML1uWw8Uz/H/B34L1yBydSggfuiqYLsHPFR7B2qjtFf+uibd+m/TnjCexzLZuvD9Zh1TjsnDGG9rUIX8bOGQ1YjRgRiZESVBGRLbMIu9t+LXYRNA67ADoJa+d4UDR9Mdr+Jeyi827gYaxzldayRty9OKzq9NHRdAxWbbLQCqyN2Gysd00lpZJkOaya77+wG1n7Y+0bxwOHYzdczogmsFoY92FtH+/D2kYuLW/I3c4QrL3oMdHfMVhb0jwPzMM6TZsezYtIQihBFRHZelZhSdLsaHkH4FDsAulorF3T7tF0brRNC/ACdoGUn+ZH++qJdqAtqT8cO3aDi7ZpxY7T3dgF5t1YdV6R7uiJaPoJdl12AHa+OArrUGkwlsCeUvCct2l/zniEnls7owYb8uVgrIr/UcDe2M2tQguwc8UdWBOMhWWMUUQ2gRJUEZGusxBrAxlEy/2AI2lLWA8CBmIlhPtgnaqAJWDPYsOiPIl1uPIElXdBNYy29rv7YL3ulupoaiXwGFbyfC9WTXJZmWIUKaf8zZd5wNVYNdR9gWOx88ahwEislLWwWjBYBz+PYueK/DnjBSqrhsY2WJvz/PliDJaU1pTY9mXsfHEPlpS+XKYYRWQLKUEVESmfJqwnyMLeIEdid/4PKvi7A1YisF/R89+l7cIzPz2NlcImWW/sYvIA7H/aP5rftsS2OazH00exUqF7ovlKusgW6ayQtu/6b6LHBtJWyyB/3hiN3fAZBkwoeH4zdo7I7+NJ7GZP0jv/cdgQXvsXTAdg58tSI1C8i50n5gP3Y4np4rJEKiJbnRJUEZF4vUxbBx15w7CLzvxF2f7AHljPkx+Kpry12AXoY9H0KJbExtV78DDsovkA2sdeatzHfOzzsbjzsa8sS6Qi3dNS4N/RlNcH+64dSNv3bl9saJt8LYVCC2h/zngM64E8DvnYD8Di3x+7kdWvg+0X0Ha+yJ873uj6MEWkXJSgiogkz1vR1FjwWL4UMl+asB92MbctbSUpeR5Leh/FBpy/D6sy2LwVY0xhnUAdGE0HRX+372D7hbRVPXwymn8GtR0V2RpWAQ9EU15hKWRhzYXdsfGAdwHqC7ZfgiWq87ESyPuBd7ZynENoO1/lzx17UvoGVnHpb/78kfTSXxHZQkpQRUS6h9VYldfiIRDy7TgL22TtTVtnTB+JtmvFLu7uxZLVu9i0EpPtgCOwdnBHYheYfUtsl28/+xjt28JVWvtZ6YSJEyfuAwxLpVJPTJ8+/d244ylWV1c3Lp1OV82cOfOOuGPpAvkbVS9jvVvnbYNVCc6fL8ZgNTYGAcdH01ejbd+mre33POyGV2dvKvWK9ps/bxyGDalTylvYOeNxKrf9rIh0khJUEZHuLV/aGhQ8Noi2UoojsIRyKOtX9XsF69XydqwTkcLSkpG09SR6NKV7xVwBPIeVhKoHYllPLpf7qnPuU7lcbjLtq7EngnNuRhiGA7AaAT7ueMpkBW3f1xuix6qxm1wHYh0xHYmVug4FJkcTWDv6B7Ahcu7AammE0boBWE/l+fPGUVjNj2LqgVhENkgJqohI5VkC3BlNebthF4z5hHV/rPrfCOAc7OL8Laz951DWb/8VYtXt8mMxPoiVcISIJFB9ff3F3vsrvfdTGxsbL4s7noRroa308k/RY/1oGybriGgaQPt28MuxG1t9geEl9vsubZ0W3Y+Vkq7okv9ARCqGElQRkZ7h1Wj6c7TcH/gkcCZWalJD+wvMfML6b+AvWFKqoV1kk4Rh+H3n3O9I6BAfuVxunHMuTc8pPd0UTVhJ6b+i5RQ2LutnsWrA22Hnkf4Fz1mB1aL4K3Ab8GK5ghWRyqEEVUSk53BY9buPA5Ow4Wzy1mIXky1Y9d5+WMJ6NnZRejNwI1bCItIps2fPfg14Le44OjJ79uzH4o6hGxgOnAWcgTURKKzq/wZWSrodsDPWvvVY4HDgH9g5o5Gt20GbiFQ4JagiIpVvD6wa79lYVd+8xdjF40xsbNb88C7VwHHAaVjbs2FYpylfxdqM/Qa4hW520Tlx4sR9wjC8ELuA3h7rDfQVYGZra+v1c+bMWVOwuauvr/+o9/6TWPvbKu/9K865W1pbW6cVbUtdXd2hzrmvOefu8t7/EbgEuwmwHfAS8LMgCGYATJgwYe90Ov1t7/3hQB/n3APe+28FQfDfwn3W19eP9t7/wHv/xLBhw658++23LwI+ilXBft17/5vGxsYbACZNmrRba2vrt5xzx2LVMB8Lw/A7s2bNmle4zwkTJuybSqW+BzwUBMHPio9RJpM5GLjEOXdvNpu9uuDxk4DPOOeyNTU1M1avXv0t51wmOo4LvPfXNzY2/o6iksj6+vrzvffHA78IgqCwh9n8+uO9958FxmLje74LPOW9/2tjY+P0/Hbjx4/fqaqq6jSsaulI7ObK+8DD3vtfNzY2PlS03997748EcM5NzmQyH8ivc841ZLPZhuj/uh7oEwTBR0vEvkMYhhc558YDO2LVWR/03l9d/HrR9lO99/uk0+lvhmHY13v/LSxRqwYe9d5f3tjYeH/x8xKqL9bB2iew0tL82KM5rLruDOy8UVgyPgr7zH8Uez/ro2kx8H/A77Hvm4jIBpUa7FhERLq/FDAemIN1ZPQdLDldClwLnIBd5J8L3Er7sUdbsA5QPo+VntQDf8MS0jHAddhYhJfTvhQ2sTKZzGfCMHwcOB/rROoJrGfkY4E/1NTUDM1vO3Xq1FQmk7nJe38zdpzecs696Jw7GPhVVVXVv8ePH9+/6CWGA5O998di1aK/iZU0rcba8N2ayWTOzGQyR6dSqQe99xnn3FKg1nt/GnB3fX39sMIdOucGA5Odc8e//fbbWex493bOLQfGOuf+VF9f/6UJEybsm8vlHnbOnYV1UJUGTkmlUnMnTJiwd9E+d4j2eWSp4+ScGxr9H4cVrRoVPX5Mc3Pzfc65r0f/3zLgYOfcb+rr668s3l8YhmOj5xX33urq6up+7b2/E6tmHnrvH8c+t6c756YVblxVVfV14FdYDYA1WHvoPsDZzrl7M5nMmYXbe+/3o63H2HxP1/mp8DhPoq0DoHUmTpy4n/f+CefcxVhy+iT2vTjLOXd/XV3decXP8d4fB0xubW093Xv/ADDeObcIS1AnOOf+PXHixJLHPUFGAD8FXgeuxz7/AP/BqvbuiN28uor1q22/CPwM6yhpH+BHWOn5tsDXsTbrM6Lni4h0SAmqiEhl6QWcB/wXmA2cgpUMNWJV9IZiF5p30rkhHFqwHoI/il2cfhlry7od8C3sAvQa2l/0J0p9ff2xwB+wJOhTQRDsFATBiUEQHNzU1DTEe3+uc64pv/28efMuxKo0vgUcEATB4dls9tjq6uqRzrl5wBHpdPqqDl7udMCnUqmRQRDsFwTBSOfcFwDnnPsJcJNz7g9NTU2Ds9ns2Nra2uHOuX8CQ7z3X+tgn8cBuzjnPhAEwd7ZbHa0c+6jAN77y1Kp1M3AzNra2u2CIDh46NChw7Eq2dukUqnvbPkRbOdTwJtVVVU7BkGwbxAEezrn6rFje1Fxkt2RTCbzJefchcC7zrnjgiDYq7Gx8UNBEOxTVVU11Dn37aKn3OW9PzEIgsFBEIwJgmBcEAS7Ouc+gSXKfyi8aRAEwdHOuSsBvPdXB0Gwe34qLBkuZcqUKdVhGDZgpcPXtra27hwEwQlBEOztnDsTCJ1zv45Km9fjnPuR9/47Y8aMGZTNZsc2NTUNw24K1YRheHlnjk8MDsWSxxeBr2E3cV4Avo0lreOw/2FRJ/f3bPTckVipd0P0+ERgLjZ0TWarRC4iFUcJqohIZeiFVeN9FqtKtwdWJXEaNt5hBrtI3JJqucuAq7HStNOwjpNqgCnYhe0vsGqaieK9vwwrVby0sbHxOgqqcs6dO7e5sbHxhunTp78PVnoalQ7ivT8vCIKn89veeuutbwMfA3LOuXM6SMZawzA8a+bMma/nH8hms78FXvXe7+ScW5DNZi+eO3duK0BDQ8Nq59yl0aYdla6lnHPnZrPZ5wr22YCNSTkASDc1NZ3X0NCwGmDatGkt6XT6WxvZ5+ZaWVVVdW7+eEWxzMJuYqS994dubAfjxo2rxUr0SaVSH89ms3cVrp8+ffr72Wy2XQlqEAR/b2xs/Bftq+H6bDZ7U1Slun9VVdUpm/9vtXnrrbcmYuOEPjd06NALCqtzZ7PZW7Aq7lVYIldKY2Nj4y+mTp0aAsydO7c1nU5/HfvuHc76wzXF6XDsvXsASx4dVnviDKxq+4+w2hKbKyzY3z7A77DjcBSQxcZjPnwL9i8iFUgJqohI9+awTo9ewoaHGIlVz7sQKy39HFaaujXlsNKWo4BjsNLY3sBXoji+hCWEsZs8efIALEbCMLxmY9s//PDDo6PqqEvGjh07q3h9Npt9ERsuoyqq0lls/qxZs4qrPnrsxgHe+1uLn+C9fyaa3aWDsF7JZrOPlHg8/7xsPuHNmzFjxqtYdd/hkydP3prvxdzp06e/W+LxJ6K/u21sB/379z8CGAy8OHPmzDs25cXHjRtXW19fP6q+vv7YiRMnnjhx4sQTnXPNAM65fTZlXx1xzp0Yzf512rRpLcXrwzC8KZo9oXhd5G/FD8yYMWMpVtug9vTTT99xa8S5hfbEalXcD9RhtSn+COxFW4lnbiu/5vNYs4FdgSux6u/HYDe6/oZ1siQiogRVRKQbOwgrgbgJa2+3AKuCuyfwWyxB6Wr3YBfqJwFPYe3NfomNk1qyCmQ5rVy5cgSWLL87a9asJRvbPpVK7RbNvpovASvhpejviOIV3vuOeqxd2tH6bDbbhCUDgzp4bskSLO/9MgDn3IZesxrrWXVr6aiTmyVRLBstQffe7x7NPt/ZFz355JO3zWQyf+zXr99i7/0L3vv/hGF4exiGtwNfiPZb3C54szjndov+lhwap0+fPvmhU7afPHlyqWO7wWOUy+U6ep/LoS8wFbuhMAGrwn8jVsvi02zCe7IF3sU6Edsd6zzJY+2An8LOX7o2FenhdBIQEel+emNVbR/BOo1ZgV3w7RE9HkfvurdjCfPnsHZqY7Ak9cdYkhSLdDpdCxB1SLRR3vvaaHZDY77m19UWr3DOrVfiVqQz7X6Lldync84DhGG4OfvcLJ34/zqzj/xx69R7Mnny5HSvXr3mYOP2PoGV0E9yzh0XhuFY59wP87vehyroNQAAIABJREFU0tgAvPc1AGEYLu9gkxVEpYvNzc2lPgNlez82UR3WrvRSrGp+I1bt9pzo8XJ7G/gM1qnSvdh4qlcBd2PJq4j0UEpQRUS6l0OA+cAXsXN4I1b6cSU2lmmcWrE2r/tgVfaqgG9gPYCOiiMg7/370d9dpk6d2pnfvHwpa3Gvs+s45/JVERdvYXhll0+evPcd3TTo8tK9/HuCVfXcqObm5uOwTnzmDx069JggCH4VBMHMbDZ7VzSMzlatiuqcy5cGl/wMrF27dhhWKp9ramrqVJIdsz7YjassVu3/BSxZzWBtx+M2H6vqOwVowtpNz8MSZxHpgZSgioh0Dw7rWOY+rJ3Ym8DJ2EXmlnRi0hXew3r9PQNL4o7ALjjryh3I2LFjX8JKdGvnz5+/0SrH0VAnIbDraaedNrh4/dSpU1Pe+zHR4qNbNdgycM69Gc121Nvu/l0dQxiGD+Zfq4Mqsu147/eMZu8q1Sa04P0ofjzfNnWTSvC9949FsyU/L977Q6L9Plnc9jeBDsWGyPkiVpX2p8B+wHrtq2Pmgf/FPn//wTr/+hPWQ3hsNTBEJB5KUEVEkq8v1mnJD7BSyQbgAOC2OIPqhHycd2PV96Zj1TPLZurUqWHUyyve+5+NHz++poPtUgBRO9V/ANUtLS3fLd5u/vz5n8E6M1rQ1NR0b9dF3jVaWlpex0raD5g0aVK7apTjx4/fyXu/3vieW1vUidSdQL/Vq1dfUWqbwtLuVCr1VjS7XidI0RBC40vtwzn3RvR3t02JL5fL3YIlTB8rHkd23Lhxtd77bwOEYfiXTdlvDD6KDekyEuug6QTgYmwc2aR6FfggNkRNDitV/QfWqZaI9BBVcQcgIiIbtAswEzgQa1t6Hlay0F28AZyIddr0GawDpb2BC7CSyi6XSqV+6L3PAMdVVVXdV19ff1XUc+5AYF/n3Ccfe+yx07CLY1Kp1NfDMBznnPtSJpPZJgzDP1VVVa0NwzDjvb8YS16+1A1Kz9YzZ86cNZlM5m/A2blc7h/19fWXee/fwkquvoKVxn+gq+Nwzl3gvX/QOXdhJpPZ1Xv/v1Hp7g7AIfPmzZtEVILZ0tJyf1VV1SrgpEwm81vv/U2pVKrZe39ylCy+iA0L005ra+u8dDqd895PrqurW+Wcy3dudW8QBPd0FNvs2bOfz2QyVwNfTqVSd9bV1X3POTfPOTfUe//NKK5ne/fu/Zute1S2mhRwOVa93mE9bp+DVZ/tDkJseJvHgb9gCeuDWI2RlzbwPBGpECpBFRFJrg9gF2YHAu8Ax9O9ktO8tcBnsQ6UWqK/f6ZMN0mz2WxTa2vrcd77LHCw9/5GrMrxv4CrvffDwzBcnd9+5syZzzjnTgJeBj6dSqXuCsPwAaxUZznw8SAIZpQj9i7yFayDrVHRsfgX1jnNv4nGJ+1q2Wz2uVQqdQzW/jDjnMti78ls4LLCbefMmfOe9/5srHOiC5xz93nv5wNXOOf+BPys1GvMnj37Ne/9Z4ElzrnPYh12/bhgGJkO1dbWfs0591NgsHNuGjDPe9+IDa10Z3V19Qn5cWcTphq4Ges0DSxR/TDdJzktNAtrc/8c1mnSPZQoRReRyqMSVBGRZNofG+B+O+AxrK3pG7FGtOWmAW8B/w/4GFa6czale7b9OZYQbpUeiefMmfMeMPHUU0/ds6qq6mjv/XbAIu/9SytWrLinuDQ0m83eO2XKlL0WLlx4XBiG+2Cd4rxSW1t7R0NDw4ri/VdVVf27paVlbHV19fvF6yLfDsPw5wVDlBQ7NN8rb16vXr2eWLVq1VjnXMneZFOp1C9aW1tvSqVSJYeZcc6Nz+Vy1Vhit04QBIsmT558+KpVq050zn3AOdecSqXunjlz5pOTJ08esGrVqrGpVKpdB1Ctra0NqVTqwTAMF3bwWn/J5XJ3pdPptwsfD8Pw+86532HJfjszZ858EhibyWTGOufGhmG4DbDQOfdUEATzC7dtbGycnslkRgDHASOcc4tzudzcWbNmvZzJZIaEYfhoVVXVeuOzNjY2XgdcN2nSpIEtLS07ATWFMeZyuXHOuTRWKr5OQ0NDDrj49NNPv6q1tfUEYFgYhstTqdSD2Wy2ZNvjXC73Oedcv1Qq9Wyp9cAnwzDs29TUVHL4mq2gF3ALcBpWjfd/ouXu7HngWKyX8P2xquEnAE+X2PYUrHfimWWLTkS6hLvmmmvWnZSdc6OnTJlSjjGwRER6uv2AHwITS6w7CGtfOgQrVTqJbthj7AZMAv6KXVDfCJxL+wThbKykeB+s9EREzJ3Ap4iqoxfohX2nJmHJ6WQgKGtkXWsIdsPuAKw2yZG0H292EFbVeyZ2fESkG5k2bdqe3vt1v/eq4isiEo/RwKlYclZoFFZaMAS4HystqKTkFKxN3Iexqr+fAAo7IzoWq266gk4OQyLSg4zBOh8rrAHngJuw5HQV1lt2JSWnYD1xfxDrOXtH7P8bEK3bBqumPgCNnypSEZSgiojEYzR2kfkz7MISrBSgEeuxMt8pyLJYout6jVhbVICpWKnp14G/Y8n5NsBucQQmklD9gNVYJ2PTsVJTsN69J2Mlp3VYSWMlWozd1Hsda5//VyxhfxTYF6uGPzy26ERkq1EbVBGReOSHrxgCXAvUY8NBjMaGhJhI9+zYpDO2BXoD/8Sq+H4CuAG7+O4TbZOiDL3JinQju2KlpX2xmhVPYN+bb0XrP4t1dFWJqrAezZcCX8Da1p6M1bjoXbBdbflDE5GtTQmqiEg8RhXMD8aq+qawsf9uBcZhJSSPAU+WO7gudj5W6vMuVurhsQvvPkXb7VnmuESSbFfakrHe2M2sH2LfnYejv2dg36eGOALsQrtgCXkr1nFaOnq8d9F2VViSulU6VxOReChBFRGJx/ZFy/kmF2lsGJC12HiAx5UzqDK5HLuI/DxWrbkju5QnHJFuYSRW9b2Qi/4eAvwRO2dcS+UlqC8Dh2G1LjZWjXdn4IUuj0hEuozaoIqIxGNjVdEWA4cDD5Uhljh8FytFXbqBbfqVKRaR7mBf2hLSUlZgQzldUJ5wyu5p4BisCUTYwTa1qHM1kW5PCaqISPltQ8fnX491AnIM8HjZIorHVVhHSYs6WN+Ltqp8Ij3dHhtYtxi4FLiwTLHE5RXsxt1LlB4/uR8woqwRichWpwRVRKT88p2dFAuxqmxHYGP69QR/A84C3utg/dAyxiKSZDt18Pj7wJeAq8sYS5zeAQ4FnsGaQhRKYeMni0g3pgRVRKT8Cjs7yWsB/ou1s3qz7BHF63ZseIyFRY/XoHaoInnFnYiB3dj5H2wc1J5kKXYj7xGs9+9Co8sfjohsTUpQRUTKb3fad3ayGuut9zCsNKQnegj4IPBGwWP9UXsyEbDq7r2KHnsXOB0bU7gnWgUcD8ylfa+9o0puLSLdhhJUEZHyO4i2Kr5rsAuso7BOTnqyZ7C2t29Fy6quJ2J2ov1NrXeADwH3xBNOYqwFJgA3Y0N0wfo9HYtIN6MEVUSk/PKlgi3YmKd10bzAq8DB2DAROWCvWKMRSYZdaWtv+S5wNDYuqFjHcp8CfgoswYZQVOdqIt2YElQRkfLbGVgG/BbrIKijIRN6qoVYJyj/RVV8RcC+B9tgN272x3qxlfa+iSWpQ1DnaiLdWlXcAYiI9EADgB9Hk5S2FEtSPxR3ICIJsAfwLFZyuizmWJLsCuBJdIxEujUlqCIi5XcYVpVVNmwVMDPuIEQS4NfAD2jfGZCU1lM7jRKpGEpQRUTK79W4AxCRbuWduAMQESkXJaiSeN5719jY+ELccWzIFVdcscMLL7xQA7B69erlK1asWBp3TEX6AwOj+bUk72KnBtihYPl1rOOLxBgyZMjOzjkHcP755793yCGHFI+9lyQ3ZjKZy8rxQtls9u/Ouf3L8Vqb469//evA2267bQBAS0tL85IlS4rHWo1bLe0/+6/FFcgGFLYDXkjySvF2wI4jWNXORJ1/+/btO6Bv374DAXbZZZc1l156adLOv+2sXr36oDPOOKPLexQPguAE4A9d/Tqba82aNe6CCy5YNw7z0qVL31m7du2aOGMqYUfs9xPsc5+0qs0DsSYtYOeNRJ1/a2pqagcMGLDu/Hvttde+Fv3MJ9UnM5lMj+i5WwmqJN5ll13mxowZs3vccWxIU1MT7733Xn5xcDQlVTU2DmeSjYw7gGKLFi1aN5/L5YbFGEpnbFfG19qFBI872NzczLvvvptfrAb6xRhOZyT2WEZ2ijuAjRgSTYmxcuVKVq5cCUD//v2rSfh7PGjQoLL0gOu938Y5l9hj4b0vPHeAdW6XZNtR3nP/pkrc+XfNmjXt3mPv/agkJ6jOub5xx1Au6sVXREREREREEkElqNId/cI591zcQRRqaWm5CugDsMMOe725cOF2N8QcUpF5H4NVI2y+Zi0c+vN44yn2ygHwxqlty4f9CnqtjC+e9VVVPfDN1lYbqnTBgpX39+8/eG68EbU3YsTSD/fundszzhiWLq35z5tv9rk/zhiKLVmy+hPAcIBBg4Y1L1ky8pcxh1TkxcPgnePblo9OYM/O91zSNr/bv2GnB+OLpZT7vgxhVMV30FvwgUSdf3fccelH33nnqREAuVzY8vTTgxJ1/u3fv2XozjuvODfeKFzL008PTNRxaW5e2hf4Qn55yJDD5yxaVPV4jCGVcN9FEPay+cGvwt63xBrOeuadDaujWhe1a2DsVfHG097226895N13Hzohv9za2vr56urqXJwxFXPO/R5IbrFuF1GCKt2O935WJpO5M+44Co0aNepKogR1yJCR7y9cOGtGzCEVGXxCW4JalYO7Ehbf8WH7BPV7c+DUxfHFs75Uqvc3wRLUV19d9fyNNx6RqGN40023HxV3grp4cc3Tl1xy1PQ4Yyh22GFrJhAlqH37DmpZsuTuRMUHY7Zpn6AmLT4AV5Cg7vE03JawGKsvaEtQ+y1O2jHcbrsPjytIUHNJ+4586lPP7BN3guo9iTsuO+10/7YUJKjDhp0yf9GiS7MxhlRC1Zfa5ge+l7TPPgwa35ag9mpJWnzDhl1SW5ig3nLLLdc2NDSsjTOmYkEQ/I4emKCqiq+IiIiIiIgkghJUERERERERSQQlqCIiIiIiIpIISlBFREREREQkEZSgioiIiIiISCIoQRUREREREZFE0DAzItAX2ANIA6uBZ+INR0REREQSLAWMBsYCY4C9sOvIZ4AvbeB50glKUKWnupC2k8re2EkF4Elg/7iCEhEREZFE6wMsBLYpsa5vmWOpSEpQpaf6ddwBiIiIiEi3k6ItOV0AzAP2wUpUZStQG1TpqZ4ArsNKUo8Abo43HBERERHpBtYApwI7ALsCpwOPxxpRhVEJamX6GtamEqABuKMTzzkXODKa/w/wly6IK0kOKFr+bCxRiIiIiEh30gLMiTuISqYEtTJlgGOj+efoXIJ6HPDJaL6Vyk9QRUREREQkYVTFV0RERERERBJBCaqIiIiIiIgkghJUERERERERSQQlqCIiIiIiIpIISlBFREREREQkEdSLr1SSbwGf7mDdPti4VSIiIiIiklBKUKWSbAuM7GCdK2cgIiIiIiKy6ZSgSiW5AvhdB+tUeioiIiIiknBKUKWSvB9NIiIiIiLSDamTJMlLxx2AiIiIiIj0bCpBrUytBfPVnXzOtl0RSIL1BXoVLNdEf9PAoILHc8DycgUlIiIiIol3AXBAwfLY6O9I4JqCxxcAl5crqEqhBLUyFSZUnU089+uKQBJsGnBWicf3ARYXLD8PjC5LRCIiIiLSHZwM1Jd4fAdgSsHyfJSgbjIlqJXp9YL5gzqx/eHArl0Ui4iIiIhIJbkBuK8T2y3s6kAqkRLUyvRIwfw4YGfaJ62FqoCfd3VACfTxaBIRERER2RR/jzuASqZOkipTI9AczVcDf8LaXBbbFmgAjgR8eUITEREREREpTSWolWkx8HvgK9Hy8cB/gb8ALwK1WMPuDwMDgeeAZ4DTyh6piIiIiIhIRAlq5foucBhWOgqwE3Bxie1eATLAN8sUl4iIiIiISEmq4lu5VgInAT8DVnew/lrgYOAFYBWwJJpWlSlGERERERGRdVSCWtlWAl8HvoeVpA4HHNZh0iO0H47mwmgSERERERGJhRLUnmE18K+4gxAREREREdkQVfEVERERERGRRFCCKiIiIiIiIomgBFVEREREREQSQW1Qpdtxzn0zCIJPxR1HoW984xt98vOvvz5/Zxh+WZzxrK9p97b5Nb2SF9/Kndovf/rrkFobTyyltba2rJvff/+Bx3z603fsEGM46+nTp2WvuGMYOnTliddff8fIuOMo1NDQd+f8/OLFb9Qm77O/fET75aTFV+yRD8HwUXFH0V6ud9v8op2Sdgxfe82vO15VVenq66+/I1HxVVfnBsQdA/heSTsuq1ev6HX++W3LL730fxNh2pj4Iiol7NU2/87IpH32YcUubfOrE3f+ffHF1G6Fy2efffb155xzThhTOB1xcQcQByWo0h2dGHcAxdLp9Lr55cvfGQScEl80G9OahrcSHB/AOx+MO4JiYcFP1sCB1SMHD25OVCKWBL175/bs3Tu3Z9xxFOrdu3rd/KpVy6phWcI/+0n/bi4ZbVNSrRoIqxJ1DJcX9JefSrn04MHNiYovCZwjlbTj0tzc2m555crX9wX2jSeazlg5GFYm6hi211KVtPPbihXrPXRmDGFICariKyIiIiIiIomgElRJvEsvvdQ3NjZeF3ccG7LLLrvsV1VVNQDg/ffff33BggWvxR1TkZ2BXaP5JuDxGGMppR9wQMHy/UAuplhKOuCAA45IpVJpgJqammeB92MOaUPuL9cLpVKprPf+yXK93qYaNGjQbgcddNBOAKtWrVr63HPPPRV3TEUG0r5U5p64AtmAowvmnwKWxhVIB/bFjiPAG8Cr8YWyvuHDh++0/fbb7waw/fbbrwAeizeiDVuyZElZmlek0+nXwjBM8m97+qCDDjoiv/Diiy8+3tTU1BRnQCUcgP1+AryGjXOfJLti1x8Ay4BE/VYMHDhwwIgRI/bLL6dSqSSef9cJw/CNuGMoF3fNNdf4dQvOjZ4yZcrzcQYkIiIiIiIiPcO0adP29N4/l19WFV8RERERERFJBCWoIiIiIiIikghKUEVERERERCQRlKCKiIiIiIhIIihBFRERERERkURQgioiIiIiIiKJoARVREREREREEkEJqoiIiIiIiCSCElQRERERERFJBCWoIiIiIiIikghKUEVERERERCQRlKCKyJb4CfAI8JG4AxGRLrMf9j2fEXcgItKtjMHOHQ1xByLdS1XcAYhIt7Yr9gO0Y9yBiEiX6Y19zwfHHYiIdCt9sHPHNnEHIt2LSlBFZEvkor/pWKMQka7UGv3VTW0R2RQ6d8hmUYIqIlsin6Dqx0ekculGlIhsDp07ZLMoQRWRLZG/O6ofH5HKpRtRIrI5dO6QzaIEVUS2hO6OilQ+3YgSkc2hc4dsFiWoIrIl1L5EpPLpRpSIbA6dO2SzKEEVkS2hHx+RyqcbUSKyOXTukM2iBFVEtoSq74hUPt2IEpHNoXOHbBYlqCKyJdQBgkjlUymIiGwOnTtksyhBFZEtobujIpVP33MR2Rw6d8hmUYIqIltCd0dFKl/+ItOh6wYR6TzVspLNoh8aEdkSujsqUvlaC+b1XReRzlI/FbJZlKCKyJbQj49I5csVzKskREQ6S7UvZLPoh0ZEtoSq74hUPpWgSrdXV1d3aDqd7h+G4X1BEKyKO54eovjcEcYViHQvupshIltCVXxFKp9KUKXbS6VSvwvD8Hbv/fC4Y+lBdO6QzaIPi4hsCXWSJFL5VIIqW0UmkxkCvAe8GgTBiK213/r6+tO99393zt2QzWbP3Vr7lS2mc4dsFl1UisiWUAmqSOULAY+1I9N3XbqlXC73CaBPGIYL4o6lByksQdW5QzpNCaqIbAklqCI9Qw67ZtB1g3RLs2bNejbuGHogVfGVzaIPi4hsCVXxFekZ8gmqbkZtwNSpU1OPPPLIZOfc2cD+QG/gHeAR59yN2Wz234Xbn3zyydv26tXrIqAOGA6sBh4GfhsEwZ3F+89kMl8DDvXeX55Op8MwDL8DHAHgvb+3qqrqOzNmzHgJoK6u7uPOufOAPYAVwP9ramqaOnfu3OaifX4GOMk595tcLvdeKpX6DnAUUAM84b3/SWNj479KxPJnwAVBcFapY5HJZP7mnFudr3KbyWTO8N5/3DkHsF0mk/lbweaLgiC4INpuiPf+NOfcKcBoYAdgJfCY935aY2Pj7MLXqa+vnxqG4Yecc3jvjyva791BEPw62u+PgZHV1dVfuvXWW98u3Mf48eP7V1dXf8l7PwnY2Tm3xnv/mHPuD9lsdlbx/1ZXV3eec+6DqVTqly0tLW+m0+mpwHHANs65/3rvfx4EwcxSx6WHyaHaF7IZ1EmSiGwJlaCK9Ay6GbURkydP7j1v3rzAOXcLcCqwCHjSOVcDnOO9/37h9pMmTdq9V69ejwLfBnYGnnDOvQ+cDvwrk8l8u8TLHAFMBiaFYfggcBLwNpYYfSyXy91dX1+/QyaT+YVz7iYs6X0r2v83+vfvf2OJfR4U7XNiKpV6GDgeeNo59w6WuN6eyWQ+XeJ5pwMf3tAhiRK+vGHOuQOi+RpgTMG0X8F2ZznnpgEnYp+7J5xza4CJzrlZxcfFe7+nc273aHFw0X7XtXN1zp0ITF67du02hc+vq6sbXl1d/XD0/uwOPAUsBCZ47xszmczPiv8x59zY6P87KZ1OzwcmO+cWA6u998cA0zOZTMnEvQdSb/+yyZSgisiW0EWrSM+gm1Eb0dzc/EssMX3aObdPEARjgiA4IZvNjvbe7wr8rmBzl8vlbgZ2Af4K7Bxte5BzbjxWkvqDurq6E0q9lnPuu8BlY8aMGRwEwWFr1qzZGfgXMNR7Px04BzghCIKRQRAcHIbhGGCp9/4jEyZMGFNqn977LwMNtbW1uwVBMCGbzR4EfAR7739TX18/akuOTxAEvwTGRotvBEGwe8F0TMH/9qRzbmJTU9PgIAgOLDiGxwPLgO+feuqpexbs9yzn3AXRc28t2u9FnQjteu/9nsDs1tbWXYIg+GA2mx0LHAssB76ayWRKJuLe++8Af2ptbR2czWbHBkGwK/BNrMTwx1OnTtV1ts4dshn0xRGRLaEfHpGeQaUgG5DJZEYAnwbWAJlsNvtc4frGxsY3gyC4uWD744FDgDeBTxWOy5nNZv8B/ApwzrmLO3jJ/wRBcOXUqVNDgNtuu21lGIaXReuOcM59q7CK8KxZs54CbgRIp9NHdLDPpWvWrLmwoaFhbf6BIAj+7r3/A1Drvf98Jw7FFstms//OZrPZuXPnFvYAS2Nj41zv/eVAKp1Of2RrvFYmkzk4KlldEobh2XPmzFmeXxcEwT3AFdHiNzrYxRO1tbVfnzNnzpr8A7W1tT8FXgN2fvjhh0dvjTi7OZ07ZJMpQRWRLaEfHpGeIZ8s6GZUCVF7yTRwexAEr3TiKSdEz5tRmJzmhWE4LZo9dvLkyb1KPP/vxQ+kUql1nQB5728tXu+9fyaa3aWDmG657bbbVpbY7y2FMZfL5MmTe02YMGHkhAkTjpk4ceKJEydOPNE5lwZwzu2zNV4jSk5xzs2eNWvWkuL1YRheE82OnTRp0sDi9d77WxsaGgo7AiJafgoglUrtvDXi7OZ07pBNpgRVRLaEfnhEegbVltgA7/3uAM655zv5lN0AwjB8udTKQw455FVgLVC7cuXKocXrnXOvFT/W1NS0NJpdGQTBohLPWRa95nqJVrS+ZCxAPuHeauOWbsi4ceNqM5nML5qbm99LpVIvpVKpu8IwvD0Mw9uJSjS99/23xmtFVa/x3pf836Ok9T3ArV27drfi9alU6tUOdr0EwDlXuzXi7OZ07pBNpgRVRLaEfnhEega1N9+AgkRk6QY3bFMTPW95qZVR1d0VhdsWyuVyrcWPFdjQug1pKvVgTU1NPsayJFv9+vW7AfgK8Ib3/mJgknPuuDAMxwLnRZu5rfRyNQDe+5L/e2Q5QHV19XrvA5t/rHsSnTtkk+nDIiJbQj88Ij2DbkZt2CKAMAx36+T2+RK2nUqtPOmkk/oC2wKk0+nFWyG+jfLeDy/1eHNzcz7G94tWtQK9p06dmsq3hc2bMGHCoM2JYdKkSbvlcrnJwFu1tbVHNjQ0LCtcn8lktkrV3gL5ar0l/3csER4OkMvlyvI+VCCdO2STqQRVRLaEfnhEegbdjNoA7/2DAM65I+hc6d6j0fNK9qjbq1ev46LZ10tV1+0K3vsjO1h1VPT30aLH3wTSDz300PbFTygYTqad2tra/BispdrV4r3P9xQ8rzg5jdaXPF5hGDZHf0vudwMejeIdW2plXV3d4VjJ8bKxY8e+tIn7FqNzh2wyJagisiX0wyPSM+hm1AYMHTr0DuBVYO9MJvOFUtsUDjninPs71uPv+Lq6una96o4bN67KOfedaPHPXRPx+pxzJ2YymcOLYqkFvhqtv6XoKS8BpFKpdkOwTJkypdo5dxklNDQ0rMCGihlSX1/fr8Qmb0V/9xo3bly735VTTz11T+dcqfFYSaVSb0Qxjiy1viOtra2NWNXmo+rr608pXDd16tSUc+570eItxaXE0mk6d8gm00WliGwJ/fCI9AzqsXsDpk2b1pLJZD4N/AP4ZSaTOdA59zfv/ftYFdHj5s2bNwrIAGSz2YWZTOYHwA+dc7Pq6+svC8PwfmCIc+4i4AhgQTqdvrKM/8ZLQLa+vv47wAPAMO/9t4G9gUeWL1/eLll2zt3ovT/VOffT+vr6/mEYPgTs8vbbb1/Ahtur3g+c4r2fU1dXd7tzrtk5tzybzf5+2bJlz/fr128BsEe/fv0a6uvrf9Xa2roklUodFSWLbwB7Fe9w+fLlz/fr128xcGhdXd12xFpHAAAVo0lEQVTNzrmnsBuoTwdB0NhRIHPmzFmeyWQuAX7rvW/IZDI/TKVSc8MwHDBv3rwLgVOAd3O53NROH0UppnOHbDKVoIrIltAPj0jPoB67NyIad/QU4GXgk977OcBDwHTgy0Sd7RRs/yPg29gYo790zj3onJuFDedyfzqdPm7GjBmd7XRpa7gCuMN7f433/vEo/qO99/c55+qKxyXNZrO3AL+O4v+Rc+4O59wfnXNVuVyurqMXCcPw8865u4EjnXNTgR977y8BmDt3bmsqlToDWAhM8t7fmU6nH3XO/Qa42zn3lVL7nDt3brNz7mPAi9HfHwI/Bj66sX86CILfAV+MFn8chuEDwD+xmwmPhmE4bvbs2e9sbD/SIZ07ZJMpQRWRTVGciHb0w7N9iW1FpPso/v6Wqi2RAnYsTzjdQxAEdzY1Ne3lnDsO+DzwFefcmc65PYIg+HjR5j4Igh9VVVXt7L3/mPf+q977851zhwRBcNSMGTNeLd5/Op2+OAzDsVFpaztz587NhWE4NpfLjSsV29q1a/8ZPffyUuu99y1BEJzlvT8QOM8592Xn3HFjx449JpvNLuzg//1iGIb7R3F/OZVKfaimpmbs7NmzX4tiOa74ObNmzXo5m80eW1tbW5tOp0eEYTg2lUqtS2hnzpz54Jo1a3b33k/AevP9TBiG+wVB8JGampp7wjAcWypRzWaztwdBsAfQF9g76vU3X0WXXC73iSimBSX+j1+n0+mdgY8AF2Hv3RFjxowZO2vWrGeLt8/lcj8Iw3DsmjVrbit1XIDvRf/XXR2sr2SdOXek0blDNsBdc801ft2Cc6OnTJnS2TG8RKTn+QlwLHA+1rnEOODfwDPAB4BBWHulr2LVwl6NI0gR2SLbAi8AlwH/C6wGHgbGAhOAOcAHgd8DdwMl2wVK95DJZH4LXOC9P7exsfGGuOORbm1H4GngO8B1QDPwBLAfcCJwJ/Ah7NzxT+CCeMKUpJk2bdqe3vvn8ssq4RCRTfFn7Afln9hF69vR4ztjP0LbYxe3r6Hk9P+3d+9Rctb1Hcffs5ck5LZgkAYwgDXREBOMiEhpvWCItVYkeOmRHoxaFeyBY1UsJV6jHEtVopYeL8mxogVKKZATslC1hFBPhMZCIiRcAkFNsoZLICHJJiHZS57+8ZvJPjM7M3thd57f7L5f58zZ32+eZ2e+2cw+O5/5/Z7fI9WrXUAb8A1gEeG8v1Py2/6REFqPApqBn9S+PEmRegbYASwBvgJsAwqXKbqG8B7hKEL++GkWBao+GFAlDcRDhFBauBTASfmvkwifkAJ0Aj+qcV2ShtY1hFGOqRRPxUtfPmQ78KtaFiUpekuA7wJ/lL8VzE212wjnZ0tleQ6qpIH6AeHyCJXswk9GpXp3K3CgyvYEaM1/VX37DXBLQ0PDlqwL0YhwE7C/yvYEuA2PHarCEVRJA/Vj4AqKPxlN206Y5iOpfh0EVgF/XWH788D3a1eOhktra+uPcNaLhs5+4F7gggrbnyOcJiBV5AiqpIHaDTxcYduL+KZVGim+Qwii5ewHNtawFkn14xpgZ4VtewgLK0oVGVAlDcY3gBfK3L+XMDVQUv17gJJrd+Z1A/9e41ok1Y/7gH1l7u8Crq9xLapDBlRJg3E3YbS01CbCp6OSRoal9D7nfCdOCZVU3XVAR8l9O/P3S1UZUCUNxmHCCEp36r524NpsypE0TP6VMK0/7Tng9xnUIql+/JDex45nCJetkqoyoEoarO9R/MdnH3BnRrVIGh47gcdS/S5CaJWkap4FfpvqdxJmZEh9MqBKGqwthD9ABb+m+uVnJNWnbxHeXEKY2n9jhrVIqh/fpPjYcXOGtaiOGFAlvRTXEFbz3IPTe6WR6ueE6xsDbAZ2ZFiLpPpxBz0LKj5Kz3FEqsqAKuml+A/C1N4DwC8zrkXS8DhMGPnoxA+iJPVfF7CCcOz454xrUR0xoEp6KV4EVgP/RXgTK2lkupawKNptWRciqa58hxBQb8+6ENWPpqwLkFT3/g7DqTTS/RZ4PeWvbShJlWwC3kD5S9NJZRlQJb1Uz2VdgKSa2JR1AZLqkscODYhTfCVJkiRJUTCgSpIkSZKiYECVJEmSJEXBgCpJkiRJioIBVZIkSZIUBQOqJEmSJCkKXmZGdWHFihXTsq6hmhUrVkxpb28fC7Bjx472NWvWtGddU4mJwOR8+xCwM8NaymkGXp7qPw0kGdVS1nvf+96puVyuAWDu3Lm75syZczDrmipJkqT9ggsu2F2L51q+fPlxDQ0NY2vxXIOxZs2aSVu2bJkEsHfv3kN33XVXbK/9McCxqf5TWRVSxQmp9vNAR1aFVDAFKLwG2/O3aJx++ukTX/nKV04GmDRp0qEFCxbE9hoscv755/8hl8sN+/H3nnvuGbdnz56X971nNjo6OnI333zz8YX+/fff/9y2bds6s6ypjPRrfy/xXac46vce06ZNG3PmmWceOf5+6EMfivH4e0RLS8tz55xzTrTvPYaSAVXRW7x4cUNjY+O2rOuoZt26dWzcuDHrMjSMli9ffqQ9c+ZMGhsbM6ymT98DLqvFEzU1Nd2Zy+XOqMVzDcbWrVu57bbbsi5Do9j69etZv349ANOnT+d973tfxhVVt2rVqqOBPcP9PO3t7X/e2Ni4YrifZ7ByuZzHjhGura2Ntra2I/2FCxfS0BDv5NL9+/e/E/hF1nXUQrz/C5IkSZKkUcWAKkmSJEmKglN8VXeSJFmYy+XWZl1H2sGDBx8gf57F9Olv2fTkk+d/MeOSSnzty7DntNAefxCuuijbekpdPw8e/Nue/mc/BicO+xSzgWhu/vytnZ2HANi8efPSs846a0nGJZX6PnBuxjUsAZZmXEORHTt2/CcwF2Dq1Ffve+aZSz6SbUWlli2Ax1O/j0ven10tlVx+a0/7jTfAByOblnnFT6B7Ymif8ARc/vlMyykxY8bPvrh586q5AJ2dnS8Cr8u4pCK5XG5OkiRZz2WN7ueyZcuWY4H7Cv3p0z/6gyefnH13hiWV8fc3wOFxoT1tI3z6q9nWU2rxVdB+amhP2A9f+3C29RSbMePBd2/efP1HCv329vbXtrS0xHae8SZG4YCiAVX1aPt55523Oesi0qZPn3640B47dmIHfLat2v619/XUSfW5JL76WncV9+c9Be/aVX7fbORyXzjS3rt37wuxvQbvuOOO/UmS+bpSO2P7uZx99tlHXvtNTc0RvvZvLFnMKrb6AC5PtY/eHV+N/3C4p90U3fF33Lh7j7wGkyQ5HNvvyMqVK4/O5XJZl5HE9nO58MIL96b748efvCu21xZckTrojzkYX31XpRZUa4zu+DthwpVFx9/bb7/9yVtuuSWqReBaW1uzLiEToy6RS5IkSZLiZECVJEmSJEXBgCpJkiRJioIBVZIkSZIUBQOqJEmSJCkKBlRJkiRJUhS8zIxGo0bgNcAbgDPyX4/Pb/sEsDqjuiRJklR/JhFyVRfQnnEtdc+AqtHoKmBRhW3ja1mIJEmS6koDMJ/igY6T8tv+Fzg7o7pGDAOqRrOtwHqgDfhUxrVIkiQpfuOBn2ddxEhmQB2cBuCPgROBHPA08PgQPOargOOAo4AXgCeBPS/xcdXbUuDbwPP5/gwMqJIkSeqffcCDwLr87SPA27MsaCTJKqCeCXw/394HvK0f33MCsDLVPxfYPbRlFfkCcEG+fQewGBgDfA64GDi5ZP82YAnwPcL88/56NXAl8JeEcJrWBdwHXAO0VnmMqwlTDQBuAL5bZd9vAeek+ldQ/ZzLXwBT8u3PAGuq7FsvtmZdgCRJkurSfuBooDt137szqmVEyiqgTibM14b+jxCOTX0PDH/tJ6WebyNhEZ0VhHBdzjRCMDwH+ADQ2cfj54CvEIJwpX9LE/CW/O0WYCFwsMx+21O1dlE5oObyj5EOwudROaBOB96Rbx8GHq2wnyRJkjQaJBSHUw0xp/j2zzhgOSGcdgD3EELrAcL00POAifl9zyeMSn69j8dcBnw81d8N3EkIgfsJI8bvBmblt3+AMPX3PYRfjLR0wDyD8KlOudHlOfQepZ1Xpcb0tgeBnVX2lSRJkqSXxIDaP+8n/Kx+RRiB/H3J9hMJJ0vPzvevAL5DCLDlfJLicPovhJHU0mWpFwGXEc6XbCQE1svy+6c9CjxFCLWNwFuB28s8bzpwHiKMSs8GpgLP9LH/3RX+LZIkSZI0JBqyLqBONBFGTN9B73AKYYrthfSMbE4G3lXhsY4hnAda8G3CAj3lrpl0GLgW+FLqvkWEYFkqPYp6boXnTgfOpfmvOcqf1N1A8bmqBlRJkiRJw8qA2n+fBl6ssv1hYG2qf0aF/S6mZzpwG2HktC9LCCEYwrmw7yyzTzpAlpu220w4lxXCisM/7WP/1wHH5tsdhNFjSZIkSRo2TvHtn6eovtJtwQPAn+Tbp1TYZ0GqfSPlFz0q1UFYxfeT+f6b6T2FNx1QTyVMO96euu9MYFJq38I5pVMoH1DT960lnBcbs6OofKmYezFgS5IkSdEzoPbPb/q5345Uu6XM9vEUr0R83wBq2JRqzyqzvQ14gnDZGgjTdq9PbS89n/QwYbGn9xMumTMD2Jza5+0l+8duPPBPFbZ9FQOqJEmSFD0Dav+80M/9OlLtMWW2v4Iw1bbgm4RLzfTHlFT7ZRX2WU1PQJ1H+YDaTQimAKsIAbWwvRBQx9AzHRjqI6C+CHyjwrZ7a1mIJEmSpMExoPZP1xA9zjEl/ZmDfJwJFe6/m55pwOkR0wnAWfn2enoCd3ra8jzgh/n2m1LP0Q783yDrrKUDwJVZFyFJkiRp8AyotdVY0r+D6gsvVdJW4f57CFN3GwijtTMJU4P/jJ4R3fRo6GZgK2GK7zn57ztMcbhdA3QOokZJkiRJGpB6CqjlpszWm+dL+l+m/+e39sdOwuJHp+f78wgBtdr1TFcDHyVMIZ5LGGH1+qeSJEmSai6ry8ykV64d18/vOW44CqmxpyieLjxnGJ6jdNpu+utBep+Puapk/4mEKb4FIzGgngLclbr9JLXtqpJtb65xbZIkSdKoldUI6p5Ueyxh0Z9dfXzPm/rYXg/2AffTcyma84F/G+LnuBv4XL79NkKwn5vv30fvKcWrgQTIAecCj9CzkNNzwIYhri8GEwn/1nLmlvR/WHYvSZIkjVY3An+R6hfWbnkjxZlmA+H9uAYgq4D6O8JqsoVzMs8mnI9ZSQPwN8NdVI3cSnFAnQ08PISPv4awmvAYwqJMl9MzUr6qzP7PEELpbMK5qk+kthXC60jzFHBJP/ddP5yFSJIkqe5MpPfipxCyVfr+SbUpZ2TJKqDuJ3yi8Pp8/1KqB9QvAacOd1E1spSw2uzLCQH9ZsI00r5GkAumEa63eqjC9v3AWnouE3NZalul6bp3EwLqeIo/CFhdfve6twtYlnURkiRJqkuXAJ/px36V3q+riqzOQYXia3S+E7iGMN037RjgWmAxYXrsSLAf+DhhtVyAWYRRuguo/P8xhjCN4Abgt/T9aUw6iI7Pf30BWDeA/UvvlyRJkhRmIP6uH7ftWRVYz7JcxXcpYeT0Vfn+5cCHCedJ7iNcJuVNhNB6EPgEcFPtyxwWKwn/3iWEUHoysJwwMvpr4A+EAHs0MB04DThqAI+/GvhqyX3/Q5hWXc4vCYs3pV8PWwhhWJIkSZJqIsuAegBYAPw3cHz+vmOB95Tstwu4iHC5lJHku8DjhEV4TsrfdxxwXh/ft4W+pwv8mhDyJ6buqzYaupfixZv62l+SJEmShlzW10F9mDA6uAj4AOH8yoJnCSOm3wbaCOHtltT24Z7TvS71fPf383s2pb6nPwsf/QyYQQjgCwjnjbaU7LMHeJQwyvkzwiJIfS1c1EkYnZ1V8lzVLCWM3BaMlNFqSZIkSXUi64AK8DxhuuvlwBRCQNtJ8aVoIEx//asa1rWMgS+kszJ/G4gO4Mf5G8Bkws8hRxg93j3AxytYPMD9f5q/SZIkSVImYgioaTvzt9Fsb/4mSZIkSaNKlqv4SpIkSZJ0hAFVkiRJkhQFA6okSZIkKQqxnYM6GI3Ax4bosdYDDwzRY2mY5HK5m1tbWzuyriNt0aJFkwvtTZtWnQrj7syynt46J/e0D4yLr76uscX9BTdAQ1+rVddUZ2fPS27u3LmXXXrppQszLKeXJElelnUNwJWtra2XZV1E2nXXXXdsof30049PjPC1X3KN69jqK3XPR2HcB7OuolhX6pJq22fG9jN87DGOHH/HjBkzvrW1dXuW9ZTRnHUBwFGx/VwOHDjQcNNNPRc0eOSRqz8FV1+SYUlldI/raW+ZG9trHzpSV6ZonxBbfRs3Mi7dv+iii36/cGFUf9phlA4mjoSA2ky4RMpQ+BoG1HpwbN+71FYulzvS7u7uaCbCGnskOTgUcX0AnVOyrqBUkorLjY2NEym+zrCCyflbNBoaev62d3d35aAr8td+7L+bXRPCLVbdTdAd1c+wq6uomwNOyKaSqEX3c0kfOwC6uw9NAiZlU01/dDfH9tovFt97j+7uXndF9RoczUZlKpckSZIkxWckjKAeAoZqatvBIXocDaHFixcfXrlyZXRzLtJmzZo1e+rUqS0Azz77bNuGDRu2ZV1TiVcAJ+fb7cCGDGspZxJwWqq/Fuj92WaG5s2bd1ZDQ0MjwPjx4x9LkmRX1jVVsamGz/XlJEmi+lQ87fjjjz95/vz5rwBob2/fvXbt2keyrqlECzA71b83q0Kq+NNU+2F6X6c8a68Fjs63/wBszbCWXmbOnHnitGnTTgFoaWlpT5IktuNvkWOOOeZALZ6nqalpXVdXV7R/25MkaZw/f/5Zhf5DDz20YceOHe1Z1lTGafSM6m4D2jKspZyTgGn59h7C8SMaU6dObZkzZ86R42+SJPclSRLV6UVpzc3NUR87hlJu6dKlR/4jcrncay6++OInsixIkiRJkjQ6LFu27NVJkjxe6DvFV5IkSZIUBQOqJEmSJCkKBlRJkiRJUhQMqJIkSZKkKBhQJUmSJElRMKBKkiRJkqJgQJUkSZIkRcGAKkmSJEmKggFVkiRJkhQFA6okSZIkKQoGVEmSJElSFAyokiRJkqQoGFAlSZIkSVEwoEqSJEmSomBAlSRJkiRFwYAqSZIkSYqCAVWSJEmSFAUDqiRJkiQpCgZUSZIkSVIUDKiSJEmSpCgYUCVJkiRJUTCgSpIkSZKiYECVJEmSJEXBgCpJkiRJioIBVZIkSZIUBQOqJEmSJCkKBlRJkiRJUhQMqJIkSZKkKBhQJUmSJElRMKBKkiRJkqJgQJUkSZIkRcGAKkmSJEmKQlO6kyTJKcuWLcuqFkmSJEnSKJIkySnpflPJ9l8kSVK7aiRJkiRJynOKryRJkiQpCgZUSZIkSVIU/h9zhnJWRK6mqgAAAABJRU5ErkJggg==\")
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c1bd8413",
+ "metadata": {},
+ "source": [
+ "The communication step happens in function `ghost_exchange!`. This one modifies the ghost cells in the input vector `u` by importing data using MPI point-to-point communication. See the following code:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "688638a1",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "code3 = quote\n",
+ " function ghost_exchange!(u,comm)\n",
+ " load = length(u)-2\n",
+ " rank = MPI.Comm_rank(comm)\n",
+ " nranks = MPI.Comm_size(comm)\n",
+ " if rank != 0\n",
+ " neig_rank = rank-1\n",
+ " u_snd = view(u,2:2)\n",
+ " u_rcv = view(u,1:1)\n",
+ " dest = neig_rank\n",
+ " source = neig_rank\n",
+ " MPI.Sendrecv!(u_snd,u_rcv,comm;dest,source)\n",
+ " end\n",
+ " if rank != (nranks-1)\n",
+ " neig_rank = rank+1\n",
+ " u_snd = view(u,(load+1):(load+1))\n",
+ " u_rcv = view(u,(load+2):(load+2))\n",
+ " dest = neig_rank\n",
+ " source = neig_rank\n",
+ " MPI.Sendrecv!(u_snd,u_rcv,comm;dest,source)\n",
+ " end\n",
+ " end\n",
+ "end;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5bf5408c",
+ "metadata": {},
+ "source": [
+ "Note that we have used `MPI.Sendrecv!` to send and receive values."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d2ce54e4",
+ "metadata": {},
+ "source": [
+ "\n",
+ "Question: Is this other implementation based on `MPI.Send` and `MPI.Recv!` correct?\n",
+ "
\n",
+ "\n",
+ "```julia\n",
+ " function ghost_exchange!(u,comm)\n",
+ " load = length(u)-2\n",
+ " rank = MPI.Comm_rank(comm)\n",
+ " nranks = MPI.Comm_size(comm)\n",
+ " if rank != 0\n",
+ " neig_rank = rank-1\n",
+ " u_snd = view(u,2:2)\n",
+ " u_rcv = view(u,1:1)\n",
+ " dest = neig_rank\n",
+ " source = neig_rank\n",
+ " MPI.Send(u_snd,comm;dest)\n",
+ " MPI.Recv!(u_rcv,comm;source)\n",
+ " end\n",
+ " if rank != (nranks-1)\n",
+ " neig_rank = rank+1\n",
+ " u_snd = view(u,(load+1):(load+1))\n",
+ " u_rcv = view(u,(load+2):(load+2))\n",
+ " dest = neig_rank\n",
+ " source = neig_rank\n",
+ " MPI.Send(u_snd,comm;dest)\n",
+ " MPI.Recv!(u_rcv,comm;source)\n",
+ " end\n",
+ " end\n",
+ "```\n",
+ "\n",
+ " a) It is correct.\n",
+ " c) It is incorrect, but it might provide the right result depending on the MPI implementation.\n",
+ " b) It is incorrect, and it is guaranteed that it will result in a dead lock.\n",
+ " d) This implementation does not work when distributing over just a single MPI rank.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "8498c7e2",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "answer = \"x\" # replace x with a, b, c or d\n",
+ "sndrcv_check(answer)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d863cb89",
+ "metadata": {},
+ "source": [
+ "\n",
+ "Question: How would you fix the implementation above, while still using `MPI.Send` and `MPI.Recv!` instead of `MPI.Sendrecv!` ?\n",
+ "
\n",
+ "\n",
+ "Think about the answer. To uncover the the solution run next cell."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "629a3356",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "sndrcv_fix_answer()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "82e21f4e",
+ "metadata": {},
+ "source": [
+ "### Local computation\n",
+ "\n",
+ "Once the ghost values have the right values, we can perform the Jacobi update locally at each process. This is done in function `local_update!`. Note that here we only update the data *owned* by the current MPI rank, i.e. we do not modify the ghost values. There is no need to modify the ghost values since they will updated by another rank, i.e. the rank that own the value. In the code this is reflected in the loop over `i`. We do not visit the first nor the last entry in `u_new`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f43560e5",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "code4 = quote\n",
+ " function local_update!(u,u_new)\n",
+ " load = length(u)-2\n",
+ " for i in 2:(load+1)\n",
+ " u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ " end\n",
+ "end;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "47861c19",
+ "metadata": {},
+ "source": [
+ "### Running the code\n",
+ "\n",
+ "Not let us put all pieces together and run the code. If not done yet, install MPI."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "53674411",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "] add MPI"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c966375a",
+ "metadata": {},
+ "source": [
+ "The following cells includes all previous code snippets into a final one. Note that we are eventually calling function `jacobi_mpi` and showing the result vector `u`. Run the following code for 1 MPI rank, then for 2 and 3 MPI ranks. Look into the values of `u`. Does it make sense?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f91e4759",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "using MPI\n",
+ "code = quote\n",
+ " using MPI\n",
+ " MPI.Init()\n",
+ " $code1\n",
+ " $code2\n",
+ " $code3\n",
+ " $code4\n",
+ " n = 9\n",
+ " niters = 200\n",
+ " comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
+ " u = jacobi_mpi(n,niters,comm)\n",
+ " @show u\n",
+ "end\n",
+ "run(`$(mpiexec()) -np 1 julia --project=. -e $code`);"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8e4fe21d",
+ "metadata": {},
+ "source": [
+ "### Checking the result\n",
+ "\n",
+ "Checking the result visually is not enough in general. To check the parallel implementation we want to compare the result against the sequential implementation. The way we do the computations (either in parallel or sequential) should not affect the result. However, how can we compare the sequential and the parallel result? The parallel version gives a distributed vector. We cannot compare this one directly with the result of the sequential function. A possible solution is to gather all the pieces of the parallel result in a single rank and compare there against the parallel implementation.\n",
+ "\n",
+ "\n",
+ "The following function gather the distributed vector in rank 0."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "49a220c6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "code5 = quote\n",
+ " function gather_final_result(u,comm)\n",
+ " load = length(u)-2\n",
+ " rank = MPI.Comm_rank(comm)\n",
+ " if rank !=0\n",
+ " # If I am not rank 0\n",
+ " # I send my own data to rank 0\n",
+ " u_snd = view(u,2:(load+1))\n",
+ " MPI.Send(u_snd,comm,dest=0)\n",
+ " u_root = zeros(0) # This will nevel be used\n",
+ " else\n",
+ " # If I am rank 0\n",
+ " nranks = MPI.Comm_size(comm)\n",
+ " n = load*nranks\n",
+ " u_root = zeros(n+2)\n",
+ " # Set boundary\n",
+ " u_root[1] = -1\n",
+ " u_root[end] = 1\n",
+ " # Set data for rank 0\n",
+ " lb = 2\n",
+ " ub = load+1\n",
+ " u_root[lb:ub] = view(u,lb:ub)\n",
+ " # Receive and set data from other ranks\n",
+ " for other_rank in 1:(nranks-1)\n",
+ " lb += load\n",
+ " ub += load\n",
+ " u_rcv = view(u_root,lb:ub)\n",
+ " MPI.Recv!(u_rcv,comm;source=other_rank)\n",
+ " end\n",
+ " end\n",
+ " # NB onle the root (rank 0) will\n",
+ " # contain meaningfull data\n",
+ " return u_root\n",
+ " end\n",
+ "end;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e66bad1b",
+ "metadata": {},
+ "source": [
+ "Run the following cell to see the result. Is the result as expected?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4a9f4a54",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "code = quote\n",
+ " using MPI\n",
+ " MPI.Init()\n",
+ " $code1\n",
+ " $code2\n",
+ " $code3\n",
+ " $code4\n",
+ " $code5\n",
+ " n = 9\n",
+ " niters = 200\n",
+ " comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
+ " u = jacobi_mpi(n,niters,comm)\n",
+ " u_root = gather_final_result(u,comm)\n",
+ " @show u_root\n",
+ "end\n",
+ "run(`$(mpiexec()) -np 3 julia --project=. -e $code`);"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dc9b2b51",
+ "metadata": {},
+ "source": [
+ "Now that we have collected the parallel vector into a single array, we can compare it against the sequential implementation. This is done in the following cells."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "cb053382",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "code6 = quote\n",
+ " function jacobi(n,niters)\n",
+ " u = zeros(n+2)\n",
+ " u[1] = -1\n",
+ " u[end] = 1\n",
+ " u_new = copy(u)\n",
+ " for t in 1:niters\n",
+ " for i in 2:(n+1)\n",
+ " u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ " u, u_new = u_new, u\n",
+ " end\n",
+ " u\n",
+ " end\n",
+ "end;"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ffe6c605",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "code = quote\n",
+ " using MPI\n",
+ " MPI.Init()\n",
+ " $code1\n",
+ " $code2\n",
+ " $code3\n",
+ " $code4\n",
+ " $code5\n",
+ " $code6\n",
+ " n = 12\n",
+ " niters = 100\n",
+ " comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
+ " u = jacobi_mpi(n,niters,comm)\n",
+ " u_root = gather_final_result(u,comm)\n",
+ " rank = MPI.Comm_rank(comm)\n",
+ " if rank == 0\n",
+ " # Compare agains serial\n",
+ " u_seq = jacobi(n,niters)\n",
+ " if isapprox(u_root,u_seq)\n",
+ " println(\"Test passed 🥳\")\n",
+ " else\n",
+ " println(\"Test failed\")\n",
+ " end\n",
+ " end\n",
+ "end\n",
+ "run(`$(mpiexec()) -np 3 julia --project=. -e $code`);"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "73cd4d73",
+ "metadata": {},
+ "source": [
+ "Note that we have used function `isapprox` to compare the results. This function checks if two values are the same within machine precision. Using `==` is generally discouraged when working with floating point numbers as they can be affected by rounding-off errors."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d73c838c",
+ "metadata": {},
+ "source": [
+ "\n",
+ "Question: What happens if we use `u_root == u_seq` to compare the parallel and the sequential result?\n",
+ "
\n",
+ "\n",
+ " a) The test will still pass.\n",
+ " b) The test will fail due to rounding-off errors.\n",
+ " c) The test might pass or fail depending on `n`.\n",
+ " d) The test might pass or fail depending on the number of MPI ranks."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "cd2427f1",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "answer = \"x\" # replace x with a, b, c or d\n",
+ "jacobitest_check(answer)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "790e7064",
+ "metadata": {},
+ "source": [
+ "Run cell below for an explanation of the correct answer."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "72ed2aa1",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "jacobitest_why()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c9aa2901",
+ "metadata": {},
+ "source": [
+ "## Latency hiding\n",
+ "\n",
+ "Can our implementation above be improved? Note that we only need communications to update the values at the boundary of the portion owned by each process. The other values (the one in green in the figure below) can be updated without communications. This provides the opportunity of overlapping the computation of the interior values (green cells in the figure) with the communication of the ghost values. This technique is called latency hiding, since we are hiding communication latency by overlapping it with computation that we need to do anyway. The actual implementation is left as an exercise (see Exercise 1)."
+ ]
+ },
+ {
+ "attachments": {
+ "fig16.png": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6gAAADyCAYAAABAvOgkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7d17nFxlYf/xz9lssrmQm4EQkhACRi7h1kD4YSIqEGuwEi4tSNEakWpqBWuLtEKrgtW2WkDbn1VJ0Cra3w8lKMKGCIJc1BBCxCgghGsSJIEk5ELut93pH89s9uxkZnez7M7z7O7n/XqdV84zc2bmm9nNZL97Lk82e/bsApIkSZIkRVYTO4AkSZIkSWBBlSRJkiQlorZkPD3LsuUxgkiSJEmSepdCoTAeuKdp3KKgZlm2fNasWc9WO5QkSZIkqfeZM2cOhULzZZE8xFeSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCTUxg4gSZKSMxB4b3F9PrA1YpZq619cdtO7/t6SlAQLqiRJKjUCuLW4/mbgxQ4+Tw3wFmAyMAkYXLz9CtItf/8IfBa4H5gWOYsk9ToWVEmS1BW+AXwAGFLmvqtJt6DG8F7gHOA54PrIWSQpKguqJEkqtRG4qri+voPPMZHmcrqy+JzHvsFc1fAH4BHg91V8zZOAWcAvqF5BPRQ4ObccCWTAbcCnq5RBkvZhQZUkSaU2A19+g89xE3Ad8BjwKvAh4Ltv8Dmr4abi0pNNBhZXuO+gagaRpFIWVEmS1BX+X+wA3ci3CBej2rKfjzsSOJFwjvBjHXjdDcCvi4/9IDCmA88hSZ3KaWYkSVKpQ4FCcTkicpZquwR4ALihiq/5CqEkPrOfj5tBuJjVrP183LOEi1+9CXg34Zzgdfv5HJLUJdyDKkmS1OwI4HSgsYqvOZFwfu5a4MEqvN6m4iJJyXEPqiRJUlx/RtgT+vnYQSQpNguqJEmSJCkJFlRJkiRJUhI8B1WSJLXXUGBChfuew/MaJUlvkAVVkiS11zuAOyvcdxZwTxWz9BZHUPkqvacU/zwV+FKFbW4HFnV2KEnqKhZUSZLUXhupPN+me0+7xjjg021sc2JxKWcZFlRJ3YgFVZIktdcvgcmxQ/QyLwFfrnDfKcCZwO+Auyts89uuCCVJXcWCKkmSlK4Xgasq3PcpQkFd1Mo2ktSteBVfSZIkSVIS3IMqSZK6womECyc1OSm3/klgW278dWBLNUJJktJmQZUkSV2htSvLfq5k/D0sqJIkLKiSJGlfW4A5xfWOXp336dxztGVb25tUzRZgLbAhdpAu9h1gbG58RPHP6cC9udsfAP61WqEkyYIqSZJKbQD+6g0+xy+LS3fz78Wlp5sCHFXm9tHFpcna6sSRpMCCKkmSFNePgKXsfxmsJ0xD82IHXvNvgcHt2O6lDjy3JHWYBVWSJCmup4rL/nq2uHREpXlTJSkqp5mRJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJDjNjJJXKBSy+vr6R2PnaM0NN9xw2LJlywYCbN68+bUNGzakNrH5gcBBxfXtwPJ4UcoaAIzPjZ8BGuNEKW/cuHFHUfyl3iWXXPLy5MmTN0eOVFGWZbfOmDHjumq8Vn19/XcLhcKx1Xitjpg7d+7Ihx56aATAzp07t65evTq1OR0HAeNy46djBWnFMbn1l4CtsYJUMI7wPgKsA9ZEzLKPYcOGjRgyZMhIgDFjxuy4+uqrl8XO1Jq6urrTp0+f3uVf47vuuusdDQ0NN3T163TUjh07aq688sqjmsarV69evnPnzu0xM5UxnvD/J4Q5dF+LF6Wsgwg/fwBsA1ZEzLKP/v37Dxw5cuRhTeOvfe1rT2dZFjNSWz5xzjnnPBI7RDVYUJW8z3/+89nJJ588OXaO1qxfv56XXtr7c+8g4LBWNo9tEM3/YaTqpNgBSuW+vjQ0NByV+H9ii6r1QoVC4dgsy5L997l9+/bSf5sjI8Zpj2Tfy6Jj2t4kqtLCH93GjRvZuHEjAP369RuUZdmIyJFaVVNTU5WfDRsaGoan/NmRZVmLz30g2V/EFQ2i5S96UzOI5l+UJ2HHjh2lX+PJKf/fnmXZ0NgZqsVDfCVJkiRJSXAPqrqjm7IseyF2iLzdu3dfQ/EwmzFjxqxYuXLlNyNHKvWXwFuK67uAz0XMUs5k4ILc+IvAlkhZyqqtrf3Snj17AFi/fv0DWZbdEzlSqZmFQmFizACFQuHumpqaB2NmKLV58+aPU9yjNmLEiO3r1q37fORIpd4B/ElufFWsIK34UvPqmIfh4CXxopTz249AY11x8BLwjZhpSo0dO/bSl19++UiAhobCnhdfHDY7dqa8gQN3jRw1atuFkWPszrLss5EztLBz584DgM80jUeNGnXbq6+++uuIkcr5Z6Bfcf154FsRs5TzMZr36u4Aro2WpIzRo0eftmrVqrObxnv27Pmnurq6hpiZShUKhX8D0t2t20UsqOp2CoXCD2bMmHF/7Bx5EyZMuIpiQR03btzKlStXfjlypFLvormg7gZSy/cRWhbU/wJWR8pSVp8+ffYW1GXLli06++yzk3oP582bNwWIWlCzLPtFau/L1KlTz6NYUIcPH75j3bp1SeUjnGudL6j/DhQiZakkV1An/gZ+9v14Ucrp+8FcQV1FYp9vRxxxxJnNBbVhzyc/edr3YmfKu/TSpyaef/6L0Qtqap8dF1988cHkCuopp5xyT319fWoF8DM0F9QVJPa9D8ygZUFNKt/JJ5+8K19Qf/CDH1w/d+7cXTEzlaqvr/9XemFB9RBfSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEmpjB1CXGQBMzI2XAI1tPKYGmJQbPwVs7+RckiRJklSWBbXnOgL4dW48kLbLZr+SxxwPPNnJuSRJkiSpLA/xlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFtecqlIyzdjzmgK4IIkmSJEntYUHtubaUjNtTPg/riiCSJEmS1B4W1J5rPS33ok5ox2Pe1UVZJEmSJKlNFtSeawuwLDc+q43tBwOXdV0cSZIkSWqdBbVnuy+3fjkwpsJ2dcDNwKFdnkiSJEmSKrCg9mxzgMbi+nDgl8AFNJ+PeiBwEfAocD7w22oHlCRJkqQmtbEDqEs9BnwN+GRxfDgwt7jeSMtfUKwilNfnq5au4y6rr68/N3aIvH/4h38Y0LT+/PPPHw78Z8Q45RyZW+9HevmOLRl/EdgWI0gle/bs2bs+YcKEd9XX1w+MGGcfhULh+NgZgLPr6+tHxQ6R981vfnN80/ratWsHkN73/kkl4/+IkqLdfvt2mHBQ7BQtNfbPDcaT2Nd46dKlRzet9+lT0/emmx64ImaeUn377nlT7AxAv/r6+qS+bps2bRp4yy237B0vWLDgz4EUPmfz6nLrR5LY9z7h584myX3+Lly4cFJ+/P73v/+GmTNnNlbaPpL2zMLR41hQe75PFf/8BC0LaX79V8CfA+uqFeqNyLLsT2NnKFVb2/xPae3atYcAfxMvTZv6knY+gI/EDlCqoaFh7/rQoUMnA5PjpUnW1OKSjIEDm3+P8Prrr/cn/e/9xPOtPTEsyRpFYu/hmjVr9q736VPTZ9SorRdFjJOqWhL7uvXr16/FeP369dOAaXHStMuhJPYelqgjsXyvvfZai3FNTc3lkaKohAW152sA/hb4NqGEngQcRLjK7wuEPaoPEvao9gHel3vsH6oZVJIkSVLvZkHtPZ4oLq1poPkQ4GRcc801hXnz5t0ZO0drRo8efXRDQ8MQgA0bNqx65ZVXXo6dqcRoYGxxfQvwVMQs5RwATMyNf03z+dNJOProo0+uqanpA9C3b9/ngA2RI1WUZdnjVXythwinCCRpyJAhYydOnDgaYPv27ZuWLVu2NHamEkOAo3PjR2MFacX/ya0vBTbFClLBUcDQ4vorJPbL1YMPPviQESNGHFpc3wr8PnKkVu3YsWN3NV4ny7JXgJT/b6+ZOHHi3iNlVqxY8dTWrVtL55iPbSLN1xV5mfQ+i8cSfv6A8LmR1OfvAQccMHjcuHHHNI0LhcJiWk7RmJSGhoY1bW/VM2SzZ8/e+4XIsuyoWbNmPRszkCRJkiSpd5gzZ86RhULhmaaxV/GVJEmSJCXBgipJkpS+fm1vIkndnwVVkiQpfZ8lXHX/wNhBJKkrWVAlSZLS9yvgZOBx4MzIWSSpy1hQJUmS0rcQeB04BLiLMH1c36iJJKkLWFAlSZLStwnYU1zvD7wfWAIcES2ROqp/7ABSyiyokiRJ3cP9ufX+hHkwFwJ/ESeOOqA/4XDt2XjhK6ksC6okSVL3cCewNTfOgJHAfxXvGxgjlPbLjYRzic8DDoqcRUqSBVWSJKl7eAjYUub2ocB04GnghKom0v74OPAhoIGw13tl3DhSmiyokiRJ3cNaYFeF+/oB4wiHAV9VtURqrynAV4vrnwbujZhFSpoFVZIkqft4rI373wR8Abi9ClnUPgcDtxJ+iXA78JW4caS01cYOIEmSpHa7A3gPUFdy+zbgeeA24DfA+irnUnm1wA+BscAzwCVAIWYgKXUWVEmSpO7jQWAjYa8chEN++xGuDnslHjqamn8H3glsBs4nTBckqRUe4itJktR9LCdcZAfCFX1/Cnyb8DPd94FD4sRSGRcBf0fYY/phwkWsJLXBgipJktS9PEUoPU8DfwZ8AniCsFf1Zvz5LgWTCb84APg34EcRs0jdih9gkiRJ3cudhL2oFxT/3E7YW7cV+GPgc/GiCRgP1AODgPn49ZD2iwVVkiSpe5kP/F9gRe62p4GPFdc/B5xX7VACYDjh6zOKcLGqi2g+JFtSO1hQJUmSupcXgE+Vuf1/CHNtZoTzUY+rZijRD5gLHAOsBM4FtkRNJHVDFlRJkqSe4++Bu4EDgB8Dw+LG6TUy4FvANMKVev8EeDlqIqmbsqBKkiT1HA3AXwDLgLcA/x9/3quGLwIfJEz7cz7weNw4UvflB5YkSVLPso5wDupW4D3A1+PG6fH+EvhHwpWVZwH3x40jdW8WVEmSpJ7nceASoJFw8aQroqbpud4L3Fhcv4YwzY+kN8CCKkmS1DPdRjgnFeA64MKIWXqiMwnvcS3w38AX4saRegYLqiRJUs/1FeA/CT/zfQ84LW6cHmMKcAfQH/gJ8Fdx40g9hwVVkiSpZ7uCcEXfpjLl9DNvzCnATwlXSp5PmOt0T9REUg9iQZUkSerZGglX9n0YGAHcCxwVNVH3dSrwM2Ao4WJIFxCu3Cupk1hQJUmSer7thAv6/AYYBfwceHPURN3PaYRyOgz4BXAu4X2V1IksqJIkSb3DRmAasAQYAzwAjI8ZqBt5B+Fw3iHAQ4SyvyVqIqmHsqBKkiT1HhsJc6MuBQ4F7sOS2pZzgLuBwcU/34PlVOoyFlRJkqTeZTXwLuB5wmG+vwSOiZooXR8lXGBqAOGqvefhYb1Sl7KgSpIk9T4rgbcDvwPGAguAt0ZNlJ5PA3OAPsB3CRdE2hkzkNQbWFAlSZJ6p1eBM4FFwHDCBYDOiJooDXXAd4AvFcf/DHwYp5KRqsKCKkmS1HutJxzuex/N51heGjVRXAcTpo+5hFBIPwZcEzOQ1NtYUCVJknq3LcDZwA+BfsC3gRsIh7b2JpOAR4GphOL+HmB21ERSL2RBlSRJ0k7gYuAqoBG4AriLMOdnbzAT+BUwDngOeBthr7KkKrOgSpIkCaAAfBl4H7AVmA4sBibHDNXFhgFzgZuBgcCdwCmEaXgkRWBBlSRJUt6PCFf4XQ5MIFzh91NAFjFTVzgd+A3h6ry7gL8jTCPzesRMUq9nQZUkSWr2l4R5Qb8aO0hkSwjnZN5KOC/1emA+cEjMUJ3kAOC/CBdDOhx4gXBI738Q9iJLisiCKkmS1Oww4DTghNhBErARuAj4KLANOAt4Gvg43fdnyLOAx4HLiuMbCUX819ESSWqhu364SJIkqTq+RTgP9RFgKPB1wgWFjo8Zaj+9hXB+6U8Je02XEabX+Wtgc8RckkpYUCVJkpp9CTgIOD92kMQ8TTgM9jLCOZpTCOdv3gSMjZirLQcRDk9+EphBONf0esIe8vsj5pJUgQVVkiSp2TbgNWBT7CAJagS+AUwkXPm2FvgIYVqW64ED40Xbx0jgOsKe0k8RzqO9i7DX9+8Jc79KSpAFVZIkqdkUwhygF8QOkrBVhKlopgAPAP0JJXAFMIe4h/5OKmZYBlwJDAIWEabMORt4Nl40Se1hQZUkSWr2HuAGwrmJat0jwJnAuwklcCDhgkqPE4rrTMI8o13toOLrPkI47PijxSyPAH8CvBX4WRVySOoEtbEDSJIkqVu7t7hMBf4G+FPCHKOnE875vBf4MfAQYUqXN6oGOBY4g3Cu8NuBPsX7dhLmcb2RMF2QpG7GgipJkqTO8HBxGU2YT/YCwsWI3ltcAF4FFhD2dL5QXFYA6wjnuOb1J1w1+AjgSMKVeCcRinB+z2wBWEyYs/VmYG3n/rUkVZMFVZIkSZ1pFfCF4nIUoaieRZiqZhTwZ8Wl1G6aL140mNZ/Tt0MLCRMG/Nj4KXOCC4pPguqJEmSusozwL8UlzpCSZ0KHAO8mbB3dAyQAX2B4SWPbySUz+cJVwt+irAH9nGgoevjS6o2C6okSZKqYSehXC4oub0PMAQYQDisNyNM97MVp/uReh0LqiRJkmJqADYUF0m9nNPMSJIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhKcZkaSJKnZbOAu4PXYQSSpN7KgSpIkNVtZXCRJEXiIryRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBaWbULdx+++3jY2doTX19/YEbNmzoD7Bq1apNixYt2hQ7U4nBwNDi+i5gTcQs5fQDRubGK4FCpCxlnXvuuaNrampqAE466aR1xx133PbYmSoZMGDAprPOOmt9NV7r7rvvPmT79u111XitjliwYMGQF154YQjAli1bdtx7772vxc5Uog44KDd+OVaQVozNra8FdsYKUsGBQP/i+mYSm790ypQpg0eNGjUUYPDgwbvOP//81D5/WzjvvPNWZFnW5Z+/t95664C+ffse3NWv01G7d+/ObrnlljFN40cffXTNypUrd8XMVMZIwv+fEL7vN0fMUs6Q4gLhc2NtxCz7OPzww+v+6I/+aO/n78yZM1P8/N1r2LBhr55xxhk7YueoBguqknfttdfW1NbWLoudozWLFy/miSeeiB1DXeiOO+7Yuz5x4kRqa9P9+Ny9e/fXgcur8Vq7du26s7a2dnI1Xqsjli9fzu233x47hnqxhQsX7l2fMGECF154YcQ0bbvvvvuGUYWS379//3dnWfaTrn6djtqzZ4+fHT3csmXLWLas+cfLSy65hOLvoZO0devWs4B7YueohnS/CpIkSZKkXsWCKkmSJElKQrrHqEmV/UWWZQvb3qx6duzYsYTieRaTJk16bMmSJe+LHKnUzcBpxfVtwPERs5TzPuDfcuNTgaTOFayrq3th585w6t3SpUtvnDJlynWRI5W6sVAo/HHkDNdlWXZj5AwtrFmz5jZgEsD48eNfX758+UmRI5X6KHDV3tFX+Xi8KBX8Hd/Yu/5Obuc87o2YZl9/z/XsYWAYjH4W/vGq1h9QXUceecdnn3323kkAu3fv3p5l2XGxM+U1NjYen8Chtsm9L3/4wx8OBBY1jU899dSrFy1adGvESOU8AU3f+ywAZkbMUs4PgaZTQDZR/CxOxdSpUy99+OGH/6lpvGnTpqOHDx++O2amUoVC4Tl64Q5FC6q6nUKh8MqMGTNejJ0jb8KECY1N6/37998JJJUPyJ9UXyC9fKVldAWwOkaQ9ti6devGs88+O6n3cN68edtiZwA2pPa+TJ06de8FfWpraxtJ73u/5cWszmQNNWldIKyFYWzhXYn928xobB7U7oLLVsYLs6+6uvv3fg8WCoXG1P6N3HnnnSNiZwAKqb0vF1988db8eOTIka+R3udH7nufHaSXL39BteQ+f0eMGLEuP77zzjuXzZ07N6kLYdXX18eOEEWva+SSJEmSpDRZUCVJkiRJSbCgSpIkSZKSYEGVJEmSJCXBgipJkiRJSoIFVZIkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJNTGDtDDZcCbgTFAf2AV8BTQ8AafdwIwChgMbAReBFa/weeUJEmSpKhS34M6Hnghtwxux2NqSh5zXFeFK/p47rX+J5fhE8BS4DngQeBu4HFCkfwCMHA/X+dQ4OvAy8Xn/CUwH3gYeAX4DfBhWv+aXp3L+p9tvN4/0fJ9/HAb29+W2/ZDbWwrSZIkSftIfQ9qX+CI3Li9hTr/mLrOi1PW8NzrLSeU6NuAd1fYfgTwGeBdwHRgUzte4zLgesJe2HIyYBLw34QieR6wvsx2T+ayXgj8LVCo8Jzn0/J9PAf4ToVtDwBmAP2K499W2E6SJEmSKkq9oHY3NYS9qO8mHMb7EPAYsBk4DDgXOLC47VsJpXNWG895LXBNbrwV+CmwBHgdGEkouqcW7387cC/wNmBHyXM9BOwhfN0PASYCvy/zmm8iFN6804E+lD88+e00l9O1hD3FkiRJkrRfLKid6zTCe/p74CL2LX9XAj8CziyOLwW+CLxU4flmAJ/LjX9I2Ju6rmS7awh7RL9LOHT4JOBfgE+VbLcJWAxMKY6nlckIcAbNe6t3EvZCDwNOBh4ts/203Pr9VN4rK0mSJEkVpX4OandTC6wkFLxyxW8j8H5gS3Hch1Asy+lLOOc0K45vBS5m33LaZC7wkdz444S9q6V+nlufVub+0ttv3M/t76+wjSRJkiS1yoLa+T5DOMy1ktXAvNx4coXtLiRcGAlCob2ctvdM3kLYQwrhfNWLy2yTL5DvpPxe9KbCuRn4MtBYcnvegcAJufHPy2wjSZIkSW2yoHau3YQ9nW15LLc+vsI25+TW76D10pv349z628vc/zCwvbg+FDil5P5DgSOL678gXCH4d8Xx24ABJdufSfP30XLCVXwlSZIkab9ZUDvXM8C2dmy3Jrc+tMI2+XK5cD8yLM2tH1Pm/p3Ar3Lj0r2i+fF9xT+b9or2B6a2sr17TyVJkiR1mBdJ6lzlpnYpZ1duvV+Z++uA0bnxFbQ9DymEKW7yc8W+qcJ2Pwf+uLg+jXChJnLj/HZNf16Zuz9fRM8ss70kSZIk7TcLaufa00nPM7xkfETZrdo2qMLt+fNQpxCu/Nu057epcL5KmDcV4JeEUt2PlgX2MGBCcb0APNDBnJIkSZJkQU1Un5Lx/VS+em9rtle4/TfABkIRriNMj/MzwryoTXtu89PFbAUeAd5BmGpmePHx+bL6e0KplSRJkqQO6YkFtdwhs91NaRn9GvCTTnz+BuBB4PzieBqhoLZ2uO7PCQW1D3A6cDuefypJkiSpE6V+kaQdJeP+7XhMubk/u5sdtNwbeUKlDd+AcvOhlrtAUqXtMzz/VJIkSVInSr2gbioZH9KOx5zaFUEiyJ/PeU7FrTouXygnEYr96cXxc8BLJds/SpgXFUJBPRYYVRzvAR7qgoySJEmSepHUC+rrwOrc+G3teMxHuyhLtf0ot34y8J5Ofv6lwMrieg3hKr3DiuNye0N3E+ZFBTgamJm779fs+8sESZIkSdovqRdUaDkH6Mdo/bzZD9M8fUp3dzvhwkNN/hs4fD8efxDNhbOS/NV8L8+tlx7e2yRfXC+vcLskSZIkdUh3KKjfz60fB9wMDCnZZiDwOeAmYEuVcnW1RkLhbjoPdxSwGLiUyheC6gOcAcwBVgBvbuM18sVyQO51K00XU2770tslSZIkqUO6w1V8fwIsoPnw3vcDZxdve52wp/CthDk/GwmHnv64+jG7xGLC3+d7hAtEjQC+DXyFsGf5ZUKBHUbYu3oicMB+PH+5YrkEWF9h+yeANbS8ENV2Wu7lliRJkqQO6Q4FtRG4CLgXOKZ42xD2PSdzG/DXwB3Vi1YVc4HlwHcJ85QCDAXOauNxrwAb29jmZeAZ4KjcbZUO74UwL+r9wJ/nblvAvldbliRJkqT91h0KKoSL+ZwCXAF8gJaFagOhxP0H8DRh+pO5Jfd3padzr/dkOx/zUu4xq1vbsGgxcDzwp8AFhMN4S6fT2UK48NEvgHsIe0cb2vHcXwPemRv/qNKGRd8nHErc3u0lSZIkqV26S0EF2Ap8obgMJhzau5F9D0ctAO+rYq4fs/+HFD9cXPZHI3BbcYFwSPMIwtew3PvQXl8vLu01v7hIkiRJUqfqTgU1bzPNc3L2VluLiyRJkiT1CN3hKr6SJEmSpF7AgipJkiRJSoIFVZIkSZKUhO56DmpHfRAY0AnP8yzwYCc8jzogy7Jb6+vrd8XOkXf11VcPaVpfvHjxZGBVxDjlvCm3PpD08pX+u/wd4cJgydi1q/lbbtKkSZdfdtllH4oYZx+FQmF47AzA1fX19Z+IHSLvO9/5zoim9WXLlg0lve/9QS1Gp3JTpBztM58LuIezY8doYU/+PVx5NPS7K16YfT31VDa0ab1fv34D6+vrNL98BAAAAwFJREFUU/se7Bs7ADAgtfdl27ZtNbfccsve8fz5868D/jleorLynx+nkd7n24jcenKfv/Pnz2/x+fuBD3xg+cyZM2PFqaRX7kzsbQX1OuDgTniem7GgxjSi7U2qK8uyvet79uzpBxwSL02bMtLOB53z77RTFQqFves1NTUHAAfES5OswcUlGTU1zf+3NzQ01JD69/4OUvhFQ2W7GcDuTvlFbxdpqIWGA2OnyGtoOeFbd/j8jSG59yX/2QHQ0NAwDBgWJ0271JHYe1giua9xQ+k/zixLKl9v1itbuSRJkiQpPb1tD+pRdE4pT+rw0p7u2muvbZw3b94HY+dozQknnHDs6NGjhwGsXLnyD08++eRLsTOVGAscVlzfDDweMUs5g4ETcuNHgIYK20Yxbdq0t9bW1vYBGDRo0NNZlnV07uEu19jY+Ey1XqtPnz6fLRQKSe2xyhs9evRh06dPHwuwadOmjQsXLvx97EwlhgLH5cYLYgVpxdty608Cr8cKUsGxNO/ZehlYETHLPo455pgx48aNGw8wbNiwzVmWpfb528Lw4cO3VeN1amtrH2tsbEz2//Ysy/pMnz79rU3jJUuWPL5mzZrUpjg8Hmg6xWgF4fs/JeOAQ4vrrxM+P5IxatSooSeeeOLez98syx7OsqzQ2mNiqq2tTfqzozNls2fP3vuFyLLsqFmzZj0bM5AkSZIkqXeYM2fOkYVCYe8v1z3EV5IkSZKUBAuqJEmSJCkJFlRJkiRJUhIsqJIkSZKkJFhQJUmSJElJsKBKkiRJkpJgQZUkSZIkJcGCKkmSJElKggVVkiRJkpQEC6okSZIkKQkWVEmSJElSEiyokiRJkqQkWFAlSZIkSUmwoEqSJEmSkmBBlSRJkiQlwYIqSZIkSUqCBVWSJEmSlAQLqiRJkiQpCRZUSZIkSVISLKiSJEmSpCRYUCVJkiRJSajNDwqFwvg5c+bEyiJJkiRJ6kUKhcL4/Li25P57CoVC9dJIkiRJklTkIb6SJEmSpCRYUCVJkiRJSfhf0SSRmgN3HHEAAAAASUVORK5CYII="
+ }
+ },
+ "cell_type": "markdown",
+ "id": "7d66b1a2",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "
\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c2d48c45",
+ "metadata": {},
+ "source": [
+ "\n",
+ "Question: Which MPI directives would you use to implement latency hiding in the communication the ghost values?\n",
+ "
\n",
+ "\n",
+ " a) MPI_Send and MPI_Recv\n",
+ " b) MPI_Bsend and MPI_Recv\n",
+ " c) MPI_Isend and MPI_Irecv \n",
+ " d) MPI_Sendrecv"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "29ade8f6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "answer = \"x\" # replace x with a, b, c or d\n",
+ "lh_check(answer)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9bd014b0",
+ "metadata": {},
+ "source": [
+ "## Conclusion\n",
+ "\n",
+ "- We learned how to parallelize the Jacobi method efficiently.\n",
+ "- Using block partitions the communication overhead is small if the problem size is larger than the number of processors.\n",
+ "- In addition, one can use latency hiding to further reduce the overhead caused by communication.\n",
+ "- Both are not true if we use cyclic data partitions.\n",
+ "- Gauss-Seidel is a simple variation of Jacobi but is much more challenging to parallelize.\n",
+ "- One needs to consider a special order (red-black) when updating the values to parallelize the method.\n",
+ "- We learned how to implement a 1D Jacobi method with MPI.\n",
+ "- Once needs to be careful when using blocking directives to avoid dead locks."
+ ]
+ },
{
"cell_type": "markdown",
"id": "47643bf6",
@@ -1005,12 +1722,152 @@
},
{
"cell_type": "markdown",
- "id": "0edeee84",
+ "id": "6527d24f",
"metadata": {},
"source": [
"### Exercise 1\n",
"\n",
- "Transform the parallel implementation of the 1d Jacobi method (function `jacopi_mpi`) to use latency hiding (overlap between computation of interior values and communication)."
+ "The following code implements the 1D Jacobi method studied in this notebook, but using non-blocking sends and receives. Modify this code so that you overlap the communication of the ghost values with local computations as explained above in the section \"latency hiding\". You only need to modify function `jacobi_mpi_latency_hiding`. Copy the code below to a file called `ex1.jl`. Modify the file (e.g. with vscode). Run it from the Julia REPL using the `run` function as explained in the [Getting Started tutorial](https://www.francescverdugo.com/XM_40017/dev/getting_started_with_julia/#Running-MPI-code)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9b57f876",
+ "metadata": {},
+ "source": [
+ "\n",
+ "```julia\n",
+ "# file ex1.jl (begin)\n",
+ "\n",
+ "using MPI\n",
+ "MPI.Init()\n",
+ "\n",
+ "function jacobi_mpi_latency_hiding(n,niters,comm)\n",
+ " u, u_new = init(n,comm)\n",
+ " load = length(u)-2\n",
+ " rank = MPI.Comm_rank(comm)\n",
+ " nranks = MPI.Comm_size(comm)\n",
+ " nreqs = 2*((rank != 0) + (rank != (nranks-1)))\n",
+ " reqs = MPI.MultiRequest(nreqs)\n",
+ " for t in 1:niters\n",
+ " ireq = 0\n",
+ " if rank != 0\n",
+ " neig_rank = rank-1\n",
+ " u_snd = view(u,2:2)\n",
+ " u_rcv = view(u,1:1)\n",
+ " dest = neig_rank\n",
+ " source = neig_rank\n",
+ " ireq += 1\n",
+ " MPI.Isend(u_snd,comm,reqs[ireq];dest)\n",
+ " ireq += 1\n",
+ " MPI.Irecv!(u_rcv,comm,reqs[ireq];source)\n",
+ " end\n",
+ " if rank != (nranks-1)\n",
+ " neig_rank = rank+1\n",
+ " u_snd = view(u,(load+1):(load+1))\n",
+ " u_rcv = view(u,(load+2):(load+2))\n",
+ " dest = neig_rank\n",
+ " source = neig_rank\n",
+ " ireq += 1\n",
+ " MPI.Isend(u_snd,comm,reqs[ireq];dest)\n",
+ " ireq += 1\n",
+ " MPI.Irecv!(u_rcv,comm,reqs[ireq];source)\n",
+ " end\n",
+ " MPI.Waitall(reqs)\n",
+ " for i in 2:load+1\n",
+ " u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ " u, u_new = u_new, u\n",
+ " end\n",
+ " return u\n",
+ "end\n",
+ "\n",
+ "function init(n,comm)\n",
+ " nranks = MPI.Comm_size(comm)\n",
+ " rank = MPI.Comm_rank(comm)\n",
+ " if mod(n,nranks) != 0\n",
+ " println(\"n must be a multiple of nranks\")\n",
+ " MPI.Abort(comm,1)\n",
+ " end\n",
+ " load = div(n,nranks)\n",
+ " u = zeros(load+2)\n",
+ " if rank == 0\n",
+ " u[1] = -1\n",
+ " end\n",
+ " if rank == nranks-1\n",
+ " u[end] = 1\n",
+ " end\n",
+ " u_new = copy(u)\n",
+ " u, u_new\n",
+ "end\n",
+ "\n",
+ "function gather_final_result(u,comm)\n",
+ " load = length(u)-2\n",
+ " rank = MPI.Comm_rank(comm)\n",
+ " if rank !=0\n",
+ " u_snd = view(u,2:(load+1))\n",
+ " MPI.Send(u_snd,comm,dest=0)\n",
+ " u_root = zeros(0)\n",
+ " else\n",
+ " nranks = MPI.Comm_size(comm)\n",
+ " n = load*nranks\n",
+ " u_root = zeros(n+2)\n",
+ " u_root[1] = -1\n",
+ " u_root[end] = 1\n",
+ " lb = 2\n",
+ " ub = load+1\n",
+ " u_root[lb:ub] = view(u,lb:ub)\n",
+ " for other_rank in 1:(nranks-1)\n",
+ " lb += load\n",
+ " ub += load\n",
+ " u_rcv = view(u_root,lb:ub)\n",
+ " MPI.Recv!(u_rcv,comm;source=other_rank)\n",
+ " end\n",
+ " end\n",
+ " return u_root\n",
+ "end\n",
+ "\n",
+ "function jacobi(n,niters)\n",
+ " u = zeros(n+2)\n",
+ " u[1] = -1\n",
+ " u[end] = 1\n",
+ " u_new = copy(u)\n",
+ " for t in 1:niters\n",
+ " for i in 2:(n+1)\n",
+ " u_new[i] = 0.5*(u[i-1]+u[i+1])\n",
+ " end\n",
+ " u, u_new = u_new, u\n",
+ " end\n",
+ " u\n",
+ "end\n",
+ "\n",
+ "n = 12\n",
+ "niters = 100\n",
+ "comm = MPI.Comm_dup(MPI.COMM_WORLD)\n",
+ "u = jacobi_mpi_latency_hiding(n,niters,comm)\n",
+ "u_root = gather_final_result(u,comm)\n",
+ "rank = MPI.Comm_rank(comm)\n",
+ "if rank == 0\n",
+ " u_seq = jacobi(n,niters)\n",
+ " if isapprox(u_root,u_seq)\n",
+ " println(\"Test passed 🥳\")\n",
+ " else\n",
+ " println(\"Test failed 😢\")\n",
+ " end\n",
+ "end\n",
+ "\n",
+ "# file ex1.jl (end)\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1af362aa",
+ "metadata": {},
+ "source": [
+ "### Exercise 2\n",
+ "\n",
+ "In the parallel implementation of the Jacobi method, we assumed that the method runs for a given number of iterations. In function, `jacobi_with_tol` at the beginning of the notebook shows how the Jacobi iterations can be stopped when the difference between iterations is small. Implement a parallel version of this function. Start with the in Exercise 1 and add the stopping criterion implemented in `jacobi_with_tol`. Use a text editor and the Julia REPL. Do not try to implement the code in a notebook."
]
},
{
diff --git a/dev/jacobi_method/index.html b/dev/jacobi_method/index.html
index f6abdec..129da4e 100644
--- a/dev/jacobi_method/index.html
+++ b/dev/jacobi_method/index.html
@@ -1,5 +1,5 @@
-
+Jacobi method · XM_40017
Tip
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
-
In our version of the jacobi method, we return after a given number of iterations. Other stopping criteria are possible. For instance, iterate until the difference between u and u_new is below a tolerance:
+In our version of the Jacobi method, we return after a given number of iterations. Other stopping criteria are possible. For instance, iterate until the maximum difference between u and u_new is below a tolerance:
In [ ]:
-
-
-using LinearAlgebra: norm
-function jacobi_with_tol(n,tol)
+function jacobi_with_tol(n,tol)
u = zeros(n+2)
u[1] = -1
u[end] = 1
u_new = copy(u)
- increment = similar(u)
while true
+ diff = 0.0
for i in 2:(n+1)
- u_new[i] = 0.5*(u[i-1]+u[i+1])
+ ui_new = 0.5*(u[i-1]+u[i+1])
+ u_new[i] = ui_new
+ diff_i = abs(ui_new-u[i])
+ diff = max(diff_i,diff)
end
- increment .= u_new .- u
- if norm(increment)/norm(u_new) < tol
+ if diff < tol
return u_new
end
u, u_new = u_new, u
@@ -7780,127 +7810,7 @@ a.anchor-link {
-However, we are not going to parallelize this more complex in this notebook (we will consider it later in this course).
-
-
-
-
-
-
-
-
-
-
-
-
-
-Where can we exploit parallelism?¶
Look at the two nested loops in the sequential implementation:
-for t in 1:nsteps
- for i in 2:(n+1)
- u_new[i] = 0.5*(u[i-1]+u[i+1])
- end
- u, u_new = u_new, u
-end
--
-
- The outer loop over
tcannot be parallelized. The value ofuat stept+1depends on the value at the previous stept.
- - The inner loop is trivially parallel. The loop iterations are independent (any order is possible). -
-
-
-
-
-
-
-
-
-The Gauss-Seidel method¶
The usage of u_new seems a bit unnecessary at first sight, right?. If we remove it, we get another method called Gauss-Seidel.
-
-
-
-
-
-
-
-In [ ]:
-
-
-
-
-
-function gauss_seidel(n,niters)
- u = zeros(n+2)
- u[1] = -1
- u[end] = 1
- for t in 1:niters
- for i in 2:(n+1)
- u[i] = 0.5*(u[i-1]+u[i+1])
- end
- end
- u
-end
-
-
-
-
-
-
-
-
-
-Note that the final solution is nearly the same for a large enough number of iterations.
-
-
-
-
-
-
-
-
-In [ ]:
-
-
-
-
-
-gauss_seidel(5,1000)
-
-
-
-
-
-
-
-
-
-
-
-Question: Which of the two loops in the Gauss-Seidel method are trivially parallelizable?
-
-for t in 1:niters
- for i in 2:(n+1)
- u[i] = 0.5*(u[i-1]+u[i+1])
- end
-end
-a) Both of them
-b) The outer, but not the inner
-c) None of them
-d) The inner, but not the outer
-
-
@@ -7916,6 +7826,28 @@ d) The inner, but not the outer
-
-
-
In [ ]:
-
-
-
-
+answer = "x" # replace x with a, b, c or d
-gauss_seidel_1_check(answer)
-However, we are not going to parallelize this more complex in this notebook (left as an exercise).
+
+
+
+
+
+
+
+
+
+Where can we exploit parallelism?¶
Look at the two nested loops in the sequential implementation:
+for t in 1:nsteps
+ for i in 2:(n+1)
+ u_new[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ u, u_new = u_new, u
+end
+-
+
- The outer loop over
tcannot be parallelized. The value ofuat stept+1depends on the value at the previous stept.
+ - The inner loop is trivially parallel. The loop iterations are independent (any order is possible). +
@@ -8098,221 +8030,19 @@ d) The inner, but not the outer
+
-
-
-
-MPI implementation¶
We consider the implementation using MPI. The programming model of MPI is generally better suited for data-parallel algorithms like this one than the task-based model provided by Distributed.jl. In any case, one can also implement it using Distributed.jl, but it requires some extra effort to setup the remote channels right for the communication between neighbor processes.
-Take a look at the implementation below and try to understand it.
-
-
-
-
-
-
-
In [ ]:
-
-
-
+
] add MPI
-Extension to 2D¶
The Jacobi method studied so far was for a one dimensional Laplace equation. In real-world applications however, one solve equations in multiple dimensions. Typically 2D and 3D. The 2D and 3D cases are conceptually equivalent, but we will discuss the 2D case here for simplicity.
+Now the goal is to find the interior points of a 2D grid given the values at the boundary.
-
-
-
-
-
-
-In [ ]:
-
-
-
-
-
-using MPI
-
-
-
-
-
-
-
-
-In [ ]:
-
-
-
-
-
-code = quote
- using MPI
- MPI.Init()
- function jacobi_mpi(n,niters)
- comm = MPI.COMM_WORLD
- nranks = MPI.Comm_size(comm)
- rank = MPI.Comm_rank(comm)
- if mod(n,nranks) != 0
- println("n must be a multiple of nranks")
- MPI.Abort(comm,1)
- end
- load = div(n,nranks)
- u = zeros(load+2)
- u[1] = -1
- u[end] = 1
- u_new = copy(u)
- for t in 1:niters
- # Communication
- if rank != 0
- neig_rank = rank-1
- s = 2
- r = 1
- MPI.Sendrecv!(view(u,s:s),view(u,r:r),comm;dest=neig_rank,source=neig_rank)
- end
- if rank != (nranks-1)
- neig_rank = rank+1
- s = load+1
- r = load+2
- MPI.Sendrecv!(view(u,s:s),view(u,r:r),comm;dest=neig_rank,source=neig_rank)
- end
- # Local computation
- for i in 2:(load+1)
- u_new[i] = 0.5*(u[i-1]+u[i+1])
- end
- u, u_new = u_new, u
- end
- # Gather the results
- if rank !=0
- lb = 2
- ub = load+1
- MPI.Send(view(u,lb:ub),comm,dest=0)
- u_all = zeros(0) # This will nevel be used
- else
- u_all = zeros(n+2)
- # Set boundary
- u_all[1] = -1
- u_all[end] = 1
- # Set data for rank 0
- lb = 2
- ub = load+1
- u_all[lb:ub] = view(u,lb:ub)
- # Set data for other ranks
- for other_rank in 1:(nranks-1)
- lb += load
- ub += load
- MPI.Recv!(view(u_all,lb:ub),comm;source=other_rank)
- end
- end
- return u_all
- end
- function jacobi(n,niters)
- u = zeros(n+2)
- u[1] = -1
- u[end] = 1
- u_new = copy(u)
- for t in 1:niters
- for i in 2:(n+1)
- u_new[i] = 0.5*(u[i-1]+u[i+1])
- end
- u, u_new = u_new, u
- end
- u
- end
- function testit(load)
- comm = MPI.COMM_WORLD
- nranks = MPI.Comm_size(comm)
- rank = MPI.Comm_rank(comm)
- n = load*nranks
- niters = 100
- u_par = jacobi_mpi(n,niters)
- if rank == 0
- # Compare agains serial
- u_seq = jacobi(n,niters)
- if u_par ≈ u_seq
- println("Test passed 🥳")
- else
- println("Test failed")
- end
- end
- end
- testit(3)
-end
-run(`$(mpiexec()) -np 4 julia --project=. -e $code`);
-
-
-
-
-
-
-
-
-
-
-Question: In function jacobi_mpi, how many messages per iteration are sent from a process away from the boundary (excluding the messages to collect the data on rank 0)?
-
-a) 1
-b) 2
-c) 3
-d) 4
-
-
-
-
-
-
-
-In [ ]:
-
-
-
-
-
-answer = "x" # replace x with a, b, c or d
-jacobi_2_check(answer)
-
-
-
-
-
-
-
-
-
-
-Latency hiding¶
Can our implementation above be improved? Note that we only need communications to update the values at the boundary of the portion owned by each process. The other values (the one in green in the figure below) can be updated without communications. This provides the opportunity of overlapping the computation of the interior values (green cells in the figure) with the communication of the ghost values. This technique is called latency hiding, since we are hiding communication latency by overlapping it with computation that we need to do anyway.
-The modification of the implementation above to include latency hiding is leaved as an exercise (see below).
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Extension to 2D¶
Now, let us study the method for a 2D example.
-
+
@@ -8325,7 +8055,7 @@ d) 4
-
+
@@ -8338,7 +8068,7 @@ d) 4
-
+
@@ -8348,7 +8078,7 @@ d) 4
-
+
+
+
+
+
+
+
+
+
+
+
+If you run the function for zero iterations you will see the initial condition.
+In [ ]:
-
u = jacobi_2d(10,0)
+n = 10
+niters = 0
+u = jacobi_2d(n,niters)
+
@@ -8421,7 +8191,7 @@ d) 4
-
+
+
+
+
+
+
+
+
+If you run the problem for enough iterations, you should converge to the exact solution. In this case, the exact solution is a matrix with all values equal to one.
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+n = 10
+niters = 300
+u = jacobi_2d(n,niters)
+
+
@@ -8438,7 +8208,7 @@ d) 4
-
+
@@ -8451,7 +8221,7 @@ d) 4
-
+
@@ -8462,7 +8232,7 @@ d) 4
-
+
@@ -8473,7 +8243,7 @@ d) 4
-
+
@@ -8486,7 +8256,7 @@ d) 4
-
+
@@ -8502,7 +8272,7 @@ d) 4
-
+
@@ -8513,7 +8283,7 @@ d) 4
-
+
@@ -8526,7 +8296,7 @@ d) 4
-
+
@@ -8542,7 +8312,7 @@ d) 4
-
+
@@ -8553,7 +8323,7 @@ d) 4
-
+
@@ -8566,7 +8336,7 @@ d) 4
-
+
@@ -8582,7 +8352,7 @@ d) 4
-
+
@@ -8626,7 +8396,7 @@ d) 4
-
+
@@ -8643,6 +8413,1083 @@ d) 4
+
+
+
+
+
+
+
+
+
+The Gauss-Seidel method¶
Now let us study a slightly more challenging method. The implementation of Jacobi used an auxiliary array u_new. The usage of u_new seems a bit unnecessary at first sight, right? If we remove it, we get another method called Gauss-Seidel.
The following cell contains the sequential implementation of Gauss-Seidel. It is obtained taking the sequential implementation of Jacobi, removing u_new, and only using u. This method is more efficient in terms of memory consumed (only one vector required instead of two). However it is harder to parallelize.
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+function gauss_seidel(n,niters)
+ u = zeros(n+2)
+ u[1] = -1
+ u[end] = 1
+ for t in 1:niters
+ for i in 2:(n+1)
+ u[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ end
+ u
+end
+
+
+
+
+
+
+
+
+
+Note that the final solution is nearly the same as for Jacobi for a large enough number of iterations.
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+n = 5
+niters = 1000
+gauss_seidel(n,niters)
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+jacobi(n,niters)
+
+
+
+
+
+
+
+
+
+
+
+Question: Which of the two loops in the Gauss-Seidel method are trivially parallelizable?
+
+for t in 1:niters
+ for i in 2:(n+1)
+ u[i] = 0.5*(u[i-1]+u[i+1])
+ end
+end
+a) Both of them
+b) The outer, but not the inner
+c) None of them
+d) The inner, but not the outer
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+answer = "x" # replace x with a, b, c or d
+gauss_seidel_1_check(answer)
+
+
+
+
+
+
+
+
+
+
+If you look into the algorithm closely, you will realize that the steps in loop over i are not independent. To update u[i] we first need to update u[i-1], which requires to update u[i-2] etc. This happens for Gauss-Seidel but not for Jacobi. For Jacobi the updates are written into the temporary array u_new instead of updating u directly. In other words, in Jacobi, updating u_new[i] does not require any entry updated entry of u_new. It only requires entries in vector u, which is not modified in the loop over i.
+
+
+
+
+
+
+
+
+Backwards Gauss-Seidel¶
In addition, the the result of the Gauss-Seidel method depends on the order of the steps in the loop over i. This is another symptom that tells you that this loop is hard to parallelize. For instance, if you do the iterations over i by reversing the loop order, you get another method called backward Gauss-Seidel.
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+function backward_gauss_seidel(n,niters)
+ u = zeros(n+2)
+ u[1] = -1
+ u[end] = 1
+ for t in 1:niters
+ for i in (n+1):-1:2
+ u[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ end
+ u
+end
+
+
+
+
+
+
+
+
+
+Both Jacobi and forward and backward Gauss-Seidel converge to the same result, but they lead to slightly different values during the iterations. Check it with the following cells. First, run it with one niters=1 and then with niters=100.
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+n = 5
+niters = 1
+u_forward = gauss_seidel(n,niters)
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+u_backward = backward_gauss_seidel(n,niters)
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+u_jacobi = jacobi(n,niters)
+
+
+
+
+
+
+
+
+
+
+Red-black Gauss-Seidel¶
There is another version called red-black Gauss-Seidel. This uses a very clever order for the steps in the loop over i. It does this loop in two phases. First, one updates the entries with even index, and then the entries with odd index.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+The implementation is given in the following cell. Note that we introduced an extra loop that represent +each one of the two phases.
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+function red_black_gauss_seidel(n,niters)
+ u = zeros(n+2)
+ u[1] = -1
+ u[end] = 1
+ for t in 1:niters
+ for color in (0,1)
+ for i in (n+1):-1:2
+ if color == mod(i,2)
+ u[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ end
+ end
+ end
+ u
+end
+
+
+
+
+
+
+
+
+
+Run the method for several values of niters.
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+n = 5
+niters = 1000
+red_black_gauss_seidel(n,niters)
+
+
+
+
+
+
+
+
+
+
+
+Question: Which of the loops in the red-black Gauss-Seidel method are trivially parallelizable?
+
+for t in 1:niters
+ for color in (0,1)
+ for i in (n+1):-1:2
+ if color == mod(i,2)
+ u[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ end
+ end
+end
+a) All loops
+b) Loop over t only
+c) Loop over color only
+d) Loop over i only
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+answer = "x" # replace x with a, b, c or d
+gauss_seidel_2_check(answer)
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+gauss_seidel_2_why()
+
+
+
+
+
+
+
+
+
+
+MPI implementation¶
We consider the implementation of the Jacobi method using MPI. We will consider the 1D version for simplicity.
+
+Note: The programming model of MPI is generally better suited for data-parallel algorithms like this one than the task-based model provided by Distributed.jl. In any case, one can also implement it using Distributed.jl, but it requires some extra effort to setup the remote channels right for the communication between neighbor processes.
+
+We are going to implement the method in a function with the following signature
+u = jacobi_mpi(n,niters,comm)
+The signature will be very similar to the sequential function jacobi, but there are some important differences:
-
+
- The parallel one takes an MPI communicator object. This allows the user to decide which communicator to use. For instance,
MPI.COMM_WORLDdirectly or a duplicate of this one (which is the recommended approach).
+ - Function
jacobi_mpiwill be called on multiple MPI ranks. Thus, the values ofnandnitersshould be the same in all ranks. The caller of this function has to make sure that this is true. Otherwise, the parallel code will not work.
+ - The result
uis NOT the same as the result of the sequential functionjacobi. It will contained only the local portion ofustored at each rank. To recover the same vector as in the sequential case, one needs to gather these pieces in a single rank. We will do this later.
+
The implementation of the parallel function is given in the following code snipped:
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+code1 = quote
+ function jacobi_mpi(n,niters,comm)
+ # Initialization
+ u, u_new = init(n,comm)
+ for t in 1:niters
+ # Communication
+ ghost_exchange!(u,comm)
+ # Local computation
+ local_update!(u,u_new)
+ u, u_new = u_new, u
+ end
+ return u
+ end
+end;
+
+
+
+
+
+
+
+
+
+
+In the following cells, we discuss the implementation of the helper functions init, ghost_exchange, and local_update.
+
+
+
+
+
+
+
+
+Initialization¶
Let us start with function init. This is its implementation:
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+code2 = quote
+ function init(n,comm)
+ nranks = MPI.Comm_size(comm)
+ rank = MPI.Comm_rank(comm)
+ if mod(n,nranks) != 0
+ println("n must be a multiple of nranks")
+ MPI.Abort(comm,1)
+ end
+ load = div(n,nranks)
+ u = zeros(load+2)
+ if rank == 0
+ u[1] = -1
+ end
+ if rank == nranks-1
+ u[end] = 1
+ end
+ u_new = copy(u)
+ u, u_new
+ end
+end;
+
+
+
+
+
+
+
+
+
+
+This function crates and initializes the vector u and the auxiliary vector u_new and fills in the boundary values. Note that we are not creating the full arrays like in the sequential case. We are only creating the parts to be managed by the current rank. To this end, we start by computing the number of entries to be updated in this rank, i.e., variable load. We have assumed that n is a multiple of the number of ranks for simplicity. If this is not the case, we stop the computation with MPI.Abort. Note that we are allocating two extra elements in u (and u_new) for the ghost cells or boundary conditions. The following figure displays the arrays created for n==9 and nranks==3 (thus load == 3). Note that the first and last elements of the arrays are displayed with gray edges denoting that they are the extra elements allocated for ghost cells or boundary conditions.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Communication¶
Once the working arrays have been created, we can start with the iterations of the Jacobi method. Remember that this is implemented in parallel by updating the ghost cells with data from neighboring processes and then performing the sequential Jacobi update. See figure:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+The communication step happens in function ghost_exchange!. This one modifies the ghost cells in the input vector u by importing data using MPI point-to-point communication. See the following code:
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+code3 = quote
+ function ghost_exchange!(u,comm)
+ load = length(u)-2
+ rank = MPI.Comm_rank(comm)
+ nranks = MPI.Comm_size(comm)
+ if rank != 0
+ neig_rank = rank-1
+ u_snd = view(u,2:2)
+ u_rcv = view(u,1:1)
+ dest = neig_rank
+ source = neig_rank
+ MPI.Sendrecv!(u_snd,u_rcv,comm;dest,source)
+ end
+ if rank != (nranks-1)
+ neig_rank = rank+1
+ u_snd = view(u,(load+1):(load+1))
+ u_rcv = view(u,(load+2):(load+2))
+ dest = neig_rank
+ source = neig_rank
+ MPI.Sendrecv!(u_snd,u_rcv,comm;dest,source)
+ end
+ end
+end;
+
+
+
+
+
+
+
+
+
+
+Note that we have used MPI.Sendrecv! to send and receive values.
+
+
+
+
+
+
+
+
+
+
+Question: Is this other implementation based on `MPI.Send` and `MPI.Recv!` correct?
+
+function ghost_exchange!(u,comm)
+ load = length(u)-2
+ rank = MPI.Comm_rank(comm)
+ nranks = MPI.Comm_size(comm)
+ if rank != 0
+ neig_rank = rank-1
+ u_snd = view(u,2:2)
+ u_rcv = view(u,1:1)
+ dest = neig_rank
+ source = neig_rank
+ MPI.Send(u_snd,comm;dest)
+ MPI.Recv!(u_rcv,comm;source)
+ end
+ if rank != (nranks-1)
+ neig_rank = rank+1
+ u_snd = view(u,(load+1):(load+1))
+ u_rcv = view(u,(load+2):(load+2))
+ dest = neig_rank
+ source = neig_rank
+ MPI.Send(u_snd,comm;dest)
+ MPI.Recv!(u_rcv,comm;source)
+ end
+ end
+a) It is correct.
+c) It is incorrect, but it might provide the right result depending on the MPI implementation.
+b) It is incorrect, and it is guaranteed that it will result in a dead lock.
+d) This implementation does not work when distributing over just a single MPI rank.
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+answer = "x" # replace x with a, b, c or d
+sndrcv_check(answer)
+
+
+
+
+
+
+
+
+
+
+Question: How would you fix the implementation above, while still using `MPI.Send` and `MPI.Recv!` instead of `MPI.Sendrecv!` ?
+
+Think about the answer. To uncover the the solution run next cell.
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+sndrcv_fix_answer()
+
+
+
+
+
+
+
+
+
+Local computation¶
Once the ghost values have the right values, we can perform the Jacobi update locally at each process. This is done in function local_update!. Note that here we only update the data owned by the current MPI rank, i.e. we do not modify the ghost values. There is no need to modify the ghost values since they will updated by another rank, i.e. the rank that own the value. In the code this is reflected in the loop over i. We do not visit the first nor the last entry in u_new.
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+code4 = quote
+ function local_update!(u,u_new)
+ load = length(u)-2
+ for i in 2:(load+1)
+ u_new[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ end
+end;
+
+
+
+
+
+
+
+
+
+Running the code¶
Not let us put all pieces together and run the code. If not done yet, install MPI.
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+] add MPI
+
+
+
+
+
+
+
+
+
+The following cells includes all previous code snippets into a final one. Note that we are eventually calling function jacobi_mpi and showing the result vector u. Run the following code for 1 MPI rank, then for 2 and 3 MPI ranks. Look into the values of u. Does it make sense?
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+using MPI
+code = quote
+ using MPI
+ MPI.Init()
+ $code1
+ $code2
+ $code3
+ $code4
+ n = 9
+ niters = 200
+ comm = MPI.Comm_dup(MPI.COMM_WORLD)
+ u = jacobi_mpi(n,niters,comm)
+ @show u
+end
+run(`$(mpiexec()) -np 1 julia --project=. -e $code`);
+
+
+
+
+
+
+
+
+
+Checking the result¶
Checking the result visually is not enough in general. To check the parallel implementation we want to compare the result against the sequential implementation. The way we do the computations (either in parallel or sequential) should not affect the result. However, how can we compare the sequential and the parallel result? The parallel version gives a distributed vector. We cannot compare this one directly with the result of the sequential function. A possible solution is to gather all the pieces of the parallel result in a single rank and compare there against the parallel implementation.
+The following function gather the distributed vector in rank 0.
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+code5 = quote
+ function gather_final_result(u,comm)
+ load = length(u)-2
+ rank = MPI.Comm_rank(comm)
+ if rank !=0
+ # If I am not rank 0
+ # I send my own data to rank 0
+ u_snd = view(u,2:(load+1))
+ MPI.Send(u_snd,comm,dest=0)
+ u_root = zeros(0) # This will nevel be used
+ else
+ # If I am rank 0
+ nranks = MPI.Comm_size(comm)
+ n = load*nranks
+ u_root = zeros(n+2)
+ # Set boundary
+ u_root[1] = -1
+ u_root[end] = 1
+ # Set data for rank 0
+ lb = 2
+ ub = load+1
+ u_root[lb:ub] = view(u,lb:ub)
+ # Receive and set data from other ranks
+ for other_rank in 1:(nranks-1)
+ lb += load
+ ub += load
+ u_rcv = view(u_root,lb:ub)
+ MPI.Recv!(u_rcv,comm;source=other_rank)
+ end
+ end
+ # NB onle the root (rank 0) will
+ # contain meaningfull data
+ return u_root
+ end
+end;
+
+
+
+
+
+
+
+
+
+Run the following cell to see the result. Is the result as expected?
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+code = quote
+ using MPI
+ MPI.Init()
+ $code1
+ $code2
+ $code3
+ $code4
+ $code5
+ n = 9
+ niters = 200
+ comm = MPI.Comm_dup(MPI.COMM_WORLD)
+ u = jacobi_mpi(n,niters,comm)
+ u_root = gather_final_result(u,comm)
+ @show u_root
+end
+run(`$(mpiexec()) -np 3 julia --project=. -e $code`);
+
+
+
+
+
+
+
+
+
+Now that we have collected the parallel vector into a single array, we can compare it against the sequential implementation. This is done in the following cells.
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+code6 = quote
+ function jacobi(n,niters)
+ u = zeros(n+2)
+ u[1] = -1
+ u[end] = 1
+ u_new = copy(u)
+ for t in 1:niters
+ for i in 2:(n+1)
+ u_new[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ u, u_new = u_new, u
+ end
+ u
+ end
+end;
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+code = quote
+ using MPI
+ MPI.Init()
+ $code1
+ $code2
+ $code3
+ $code4
+ $code5
+ $code6
+ n = 12
+ niters = 100
+ comm = MPI.Comm_dup(MPI.COMM_WORLD)
+ u = jacobi_mpi(n,niters,comm)
+ u_root = gather_final_result(u,comm)
+ rank = MPI.Comm_rank(comm)
+ if rank == 0
+ # Compare agains serial
+ u_seq = jacobi(n,niters)
+ if isapprox(u_root,u_seq)
+ println("Test passed 🥳")
+ else
+ println("Test failed")
+ end
+ end
+end
+run(`$(mpiexec()) -np 3 julia --project=. -e $code`);
+
+
+
+
+
+
+
+
+
+
+Note that we have used function isapprox to compare the results. This function checks if two values are the same within machine precision. Using == is generally discouraged when working with floating point numbers as they can be affected by rounding-off errors.
+
+
+
+
+
+
+
+
+
+Question: What happens if we use `u_root == u_seq` to compare the parallel and the sequential result?
+
+a) The test will still pass.
+b) The test will fail due to rounding-off errors.
+c) The test might pass or fail depending on `n`.
+d) The test might pass or fail depending on the number of MPI ranks.
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+answer = "x" # replace x with a, b, c or d
+jacobitest_check(answer)
+
+
+
+
+
+
+
+
+
+Run cell below for an explanation of the correct answer.
+
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+jacobitest_why()
+
+
+
+
+
+
+
+
+
+
+Latency hiding¶
Can our implementation above be improved? Note that we only need communications to update the values at the boundary of the portion owned by each process. The other values (the one in green in the figure below) can be updated without communications. This provides the opportunity of overlapping the computation of the interior values (green cells in the figure) with the communication of the ghost values. This technique is called latency hiding, since we are hiding communication latency by overlapping it with computation that we need to do anyway. The actual implementation is left as an exercise (see Exercise 1).
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Question: Which MPI directives would you use to implement latency hiding in the communication the ghost values?
+
+a) MPI_Send and MPI_Recv
+b) MPI_Bsend and MPI_Recv
+c) MPI_Isend and MPI_Irecv
+d) MPI_Sendrecv
+
+
+
+
+
+
+
+In [ ]:
+
+
+
+
+
+answer = "x" # replace x with a, b, c or d
+lh_check(answer)
+
+
+
+
+
+
+
+
+
+Conclusion¶
-
+
- We learned how to parallelize the Jacobi method efficiently. +
- Using block partitions the communication overhead is small if the problem size is larger than the number of processors. +
- In addition, one can use latency hiding to further reduce the overhead caused by communication. +
- Both are not true if we use cyclic data partitions. +
- Gauss-Seidel is a simple variation of Jacobi but is much more challenging to parallelize. +
- One needs to consider a special order (red-black) when updating the values to parallelize the method. +
- We learned how to implement a 1D Jacobi method with MPI. +
- Once needs to be careful when using blocking directives to avoid dead locks. +
@@ -8654,13 +9501,155 @@ d) 4
+
+
+
+
-
+Exercise 1¶
Transform the parallel implementation of the 1d Jacobi method (function jacopi_mpi) to use latency hiding (overlap between computation of interior values and communication).
Exercise 1¶
The following code implements the 1D Jacobi method studied in this notebook, but using non-blocking sends and receives. Modify this code so that you overlap the communication of the ghost values with local computations as explained above in the section "latency hiding". You only need to modify function jacobi_mpi_latency_hiding. Copy the code below to a file called ex1.jl. Modify the file (e.g. with vscode). Run it from the Julia REPL using the run function as explained in the Getting Started tutorial.
+
+
+
+
+
+
+
+
+
+
+# file ex1.jl (begin)
+
+using MPI
+MPI.Init()
+
+function jacobi_mpi_latency_hiding(n,niters,comm)
+ u, u_new = init(n,comm)
+ load = length(u)-2
+ rank = MPI.Comm_rank(comm)
+ nranks = MPI.Comm_size(comm)
+ nreqs = 2*((rank != 0) + (rank != (nranks-1)))
+ reqs = MPI.MultiRequest(nreqs)
+ for t in 1:niters
+ ireq = 0
+ if rank != 0
+ neig_rank = rank-1
+ u_snd = view(u,2:2)
+ u_rcv = view(u,1:1)
+ dest = neig_rank
+ source = neig_rank
+ ireq += 1
+ MPI.Isend(u_snd,comm,reqs[ireq];dest)
+ ireq += 1
+ MPI.Irecv!(u_rcv,comm,reqs[ireq];source)
+ end
+ if rank != (nranks-1)
+ neig_rank = rank+1
+ u_snd = view(u,(load+1):(load+1))
+ u_rcv = view(u,(load+2):(load+2))
+ dest = neig_rank
+ source = neig_rank
+ ireq += 1
+ MPI.Isend(u_snd,comm,reqs[ireq];dest)
+ ireq += 1
+ MPI.Irecv!(u_rcv,comm,reqs[ireq];source)
+ end
+ MPI.Waitall(reqs)
+ for i in 2:load+1
+ u_new[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ u, u_new = u_new, u
+ end
+ return u
+end
+
+function init(n,comm)
+ nranks = MPI.Comm_size(comm)
+ rank = MPI.Comm_rank(comm)
+ if mod(n,nranks) != 0
+ println("n must be a multiple of nranks")
+ MPI.Abort(comm,1)
+ end
+ load = div(n,nranks)
+ u = zeros(load+2)
+ if rank == 0
+ u[1] = -1
+ end
+ if rank == nranks-1
+ u[end] = 1
+ end
+ u_new = copy(u)
+ u, u_new
+end
+
+function gather_final_result(u,comm)
+ load = length(u)-2
+ rank = MPI.Comm_rank(comm)
+ if rank !=0
+ u_snd = view(u,2:(load+1))
+ MPI.Send(u_snd,comm,dest=0)
+ u_root = zeros(0)
+ else
+ nranks = MPI.Comm_size(comm)
+ n = load*nranks
+ u_root = zeros(n+2)
+ u_root[1] = -1
+ u_root[end] = 1
+ lb = 2
+ ub = load+1
+ u_root[lb:ub] = view(u,lb:ub)
+ for other_rank in 1:(nranks-1)
+ lb += load
+ ub += load
+ u_rcv = view(u_root,lb:ub)
+ MPI.Recv!(u_rcv,comm;source=other_rank)
+ end
+ end
+ return u_root
+end
+
+function jacobi(n,niters)
+ u = zeros(n+2)
+ u[1] = -1
+ u[end] = 1
+ u_new = copy(u)
+ for t in 1:niters
+ for i in 2:(n+1)
+ u_new[i] = 0.5*(u[i-1]+u[i+1])
+ end
+ u, u_new = u_new, u
+ end
+ u
+end
+
+n = 12
+niters = 100
+comm = MPI.Comm_dup(MPI.COMM_WORLD)
+u = jacobi_mpi_latency_hiding(n,niters,comm)
+u_root = gather_final_result(u,comm)
+rank = MPI.Comm_rank(comm)
+if rank == 0
+ u_seq = jacobi(n,niters)
+ if isapprox(u_root,u_seq)
+ println("Test passed 🥳")
+ else
+ println("Test failed 😢")
+ end
+end
+
+# file ex1.jl (end)
+
+Asynchronous programming in Julia · XM_40017
+
diff --git a/dev/julia_async/index.html b/dev/julia_async/index.html
index fc3a355..6aaff10 100644
--- a/dev/julia_async/index.html
+++ b/dev/julia_async/index.html
@@ -1,5 +1,5 @@
-
+
+
+
+
Exercise 2¶
In the parallel implementation of the Jacobi method, we assumed that the method runs for a given number of iterations. In function, jacobi_with_tol at the beginning of the notebook shows how the Jacobi iterations can be stopped when the difference between iterations is small. Implement a parallel version of this function. Start with the in Exercise 1 and add the stopping criterion implemented in jacobi_with_tol. Use a text editor and the Julia REPL. Do not try to implement the code in a notebook.
+Asynchronous programming in Julia · XM_40017
Tip
Julia Basics · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+Julia Basics · XM_40017
Tip
Distributed computing in Julia · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+Distributed computing in Julia · XM_40017
Tip
- · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+- · XM_40017
Tip
- · XM_40017
-
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+- · XM_40017
Tip
MPI (point-to-point) · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+MPI (point-to-point) · XM_40017
Tip
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
Exercise 1¶
Implement the parallel matrix-matrix multiplication (Algorithm 3) in previous notebook using MPI instead of Distributed. Use this function signature:
function matmul_mpi_3!(C,A,B)
Assume that the input matrices A and B are given only on rank 0, the other ranks get dummy matrices with zero rows and zero columns to save memory. You need to communicate the required parts to other ranks. For simplicity you can assume that A and B are square matrices and that the number of rows is a multiple of the number of processes (on rank 0). The result C should be overwritten only on rank 0. You can use the following cell to implement and check your result.
Assume that the input matrices A and B are given only on rank 0, the other ranks get dummy matrices with zero rows and zero columns to save memory. You need to communicate the required parts to other ranks. For simplicity you can assume that A and B are square matrices and that the number of rows is a multiple of the number of processes (on rank 0). The result C should be overwritten only on rank 0. You can use the following cell to implement and check your result. Copy the code below to a file called ex1.jl. Modify the file (e.g. with vscode). Run it from the Julia REPL using the run function as explained in the Getting Started tutorial.
+
+
-
- · XM_40017
In [ ]:
-
-
-
-code = quote
- using MPI
- MPI.Init()
- function matmul_mpi_3!(C,A,B)
- # Implement here
- end
- function testit(load)
- comm = MPI.COMM_WORLD
- rank = MPI.Comm_rank(comm)
- if rank == 0
- P = MPI.Comm_size(comm)
- N = load*P
- else
- N = 0
- end
- A = rand(N,N)
- B = rand(N,N)
- C = similar(A)
- matmul_mpi_3!(C,A,B)
- if rank == 0
- if !(C ≈ A*B)
- println("Check not passed")
- else
- println("Check passed!")
- end
- end
- end
- testit(100)
+
+
+# ex1.jl (begin)
+using MPI
+MPI.Init()
+function matmul_mpi_3!(C,A,B)
+ # Implement here
end
-run(`$(mpiexec()) -np 4 julia --project=. -e $code`);
+function testit(load)
+ comm = MPI.COMM_WORLD
+ rank = MPI.Comm_rank(comm)
+ if rank == 0
+ P = MPI.Comm_size(comm)
+ N = load*P
+ else
+ N = 0
+ end
+ A = rand(N,N)
+ B = rand(N,N)
+ C = similar(A)
+ matmul_mpi_3!(C,A,B)
+ if rank == 0
+ if !(C ≈ A*B)
+ println("Test failed 😢")
+ else
+ println("Test passed 🥳")
+ end
+ end
+end
+testit(100)
+end
+# ex1.jl (end)
@@ -9226,7 +9224,7 @@ a.anchor-link {
diff --git a/dev/julia_tutorial/index.html b/dev/julia_tutorial/index.html
index e9c0714..cdc8225 100644
--- a/dev/julia_tutorial/index.html
+++ b/dev/julia_tutorial/index.html
@@ -1,5 +1,5 @@
-Exercise 2¶
Implement this "simple" algorithm (the telephone game):
-Rank 0 generates a message (an integer). Rank 0 sends the message to rank 1. Rank 1 receives the message, increments the message by 1, and sends the result to rank 2. Rank 2 receives the message, increments the message by 1, and sends the result to rank 3. Etc. The last rank sends back the message to rank 0 closing the ring. See the next figure. Implement the communications using MPI. Do not use Distributed.
Rank 0 generates a message (an integer). Rank 0 sends the message to rank 1. Rank 1 receives the message, increments the message by 1, and sends the result to rank 2. Rank 2 receives the message, increments the message by 1, and sends the result to rank 3. Etc. The last rank sends back the message to rank 0 closing the ring. See the next figure. Implement the communications using MPI. Do not use Distributed. Use a text editor and the Julia REPL. Do not try to implement the code in a notebook.
+- · XM_40017
Tip
Matrix-matrix multiplication · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+Matrix-matrix multiplication · XM_40017
Tip
MPI (collectives) · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+MPI (collectives) · XM_40017
Tip
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
-
-
-
-
-
-
-
-
-
-Exercises¶
-
-
-
-
-
-
-
-
-
-Exercise 1¶
In the parallel implementation of the Jacobi method in previous notebook, we assumed that the method runs for a given number of iterations. However, other stopping criteria are used in practice. The following sequential code implements a version of Jacobi in which the method iterates until the norm of the difference between u and u_new is below a tolerance.
-
-
-
-
-
-
-
-In [ ]:
-
-
-
-
-
-function jacobi_with_tol(n,tol)
- u = zeros(n+2)
- u[1] = -1
- u[end] = 1
- u_new = copy(u)
- increment = similar(u)
- while true
- for i in 2:(n+1)
- u_new[i] = 0.5*(u[i-1]+u[i+1])
- end
- increment .= u_new .- u
- norm_increment = 0.0
- for i in 1:n
- increment_i = increment[i]
- norm_increment += increment_i*increment_i
- end
- norm_increment = sqrt(norm_increment)
- if norm_increment < tol*n
- return u_new
- end
- u, u_new = u_new, u
- end
- u
-end
-
-
-
-
-
-
-
-
-In [ ]:
-
-
-
-
-
-n = 10
-tol = 1e-12
-jacobi_with_tol(n,tol)
-
-
-
-
-
-
-
-
-
-Implement a parallel version of this algorithm. Recommended: start with the parallel implementation given in the previous notebook (see function jacobi_mpi) and introduce the new stopping criteria. Think carefully about which MPI operations you need to use in this case.
diff --git a/dev/mpi_tutorial/index.html b/dev/mpi_tutorial/index.html
index 4202a84..9c40ecd 100644
--- a/dev/mpi_tutorial/index.html
+++ b/dev/mpi_tutorial/index.html
@@ -1,5 +1,5 @@
-- · XM_40017
+- · XM_40017
Tip
- · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+- · XM_40017
Tip
- · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+- · XM_40017
Tip
- · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+- · XM_40017
Tip
Solutions · XM_40017
- · XM_40017
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
Solutions
Julia Basics
Exercise 1
function ex1(a)
+Solutions · XM_40017 Solutions
Julia Basics
Exercise 1
function ex1(a)
j = 1
m = a[j]
for (i,ai) in enumerate(a)
@@ -172,4 +172,4 @@ end
« MPI (collectives)
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.
+- · XM_40017
Tip
-
@@ -14,4 +14,4 @@
var myIframe = document.getElementById("notebook");
iFrameResize({log:true}, myIframe);
});
-
Settings
This document was generated with Documenter.jl version 1.6.0 on Monday 2 September 2024. Using Julia version 1.10.5.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 9 September 2024. Using Julia version 1.10.5.