diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json
index c0b9997..749f819 100644
--- a/dev/.documenter-siteinfo.json
+++ b/dev/.documenter-siteinfo.json
@@ -1 +1 @@
-{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-23T05:44:47","documenter_version":"1.7.0"}}
\ No newline at end of file
+{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-23T11:58:25","documenter_version":"1.7.0"}}
\ No newline at end of file
diff --git a/dev/LEQ/index.html b/dev/LEQ/index.html
index 27c5e9c..4bae58d 100644
--- a/dev/LEQ/index.html
+++ b/dev/LEQ/index.html
@@ -1,5 +1,5 @@
-
The programming of this course will be done using the Julia programming language. Thus, we start by explaining how to get up and running with Julia. After studying this page, you will be able to:
Courses related with high-performance computing (HPC) often use languages such as C, C++, or Fortran. We use Julia instead to make the course accessible to a wider set of students, including the ones that have no experience with C/C++ or Fortran, but are willing to learn parallel programming. Julia is a relatively new programming language specifically designed for scientific computing. It combines a high-level syntax close to interpreted languages like Python with the performance of compiled languages like C, C++, or Fortran. Thus, Julia will allow us to write efficient parallel algorithms with a syntax that is convenient in a teaching setting. In addition, Julia provides easy access to different programming models to write distributed algorithms, which will be useful to learn and experiment with them.
Tip
You can run the code in this link to learn how Julia compares to other languages (C and Python) in terms of performance.
There are several ways of opening Julia depending on your operating system and your IDE, but it is usually as simple as launching the Julia app. With VSCode, open a folder (File > Open Folder). Then, press Ctrl+Shift+P to open the command bar, and execute Julia: Start REPL. If this does not work, make sure you have the Julia extension for VSCode installed. Independently of the method you use, opening Julia results in a window with some text ending with:
julia>
You have just opened the Julia read-evaluate-print loop, or simply the Julia REPL. Congrats! You will spend most of time using the REPL, when working in Julia. The REPL is a console waiting for user input. Just as in other consoles, the string of text right before the input area (julia> in the case) is called the command prompt or simply the prompt.
Curious about what the function println does? Enter into help mode to look into the documentation. This is done by typing a question mark (?) into the input field:
julia> ?
After typing ?, the command prompt changes to help?>. It means we are in help mode. Now, we can type a function name to see its documentation.
The REPL comes with two more modes, namely package and shell modes. To enter package mode type
julia> ]
Package mode is used to install and manage packages. We are going to discuss the package mode in greater detail later. To return back to normal mode press the backspace key several times.
To enter shell mode type semicolon (;)
julia> ;
The prompt should have changed to shell> indicating that we are in shell mode. Now you can type commands that you would normally do on your system command line. For instance,
shell> ls
will display the contents of the current folder in Mac or Linux. Using shell mode in Windows is not straightforward, and thus not recommended for beginners.
Real-world Julia programs are not typed in the REPL in practice. They are written in one or more files and included in the REPL. To try this, create a new file called hello.jl, write the code of the "Hello world" example above, and save it. If you are using VSCode, you can create the file using File > New File > Julia File. Once the file is saved with the name hello.jl, execute it as follows
julia> include("hello.jl")
Warning
Make sure that the file "hello.jl" is located in the current working directory of your Julia session. You can query the current directory with function pwd(). You can change to another directory with function cd() if needed. Also, make sure that the file extension is .jl.
The recommended way of running Julia code is using the REPL as we did. But it is also possible to run code directly from the system command line. To this end, open a terminal and call Julia followed by the path to the file containing the code you want to execute.
$ julia hello.jl
The previous line assumes that you have Julia properly installed in the system and that it's usable from the terminal. In UNIX systems (Linux and Mac), the Julia binary needs to be in one of the directories listed in the PATH environment variable. To check that Julia is properly installed, you can use
$ julia --version
If this runs without error and you see a version number, you are good to go!
You can also run julia code from the terminal using the -e flag:
$ julia -e 'println("Hello, world!")'
Note
In this tutorial, when a code snipped starts with $, it should be run in the terminal. Otherwise, the code is to be run in the Julia REPL.
Tip
Avoid calling Julia code from the terminal, use the Julia REPL instead! Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch. This can be time consuming for large projects. In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster. Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code from the terminal. Visit this link (Julia workflow tips) from the official Julia documentation for further information about how to develop Julia code effectivelly.
The programming of this course will be done using the Julia programming language. Thus, we start by explaining how to get up and running with Julia. After studying this page, you will be able to:
Courses related with high-performance computing (HPC) often use languages such as C, C++, or Fortran. We use Julia instead to make the course accessible to a wider set of students, including the ones that have no experience with C/C++ or Fortran, but are willing to learn parallel programming. Julia is a relatively new programming language specifically designed for scientific computing. It combines a high-level syntax close to interpreted languages like Python with the performance of compiled languages like C, C++, or Fortran. Thus, Julia will allow us to write efficient parallel algorithms with a syntax that is convenient in a teaching setting. In addition, Julia provides easy access to different programming models to write distributed algorithms, which will be useful to learn and experiment with them.
Tip
You can run the code in this link to learn how Julia compares to other languages (C and Python) in terms of performance.
There are several ways of opening Julia depending on your operating system and your IDE, but it is usually as simple as launching the Julia app. With VSCode, open a folder (File > Open Folder). Then, press Ctrl+Shift+P to open the command bar, and execute Julia: Start REPL. If this does not work, make sure you have the Julia extension for VSCode installed. Independently of the method you use, opening Julia results in a window with some text ending with:
julia>
You have just opened the Julia read-evaluate-print loop, or simply the Julia REPL. Congrats! You will spend most of time using the REPL, when working in Julia. The REPL is a console waiting for user input. Just as in other consoles, the string of text right before the input area (julia> in the case) is called the command prompt or simply the prompt.
Curious about what the function println does? Enter into help mode to look into the documentation. This is done by typing a question mark (?) into the input field:
julia> ?
After typing ?, the command prompt changes to help?>. It means we are in help mode. Now, we can type a function name to see its documentation.
The REPL comes with two more modes, namely package and shell modes. To enter package mode type
julia> ]
Package mode is used to install and manage packages. We are going to discuss the package mode in greater detail later. To return back to normal mode press the backspace key several times.
To enter shell mode type semicolon (;)
julia> ;
The prompt should have changed to shell> indicating that we are in shell mode. Now you can type commands that you would normally do on your system command line. For instance,
shell> ls
will display the contents of the current folder in Mac or Linux. Using shell mode in Windows is not straightforward, and thus not recommended for beginners.
Real-world Julia programs are not typed in the REPL in practice. They are written in one or more files and included in the REPL. To try this, create a new file called hello.jl, write the code of the "Hello world" example above, and save it. If you are using VSCode, you can create the file using File > New File > Julia File. Once the file is saved with the name hello.jl, execute it as follows
julia> include("hello.jl")
Warning
Make sure that the file "hello.jl" is located in the current working directory of your Julia session. You can query the current directory with function pwd(). You can change to another directory with function cd() if needed. Also, make sure that the file extension is .jl.
The recommended way of running Julia code is using the REPL as we did. But it is also possible to run code directly from the system command line. To this end, open a terminal and call Julia followed by the path to the file containing the code you want to execute.
$ julia hello.jl
The previous line assumes that you have Julia properly installed in the system and that it's usable from the terminal. In UNIX systems (Linux and Mac), the Julia binary needs to be in one of the directories listed in the PATH environment variable. To check that Julia is properly installed, you can use
$ julia --version
If this runs without error and you see a version number, you are good to go!
You can also run julia code from the terminal using the -e flag:
$ julia -e 'println("Hello, world!")'
Note
In this tutorial, when a code snipped starts with $, it should be run in the terminal. Otherwise, the code is to be run in the Julia REPL.
Tip
Avoid calling Julia code from the terminal, use the Julia REPL instead! Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch. This can be time consuming for large projects. In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster. Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code from the terminal. Visit this link (Julia workflow tips) from the official Julia documentation for further information about how to develop Julia code effectivelly.
Since we are in a parallel computing course, let's run a parallel "Hello world" example in Julia. Open a Julia REPL and write
julia> using Distributed
julia> @everywhere println("Hello, world! I am proc $(myid()) from $(nprocs())")
Here, we are using the Distributed package, which is part of the Julia standard library that provides distributed memory parallel support. The code prints the process id and the number of processes in the current Julia session.
You will probably only see output from 1 process. We need to add more processes to run the example in parallel. This is done with the addprocs function.
julia> addprocs(3)
We have added 3 new processes. Plus the old one, we have 4 processes. Run the code again.
julia> @everywhere println("Hello, world! I am proc $(myid()) from $(nprocs())")
Now, you should see output from 4 processes.
It is possible to specify the number of processes when starting Julia from the terminal with the -p argument (useful, e.g., when running in a cluster). If you launch Julia from the terminal as
$ julia -p 3
and then run
julia> @everywhere println("Hello, world! I am proc $(myid()) from $(nprocs())")
One of the most useful features of Julia is its package manager. It allows one to install Julia packages in a straightforward and platform independent way. To illustrate this, let us consider the following parallel "Hello world" example. This example uses the Message Passing Interface (MPI). We will learn more about MPI later in the course.
Copy the following block of code into a new file named "hello_mpi.jl"
Copy the contents of previous code block into a file called Project.toml and place it in an empty folder named newproject. It is important that the file is named Project.toml. You can create a new folder from the REPL with
julia> mkdir("newproject")
To install all the packages registered in this file you need to activate the folder containing your Project.toml file
(@v1.10) pkg> activate newproject
and then instantiating it
(newproject) pkg> instantiate
The instantiate command will download and install all listed packages and their dependencies in just one click.
In some situations it is required to use package commands in Julia code, e.g., to automatize installation and deployment of Julia applications. This can be done using the Pkg package. For instance
In many situations, it is useful to create your own package, for instance, when working with a large code base, when you want to reduce compilation latency using Revise.jl, or if you want to eventually register your package and share it with others.
The simplest way of generating a package (called MyPackage) is as follows. Open Julia, go to package mode, and type
(@v1.10) pkg> generate MyPackage
This will crate a minimal package consisting of a new folder MyPackage with two files:
MyPackage/Project.toml: Project file defining the direct dependencies of your package.
MyPackage/src/MyPackage.jl: Main source file of your package. You can split your code in several files if needed, and include them in the package main file using function include.
Tip
This approach only generates a very minimal package. To create a more sophisticated package skeleton (including unit testing, code coverage, readme file, licence, etc.) use PkgTemplates.jl or BestieTemplate.jl. The later one is developed in Amsterdam at the Netherlands eScience Center.
You can add dependencies to the package by activating the MyPackage folder in package mode and adding new dependencies as always:
To use your package you first need to add it to a package environment of your choice. This is done by changing to package mode and typing develop followed by the path to the folder containing the package. For instance:
(@v1.10) pkg> develop MyPackage
Note
You do not need to "develop" your package if you activated the package folder MyPackage.
Now, we can go back to standard Julia mode and use it as any other package:
using MyPackage
-MyPackage.greet()
Here, we just called the example function defined in MyPackage/src/MyPackage.jl.
We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:
We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:
This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam. We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems. Further information about the course is found in the study guide (click here) and our Canvas page (for registered students).
Note
Material will be added incrementally to the website as the course advances.
Warning
This page will eventually contain only a part of the course material. The rest will be available on Canvas. In particular, the material in this public webpage does not fully cover all topics in the final exam.
Download the notebooks and run them locally on your computer (recommended). At each notebook page you will find a green box with links to download the notebook.
You also have the static version of the notebooks displayed in this webpage for quick reference.
This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam. We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems. Further information about the course is found in the study guide (click here) and our Canvas page (for registered students).
Note
Material will be added incrementally to the website as the course advances.
Warning
This page will eventually contain only a part of the course material. The rest will be available on Canvas. In particular, the material in this public webpage does not fully cover all topics in the final exam.
Download the notebooks and run them locally on your computer (recommended). At each notebook page you will find a green box with links to download the notebook.
You also have the static version of the notebooks displayed in this webpage for quick reference.
This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project "Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course" funded by the "Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning".
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 23 September 2024. Using Julia version 1.10.5.
+julia> notebook()
These commands will open a jupyter in your web browser. Navigate in jupyter to the notebook file you have downloaded and open it.
This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project "Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course" funded by the "Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning".
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 23 September 2024. Using Julia version 1.10.5.
This document was generated with Documenter.jl version 1.7.0 on Monday 23 September 2024. Using Julia version 1.10.5.
+
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 23 September 2024. Using Julia version 1.10.5.
diff --git a/dev/search_index.js b/dev/search_index.js
index 93fc0b2..149ec2e 100644
--- a/dev/search_index.js
+++ b/dev/search_index.js
@@ -1,3 +1,3 @@
var documenterSearchIndex = {"docs":
-[{"location":"getting_started_with_julia/#Getting-started","page":"Getting started","title":"Getting started","text":"","category":"section"},{"location":"getting_started_with_julia/#Introduction","page":"Getting started","title":"Introduction","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The programming of this course will be done using the Julia programming language. Thus, we start by explaining how to get up and running with Julia. After studying this page, you will be able to:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Use the Julia REPL,\nRun serial and parallel code,\nInstall and manage Julia packages.","category":"page"},{"location":"getting_started_with_julia/#Why-Julia?","page":"Getting started","title":"Why Julia?","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Courses related with high-performance computing (HPC) often use languages such as C, C++, or Fortran. We use Julia instead to make the course accessible to a wider set of students, including the ones that have no experience with C/C++ or Fortran, but are willing to learn parallel programming. Julia is a relatively new programming language specifically designed for scientific computing. It combines a high-level syntax close to interpreted languages like Python with the performance of compiled languages like C, C++, or Fortran. Thus, Julia will allow us to write efficient parallel algorithms with a syntax that is convenient in a teaching setting. In addition, Julia provides easy access to different programming models to write distributed algorithms, which will be useful to learn and experiment with them.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"tip: Tip\nYou can run the code in this link to learn how Julia compares to other languages (C and Python) in terms of performance.","category":"page"},{"location":"getting_started_with_julia/#Installing-Julia","page":"Getting started","title":"Installing Julia","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"This is a tutorial-like page. Follow these steps before you continue reading the document.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Download and install Julia from julialang.org;\nFollow the specific instructions for your operating system: Windows, MacOS, or Linux\nDownload and install VSCode and its Julia extension;","category":"page"},{"location":"getting_started_with_julia/#The-Julia-REPL","page":"Getting started","title":"The Julia REPL","text":"","category":"section"},{"location":"getting_started_with_julia/#Starting-Julia","page":"Getting started","title":"Starting Julia","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"There are several ways of opening Julia depending on your operating system and your IDE, but it is usually as simple as launching the Julia app. With VSCode, open a folder (File > Open Folder). Then, press Ctrl+Shift+P to open the command bar, and execute Julia: Start REPL. If this does not work, make sure you have the Julia extension for VSCode installed. Independently of the method you use, opening Julia results in a window with some text ending with:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia>","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You have just opened the Julia read-evaluate-print loop, or simply the Julia REPL. Congrats! You will spend most of time using the REPL, when working in Julia. The REPL is a console waiting for user input. Just as in other consoles, the string of text right before the input area (julia> in the case) is called the command prompt or simply the prompt.","category":"page"},{"location":"getting_started_with_julia/#Basic-usage","page":"Getting started","title":"Basic usage","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The usage of the REPL is as follows:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You write some input\npress enter\nyou get the output","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"For instance, try this","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> 1 + 1","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"A \"Hello world\" example looks like this in Julia","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> println(\"Hello, world!\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Try to run it in the REPL.","category":"page"},{"location":"getting_started_with_julia/#Help-mode","page":"Getting started","title":"Help mode","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Curious about what the function println does? Enter into help mode to look into the documentation. This is done by typing a question mark (?) into the input field:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> ?","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"After typing ?, the command prompt changes to help?>. It means we are in help mode. Now, we can type a function name to see its documentation.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"help?> println","category":"page"},{"location":"getting_started_with_julia/#Package-and-shell-modes","page":"Getting started","title":"Package and shell modes","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The REPL comes with two more modes, namely package and shell modes. To enter package mode type","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> ]","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Package mode is used to install and manage packages. We are going to discuss the package mode in greater detail later. To return back to normal mode press the backspace key several times.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To enter shell mode type semicolon (;)","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> ;","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The prompt should have changed to shell> indicating that we are in shell mode. Now you can type commands that you would normally do on your system command line. For instance,","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"shell> ls","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"will display the contents of the current folder in Mac or Linux. Using shell mode in Windows is not straightforward, and thus not recommended for beginners.","category":"page"},{"location":"getting_started_with_julia/#Running-Julia-code","page":"Getting started","title":"Running Julia code","text":"","category":"section"},{"location":"getting_started_with_julia/#Running-more-complex-code","page":"Getting started","title":"Running more complex code","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Real-world Julia programs are not typed in the REPL in practice. They are written in one or more files and included in the REPL. To try this, create a new file called hello.jl, write the code of the \"Hello world\" example above, and save it. If you are using VSCode, you can create the file using File > New File > Julia File. Once the file is saved with the name hello.jl, execute it as follows","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> include(\"hello.jl\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"warning: Warning\nMake sure that the file \"hello.jl\" is located in the current working directory of your Julia session. You can query the current directory with function pwd(). You can change to another directory with function cd() if needed. Also, make sure that the file extension is .jl.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The recommended way of running Julia code is using the REPL as we did. But it is also possible to run code directly from the system command line. To this end, open a terminal and call Julia followed by the path to the file containing the code you want to execute.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ julia hello.jl","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The previous line assumes that you have Julia properly installed in the system and that it's usable from the terminal. In UNIX systems (Linux and Mac), the Julia binary needs to be in one of the directories listed in the PATH environment variable. To check that Julia is properly installed, you can use","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ julia --version","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"If this runs without error and you see a version number, you are good to go!","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can also run julia code from the terminal using the -e flag:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ julia -e 'println(\"Hello, world!\")'","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nIn this tutorial, when a code snipped starts with $, it should be run in the terminal. Otherwise, the code is to be run in the Julia REPL.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"tip: Tip\nAvoid calling Julia code from the terminal, use the Julia REPL instead! Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch. This can be time consuming for large projects. In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster. Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code from the terminal. Visit this link (Julia workflow tips) from the official Julia documentation for further information about how to develop Julia code effectivelly.","category":"page"},{"location":"getting_started_with_julia/#Running-parallel-code","page":"Getting started","title":"Running parallel code","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Since we are in a parallel computing course, let's run a parallel \"Hello world\" example in Julia. Open a Julia REPL and write","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using Distributed\njulia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Here, we are using the Distributed package, which is part of the Julia standard library that provides distributed memory parallel support. The code prints the process id and the number of processes in the current Julia session.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You will probably only see output from 1 process. We need to add more processes to run the example in parallel. This is done with the addprocs function.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> addprocs(3)","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"We have added 3 new processes. Plus the old one, we have 4 processes. Run the code again.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, you should see output from 4 processes.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"It is possible to specify the number of processes when starting Julia from the terminal with the -p argument (useful, e.g., when running in a cluster). If you launch Julia from the terminal as","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ julia -p 3","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"and then run","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You should get output from 4 processes as before.","category":"page"},{"location":"getting_started_with_julia/#Installing-packages","page":"Getting started","title":"Installing packages","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"One of the most useful features of Julia is its package manager. It allows one to install Julia packages in a straightforward and platform independent way. To illustrate this, let us consider the following parallel \"Hello world\" example. This example uses the Message Passing Interface (MPI). We will learn more about MPI later in the course.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Copy the following block of code into a new file named \"hello_mpi.jl\"","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"# file hello_mpi.jl\nusing MPI\nMPI.Init()\ncomm = MPI.COMM_WORLD\nrank = MPI.Comm_rank(comm)\nnranks = MPI.Comm_size(comm)\nprintln(\"Hello world, I am rank $rank of $nranks\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"As you can see from this example, one can access MPI from Julia in a clean way, without type annotations and other complexities of C/C++ code.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, run the file from the REPL","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> include(\"hello_mpi.jl\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"It probably didn't work, right? Read the error message and note that the MPI package needs to be installed to run this code.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To install a package, we need to enter package mode. Remember that we entered into help mode by typing ?. Package mode is activated by typing ] : ","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> ]","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"At this point, the prompt should have changed to (@v1.10) pkg> indicating that we are in package mode. The text between the parentheses indicates which is the active project, i.e., where packages are going to be installed. In this case, we are working with the global project associated with our Julia installation (which is Julia 1.10 in this example, but it can be another version in your case).","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To install the MPI package, type","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> add MPI","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Congrats, you have installed MPI!","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nMany Julia package names end with .jl. This is just a way of signaling that a package is written in Julia. When using such packages, the .jl needs to be omitted. In this case, we have installed the MPI.jl package even though we have only typed MPI in the REPL.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nThe package you have installed is the Julia interface to MPI, called MPI.jl. Note that it is not a MPI library by itself. It is just a thin wrapper between MPI and Julia. To use this interface, you need an actual MPI library installed in your system such as OpenMPI or MPICH. Julia downloads and installs a MPI library for you, but it is also possible to use a MPI library already available in your system. This is useful, e.g., when running on HPC clusters. See the documentation of MPI.jl for further details.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To check that the package was installed properly, exit package mode by pressing the backspace key several times, and run it again","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> include(\"hello_mpi.jl\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, it should work, but you probably get output from a single MPI rank only.","category":"page"},{"location":"getting_started_with_julia/#Running-MPI-code","page":"Getting started","title":"Running MPI code","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To run MPI applications in parallel, you need a launcher like mpiexec. MPI codes written in Julia are not an exception to this rule. From the system terminal, you can run","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ mpiexec -np 4 julia hello_mpi.jl","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"But it will probably not work since the version of mpiexec needs to match with the MPI version we are using from Julia. Don't worry if you could not make it work! A more elegant way to run MPI code is from the Julia REPL directly, by using these commands:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using MPI\njulia> run(`$(mpiexec()) -np 4 julia hello_mpi.jl`)","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, you should see output from 4 ranks.","category":"page"},{"location":"getting_started_with_julia/#Package-manager","page":"Getting started","title":"Package manager","text":"","category":"section"},{"location":"getting_started_with_julia/#Installing-packages-locally","page":"Getting started","title":"Installing packages locally","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"We have installed the MPI package globally and it will be available in all Julia sessions. However, in some situations, we want to work with different versions of the same package or to install packages in an isolated way to avoid potential conflicts with other packages. This can be done by using local projects.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"A project is simply a folder in your file system. To use a particular folder as your project, you need to activate it. This is done by entering package mode and using the activate command followed by the path to the folder you want to activate.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> activate .","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The previous command will activate the current working directory. Note that the dot . is indeed the path to the current folder.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The prompt has changed to (lessons) pkg> indicating that we are in the project within the lessons folder. The particular folder name can be different in your case.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"tip: Tip\nYou can activate a project directly when opening Julia from the terminal using the --project flag. The command $ julia --project=. will open Julia and activate a project in the current directory. You can also achieve the same effect by setting the environment variable JULIA_PROJECT with the path of the folder you want to activate.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nThe active project folder and the current working directory are two independent concepts! For instance, (@v1.10) pkg> activate folderB and then julia> cd(\"folderA\"), will activate the project in folderB and change the current working directory to folderA.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"At this point all package-related operations will be local to the new project. For instance, install the DataFrames package.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(lessons) pkg> add DataFrames","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Use the package to check that it is installed","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using DataFrames\njulia> DataFrame(a=[1,2],b=[3,4])","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, we can return to the global project to check that DataFrames has not been installed there. To return to the global environment, use activate without a folder name.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(lessons) pkg> activate","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The prompt is again (@v1.10) pkg>","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, try to use DataFrames.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using DataFrames\njulia> DataFrame(a=[1,2],b=[3,4])","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You should get an error or a warning unless you already had DataFrames installed globally.","category":"page"},{"location":"getting_started_with_julia/#Project-and-Manifest-files","page":"Getting started","title":"Project and Manifest files","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The information about a project is stored in two files Project.toml and Manifest.toml.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Project.toml contains the packages explicitly installed (the direct dependencies)\nManifest.toml contains direct and indirect dependencies along with the concrete version of each package.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"In other words, Project.toml contains the packages relevant for the user, whereas Manifest.toml is the detailed snapshot of all dependencies. The Manifest.toml can be used to reproduce the same environment in another machine.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can see the path to the current Project.toml file by using the status operator (or st in its short form) while in package mode","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> status","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The information about the Manifest.toml can be inspected by passing the -m flag.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> status -m","category":"page"},{"location":"getting_started_with_julia/#Installing-packages-from-a-project-file","page":"Getting started","title":"Installing packages from a project file","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Project files can be used to install lists of packages defined by others. E.g., to install all the dependencies of a Julia application.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Assume that a colleague has sent to you a Project.toml file with this content:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"[deps]\nBenchmarkTools = \"6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf\"\nDataFrames = \"a93c6f00-e57d-5684-b7b6-d8193f3e46c0\"\nMPI = \"da04e1cc-30fd-572f-bb4f-1f8673147195\"","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Copy the contents of previous code block into a file called Project.toml and place it in an empty folder named newproject. It is important that the file is named Project.toml. You can create a new folder from the REPL with","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> mkdir(\"newproject\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To install all the packages registered in this file you need to activate the folder containing your Project.toml file","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> activate newproject","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"and then instantiating it","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(newproject) pkg> instantiate","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The instantiate command will download and install all listed packages and their dependencies in just one click.","category":"page"},{"location":"getting_started_with_julia/#Getting-help-in-package-mode","page":"Getting started","title":"Getting help in package mode","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can get help about a particular package operator by writing help in front of it","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> help activate","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can get an overview of all package commands by typing help alone","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> help","category":"page"},{"location":"getting_started_with_julia/#Package-operations-in-Julia-code","page":"Getting started","title":"Package operations in Julia code","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"In some situations it is required to use package commands in Julia code, e.g., to automatize installation and deployment of Julia applications. This can be done using the Pkg package. For instance","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using Pkg\njulia> Pkg.status()","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"is equivalent to calling status in package mode.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> status","category":"page"},{"location":"getting_started_with_julia/#Creating-you-own-package","page":"Getting started","title":"Creating you own package","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"In many situations, it is useful to create your own package, for instance, when working with a large code base, when you want to reduce compilation latency using Revise.jl, or if you want to eventually register your package and share it with others.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The simplest way of generating a package (called MyPackage) is as follows. Open Julia, go to package mode, and type","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> generate MyPackage","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"This will crate a minimal package consisting of a new folder MyPackage with two files:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"MyPackage/Project.toml: Project file defining the direct dependencies of your package.\nMyPackage/src/MyPackage.jl: Main source file of your package. You can split your code in several files if needed, and include them in the package main file using function include.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"tip: Tip\nThis approach only generates a very minimal package. To create a more sophisticated package skeleton (including unit testing, code coverage, readme file, licence, etc.) use PkgTemplates.jl or BestieTemplate.jl. The later one is developed in Amsterdam at the Netherlands eScience Center.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can add dependencies to the package by activating the MyPackage folder in package mode and adding new dependencies as always:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> activate MyPackage\n(MyPackage) pkg> add MPI","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"This will add MPI to your package dependencies.","category":"page"},{"location":"getting_started_with_julia/#Using-your-own-package","page":"Getting started","title":"Using your own package","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To use your package you first need to add it to a package environment of your choice. This is done by changing to package mode and typing develop followed by the path to the folder containing the package. For instance:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> develop MyPackage","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nYou do not need to \"develop\" your package if you activated the package folder MyPackage.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, we can go back to standard Julia mode and use it as any other package:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"using MyPackage\nMyPackage.greet()","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Here, we just called the example function defined in MyPackage/src/MyPackage.jl.","category":"page"},{"location":"getting_started_with_julia/#Conclusion","page":"Getting started","title":"Conclusion","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Julia Manual\nPackage manager","category":"page"},{"location":"tsp/","page":"-","title":"-","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/tsp.ipynb\"","category":"page"},{"location":"tsp/","page":"-","title":"-","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"jacobi_2D/","page":"-","title":"-","text":"\n","category":"page"},{"location":"julia_async/","page":"Asynchronous programming in Julia","title":"Asynchronous programming in Julia","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/julia_async.ipynb\"","category":"page"},{"location":"julia_async/","page":"Asynchronous programming in Julia","title":"Asynchronous programming in Julia","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"julia_async/","page":"Asynchronous programming in Julia","title":"Asynchronous programming in Julia","text":"\n","category":"page"},{"location":"solutions/","page":"-","title":"-","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/solutions.ipynb\"","category":"page"},{"location":"solutions/","page":"-","title":"-","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"LEQ/","page":"Gaussian elimination","title":"Gaussian elimination","text":"\n","category":"page"},{"location":"julia_distributed/","page":"Distributed computing in Julia","title":"Distributed computing in Julia","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/julia_distributed.ipynb\"","category":"page"},{"location":"julia_distributed/","page":"Distributed computing in Julia","title":"Distributed computing in Julia","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"julia_distributed/","page":"Distributed computing in Julia","title":"Distributed computing in Julia","text":"\n","category":"page"},{"location":"julia_jacobi/","page":"-","title":"-","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/julia_jacobi.ipynb\"","category":"page"},{"location":"julia_jacobi/","page":"-","title":"-","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"matrix_matrix/","page":"Matrix-matrix multiplication","title":"Matrix-matrix multiplication","text":"\n","category":"page"},{"location":"asp/","page":"All pairs of shortest paths","title":"All pairs of shortest paths","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/asp.ipynb\"","category":"page"},{"location":"asp/","page":"All pairs of shortest paths","title":"All pairs of shortest paths","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"asp/","page":"All pairs of shortest paths","title":"All pairs of shortest paths","text":"\n","category":"page"},{"location":"","page":"Home","title":"Home","text":"CurrentModule = XM_40017","category":"page"},{"location":"#Programming-Large-Scale-Parallel-Systems-(XM_40017)","page":"Home","title":"Programming Large-Scale Parallel Systems (XM_40017)","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Welcome to the interactive lecture notes of the Programming Large-Scale Parallel Systems course at VU Amsterdam!","category":"page"},{"location":"#What","page":"Home","title":"What","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam. We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems. Further information about the course is found in the study guide (click here) and our Canvas page (for registered students).","category":"page"},{"location":"","page":"Home","title":"Home","text":"note: Note\nMaterial will be added incrementally to the website as the course advances.","category":"page"},{"location":"","page":"Home","title":"Home","text":"warning: Warning\nThis page will eventually contain only a part of the course material. The rest will be available on Canvas. In particular, the material in this public webpage does not fully cover all topics in the final exam.","category":"page"},{"location":"#How-to-use-this-page","page":"Home","title":"How to use this page","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"You have two main ways of studying the notebooks:","category":"page"},{"location":"","page":"Home","title":"Home","text":"Download the notebooks and run them locally on your computer (recommended). At each notebook page you will find a green box with links to download the notebook.\nYou also have the static version of the notebooks displayed in this webpage for quick reference.","category":"page"},{"location":"#How-to-run-the-notebooks-locally","page":"Home","title":"How to run the notebooks locally","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"To run a notebook locally follow these steps:","category":"page"},{"location":"","page":"Home","title":"Home","text":"Install Julia (if not done already). More information in Getting started.\nDownload the notebook.\nLaunch Julia. More information in Getting started.\nExecute these commands in the Julia command line:","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> using Pkg\njulia> Pkg.add(\"IJulia\")\njulia> using IJulia\njulia> notebook()","category":"page"},{"location":"","page":"Home","title":"Home","text":"These commands will open a jupyter in your web browser. Navigate in jupyter to the notebook file you have downloaded and open it.","category":"page"},{"location":"#Authors","page":"Home","title":"Authors","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This material is created by Francesc Verdugo with the help of Gelieza Kötterheinrich. Part of the notebooks are based on the course slides by Henri Bal.","category":"page"},{"location":"#License","page":"Home","title":"License","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"All material on this page that is original to this course may be used under a CC BY 4.0 license.","category":"page"},{"location":"#Acknowledgment","page":"Home","title":"Acknowledgment","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project \"Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course\" funded by the \"Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning\".","category":"page"}]
+[{"location":"getting_started_with_julia/#Getting-started","page":"Getting started","title":"Getting started","text":"","category":"section"},{"location":"getting_started_with_julia/#Introduction","page":"Getting started","title":"Introduction","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The programming of this course will be done using the Julia programming language. Thus, we start by explaining how to get up and running with Julia. After studying this page, you will be able to:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Use the Julia REPL,\nRun serial and parallel code,\nInstall and manage Julia packages.","category":"page"},{"location":"getting_started_with_julia/#Why-Julia?","page":"Getting started","title":"Why Julia?","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Courses related with high-performance computing (HPC) often use languages such as C, C++, or Fortran. We use Julia instead to make the course accessible to a wider set of students, including the ones that have no experience with C/C++ or Fortran, but are willing to learn parallel programming. Julia is a relatively new programming language specifically designed for scientific computing. It combines a high-level syntax close to interpreted languages like Python with the performance of compiled languages like C, C++, or Fortran. Thus, Julia will allow us to write efficient parallel algorithms with a syntax that is convenient in a teaching setting. In addition, Julia provides easy access to different programming models to write distributed algorithms, which will be useful to learn and experiment with them.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"tip: Tip\nYou can run the code in this link to learn how Julia compares to other languages (C and Python) in terms of performance.","category":"page"},{"location":"getting_started_with_julia/#Installing-Julia","page":"Getting started","title":"Installing Julia","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"This is a tutorial-like page. Follow these steps before you continue reading the document.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Download and install Julia from julialang.org;\nFollow the specific instructions for your operating system: Windows, MacOS, or Linux\nDownload and install VSCode and its Julia extension;","category":"page"},{"location":"getting_started_with_julia/#The-Julia-REPL","page":"Getting started","title":"The Julia REPL","text":"","category":"section"},{"location":"getting_started_with_julia/#Starting-Julia","page":"Getting started","title":"Starting Julia","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"There are several ways of opening Julia depending on your operating system and your IDE, but it is usually as simple as launching the Julia app. With VSCode, open a folder (File > Open Folder). Then, press Ctrl+Shift+P to open the command bar, and execute Julia: Start REPL. If this does not work, make sure you have the Julia extension for VSCode installed. Independently of the method you use, opening Julia results in a window with some text ending with:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia>","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You have just opened the Julia read-evaluate-print loop, or simply the Julia REPL. Congrats! You will spend most of time using the REPL, when working in Julia. The REPL is a console waiting for user input. Just as in other consoles, the string of text right before the input area (julia> in the case) is called the command prompt or simply the prompt.","category":"page"},{"location":"getting_started_with_julia/#Basic-usage","page":"Getting started","title":"Basic usage","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The usage of the REPL is as follows:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You write some input\npress enter\nyou get the output","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"For instance, try this","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> 1 + 1","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"A \"Hello world\" example looks like this in Julia","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> println(\"Hello, world!\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Try to run it in the REPL.","category":"page"},{"location":"getting_started_with_julia/#Help-mode","page":"Getting started","title":"Help mode","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Curious about what the function println does? Enter into help mode to look into the documentation. This is done by typing a question mark (?) into the input field:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> ?","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"After typing ?, the command prompt changes to help?>. It means we are in help mode. Now, we can type a function name to see its documentation.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"help?> println","category":"page"},{"location":"getting_started_with_julia/#Package-and-shell-modes","page":"Getting started","title":"Package and shell modes","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The REPL comes with two more modes, namely package and shell modes. To enter package mode type","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> ]","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Package mode is used to install and manage packages. We are going to discuss the package mode in greater detail later. To return back to normal mode press the backspace key several times.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To enter shell mode type semicolon (;)","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> ;","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The prompt should have changed to shell> indicating that we are in shell mode. Now you can type commands that you would normally do on your system command line. For instance,","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"shell> ls","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"will display the contents of the current folder in Mac or Linux. Using shell mode in Windows is not straightforward, and thus not recommended for beginners.","category":"page"},{"location":"getting_started_with_julia/#Running-Julia-code","page":"Getting started","title":"Running Julia code","text":"","category":"section"},{"location":"getting_started_with_julia/#Running-more-complex-code","page":"Getting started","title":"Running more complex code","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Real-world Julia programs are not typed in the REPL in practice. They are written in one or more files and included in the REPL. To try this, create a new file called hello.jl, write the code of the \"Hello world\" example above, and save it. If you are using VSCode, you can create the file using File > New File > Julia File. Once the file is saved with the name hello.jl, execute it as follows","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> include(\"hello.jl\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"warning: Warning\nMake sure that the file \"hello.jl\" is located in the current working directory of your Julia session. You can query the current directory with function pwd(). You can change to another directory with function cd() if needed. Also, make sure that the file extension is .jl.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The recommended way of running Julia code is using the REPL as we did. But it is also possible to run code directly from the system command line. To this end, open a terminal and call Julia followed by the path to the file containing the code you want to execute.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ julia hello.jl","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The previous line assumes that you have Julia properly installed in the system and that it's usable from the terminal. In UNIX systems (Linux and Mac), the Julia binary needs to be in one of the directories listed in the PATH environment variable. To check that Julia is properly installed, you can use","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ julia --version","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"If this runs without error and you see a version number, you are good to go!","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can also run julia code from the terminal using the -e flag:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ julia -e 'println(\"Hello, world!\")'","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nIn this tutorial, when a code snipped starts with $, it should be run in the terminal. Otherwise, the code is to be run in the Julia REPL.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"tip: Tip\nAvoid calling Julia code from the terminal, use the Julia REPL instead! Each time you call Julia from the terminal, you start a fresh Julia session and Julia will need to compile your code from scratch. This can be time consuming for large projects. In contrast, if you execute code in the REPL, Julia will compile code incrementally, which is much faster. Running code in a cluster (like in DAS-5 for the Julia assignment) is among the few situations you need to run Julia code from the terminal. Visit this link (Julia workflow tips) from the official Julia documentation for further information about how to develop Julia code effectivelly.","category":"page"},{"location":"getting_started_with_julia/#Running-parallel-code","page":"Getting started","title":"Running parallel code","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Since we are in a parallel computing course, let's run a parallel \"Hello world\" example in Julia. Open a Julia REPL and write","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using Distributed\njulia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Here, we are using the Distributed package, which is part of the Julia standard library that provides distributed memory parallel support. The code prints the process id and the number of processes in the current Julia session.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You will probably only see output from 1 process. We need to add more processes to run the example in parallel. This is done with the addprocs function.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> addprocs(3)","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"We have added 3 new processes. Plus the old one, we have 4 processes. Run the code again.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, you should see output from 4 processes.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"It is possible to specify the number of processes when starting Julia from the terminal with the -p argument (useful, e.g., when running in a cluster). If you launch Julia from the terminal as","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ julia -p 3","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"and then run","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> @everywhere println(\"Hello, world! I am proc $(myid()) from $(nprocs())\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You should get output from 4 processes as before.","category":"page"},{"location":"getting_started_with_julia/#Installing-packages","page":"Getting started","title":"Installing packages","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"One of the most useful features of Julia is its package manager. It allows one to install Julia packages in a straightforward and platform independent way. To illustrate this, let us consider the following parallel \"Hello world\" example. This example uses the Message Passing Interface (MPI). We will learn more about MPI later in the course.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Copy the following block of code into a new file named \"hello_mpi.jl\"","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"# file hello_mpi.jl\nusing MPI\nMPI.Init()\ncomm = MPI.COMM_WORLD\nrank = MPI.Comm_rank(comm)\nnranks = MPI.Comm_size(comm)\nprintln(\"Hello world, I am rank $rank of $nranks\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"As you can see from this example, one can access MPI from Julia in a clean way, without type annotations and other complexities of C/C++ code.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, run the file from the REPL","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> include(\"hello_mpi.jl\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"It probably didn't work, right? Read the error message and note that the MPI package needs to be installed to run this code.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To install a package, we need to enter package mode. Remember that we entered into help mode by typing ?. Package mode is activated by typing ] : ","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> ]","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"At this point, the prompt should have changed to (@v1.10) pkg> indicating that we are in package mode. The text between the parentheses indicates which is the active project, i.e., where packages are going to be installed. In this case, we are working with the global project associated with our Julia installation (which is Julia 1.10 in this example, but it can be another version in your case).","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To install the MPI package, type","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> add MPI","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Congrats, you have installed MPI!","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nMany Julia package names end with .jl. This is just a way of signaling that a package is written in Julia. When using such packages, the .jl needs to be omitted. In this case, we have installed the MPI.jl package even though we have only typed MPI in the REPL.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nThe package you have installed is the Julia interface to MPI, called MPI.jl. Note that it is not a MPI library by itself. It is just a thin wrapper between MPI and Julia. To use this interface, you need an actual MPI library installed in your system such as OpenMPI or MPICH. Julia downloads and installs a MPI library for you, but it is also possible to use a MPI library already available in your system. This is useful, e.g., when running on HPC clusters. See the documentation of MPI.jl for further details.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To check that the package was installed properly, exit package mode by pressing the backspace key several times, and run it again","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> include(\"hello_mpi.jl\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, it should work, but you probably get output from a single MPI rank only.","category":"page"},{"location":"getting_started_with_julia/#Running-MPI-code","page":"Getting started","title":"Running MPI code","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To run MPI applications in parallel, you need a launcher like mpiexec. MPI codes written in Julia are not an exception to this rule. From the system terminal, you can run","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"$ mpiexec -np 4 julia hello_mpi.jl","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"But it will probably not work since the version of mpiexec needs to match with the MPI version we are using from Julia. Don't worry if you could not make it work! A more elegant way to run MPI code is from the Julia REPL directly, by using these commands:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using MPI\njulia> run(`$(mpiexec()) -np 4 julia hello_mpi.jl`)","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, you should see output from 4 ranks.","category":"page"},{"location":"getting_started_with_julia/#Package-manager","page":"Getting started","title":"Package manager","text":"","category":"section"},{"location":"getting_started_with_julia/#Installing-packages-locally","page":"Getting started","title":"Installing packages locally","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"We have installed the MPI package globally and it will be available in all Julia sessions. However, in some situations, we want to work with different versions of the same package or to install packages in an isolated way to avoid potential conflicts with other packages. This can be done by using local projects.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"A project is simply a folder in your file system. To use a particular folder as your project, you need to activate it. This is done by entering package mode and using the activate command followed by the path to the folder you want to activate.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> activate .","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The previous command will activate the current working directory. Note that the dot . is indeed the path to the current folder.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The prompt has changed to (lessons) pkg> indicating that we are in the project within the lessons folder. The particular folder name can be different in your case.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"tip: Tip\nYou can activate a project directly when opening Julia from the terminal using the --project flag. The command $ julia --project=. will open Julia and activate a project in the current directory. You can also achieve the same effect by setting the environment variable JULIA_PROJECT with the path of the folder you want to activate.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nThe active project folder and the current working directory are two independent concepts! For instance, (@v1.10) pkg> activate folderB and then julia> cd(\"folderA\"), will activate the project in folderB and change the current working directory to folderA.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"At this point all package-related operations will be local to the new project. For instance, install the DataFrames package.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(lessons) pkg> add DataFrames","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Use the package to check that it is installed","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using DataFrames\njulia> DataFrame(a=[1,2],b=[3,4])","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, we can return to the global project to check that DataFrames has not been installed there. To return to the global environment, use activate without a folder name.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(lessons) pkg> activate","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The prompt is again (@v1.10) pkg>","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, try to use DataFrames.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using DataFrames\njulia> DataFrame(a=[1,2],b=[3,4])","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You should get an error or a warning unless you already had DataFrames installed globally.","category":"page"},{"location":"getting_started_with_julia/#Project-and-Manifest-files","page":"Getting started","title":"Project and Manifest files","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The information about a project is stored in two files Project.toml and Manifest.toml.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Project.toml contains the packages explicitly installed (the direct dependencies)\nManifest.toml contains direct and indirect dependencies along with the concrete version of each package.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"In other words, Project.toml contains the packages relevant for the user, whereas Manifest.toml is the detailed snapshot of all dependencies. The Manifest.toml can be used to reproduce the same environment in another machine.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can see the path to the current Project.toml file by using the status operator (or st in its short form) while in package mode","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> status","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The information about the Manifest.toml can be inspected by passing the -m flag.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> status -m","category":"page"},{"location":"getting_started_with_julia/#Installing-packages-from-a-project-file","page":"Getting started","title":"Installing packages from a project file","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Project files can be used to install lists of packages defined by others. E.g., to install all the dependencies of a Julia application.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Assume that a colleague has sent to you a Project.toml file with this content:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"[deps]\nBenchmarkTools = \"6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf\"\nDataFrames = \"a93c6f00-e57d-5684-b7b6-d8193f3e46c0\"\nMPI = \"da04e1cc-30fd-572f-bb4f-1f8673147195\"","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Copy the contents of previous code block into a file called Project.toml and place it in an empty folder named newproject. It is important that the file is named Project.toml. You can create a new folder from the REPL with","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> mkdir(\"newproject\")","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To install all the packages registered in this file you need to activate the folder containing your Project.toml file","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> activate newproject","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"and then instantiating it","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(newproject) pkg> instantiate","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The instantiate command will download and install all listed packages and their dependencies in just one click.","category":"page"},{"location":"getting_started_with_julia/#Getting-help-in-package-mode","page":"Getting started","title":"Getting help in package mode","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can get help about a particular package operator by writing help in front of it","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> help activate","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can get an overview of all package commands by typing help alone","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> help","category":"page"},{"location":"getting_started_with_julia/#Package-operations-in-Julia-code","page":"Getting started","title":"Package operations in Julia code","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"In some situations it is required to use package commands in Julia code, e.g., to automatize installation and deployment of Julia applications. This can be done using the Pkg package. For instance","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"julia> using Pkg\njulia> Pkg.status()","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"is equivalent to calling status in package mode.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> status","category":"page"},{"location":"getting_started_with_julia/#Creating-you-own-package","page":"Getting started","title":"Creating you own package","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"In many situations, it is useful to create your own package, for instance, when working with a large code base, when you want to reduce compilation latency using Revise.jl, or if you want to eventually register your package and share it with others.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"The simplest way of generating a package (called MyPackage) is as follows. Open Julia, go to package mode, and type","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> generate MyPackage","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"This will crate a minimal package consisting of a new folder MyPackage with two files:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"MyPackage/Project.toml: Project file defining the direct dependencies of your package.\nMyPackage/src/MyPackage.jl: Main source file of your package. You can split your code in several files if needed, and include them in the package main file using function include.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"tip: Tip\nThis approach only generates a very minimal package. To create a more sophisticated package skeleton (including unit testing, code coverage, readme file, licence, etc.) use PkgTemplates.jl or BestieTemplate.jl. The later one is developed in Amsterdam at the Netherlands eScience Center.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"You can add dependencies to the package by activating the MyPackage folder in package mode and adding new dependencies as always:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> activate MyPackage\n(MyPackage) pkg> add MPI","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"This will add MPI to your package dependencies.","category":"page"},{"location":"getting_started_with_julia/#Using-your-own-package","page":"Getting started","title":"Using your own package","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"To use your package you first need to add it to a package environment of your choice. This is done by changing to package mode and typing develop followed by the path to the folder containing the package. For instance:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"(@v1.10) pkg> develop MyPackage","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"note: Note\nYou do not need to \"develop\" your package if you activated the package folder MyPackage.","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Now, we can go back to standard Julia mode and use it as any other package:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"using MyPackage\nMyPackage.greet()","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Here, we just called the example function defined in MyPackage/src/MyPackage.jl.","category":"page"},{"location":"getting_started_with_julia/#Conclusion","page":"Getting started","title":"Conclusion","text":"","category":"section"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"We have learned the basics of how to work with Julia, including how to run serial and parallel code, and how to manage, create, and use Julia packages. This knowledge will allow you to follow the course effectively! If you want to further dig into the topics we have covered here, you can take a look at the following links:","category":"page"},{"location":"getting_started_with_julia/","page":"Getting started","title":"Getting started","text":"Julia Manual\nPackage manager","category":"page"},{"location":"tsp/","page":"Traveling salesperson problem","title":"Traveling salesperson problem","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/tsp.ipynb\"","category":"page"},{"location":"tsp/","page":"Traveling salesperson problem","title":"Traveling salesperson problem","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"jacobi_2D/","page":"-","title":"-","text":"\n","category":"page"},{"location":"julia_async/","page":"Asynchronous programming in Julia","title":"Asynchronous programming in Julia","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/julia_async.ipynb\"","category":"page"},{"location":"julia_async/","page":"Asynchronous programming in Julia","title":"Asynchronous programming in Julia","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"julia_async/","page":"Asynchronous programming in Julia","title":"Asynchronous programming in Julia","text":"\n","category":"page"},{"location":"solutions/","page":"-","title":"-","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/solutions.ipynb\"","category":"page"},{"location":"solutions/","page":"-","title":"-","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"LEQ/","page":"Gaussian elimination","title":"Gaussian elimination","text":"\n","category":"page"},{"location":"julia_distributed/","page":"Distributed computing in Julia","title":"Distributed computing in Julia","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/julia_distributed.ipynb\"","category":"page"},{"location":"julia_distributed/","page":"Distributed computing in Julia","title":"Distributed computing in Julia","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"julia_distributed/","page":"Distributed computing in Julia","title":"Distributed computing in Julia","text":"\n","category":"page"},{"location":"julia_jacobi/","page":"-","title":"-","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/julia_jacobi.ipynb\"","category":"page"},{"location":"julia_jacobi/","page":"-","title":"-","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"matrix_matrix/","page":"Matrix-matrix multiplication","title":"Matrix-matrix multiplication","text":"\n","category":"page"},{"location":"asp/","page":"All pairs of shortest paths","title":"All pairs of shortest paths","text":"EditURL = \"https://github.com/fverdugo/XM_40017/blob/main/notebooks/asp.ipynb\"","category":"page"},{"location":"asp/","page":"All pairs of shortest paths","title":"All pairs of shortest paths","text":"
\n Tip\n
\n
\n
\n Download this notebook and run it locally on your machine [highly recommended]. Click here.\n
\n
\n
\n
","category":"page"},{"location":"asp/","page":"All pairs of shortest paths","title":"All pairs of shortest paths","text":"\n","category":"page"},{"location":"","page":"Home","title":"Home","text":"CurrentModule = XM_40017","category":"page"},{"location":"#Programming-Large-Scale-Parallel-Systems-(XM_40017)","page":"Home","title":"Programming Large-Scale Parallel Systems (XM_40017)","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Welcome to the interactive lecture notes of the Programming Large-Scale Parallel Systems course at VU Amsterdam!","category":"page"},{"location":"#What","page":"Home","title":"What","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This page contains part of the course material of the Programming Large-Scale Parallel Systems course at VU Amsterdam. We provide several lecture notes in jupyter notebook format, which will help you to learn how to design, analyze, and program parallel algorithms on multi-node computing systems. Further information about the course is found in the study guide (click here) and our Canvas page (for registered students).","category":"page"},{"location":"","page":"Home","title":"Home","text":"note: Note\nMaterial will be added incrementally to the website as the course advances.","category":"page"},{"location":"","page":"Home","title":"Home","text":"warning: Warning\nThis page will eventually contain only a part of the course material. The rest will be available on Canvas. In particular, the material in this public webpage does not fully cover all topics in the final exam.","category":"page"},{"location":"#How-to-use-this-page","page":"Home","title":"How to use this page","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"You have two main ways of studying the notebooks:","category":"page"},{"location":"","page":"Home","title":"Home","text":"Download the notebooks and run them locally on your computer (recommended). At each notebook page you will find a green box with links to download the notebook.\nYou also have the static version of the notebooks displayed in this webpage for quick reference.","category":"page"},{"location":"#How-to-run-the-notebooks-locally","page":"Home","title":"How to run the notebooks locally","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"To run a notebook locally follow these steps:","category":"page"},{"location":"","page":"Home","title":"Home","text":"Install Julia (if not done already). More information in Getting started.\nDownload the notebook.\nLaunch Julia. More information in Getting started.\nExecute these commands in the Julia command line:","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> using Pkg\njulia> Pkg.add(\"IJulia\")\njulia> using IJulia\njulia> notebook()","category":"page"},{"location":"","page":"Home","title":"Home","text":"These commands will open a jupyter in your web browser. Navigate in jupyter to the notebook file you have downloaded and open it.","category":"page"},{"location":"#Authors","page":"Home","title":"Authors","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This material is created by Francesc Verdugo with the help of Gelieza Kötterheinrich. Part of the notebooks are based on the course slides by Henri Bal.","category":"page"},{"location":"#License","page":"Home","title":"License","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"All material on this page that is original to this course may be used under a CC BY 4.0 license.","category":"page"},{"location":"#Acknowledgment","page":"Home","title":"Acknowledgment","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This page was created with the support of the Faculty of Science of Vrije Universiteit Amsterdam in the framework of the project \"Interactive lecture notes and exercises for the Programming Large-Scale Parallel Systems course\" funded by the \"Innovation budget BETA 2023 Studievoorschotmiddelen (SVM) towards Activated Blended Learning\".","category":"page"}]
}
diff --git a/dev/solutions/index.html b/dev/solutions/index.html
index 9b9603d..2b6fe45 100644
--- a/dev/solutions/index.html
+++ b/dev/solutions/index.html
@@ -1,5 +1,5 @@
-- · XM_40017
This document was generated with Documenter.jl version 1.7.0 on Monday 23 September 2024. Using Julia version 1.10.5.
+end
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 23 September 2024. Using Julia version 1.10.5.
diff --git a/dev/tsp.ipynb b/dev/tsp.ipynb
index 1ad09d8..259bd0f 100644
--- a/dev/tsp.ipynb
+++ b/dev/tsp.ipynb
@@ -22,8 +22,8 @@
"In this notebook, we will learn\n",
"\n",
"- How to parallelize the solution of the traveling sales person problem\n",
- "- How to fix dynamic load imbalance\n",
- "- The concept of search overhead\n"
+ "- The concept of search overhead\n",
+ "- A dynamic load balancing method\n"
]
},
{
@@ -51,9 +51,17 @@
" \"It's not correct. Keep trying! 💪\"\n",
" end |> println\n",
"end\n",
- "tsp_check_2(answer) = answer_checker(answer, 2)\n",
+ "tsp_check_2(answer) = answer_checker(answer, 4)\n",
"tsp_check_3(answer) = answer_checker(answer, \"d\")\n",
- "tsp_check_4(answer) = answer_checker(answer, \"a\")"
+ "tsp_check_4(answer) = answer_checker(answer, \"a\")\n",
+ "function q_superlinear_answer(bool)\n",
+ " bool || return\n",
+ " msg = \"\"\"\n",
+ " Negative search overhead can explain the superlinear speedup in this algorithm. The optimal speedup (speedup equal to the numer of processors) assumes that the work done in the sequental and parallel algorithm is the same. If the parallel code does less work, it is possible to go beyond the optimal speedup. Cache effects are not likely to have a positive impact here. Even large search spaces can be represented with rather small distance matrices. Moreover, we are not partitioning the distance matrix.\n",
+ " \"\"\"\n",
+ " println(msg)\n",
+ "end\n",
+ "println(\"🥳 Well done!\")"
]
},
{
@@ -64,9 +72,16 @@
"## The traveling sales person (TSP) problem\n",
"\n",
"\n",
+ "In this notebook we will study another algorithm that works with graphs, the [traveling sales person (TSP) problem](https://en.wikipedia.org/wiki/Travelling_salesman_problem). The classical formulation of this problem is as follows (quoted from Wikipedia) \"Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once?\" This problem as applications in combinatorial optimization, theoretical computer science, and operations research. It is very expensive problem to solve (NP-hard problem) which often needs parallel computing.\n",
+ "\n",
+ "
\n",
+ "Note: There are two key variations of this problem. One in which the sales person returns to the initial city, and another in which the sales person does not return to the initial city. We will consider the second variant for simplicity.\n",
+ "
\n",
+ "\n",
+ "\n",
"### Problem statement\n",
"\n",
- "Given a graph $G$ with a distance table $C$ and an initial node (i.e. a city) in the graph, compute the shortest route that visits all cities exactly once, without returning to the initial city."
+ "Our version of the TSP problem can be formalized as follows. Given a graph $G$ with a distance table $C$ and an initial node in the graph, compute the shortest route that visits all nodes exactly once, without returning to the initial node. The nodes on the graph can be interpreted as the \"cities\", and the solution is the optimal route for the traveling salesperson to visit all cities. The following figure shows a simple TSP problem and its solution."
]
},
{
@@ -86,14 +101,12 @@
},
{
"cell_type": "markdown",
- "id": "c303dddf",
+ "id": "fd4f87fc",
"metadata": {},
"source": [
"### Sequential algorithm (branch and bound)\n",
"\n",
- "The sequential algorithm finds a shortest path by traversing the paths tree of the problem. The root of this tree is the initial city. The children of each node in the graph are the neighbour cities that have not been visited on the path so far. When all neighbour cities are already visited, the city becomes a leaf node in the tree.\n",
- "\n",
- "The possile solutions of the problem are the paths from the root of the tree to a leaf node. Note that we assume the children are sorted using the **nearest city first heuristic**. This allows to quickly find a minimum bound for the distance which will be used to prune the remaining paths (see next section). "
+ "A well known method to solve this problem is based on a [branch and bound](https://en.wikipedia.org/wiki/Branch_and_bound) strategy. It consisting in organizing all possible routes in a tree-like structure (this is the \"branch\" part). The root of this tree is the initial city. The children of each node in the graph are the neighbor cities that have not been visited in the path so far. When all neighbor cities are already visited, the city becomes a leaf node in the tree. See figure below for the tree associated with our TSP problem example. The TSP problem consists now in finding which is the \"shortest\" branch in this tree. The tree data structure is just a convenient way of organizing all possible routes in order to search for the shortest one. We refer to it as the *search tree* or the *search space*."
]
},
{
@@ -113,10 +126,12 @@
},
{
"cell_type": "markdown",
- "id": "9da5f5ae",
+ "id": "c303dddf",
"metadata": {},
"source": [
- "Of course, visiting all paths in the tree is impractical for moderate and large numbers of cities. The number of possible paths might be up to $O(N!)$. Therefore, an essential part of the algorithm is to bound the search by remembering the current minimum distance. "
+ "### Nearest city first heuristic\n",
+ "\n",
+ "When building the search tree we are free to choose any order when defining the children of a node. A clever order is using the *nearest city first heuristic*. I.e., we sort the children according to how far they are from the current node, in ascending order. This allows to quickly find a minimum bound for the distance which will be used to prune the remaining paths (see next section). The figure above used the nearest city first heuristic. In blue you can see the distance between cities. The first child is always the one with the shortest distance."
]
},
{
@@ -126,9 +141,9 @@
"source": [
"### Pruning the search tree\n",
"\n",
- "The algorithm keeps track of the best solution of all paths visited so far. This allows to skip searching paths that already exceed this value. \n",
+ "The basic idea of the algorithm is to loop over all possible routes (all branches in the search tree) and find find the one with the shortest distance. One can optimize this process by \"pruning\" the search tree. We keep track of the best solution of all paths visited so far, which allows us to skip searching paths that already exceed this value. This is the \"bound\" part of the branch and bound strategy. \n",
"\n",
- "For example, in the following graph only 3 out of 6 possible routes need to be visted when we cut off the search after the minimum distance is exceeded. (The grey nodes are the ones we don't visit because the minimum distance had been exceeded at the previous node already.)"
+ "For example, in the following graph only 3 out of 6 possible routes need to be fully traversed to find the shortest route. In particular, we do not need to fully traverse the second branch/route (figure below left). when visiting the third city in this branch the current distance is already equal to the full previous route. It means that the solution will not be in this part of the tree for sure. In figure below (right), the gray nodes are the ones we do not visit because the minimum distance had been exceeded before completing the route."
]
},
{
@@ -148,37 +163,14 @@
},
{
"cell_type": "markdown",
- "id": "f3c78a1d",
+ "id": "9da5f5ae",
"metadata": {},
"source": [
- "Note that it is not necessary that the graph be fully connected. Variations of this algorithm work for sparse graphs or directed graphs as well. \n",
+ "### Computation complexity\n",
"\n",
- "In the previous example, the shortest route was also the leftmost path in the graph. Although it is more likely that the shortest route be found in the left part of the graph when using the nearest city first heuristic, the solution can be anywhere in the search tree."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "85d771de",
- "metadata": {},
- "source": [
- "
\n",
- " Example: Look at the following graph and its corresponding search tree. If $x\\leq 15$, the shortest route is the leftmost branch of the search tree. If $x > 16$, the route is situated on the right side of the search tree. \n",
- "
"
+ "The total number of routes we need to traverse is $O(N!)$, where $N$ is the number of cities. This comes from the fact that the number of possible routes is equal to the number of possible permutations of $N$ cities. Thus the cost of the algorithm is $O(N!)$, which becomes expensive very quickly when $N$ grows.\n",
+ "\n",
+ "In practice, however, we will not need to traverse all $O(N!)$ possible routes to find the shortest one since we consider pruning. The nearest city first heuristic also makes more likely that the shortest route is among the first routes to be traversed (left part of the tree), thus speeding the process. However, the solution can be anywhere in the search tree, and the number of routes to be traversed is $O(N!)$ in the worse case scenario."
]
},
{
@@ -188,7 +180,12 @@
"source": [
"## Serial implementation\n",
"\n",
- "Let's implement the serial algorithm. First, we sort the neighbours according to their distance. "
+ "Let's implement the serial algorithm.\n",
+ "\n",
+ "\n",
+ "
\n",
+ "Note: The implementation of this algorithm is rather challenging. Try to understand the key ideas (the explanations) instead of all code details. Having the complete functional implementation is useful to analyze the actual performance of our parallel implementation at the end of the notebook, and to check if it is consistent with the theory.\n",
+ "
"
]
},
{
@@ -196,7 +193,9 @@
"id": "f2b70f85",
"metadata": {},
"source": [
- "### Nearest-city first heuristic"
+ "### Nearest-city first heuristic\n",
+ "\n",
+ "The first step is preprocessing the distance table to create a new data structure that takes into account the nearest city first heuristic. This is done in the following function."
]
},
{
@@ -217,27 +216,12 @@
"end"
]
},
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "2eeecdd6",
- "metadata": {},
- "outputs": [],
- "source": [
- "C = [\n",
- " 0 2 3 2\n",
- " 2 0 4 1\n",
- " 3 4 0 3\n",
- " 2 1 3 0 \n",
- "]"
- ]
- },
{
"cell_type": "markdown",
- "id": "f769627c",
+ "id": "9fc1a398",
"metadata": {},
"source": [
- "The data structure we will use for the connections table is a matrix of tuples of the form (destination, distance). The tuples are sorted by their distance in ascending order (per start city). "
+ "Execute the next cell to understand the output of `sort_neighbors`."
]
},
{
@@ -247,15 +231,21 @@
"metadata": {},
"outputs": [],
"source": [
+ "C = [\n",
+ " 0 2 3 2\n",
+ " 2 0 4 1\n",
+ " 3 4 0 3\n",
+ " 2 1 3 0 \n",
+ "]\n",
"C_sorted = sort_neighbors(C)"
]
},
{
"cell_type": "markdown",
- "id": "51ed8312",
+ "id": "a63dc266",
"metadata": {},
"source": [
- "The connections matrix can be indexed by a city. This returns a `Vector{Tuple}}` of all the destinations and their corresponding distances. "
+ "The output is a vector of vector of tuples. The outer vector is indexed by a city id, for instance:"
]
},
{
@@ -271,6 +261,14 @@
"C_sorted[city]"
]
},
+ {
+ "cell_type": "markdown",
+ "id": "f769627c",
+ "metadata": {},
+ "source": [
+ " This returns a vector of tuples that contains information about the connections to this city of the form (destination, distance). In this case, city 3 is connected to city 3 at distance 0 (itself), then with city 1 at distance 3, then with city 4 at distance 3, and finally with city 2 at distance 4. Note that the connections are sorted by their distance in ascending order (here is where the nearest city first heuristic is used). "
+ ]
+ },
{
"cell_type": "markdown",
"id": "44025bd5",
@@ -284,7 +282,7 @@
"id": "6c91a99f",
"metadata": {},
"source": [
- "Next, we write an algorithm that traverses the whole search tree and prints all the possible paths. The tree is traversed in depth-first order. Before we go to a neighbouring city, we also have to verify that it has not been visited on this path yet. If we reach a leaf node, we print the complete path and continue searching. "
+ "Next, we write an algorithm that traverses the whole search tree and prints all the possible paths. To this end, the tree is traversed in [depth-first order](https://en.wikipedia.org/wiki/Depth-first_search) using a recursive function call. Before we go to a neighbouring city, we also have to verify that it has not been visited on this path yet. If we reach a leaf node, we print the complete path and continue searching. "
]
},
{
@@ -333,7 +331,7 @@
" end\n",
" return nothing\n",
" else\n",
- " println(path)\n",
+ " println(\"I just completed route $path\")\n",
" return nothing\n",
" end\n",
"end"
@@ -357,7 +355,7 @@
"source": [
"### Serial implementation without pruning\n",
"\n",
- "Now, we add the computation of the minimum distance. At each leaf node, we update the minimum distance. Furthermore, as we add another node to our path, we update the distance of the current path. That makes it necessary to include two more parameters in our recursive algorithm: `distance`, the distance of the current path, and `min_distance`, the best minimum distance found so far. "
+ "Now, we know how to traverse all possible routes. We just need a minor modification of the code below to solve the TSP problem (without pruning). We add a new variable called `min_distance` that keeps track of the distance of the shortest route so-far. This variable is updated at the end of each route, i.e., when a leaf node is visited. After traversing all routes, `min_distance` will contain the distance of the shortest route (the solution of the ASP problem)."
]
},
{
@@ -375,6 +373,16 @@
"
\n",
+ "Note: We could further modify the function so that we also return a vector containing the cities in the shortest route. However, in this notebook, we will only return the distance of the shortest route (a single value) for simplicity.\n",
+ "
"
+ ]
+ },
{
"cell_type": "code",
"execution_count": null,
@@ -382,6 +390,7 @@
"metadata": {},
"outputs": [],
"source": [
+ "verbose::Bool = true\n",
"function tsp_serial_no_prune(C_sorted,city)\n",
" num_cities = length(C_sorted)\n",
" path=zeros(Int,num_cities)\n",
@@ -411,7 +420,7 @@
" else\n",
" # Set new minimum distance in leaf nodes\n",
" min_distance = min(distance,min_distance)\n",
- " #@show path, distance, min_distance\n",
+ " verbose && println(\"I just completed route $path. Min distance so far is $min_distance\")\n",
" return min_distance\n",
" end\n",
"end"
@@ -425,6 +434,7 @@
"outputs": [],
"source": [
"city = 1\n",
+ "verbose = true\n",
"min_distance = tsp_serial_no_prune(C_sorted,city)"
]
},
@@ -435,7 +445,7 @@
"source": [
"### Final serial implementation\n",
"\n",
- "Finally, we add the pruning to our algorithm. Anytime the current distance exceeds the minimum distance, the search in this path is aborted and continued with another path. "
+ "Finally, we add the pruning to our algorithm. Anytime the current distance exceeds the minimum distance, the search in this path is aborted and continued with another path. By running the function below, you will see that only three routes will be traversed thanks to pruning as shown in next figure."
]
},
{
@@ -472,6 +482,7 @@
"function tsp_serial_recursive!(C_sorted,hops,path,distance,min_distance)\n",
" # Prune this path if its distance is too high already\n",
" if distance >= min_distance\n",
+ " verbose && println(\"I am pruning at $(view(path,1:hops))\")\n",
" return min_distance\n",
" end\n",
" num_cities = length(C_sorted)\n",
@@ -493,7 +504,7 @@
" else\n",
" # Set new minimum distance in leaf nodes\n",
" min_distance = min(distance,min_distance)\n",
- " #@show path, distance, min_distance\n",
+ " verbose && println(\"I just completed route $path. Min distance so far is $min_distance\")\n",
" return min_distance\n",
" end\n",
"end"
@@ -507,6 +518,7 @@
"outputs": [],
"source": [
"city = 1\n",
+ "verbose = true\n",
"min_distance = tsp_serial(C_sorted,city)"
]
},
@@ -526,13 +538,14 @@
"metadata": {},
"outputs": [],
"source": [
- "n = 12 # It is safe to test up to n=12\n",
+ "n = 11 # It is safe to test up to n=11 on a laptop\n",
"using Random\n",
"using Test\n",
"Random.seed!(1)\n",
"C = rand(1:10,n,n)\n",
"C_sorted = sort_neighbors(C)\n",
"city = 1\n",
+ "verbose = false\n",
"@time min_no_prune = tsp_serial_no_prune(C_sorted,city)\n",
"@time min_prune = tsp_serial(C_sorted,city)\n",
"@test min_no_prune == min_prune"
@@ -543,7 +556,7 @@
"id": "6088ddc9",
"metadata": {},
"source": [
- "You can observe that, especially for larger numbers of cities (n=11 or n=12), the performance of the algorithm with pruning is much better than the performance of the algorithm without pruning. "
+ "You can observe that, especially for larger numbers of cities (n=11), the performance of the algorithm with pruning is much better than the performance of the algorithm without pruning. "
]
},
{
@@ -556,11 +569,13 @@
},
{
"cell_type": "markdown",
- "id": "c6375465",
+ "id": "732a3ffb",
"metadata": {},
"source": [
"### Where can we extract parallelism ?\n",
- "Unlike the previous algorithms we studied, in this problem we don't know beforehand how much work is performed since we don't know where the pruning cuts off part of the search tree. Still, we want to divide the workload among multiple processes to enhance the performance. "
+ "\n",
+ "All branches of the search tree can be traversed in parallel. Let us discuss how we can distribute these branches over several processes.\n",
+ "\n"
]
},
{
@@ -585,7 +600,7 @@
"source": [
"### Option 1\n",
"\n",
- "The first idea how to parallelize the TSP algorithm is to assign a branch of our search tree to each process. However, as mentioned in an earlier section, the number of branches in the search tree can be up to $O(N!)$. This would require an unfeasibly large amount of proecesses which each do only very little work. "
+ "We can (at least in theory) assign a branch of our search tree to each process. However, as mentioned in an earlier section, the number of branches in the search tree can be up to $O(N!)$. This would require an unfeasibly large amount of processors which each do only very little work. Thus, we skip this option as it is impractical."
]
},
{
@@ -609,7 +624,8 @@
"metadata": {},
"source": [
"### Option 2\n",
- "Instead of assigning one branch per worker, we can assign a fixed number of branches to each worker. This way, each worker can perform the pruning within their own subtree and less workers are needed. \n"
+ "\n",
+ "Instead of assigning one branch per worker, we can assign a fixed number of branches to each worker. This would be a good strategy if we do not consider pruning. However, it is not efficient if we include pruning (which is essential in this algorithm). "
]
},
{
@@ -632,13 +648,14 @@
"id": "e3af7def",
"metadata": {},
"source": [
- "### Performance issues\n",
+ "### Performance issues: Load balance\n",
"\n",
- "#### Load balancing\n",
- "However, this approach has a problem with load balancing. Since we don't know beforehand how much pruning can be done in each subtree, some workers might end up doing less work than others. This uneven distribution of workload leads to some workers being idle, which impairs the speedup. \n",
+ "Pruning is essential in this algorithm but makes challenging to evenly distribute the work over available processors. Image that we assign the same number of branches per worker and that the workers use pruning locally to speed up the solution process. It is not possible to know in advance how many branches will be fully traversed by each worker since pruning depends on the actual values in the input distance matrix (runtime values). It might happen that a worker can prune many branches and finishes fast, whereas other workers are not able to prune so many branches and they need more time to finish. This is a clear example of bad load balance. We will explain later a strategy to fix it.\n",
"\n",
- "#### Search overhead \n",
- "Another disadvantage of this kind of parallel search is that the pruning is now less effective. The workers each run their own version of the search algorithm and keep track of their local minimum distances. This means that less nodes will be pruned in the parallel version than in the serial version. This is called **search overhead**."
+ "\n",
+ "### Performance issues: Search overhead \n",
+ "\n",
+ "Another disadvantage of this kind of parallel search is that the pruning is now less effective. The workers each run their own version of the search algorithm and keep track of their local minimum distances. This means that less nodes will be pruned in the parallel version than in the serial version. The parallel code might search more routes than the sequential ones. This is called *search overhead*."
]
},
{
@@ -647,7 +664,7 @@
"metadata": {},
"source": [
"
\n",
- "Question: How many nodes are pruned in total when we assign two branches to each worker? Look at the illustration below.\n",
+ "Question: How routes are fully traversed in total when we assign two branches to each worker? Look at the illustration below. Assume that each worker does pruning locally and independently of the other workers.\n",
"
"
]
},
@@ -682,33 +699,42 @@
"id": "d0bc4fdd",
"metadata": {},
"source": [
- "In this example, the parallel algorithm prunes less nodes than the serial version because not all workers are able to use the global minimum distance as a pruning bound."
+ "In this example, the parallel algorithm traverses more routes (1 more) then the serial version because not all workers are able to use the global minimum distance as a pruning bound. Remember that the sequential code only traverses 3 routes completely. See figure:"
+ ]
+ },
+ {
+ "attachments": {
+ "g26375.png": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1oAAAHpCAYAAACbY01sAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAewgAAHsIBbtB1PgAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAACAASURBVHic7N15fFxl9cfxz7mTZtKWRQrIItAmadkqyCaI7IsisikKgsiOFOVHwQLNTFrgKs3MpC1Vyr6oQBWxZRGogFAoImURQUCRpSRpWcsiFeiSSTNzfn/cDEymycwkmZk7k5z369WX5s6duac0mcy5z/N8H8EYY4wpnBpgD2BXYOuuP1sB6wAju/73f8BKYAXQCrza9edp4AUgWfKqjTHGGGOMMabMbAZMAhYAqwAdwJ//AncAJ+I1ZsYYY4wxxhgzZAhwFHAf0MnAmqve/nwK3Iw3OmaMMcYYY4wxg1YAOAH4F8Vprnr78wCwTwn+fsYYY0xBiN8FGGOMqRhfBa4hvxGmj4F/4629agU+xJtW2I43JXAksDHeGq5tgO3x1nflcjvwM+CtPtZujDHGGGOMMWVlBHAVkCD7qNMTwIXAbngjX30xHDgIaAJez3GdT4FzsJuFxhhjjDHGmAq1HdmnCS4HongjU4UiwF7ATUBHlmv/CdiggNc1xhhjjDHGmKI7Gi+CvbcGKwysV+QaRgNXA2t6qaMNGF/kGowxxhhjjDGmIE6h9+ZmLrBJievZEXi8l3o+AvYucT3GGGOMMcYY0ycX0nND8z5wqI91CXA+PU8nXAl8w7/SPhd5PbJXZGlke7/rMMYY4y9bSGyMMSbdGcD1rP37YRHwA+Dtkle0tq8B84AtMo6vAA4Enil5RV2u+8d1wz4a9dFLQD1wRyARCE8eN7nFr3qMMcb4xxotY4wxKUcBd7B2YuDdwHF40ezlYkvgL3hhHek+xAvSeK3kFQGRtshEUbk87dAa4LcBCVwyuXbyMj9qMsYY4w9rtIwxxgCMBZ5l7XCLOcBpQGfJK8ptQ+BBYJeM4/8C9gBWl7KYWEtsfRVdjLc/WKYViv5KAjIrPDq8vJR1GWOM8YfjdwHGGGN8FwT+yNpN1nzKt8kC+C9wCN6myOl2AC5f+/TikmEyUpC/4q0Zy7SOIFNJsCTSGmma+erMjUpdnzHGmNKyES1jjDHT8QIw0j0NHECJR4X6qRb4O5DZvHwHb9pjScVaYjuo6EXAMVlOWwn82qlymhu2aninRKUZY4wpIWu0jDFmaBsP/BMYlnZsOd50vCUFeP31u15rR6Am7XgH8MsCvH7KoXgjcOkzNd7EW8O1soDXyVvTkqYDnKQTxZvG2JtVKDc4SWdmw7iGt0pVmzHGmOKzRssYY4YuAf4K7JNx/LvAn/rxeusAOwG7pv3Zlp6nqa8A1u3HNbKZAVyQcSyGt7GyL1RVokuiR4mKC3wly6lrgNsEuSxUF3qhNNUZY4wpJmu0jDFm6DoMbxQo3e1kn/LWm58DU8l/7W8xGq3hwEt4UwlT2oE64N0CX6vPoi3Rg1W0SZDdc5y6SJDmhtqG+SLS03ovY4wxFcAaLWOMKQPNS5q304Tur6L7iSM3h8aE7i/BZRcBX0/7egWwPd6Uu766HJjYh/OL0WgBHMna67JmAJOLcK1+iS2JHapJvQjYM9t5gjwP/HJEYsQfJ46bGC9NdcYYYwrFGi1jjPFBdGm0jiQHo+wN7I+3L1TK7HBd+Nwil7Af8GjGsYFMs8vWaC3BS+JLH2kqVqMF8DjeXlrp19oKb+1Z2Yi2RA/GIYxyYI5TP1D0hmQyee3UsVP70wQbY4zxgTVaxhhTAtHXo2MlIAep6gF4jdUmWU5/IVwX3qnIJf0OOCHt64FOsUs1Wu/i7ceV+vM08D4QBUJp5xez0ToUuC/j2NnA1UW63oA0tTXt5KgzCW9T6GFZTk0Cjwgy26YVGmNM+bNGyxhjiiCyOLKxBGR/4GBBvqFoba7npEkGhwU3nrTlpI+KVN66wDJgRNqx64CzBvCadcAnwIe9PF7KRgvgBbykw5SnyDFVz2/TWqeNrtKq81T0dHL9t1H+o47ewDDmNG7R+N/SVGiMMaYvrNEyxpgCcF9y16mpqdlPHT0I5WDgy/T9PbZT0eccdR4NaOCyC8de+H4RSgU4Gbgp49ieeM1IsZS60ZoEXJZxbGtgcRGvWRBum/uFIMEJKD/Fm/KYTRzhThLcGKoPLbRRLmOMKR/WaBljTD+4C92q4JbBrxDg4K7Gal+guo8vk0B4HmUR8Hhc4g+5te7/Cl/tWv6AN00t5VW8GPZiKnWjtSleqEdV2rFzgCuLeM2CmqtzAy1LW46UpJyt6IHk+J2toi2SlF87w5ybbRNkY4zxnzVaxhiTB1WVyJLIVxx1DkI5CGFfYGQfXyYBPIfyiATkr9Iujzds2/BpEcrNRoB38BqRlFKk8pW60YK1UxXvBL5X5GsWRaQlsrUgp4nIBEW/kOP0JPAkwi3xVfFb3fHuilLUaIwxpjtrtIwxphfNbzRvnuxM7gUcjHIYwpf68TKtwAJgQXBY8OEirrvK1/Z4e02l+zZQ7Dh5PxqtS/H29kpZDmyE14hUJHexu17QCf4I4Qxg5zyesgK4I6nJOWvq1ix0xa3Yv7sxxlQaa7SMMaZLZHFkY6mSA0XlIODgPgZYpLwLPIzwsNPpLGgY1/BWgcscqOOBW9O+TgBfwPtAXkx+NFoH4TW56cYAS4t83ZKIvh7dFeEMhB8C6+XxlLcQ5klCbm+ob3jS1nMZY0xxWaNljBmy5urcQEtLy05d66yOwAuEcPr4MivxpmktIMGCUH3ouTL/AOsCl6R93QKMLcF1/Wi0NsFLV0z3TeChIl+3pGYsmzFyzco1x4rI6XTfPyybt1DuEOT29rr2J2ykyxhjCs8aLWPMkJK2UfDBeB+61+/jS3QivEDSmw4Yb48/5o53OwpfadHcijeqlXIfcFgJrutHowXedMH0NU0VFYjRV7GW2Fbq6PEoZ5B/A/0hcL8g89qXtt/vHuB2FrFEY4wZMqzRMsYMajOWzRjZubJzT+Bg4AiE7fvxMp+ts4on4n9xx7mfFLTI0lqIt2FyylXA/5Xgun41Ws8Cu6R9HQGmlOC6vou+Ht0Vh5OAH+KtTcuHNV3GGFMg1mgZYwYVV10n2BLceYCx6581VlqtjwyyDWH/Aeya9nUT3QMjisWvRiuzsbwCmFiC65YNt82tCSaDhyEcgzd6uU6eT31PRe8UlT+NTIz868RxE+NFLNMYYwYda7SMMRVvetv0TTuTnYeKyCF4I1cb9uX5gvxP0YcRHgQeDNeGlxSjzjLxKt7GvSlhIFaC6/rVaN0NHJn29U3AqSW4blly29yaGq35hqLHAEeRX4gGwGpgkYjMJ8ldofrQG8Wr0hhjBgdrtIwxFSdj1Ko/IRZJhH9W8DqrgWgF0tMUzwdmleC6fjVac4Fj0r7+A95UuiEvo+k6kr6tV2wF5qPcO2r5qL9O2G3CmuJUaYwxlcsaLWNMRZj56syN1gxbcwDeiNWRdN9wNx/vAY8JMr96WPX8MtjPyi//Ar6c9vUlwC9KcF2/Gq37gW+lfX0DcGYJrltRZi+eHVwVWPXN/jRdin4kyMMIC5KSvHfKmCnvFq9SY4ypHNZoGWPKUg9rrfYHqvrwEquBRRUUu14qTwJfS/t6BjC5BNf1q9F6nO6R55cBF5TguhVr9uLZwRXOim+IyPfx1nTlG6QB3r5sfxfkPhFZsHrJ6n9YoIYxZqiyRssYUzYib0U2lLh8E+HbwCHAxn18iX8L8oAgD66W1X9za932IpRZ6TJHeG4BTi7Bdf1qtF4DxqV9fTFwaQmuOyj0ME336/Tts0Ol7TNnjDEFY42WMcY3qiqxltgu4sihin4b2B0I9OElVgAPi8j9JLnfFujn5Srgp2lfP4W3xq3Y/Gi0qvE+6KePhJ6At5eY6YfI4sjGTsA5VNHD6d8+dB8AjyIs6JTOhy4ac1Fb4as0xpjyYI2WMaakZr05a3i8I74XwhHA0cAWfXyJVmCBIPNHJEY8aJHTfTYRuDzt6+XAqBJc149Ga1vg5YxjX8WLuDcDNFfnBlpbW/dU9HAcDka7bRuQr3eBxxEWJKoSf5665dS3C12nMcb4xRotY0zRzXh9xhcTTuJbXXfBDyX/fXwgba2Vonc31ja+Upwqh4xDgAcyjm0NLC7ydf1otE7Gi3NPUbwRmE+LfN0hafri6fWdVZ2HicqhwD7AyD6+hAL/VvQxR5zHJSCPNWzV8E7hKzXGmNKwRssYUxSxxbHxWqWH93Ndx2cbBsdXx+93x7srilLk0PQF4EO6T9E8C7iuyNf1o9G6GTgp7esXga8U+ZoGcBe6VcEtg19JC7PZBwj246XeBR4XkUWa0MdtjZcxppJYo2WMKYgBTglsp2v6kDp6b+Poxv8Up0rT5Rlgt7Svb6f7XlPFUOpGS4A3gS+lHbscOK+I1zS9cN9xRwRXB78OHNw1zXBn+rb3Xcr7wN8RHpekLNpg+QZP2x5exphyZY2WMabfmhc3b6FVepiqHgEcCAzvw9PfAeYLMr+9pv1hd3N3VXGqND1opnuk+ypgM+CTIl6z1I3WfsCjGceOAu4p4jVNnlL74qnqQSJyEDC2ny/1CcoihEVJJ/nEmpVrnrERcGNMubBGyxjTJwOaEqj8B7hXkPkNdQ2LbAqQb3YHns44djrwmyJes9SN1o14f6eUT/CaSWvoy1CsJbZV0kkeIMi+KHvjrRvsj07gX4o+4eA85SScJyePm9xSwFKNMSZv1mgZY7JyX3Kra0bUHJBMJr8rIkcAm/fh6auBhxHudQLOfFvYXjaCQBte45Hyd2CPIl6zlI3WKGBJxuv/HvhRka5nCqyptWmTAIHdVXUvhL3xbg4M6+fLvQc8g/Is8Hh8ePwJG0E3xpSCNVrGmLXMenPW8I41HQcreowgRyj6hT48/QO8VLt7nQ7ngYZtGyzhrXx8EW8Prf8DNuzh8W8CDxXp2qVstFzgkoxjq4EbgFnA0iJd1xSJu9hdr9qp3lsc2RtlH7yY/v6EawCsAZ4HnkR5qjPQ+ZTt52WMKQZrtIwxAETeimzodDiHdUWwf5u+RDPblMByNxY4B/gx2dfRPYa3tqm/FuClGvbkS8CmaV8ngX9mea1G4MF+1PAFvNTKDXp5PAncB0xj7emTpkK4bW5NtVZ/tWuq4V4ismcfbwhleh94WpC/C/K0qv49VB/6uFD1GmOGJmu0jBnCprVOGx2QwCFd660OIf+pOauBRSIyXzrljoZxDW8Vr0ozAPsCFwCHkX/C23HAH/t5vQ+Ajfr53EzHA7f143lXAmfnee7DwEzgL3h7OJkKFl0arZOE7K3orgh7DSDZMKUV733uWZI8OyI54hnbIN0Y0xfWaBkzxDS1Nu0YkMB3UL6r6E59eOoHCPeKyp+qh1UvmLTlpNVFK9IMhIPXWIWBPXOcuxRvzV16g/0OsB39SyD0u9HaBW+tWfoeYavJnYa5GLgKby+x9j5e05Sp5lea19Vh+hV1dK+ugI296H2kMx9rEF5EWSTIsyR4NjQu9FKByjXGDELWaBkzyLnqOsGW4M4IRyAcB2zTh6e/ATwgyPwNPtrgAduvpqzVAMfiTbnL9W/8LDAbLyCiEfhFxuN/xBvZ6is/G63hwJOsvSHxQUAH0IDXgGb7vfcecC3ef5uP+nBtUwHm6tzA4rbF4wMa2EtF9wS+Bowb4Mu+KypPA0+ro085cedZW5dqjEmxRsuYQchd6FbVbFXzNRU9Bm8j2s1yPecztt6q0uQKuEhJrU2KAYvSjlcDLwDbZpz/Y7yI9ErxW+CUjGOZSYM74k0rPJnsQQor8KLuLThjkHMXu+vVODU7po167QFsPICXTKC8ivBsasphe3v7393xbkeBSjbGVBBrtIwZJNw2t6aGmkNU9RjgCGC9PJ+aBJ5U9E9Viaq7bM+ZipFvwEUcmAtEgFd6OWc/vPVK6VPu2vHW7T024EqL7zzglxnH3scb3VrWw/mbAmcBE8k+lSzVnDYBTw28TFPuVFWirdFxCHsIsjte47UT/Y+WB69x/4eKPgU8raJPTxkz5d1C1GuMKW/WaBlTwdw2t6ZGa76h6DHAUeTfXCWAp0RkngRknu1vVVH2xmsQjqZ7Y5TpA7xRmcuBfD7UXYIXi57uE2B/sqcD+u04vJGr9NCDJHAouVML1wVOAyYBW+U4dxHQDMzHgjOGlOv+cd2w5esv31qruka9lF0RtmNgn6HeBZ5FeFySsqjdaf+HW+va+kBjBhlrtIypMANorlYBjwgyD+Vuiy6uKKmAi0a8dSXZtABX4O0Z1ZdNWQN46XsHZRx/Hy/u/9k+vFap/AivmcwcbWgCpvbhdYYB38FLaNw9x7mp4Izr8YI2zBAUa4mtr+hXgb0RdsULnsk2dTcXC9owZhCyRsuYCjCA5uq/eFOf7q0aUXXfhZteuLJoRZpiWAf4IXA+sHWOc9MDLhL9vN4GeFMFv5xx/FO8EbQF/XzdYjgfmMHav8duA07AG9Xqj73JLzjjfeAaLDjD4IUODV86fJukJvcgyR4IewA7AFUDeNm3BHkqqcknHXWeGqEjnrV4eWMqizVaxpSpWW/OGh5PxL+FcgzK4XjTnPLxDnCnIPPqausWHSvH9vdDt/FPXwMuosATBbr25njT5MZkHO/EGyn6Bf1vYgphHbxkwBN6eGwh3pTBQnwYTQVnnISX6NibFcCtwGXAawW4rhkk3HfcEdXt1bsKsrsgX1N0D2DLAbxkO95ar8edpPO4VukT4dHh5QUq1xhTBNZoGVNGBjBy9SFwvyDz2pe23+8e4HYWr0pTROPwmqtCBFwMRD3eNML6Hh57EDgTf9L4vo6XLtjT6N6DwPfwGp9CsuAMUzBNS5o2CyQDu6l2barsfU+PGMBLtgKLEB6XTllk0w2NKS/WaBnjM2uuDPlPV/sAuBq4Eu/fv5g2xWscdu7hsVXANLz481JMZfoi3qjdqfT83+fWrseKGaHd1+CMy4E76f80TjMEuAvdqpotarYpYNBGt5CNDZZv8LTtf2iMf6zRMsYHsxfPDq6oWnEoSX4gIofjTYfKTXkb4Q5Rmdde1/6EK66fU7jMwJQi4GKg1sPbS+uYXh5/G5iJFwxRjLo2wwuomACM7OHxBN5UxmmUbjpj6t9tKrmDM17Ha4otOMPkram1aZOABL6WJPk1UdkT2I2ev//z8QnwN2Bh0kkuHDd63As2ndyY0rFGy5gScdV1alprvt61ifDx5L8p5mcjV3W1dffZL8mKlwq4uABvqmA25TIycjbeGqTeNvn9CPgjMAd4coDXqga+hbc26vAs13wXb53WwgFebyD6GpxxBV5AjSk4rQY2wntf3Qj4GPgQZImfVRXCXJ0baH29ddvPRr1gX2B0P19uBfAUwgISLIjXx/9pN+yMKR5rtIwpstji2Hh19BiEE4G6vJ5kI1eD0SbAT/A2GR6V5bxiBFwUwvZ4seb75zjvXeARvAboRbyAiGxbCWwCbAvsChyI9yEyW/BLEi/SPUzxp0/mawe8tXW5gjNW4qVCWnDGgOmOwD54o8F70vN6QoD/4W3GfSPIAyUqruia32jePNmZ3AsvXn4vlJ3pvpdcvj4Fnk41XqH60HMiYvvEGVMg1mgZUwTTXp+2ZVWg6miSnKSiu+T5tLeAO625GnRSARdnkv1DeCrgogl4tQR19YfgjcZFyL1OKd17eB/oPsZLThsOfAEvUXH9PrzO08DPGPioWbH0tZmOUL5/lzKnS+j7qM4DwKkgywpfj79mvjpzo45hHV93cPZR0a+j7IY3OtxX7wGPishCRf8Srg0vKWylxgwt1mgZUyCRtyIbOnHnWBU9AS9JKp+fr3dR5qrqvI76jietuRpUBvO0smF4018bgW1KcL1FQDNwbwmuVQjrAKfjNYW5moFymR5aYXpstJbhNfOdeE38pj088TVgH5D3i1qez2a9OWt4x5qOryLsr6oH4I38ZbvR05tXReV+HB6orqp+bNKWk2ytoTF9YI2WMQOQlhh4Il5iYD53ED8G7rG0wEEpFZQwBdgjx7mDISghABwEnAh8l/4v2O/JMuAPwC3A8wV83VIaat8PJaQteCmcD+FtpP0cyKcZ52yGt45vCt4IasptIMeXps7y4C50q4JbBr9CgINRDsabdtnb+sferMaLkl9AggXhseFnC1+pMYOLNVrG9NFcnRtobW3dU0VPxLurn89GwnHgIUHmBUYE7rhw0wtXFrdKU2I2guE1WfvjrbM6APgy3shXvlYBz+Ct73oEb0rdYPrvM5hHOH2gI0HyfB/VbfHWO6b2QUsCW4G8XZzayp/7jjuiZnXNLuroXl2N1770caqhIG2KPgQsEJUHQ/WhbGsxjRmSrNEyJk9doRYnIpxMz1NSMiWBJ0VkXrIzeWvjuMYPilyiKT1bk9O7YUAtXtDFl4Cd8NappTsXeBlvOtcbwFBYhD+Y1uxVEA3hBcyknAQyx69qyk3zK83rJoYl9hGR/REO7Ee4Rocgf1P0/kAi8KfJ4ya3FKtWYyqJNVrGZDG9Zfq4Tjp/JCI/BMbm+bQXgd8lkonbpo6d+mYRyzP+sZS5vtsPeDTj2ChgeelLKQuVnkJZYXQP4Km0Aw0g0/2qptzNfHXmRmuGrTkAOBhvm4XN+/QCyn+AewWZ31DXsMiSDM1QZY2WMRncxe56NYGa73StuzqI/H5O3gLuTDrJm6eMmfJccSs0PrLpX/1njVbPUvuqnQ9snePcZ4HZeM37YJpWWQK6M5D+3nwuyGy/qqk0scWx8Vqlh3dNM9yPvk0LXgLcY4m6ZiiyRssYum0mfCLe4ul8FvUvB+aj3BKqCz1cHnfsdCSgIKv8rmQQSQUaTAV2z3GuBRr0zhqt7FLfZ414CXHZtOA18TfgrW0zOempePuvpewH8phf1VQy9yV3nZrhNQcoejhwKLBlH57+IXC/IPPaV7f/xR3vdhSnSmPKgzVaZkiLtcS2UkePF5UJitbm8ZR2vISrW+Kr43f7+0tCNwa+iTe1Y2e8DWVTdxk78KaqPQHMAXnclxIr27rAacAkcu8ZNVgDLgrJGq385Tty+gFwNV5zXy6bN5chXQ/4O59vRfAasB2IjawUQFNr044OziF4369746WR5qToR4LMV9G7Ouh4wK1124taqDE+sEbLDDnuYne9YCB4LHAKsFceT0kADwvye+mQuxq2bfg05zOKSg/BCxH4Jnn+QgMew9uos7VoZQ0emwJnARP5PKWsJ6m1M010X/themaNVt+lgjN+jLfJc29SwRkR4JUS1FUh1MFLwLwM+ErXwXbgmyB/862sQSzyVmRDp8M5TNFjgG+Qf4T8KuDPgszZ4KMNHpiw24Q1xavSmNKxRssMGdHXo7vicCbeeoh18njKqyi34XBTuDa8pLjV9YXeBXynH0/8GDgIxPY+6dmOwNnkDrhYAdyKBVz0lTVa/fdF4Kd4TdeGWc5LNf8xvFHWIUIDeHuupQTxbpKMp3vQyBvAiTZlsDTcd9wRNe01B3U1XUcB6+XzvK6RrttFZY4FaZhKZ42WGdSiS6MbkOQYlLPxPkjn8jFwT3mtu8rUrdH6EJiH9wH2ObxNXpN4sdpH4k172yjtye8C24J8UqpqK0C+07TeA67FCyP4qAR1DTbWaA2cBWf0SKuAbCMgy4GLgRtA4qWpyaRz29yaGq35BsJ3VfVIst8w+IyKtjg4t3Rq581T66YuLXKZxhScNVpm0HHVdYKtwQMRzsS7i5ZrE8Yk8Iggc9pr2m93N3fLfHG53oXXSDUBd4NkWSemm+OtKdsu7eA0kIuKWWEFGIbXrF5A7oCLxcBVWMDFQFmjVTip4IwwsGeOc4dAcEbORgu8dau/BxpBlhW/JtMbd6FbFdwyuC8O38V7H94ij6clER5V1Zs6ajruKP/f08Z4rNEyg0asJbaVoqchnEru8AKAlxFudgLOnIatGt4pdn2Fo+OB/0C+o226Ld7eXqmgjBaQfPcEG2z6GnDRDMxnaGykW2zWaBXH3njrCY8m+5rND/BS9y7HG9keRFSA76cdcICN8TbJPrLr/6e8DxwC8nzp6jO9cdV1atpq9k5q8oci8n3yG+n6BJirjv6mcUzjUNkA3lQoa7RMRZurcwOtS1oPV9Wz8MIhcu1kv1JE5pLgxtDY0BDa/FPvB76VdmADkP/5VY0PLODCf9ZoFddYvM2PLTijG63GG/m7mM9/P7wN7AhiU4DLyFydG2hpbTkA4SS8ka5183jaKwg3BauCN0zacpL9e5qyY42WqUjT26ZvmiBxMspPgNE5nyA8K8ic6qrqOUPzzVhn4q3rSNkaZLFf1ZRQKuDiZLKnX63Au9s/Cxhy6wCa32jefJgMWz5py0nFnBpZVo3W9MXT6yePm9zix7WLzIIzeqTnAzPTDkRApvhVjcluxrIZI9esWnO0iJyCsj953EQF/uCoc11DfcM/il+hMfmxRstUDFWVWGvsoK61V98h9870HwN/xOHa8JjwP4tfYTnTa/BGdFK+CPKBX9WUgAVc5MlV1wm2BR8FvqToKY11jcWKvS6LRstd6FYFxwTPR/m5ik5orG28uZTXL6Ea4Fi8DZC3yXHuEAjO0ADehuJjug60gdT5V4/JV7QtOgY4WVROzmu/S+FZRa9Yp3Od2yaOm2jhJ8ZX1miZsue2uV8IEjyWJOcibJ/zCcKzwPXxYPx3tmA2RZ8Fdun64r94jdZg26wzFXBxIfDVHOemAi6uw9tXZ8iKtEQmichlXV8mUGaOTI68pAgfUHxvtKYvnl6fCCR+C+zTdehjJ+F8uWFcw1ulqsEHfQnOaMVruAZpcIb+Gm+NZsr6lsBaOVRVYkti+6GcChxD9imyAO+hXBtwAtdOrp1sASjGF9ZombIVWRLZU5LyU7xFztn2NQL4QFR+6ySd6wfpdKAB0N2AZ9IO3AJysl/VFEEq4OJ8YMsc55ZNwMWMZTNGjt5kdPuxcqxvIwjTW6aPS0jieWBE2uFPEyR2KEKUsu+NVrQlGkaIZBy+N1wXPrJUNfgs3+CMD4Ff4zVdFRQUlIvOwEsaIpjFdQAAIABJREFUTfkSyCD6+w0dsZbY+uroD/LcuqUDuNsRZ1ZDbYOtvTUlZY2WKSvuS251cHjwKITzUL6e8wldo1fBquCcIq8vqVDqAI/z+Z3sJLDrIEncSgVcnAt8Ict5qbUo04CnS1BXVtHXo2MRzkA4Q0XPaKxt/JMfdaRNGdyn2wPCT8K14WuLcEnfGy13oVsVHB18gowRT0VPaqxrnFOqOspAPV7DlW9wRhR4uQR1FZnOAX6U+gIYbvtqVb5Ia2QfQSYA3yP3TdlHEZrDteEHSlCaMdZomfLwWbhFknMQvpTj9E+A25Ikr5pSN+XFUtRXufTneGlbKdeBnNXb2RXiK3ijV8eRfZ1e2QRcpO3tdi7d1409GK4LH+JHTdGW6M8QZqUfE5WFDXUNBxVpo27fGy2A5iXN2yWTyefo/oFsKEwh7ElfgzMux9uXrwLpMLz3gc26DiwFGeNXNabwIm9FNmQNp4vK2eTYvkOQ5xGa68bUzfNzVoEZ/KzRMr6Kvh7dFYdzyf2h+fO1V6vit7rj3RWlqK+y6feAeXz+c/4y8FWQlf7VNCAVF3DhtrlfqKHmZJRze1nEraq6bWN942ulrCv6enQsDi/Qfcrgik6nc8eLxlzUVqTLlkWjBRBrjU1V9NKMw/eE68JHlbqWMhEEfkDfgjNuBTqLXFcvVPLfR/Cz57jAJWkHrgL5vwIWZcpE195chyk6ETg427mCtCFcXl1Vfb3NijHFYI2WKbnZi2cHVwVW/SBJ8hxBdstxeruI/EGTek24PvxMjnPNZ/RAvHVIqWlBy4F9Qf7tX039Uo3XhF8A7JDj3H8BVwK34HPARXNr8+5Jkmfjpb7lmspycbgunPmhv2hUVWJtsQfJ+AAiKj8N1YeuKeKly6bRche6VdWjq5/MfP8R5AehutDcUtdTRlLBGSHIOXU7FZxxI160dgnpb4AW4Le511jpOsDP8TYpT4kDOwyRLS6GtOaW5t2SkjwX7724Osupy0Qk1k77dW6tO6QDkkxhWaNlSmbmqzM3WjNszU/w9jXaJMfpb6JcM6xz2A0XbHPBhyUobxDRPYEHgXW6DqwCDgF53L+a+qziAi5mL54dXBlYeSTemrG9cpyeBB4Brq+vrb+zlFNXIq2RHwtyffqxIk8ZTCmbRgugqbVpRwfnGbp/+Ppw2Jph29l7DgC74n0v/5DswRkfAzfj/QyWKFhC7waOxPs5ehr4O/AfvETVj/E2Jf8SsBtwBLBexgucDzILM2Q0v9G8uXbqzxSdQPaNkN8RJDYiMeJ6i4Y3hWCNlim6rkjlicAZdJ+qtDbhWVGZvcFHG/xhwm4T1pSkwEFFdwYexvugAV7a0pEgf/Gvpj7ZDJhA7oCLBHA/cCnehyzfTGudNrpKq36qoqeTfZ0LwIei8ps1gTXXFnGKXq+aljRtFkgG/qNo+n/bVQTYITw63Frky5dVowUQbYm6SLfpZKDcFK4Pn+pTSeUoFZyR6/07FZwRw2t6iuizRquvOoGpIM0FLshUCHexu15NVc2pqjoZ2DzLqe8jzIoTv9xGuMxAWKNliiZt/VWuO6Jx4B6LXh0o/QreKMmorgMdwPdA5vtXU952wpvak2ut3qfAb4HLgDdKUFev0r6/jweqsp5cJnu7xdpid6jq0RmHLwjXhS/r8QmFVcBGS9fBa8rXwwvH+QDkf319Ffcltzo4IvgcyviMhw4J14Uf7Htdg9rGeLMRzgY2ynJeCYIz9AzgTLy9AbP9bklJAA8AU0BeKE5NppLMenPW8I41HacpegGfb2LdkzdU1e2o67jZFXew7T1pSsAaLVNQny1CFQ3lEc/+Lsr1wzqHXWlTdQZKxwML8T4MAawBjgO507+a8pJvwMUyvM2FL8fHEZDPth/wmsKv5Tg9DtyjSb28cWzjouJXl12sNXaEovekHxPk+fal7V91D3BLEWowgEZL1we+BRyC9z1Tj7eeKN3reB/srwHJO4000hbZQ1QW0f0D+5L46vgOpQrdiS6N1pHgclH5Q2Bk4O4LN72wnANrUsEZYWDbHOc+h/czW6TgDB0FHIgX1z8WrwEciZc4+gneOq7ngAUg7xb++qbSXfeP64YtH7X8eEUvwvse6s3LwCXhuvC8EpVmBglrtExBzFg2Y2RideKMLOlq6Z5BuCy+JH5HiT7gDXK6Nd4H2FRscQI4CeRW30rKLhVwcSHw5Rznvghchc8BF02tTZs44pyS1/YDytvAjZrUqxrHNX5Qmgqzi7XE1lf0pYzaO3HYPTwm/M8SldGPRks3AK4GjiL7fk/dnoTXlF+Qb8JmtDV6BV7EebrLwnXhC3o6v9BirbFGRZu6vmzHaxhvia+O3+2OdztKUUM/pIIzGsi9JrENr+HyITjDmNxSDVdSkheLSn2vJwpPSEIuDI0NPVHC8kwFs0bLDEjzK83ralBPU9UQ3gayvUkCjwgyO1QXurdE5Q0BOhb4K5/PNU8CJ4P8zr+aerUecCpeguAWOc4ti4CL5tbm3ROSOFdUvk/2xCpVdAHK7I66jvvKbYpJtDV6A94am88oGmmsa5xSwjL602htDbzay4OrgA/x1iP2tLj9r8C3QXJO1ZyxbMbIxKrEvzJuEiXV0b0bxzQ+mev5AxVtjb4A7NjDQ8uB+YLMq6utu6+M9/vpa3DGdODtEtRlTJ+4L7nVNTU1p6tomN6DmBThtoQmwlPrpvq6R6Mpf9ZomX6JLI5sLI6cLSLnZiyszxQH5mpAY42jG4u8QHqo0dF4H1zHpA4AZ5GRKFcGxgBn4YVcZPteWQP8CZgB+Bblf90/rhu2fMPl31PVc8k9PXCVqs6hitnl+v0daYvsJyoL6f5+/1pwWHCnEu8bM9BGaxVwO3Av8DeQ9zLOOws4h+7r5a4EOSef4mJLYodqUu/LOPziqI9G7VbMYJ5IW2RbUXk554nK2wh3iMq8UH2oXBNE6/AarlzBGR3AHylJcIYxfTd78ezgCmfF2SLSSO8hR6tEpLm6qnqG7cFlemONlumTWEtsKxU9n9y/SD9A+U0ykLx8ypgpNje+4HQrvA+tqTvwCpwNUsx9kPoqFXCRKywiFXAxE3izBHX1KG10dhKwVY7T30W5XoN6ReMWjf8tRX390RU5/zzd19IoSQ4Ojw0/UuJy+tto/Q1vdPNGkE+yX0IPxWvWU6OPncBYkLzuOkfboreiHJ9+TJApobpQJJ/n90fzG82bJzoTZzs4x+cx7TrlZeAPgUTg1snjJrcUq7YB2Bhve4Zz+XxKc08ULyV1Nl4DbUxZcV9y1wnWBC9AmEzv05ff6nqfuKWUtZnKYI2Wycv0lunjEk4ihPIjsk+hek3RmR3SMcciUYtFt8CbFlWXOgCcA3KVfzV9RoCD8D5gHZ7j3LIIuIi2RceQ5CwRmZBjdPaz7Qfal7bfWgnrC6Mt0RhCQ/oxVb2usb7xLB/K6U+jNRxw8l1r1fWcKN6GuynngVyezzO79vr7D5+HygDEHcfZuWFMQ+5RpwFQVYkujX5NkvJDvM1Vv5jnU58WkT8kNHHblLop7+U+vaRSwRkhYLsc5/4T+BVFC84wpv+aFzdvkQwkLwJOp7fpscIj0ikTQ+NCL5W0OFPWrNEyWTUvad4ukUhcLCLHkH3u/QsI0fiY+LxyW58y+OjTwO5pB94G/tzHF7kG5PnC1fRZwMVkWCsqO9MLwCzgD3jTBX0RXRLdmSQ/I/eIWwdwd6VtPxBrjX1F0WfoHpf/blzi27u1bp+j0AugRPtorbWu69cgZ/R2dqZIa+REQTLvTD8Wqg3tX+QNnT/jquvUtNZ8XUVPxPu5ytxwtycJYKEgc9pr2m/3cxuBHqSCMyYCB+c414IzTNlqam3a0RHnlygH9nJKB8JlceK/sJvNBqzRMr2ItkXHoITJdvcGEJXnVDQWqg3dXqoPIUZb+Hw0q7+OBrmrAMVUVMCFq65Ts6TmCFU9H9gnx+nLROSahCauK8ORgqzchW5VcEzwKZRduz3gcHR4TLgQ/+79UapGK4DXHKfi3+8Cydw7LKtoa/ReMkZkReSsUG3ousLUmD+3za2p0ZpvKHoM8D1ybfru+Ri4B+WWUF3o4TJ7b843OOMT4CYsOMOUoa7tMmbTyx5cKtoiSTkrXB8u0l5yplJYo2W66VqDNQVvfn22u/yLBGm2BEE/lEWjNQYvfOAsYP0s55VFwMXsxbODqwKrfqBoPlOYXhORq6urqq+v1AXOkbbIZFFpzjg8L1wXPtaXgjylarQ2xEsjTLkB5My+vMK01mmjAwT+DayTdvgTJ+GMbxjX8FYhquyPWEtsfYSjupqub5Fro2zPqyi3UcUt4dHh1iKX2Bd9Dc5oBmxKlikb7jvuiODq4OSu6dk1PZyiwO+0Wn9Wzmt5TXFZo2UAmPb6tC0DTuACvGS4YC+nKfBnR5ymSppCNfhorvS+fNwF8lo/nrcz5DXdriwCLma+OnOjjqqOc0Tkp3ibmfZGgYeAy0K1oYfKbASgTy5dcmltVbLqX3gbt6Z8nBiWGD91y6l+jgyUqtH6DpB+EyHvNVrpIm2R80TllxmH/xyuC+dae1gSTUuaNhOVH4jKD/E27M0lifCoqNzcvrr9zlJtxpyHjfBmTkzk820qemLBGaYsTV88vT4RSPwSOKKXU5YBE22z46HJGq0hrvmN5s2Tnckw8GOyNFiC3IHyi1B96F8lLM+Uh74EXLwLXI+3qN2PdUAATG+bvmkimTgP4f/o3nBk6gDuRpkRrg/7NuJWKKoqsbbYg2Sug1FOD9eHf+NPVZ8pVaO1AO/7Fbx95epBlvT1VVx1neCS4N9Qvt7t1dFjGusabx94nYUTaYtsK0k5TkROyjO5cDXe/lxzymh/LgvOMBUtuiT6XZJcSS83DAS5vWpN1U8u2OaCD3t63AxO1mgNUW6b+4VgMhhCmEjvkaUK3JuU5CVTaqcUMjjBVIaKC7jourM4ETiTnqdypHwC3OQknBl+TgUrtEhr5HRBbsw4/GioNnRgGYzSlaDR0uPwvgdTbgE5ub+v1rXH1fN0vwm1jADbh0eHfUvK7I2rrjO8bfhBSZInAUeT33quN1B+K8hvQvWhN4pcYj76EpyxBLi268/HxS3LmNxiLbH1VfQXwP/x+TrRdO8r+uPGusZ7Slya8Yk1WkPMdf+4bthHG350KsqlZI8PXoDSOBju8ps+SwVcXAh8Kce5vgdcQLcEwawL7AVpQ7g8MDxw44WbXjioEs2aWps2cXBeBjZIO7wqkAjsWCZ7LRW50dJxwNN8/vdfDuwIMqBGOtIS+bmIXJxx+MZwXfjHA3ndYpv15qzh8TXxw/FuOhxE7t/3SeAR4Pr46vjd7ni3o9g15mEX4DxyT1VOBWfMAAbNjRNTuSKvR/YSkesRtu/llDnx1fGfltEUXlMk1mgNIV0pObOAsVlOW4QwNVwbfrREZZnyUYv3oeZ0ck+3+yNeGti/S1BXr6Kt0W+KSkhFD8hx6jMozfG6+F2DdfuBSFvk913rdT6j6IWNdY0z/aopQxEbLV0feJLuU86+B3LnQF/ZfcmtDtYE/5nxgUkRDqyU98nmxc1bJKuSJ6CcDozL4ynLgXmicmWZTBevuPcmY2Yvnh1cEVgxVZAQPdwoUNEWJ+GcFBobesKH8kyJWKM1BDS3Nu+eJDmTbHHWwhPJZPLiKfVTHi5dZaZMVNRdY1ddJ9gW/F5Xg7VLjtMXJDUZG+zf19GW6MEID6UfE5Xn2t9o36OMNlYuUqOl6wAPAHulHYyATBnY66a92OuRvcSRx0ifCiS8FF8V36VMRn7yFn09uisOJwEnABvmfILwLHB91fCq35fBKPD6wCl405mzBWfA56PtFpxhfBVtiX4V4RZg2x4eTojIL9rHtE8brDcBhzprtAax6S3TxyUk0QR8n97/rV8BLrY0nCGnLwEXS4FrgOvwMeDCVdepaas5DPiFojtlOTUJ3DdU0jFnL54dXBlY+QKwTdrhpCPOXmX29y9Co6XD8TbrTh/RvBrk7P6/Zs9irbHrFe0+XdCbXh0t9LVKYdabs4bHO+LfweEMlAPI8XlAkP8p+jtBbgzVhV4oUZm9Sa0fbYBep2alPA/8EgvOMD5y29yaYDLoIlxID2u3RGVhIpA4YcqYKe/6UJ4pImu0BqG0vR1C9J4k+KGITGtf0n5VGd3xNsVXcR9QUiNYwC/o+Y5gyhrgNsdxog1jGl4uTXX+i7RGLhVkasbh2eG68Lm+FNS7AjdaWg3ciReckDIHOAWk4HeGo0ujG5DgFbqvbV0dSAR2KJM1cP02ffH0+kRV4nSUU4DN8njKMwg3xlfFb/V5jUnF3TAyQ1vX7IPfAFv28PD7wInhuvCDJS7LFJE1WoNM1zqsK4GtejllJcqV8WQ84o5zPyllbcZXqSk3fQm48HXKTVqDNQ3YOsupK4FfJ5KJmVPHTvVtzy4/RFoiW4vIi3S/ofKuqGwXqg+VWwpbARstHQbMA45KOzgPOB6kaFHlsdbYjxSd060S9C+NdY3fKtY1S8lV1wm2Bg9EOBP4Lrk3RP4EuC2ZTF4xZewUv9dEVdQUaDN0dd20uQZvO4NMCZRp9XX1l5bJtgtmgKzRGiQiLZGtEWYLckgvpySB3yedZIMNTQ8pFbeI3H3JrQ4OD54kSGOOPYH+KyKzq6uqr5y05aSPSlZgmVBViS2JLUA5sPsDfD9cH77Dp7KyKVCjpQG8UdZj0w7+yftair61QLQ1+hAZseOC/CBUF5pb7GuXUtOSps0cdU5COROoy+Mpi4DL40vjd/k8S6Li3vPM0BRti56Jcjk9bEUiyMNVa6qOsz23Kp81WhWu+ZXmdZPVyUvw9hwZ1stpf9aATm4c3fifEpZm/FVxd3fdl9zqmuE1xyl6EdmTMf+LcmXcif/KrXWH7BSgSEvkVBHJ3IT4gXBd+FBfCsqtAI2WBvC+X3+UdvAvwFEg8QFVl6euta8v0v3D0bK4xLcbjN+PaaNcJ+Gt9+1t38WUd1DmJKoTV0zdcurbJSixN/0Zxfd1mwoz9ESWRraXpMxFe9yr8k2SfDc8NvxsyQszBWONVgWLtkSPR5hJL+lLKtriJJ1zQ/WhP5e4NOOPitzo013oVlWPrj5VkIvoed56yvvAzPjq+DVDfe+RyFuRDaVDXgY2Tju8igA7hEeHW/2qK4cBNloqeOtr0gMpHgKOBGkfeHn5i7ZFL0Fx048pelVjXeP/lbKOUossjmwsATkJ799gmxynx4F5jjhX+RzKUnEbr5uhxX3JXSc4PHgtXhJoppWKntJY13h7qesyhWGNVgVqWtK0mZN0rqH7+oR0q1Gmx514zK11S/oBxPgiiDfXO0T3fYR68k/gV5RBApeqSqwt9n1yr8H6AOXqeDI+y9YVeqIt0d8inJJxeHK4LjzDj3ryNIBGSwW4Gjgr7eDfgENBSh453jW99Xm6/7wl1dG9G8c0PlnqevwQa4ntraITyWMtl6g8p45e53NEfF+CM94Frsd7rxx0o5SmPEVaIz8WZDZrTyVUYFq8Nu5aBHzlsUarwkRbo8coeq0go3o5ZX6n0znxojEXtZW0MOOHjfDWIUwk+54yCjwMzKZM9pTpSl6aDuyc5bT3EWbFg/Er3M3dVaWqrdzFWmP7Kvoo3d+//z3qo1G7TNhtQjnfhe9no6UCXAn8NO3gIuBbIL6NbEbaIvuJykK6/zv8a9RHo3Yt83+HgprWOm20I85ZonI63UdY16LoR8C16uiVPq8V3hn4GbmnVn8K/BaYCQypoB3jj+bW5t2TmrwT6XG66z1Oh/Ojhm0bPi15YabfrNGqEM1vNG+e7ExeCxzR0+M2TXBIqcO7K3sGMCLLeanF3s3ASyWoK6euu+BNwL5ZTnsP4ZfBquDsSVtOWl2q2ipBhY+k9KPRUsG7QZA+Je9J4BAQ3z9sRFujtwAnph9T1fMb6xtn+VSSb7q+N48CziT31OUO4I+iMiNUH/pX8avr1Ri8UdKz8NZ09aYDuBsvOOMfxS/LDGVdU3Tn4b1nZvp3Ipn49lBL2K1k1miVua5ksQko04F1ezglDjSNTIycPnHcxJIsBje+2RWvwco34GI64Odi9M80tzbvniR5Edmn7HyIMDNO/HKb8tqzaEs0jBDpdlC5Nlwf/olPJfVFfxqtw1l7FPbveN/j+XoL5NQ+nJ+3rrVyr+CNLqd8mhiW2M7nIAhfNS1p2sVJOhPwQkuy3QwCWCRIc0Ntw3wR8SuIYj3gVCw4w5SJ2YtnB1cFVl2l6Ok9PPxOUpKHTamd8nzJCzN9Zo1WGYu2RccAv14rvrmLon8nwKmWJjioVWTARUrzkubtkslklN7XEwIsF2RmYETgch/Xb5S9WEtsKxX9D90jq98jwHbh0eF+bvpbUv1ptI4BBhqb/hpIruCGfou2Rs8Abkg/pqq3NdY3Hl+sa1aKWEtsfRxOUdWJ5I6If01Erq6uqr7ex5FsC84wZaUrAv5K1k6V/lREjgnVhv7iR10mf9Zolanokuh3SfJrYIMeHm5HcOvH1M+0De0GrYoMuEjputN/Md66mt5G31aiXEkVzRXSKPgq1ha7Q1WPTj+m6EmNdY1zentOmRmUjVZXqMsiYM/040lNHjylfsrDxbpuJXHVdaqXVB8pKpOAfXKcvkyQK5LVyesat2j8bynq60FfgjOW4aVhXk6/Nt82JruuNc134I28pouLymmh+tCtftRl8mONVplx29yaoAab8UYwevKkip7WWNv4SinrMiWzMXAa3i/4zbKcV3YBFwCz3pw1PN4Zn4gSpvc1Dx3ATUkn6drm2fmJtka/ibdfVLrHQ7WhfX2cbtVX/Wm0tiCjgemHT0EeGOBrZBVrie2gos/R/abC4pGJkTvYlO7umlubd0+QOF+Qo8k+BXqlojclSc6YWjd1aanq68FOwCQsOMP4qKm1aUdHnft6CMlQoKHME2eHNGu0ykhza/M2it6m6E49PLwa4ec2ijVo9TXgIgaUzZTRtKj26XgLzHuSBO4gQKiM93oqO10hAy/Sfd+iTkF2C9WFXvCrrn4owIbF5SvaGr2C7qEdIITCteFmfyoqb9G26BiSnCUiExT9QpZT1wC3OThNDXUNr5aqvh6MwQvNmADkqvdPeJvAP1P8ssxQ0fxG8+baqX/u8TOi0hyuD4d8KMvkYI1WmYi1xk5S9CpgnR4efiaggRMm109eXOq6TNGlAi5+CASynPcxcDNlFHCREm2L7g/MRNm115OUuxxxwj5/UKpI0dboRcAv0o8JMitUFzrfp5L6a1A3Wu5id71gIPgK3UeiVyGMD9eGl/hUVtlrfqV5XQ3qaao6Cdgqy6lJ4A4NqOvzuuRUcMYFwBY5zrXgDFNQza80r5usTs4FvtXDw1fHa+Pn2F5b5cUaLZ+577gjgu3Bq4GTe3hYEX4VXxUPuePdjlLXZoomFXDRAOyV49w2vLn/NwJlFRTR3Nq8TZLkdODILKc9o+j5jXWNfytVXYNJLwEYy0Rl21B9qCwCT/pgUDda8NkNs5szDt8Rrgt/35eCKoirrlPTVnOYoo3A17KcmgTuc9T5eUN9g59R66ngjAuBL+c490XgKuAWwBJVzYB0zXL4DXBC5mOC3NBe236WNVvlwxotHzUtadrMSTp3A1/t4eH/quopjfWN80tdlymaVMBFGNg2x7nP4TVYZRNwkeK+444Irg5ORgjh/Z168hbCpfEx8RvtDb//oq3Re8jYO09Ejg/Vhm7zqaSBGPSNVtcU2keA/TOOH9ZY33ifP1VVnqYlTQdIUsKCfCPLaQrcizItXB/2e4re3ng3zg4j++cqC84wBaGqEmuN/RLh3B4eu63jjY4T3QPcsvrsMFRZo+WTrsXT8+l5qsTTnU7n8ReNuait1HWZosg34CIJ3If3S3hBCerqs1hr7AhFr6T3KT4rUC6LO/GY7YU1MNG26LdQ7s84/FioNrR/BQVgpBv0jRZAbHFsvAb0n3SPY349LvEd7GeibyJtkT1EZSo5GhhF/+KoMy1UH3q8dNX1KBWccRxrx3GnSwVnXAa8UYK6zCDUdWOnGW9UNdMdoz4adfyE3SbY1gM+s0bLB7G22CGqOpe1ozoVuGLUR6MusB+OQaEeLz0yV8BFHC/CuqwCLtJFWiJbI8wW5JBeTukEfpMkefGUuinvlbK2wWjWm7OGx9fE/033vYc6RWWXUH3oX37VNUBDotECiLZFZ6H8LP2YIBeF6kLT/KqpknXdmLyQ3GtZFySdZMOUMVOeK1FpvdkMLzTjXPILzpiJtxG3MX0WbYs2oMQyj4vIne1L2n9gI1v+skarxGJtsXNV9TLW/mXxiThyXGhMKPMOtqk8fQ24aAbeKUFdfeYudterCdRcoug59H6Hdr6Dc4EFXRROpCXycxG5OP2YqEwP1Yca/KqpAIZMo9X8SvO6yWHJlzOimFcT4MuWuNl/kaWR7SUhIbJHrStwe0ADU8ogQCoVnHE+sGWOcy04w/RbrDV2tqJXsPbn+jnx2vgpNoXfP9ZolchcnRtoaWuZjbeBa6Y3ROXwCr5TbT4PuAgBX89xbive/ldlF3CRrmua4FX08gFBRVtIcp6tIyys6Yun1ycCiX8DNWmH34qvjm/njndX+FVXAQyZRgsg2hI9HiFzI9F7wnXho3wpaBC5dMmltVXJqvPwRo16Wye6BvitU+X8vGGrBr9vZKWCMy4AdshxrgVnmH6JtcR+oqJXkfHZXpAbGmobJlTolPOKZ41WCXQlxPwOOCbzMUGel4Qc0TCu4S0fSjMDNygCLtJFX4+OxeEGMhb0p1kNROMSn2FrTgov2hK9D+HQjMPHhuvC83wpqHCGVKMFEG2LPoxyYPoxVT3Cbk4URrQtOkaTGhKR0+h9xH2Vis4WR6aHR4fL4XvNgjNM0fQ2sqXoFY11jRN9KmtIs0aryLri2++ghz0PBLm9elj1SZO2nLS4BFtVAAAgAElEQVTah9LMwGwMnN31Z6Ms55V9wEWKu9CtqhlTc7aqNtE9Tjzd/E6nc6IFtRRHpDVypCB3ZxxeEK4LZ0tfqxRDrtGKtES2FpEXSRt1UdGWDjq+bDcpCmda67TRAQKNwOn0Pl37U5SrnTVOU8O2DZ+WsLzefAVvSmGu4IwVwG+w4AyTp0hb5DxR+eVaDwi/DNeGJ/lQ0pBmjVYRdW1geS+wb+ZjKhoLjwk32lBuxUkFXPwYGJ7lvFTARRR4uQR1DUhTa9OODs6N9LzVAMDrqnquRVQXz+zFs4MrAyv/DYxNOxxX1R0b6xtf86uuAhpyjRZAtCUaQ+i2tk5UwqH60FqL183AdL2PNQGHZzntHRVtDI8J31Imv3/zDc5I3bSbBjxdgrpMBYu1xS5W1Z/38NDkcF14RskLGsKs0SqS6NLoBiS4j7U3XlQVndRY2/grP+oy/bY3XoN1NNkDLj4Efo23BsvvdQE5zXpz1vB4R/wShPPpeXH5KkGaRiRGXDZx3MR4qesbSqIt0TBCJP2YisYaaxvDftVUYEOy0eqa1fAy3bdEWJEYlth26pZT3/arrsGsua35a0lNRul9+jMIz0pCJobGhp4oWWHZrYu3DYgFZ5iCiLZFIyiZvz9U0ZMb6xrn+FLUEGSNVhF0bUT8IGvvFp8Q5LRQXegWP+oyfZYKuAgDe+Y4NxVwcQOwqsh1FUSsJba3it5A72vLHlPVHw+S0ZSyNr1t+qYJTbxK9y0flsUT8W3cce4nftVVYEOy0QKItkRPQPhdxuHfhevCJ/pS0BARbYkeLEiziu7SyykK/C4ggcmTaycvK2VtWQwDvoO3N1JvMwxSFuMFZ1yHBWeYHsRaYzMUvSDj8BqUb4frw2W9nGGwsEarwJpamzZx1HkEYfuMhzoUPaGxrvF2XwozfZEKuGgEtslx7rN4DdbvgUSR6yoI9yV3neqa6pkiciY9vwcsV9Xzw3Xhm8pkas2gF22NzgF+lH5MkBNDdaHMD+eVbMg2Wl0biz5K92nkquh+jXWNf/OprCGh67/994EmYFwvp61EmTkyOTJaZiP3+QZnvAdci/e76KMS1GUqRNf3/03ASRkPfSLIvqG60As+lDWkOH4XMJhkabJWIRxlTVbZ+yLgAm/j7W/VW5OVxJuysTewG14Mb0U0Wc2tzbsHhwefE5EJ9PyLe35iWGKHxvrG31qTVRqRJZE9gRMyDj/ZUNvwez/qMYUnIorDeXR/nxBBLnfVtd/DRSQiGq4Lzxv10ajxInIe3v6FmUYiXLIysPJfkZZItvVdpfY4cASwEzAHL7K+J5sAlwBL8YKXRpekOlP2RERHfTTqDEUfynhoPVX987TXp+WapmoGyEa0CqRruuBC1v5w/rGiR9hdy7I2FjiHQRZwkc5d6FYFxwTPR7mUnhOulil6jt0MKC1XXae6rfpJQXZPO5x0cPZsqGv4u2+FFceQHdFKibXGrlf0xxmHfxyuC9/oS0FD0PS26Zsmkokowsn0/hnozwSYWIabS28KnIW3XniDLOdZcIbpJtYSW19F/8ba+7i9GE/E9xlEU9TLjt1JK4Cm1qZNnISzgJ6aLNFDrMkqW3vjNU6v4P3i6q3J+gBv4XEt3vB7RTVZ0bbomODo4EKUGGs3WQrMCQ4Ljrcmq/SCbcHTMposgN8MwibLAP/P3pmHx1VWf/zz3kmTtOwF2aFN0rJYwYXFDVFE3ABBwIoKgoCWRbYCzUxSYFiSmbRYpbJV4IcCohRQXBEXUBBBxY1VaLOUfZG9pZk0c8/vj5O0d947SZPZ7tzJ/TzPPDpnJveekM4773nPOd/jZt12g3ndMncke5Ojqc1FlJB5TfNeSLQkvobLXhhGEsI4kCyPpHpTrUtl6WjiR5XmBbTqYhpwBiPLvTuo8uIDrMuKRQfrE5h4S/wNp875NP5/M7s3xBpurrJ/5zVFFGgVSUdfxzYOzp/ylAtqkNXUFp0mVRcO+qXzF+BedIj0SAtMN/plNh2IA89XwL+S0tnTeTzCw2hQafMMwicTzYmvzt1hblTXX2GSy5IbAxdZ5jdjJnZuEP5ElJ+2mW0vu8a1JZe3bKDhvEAcmsAkZiT+EZ8e3weYDTyd5y2TEdLdvd1/7+jrGElMIyjeQksEZ6D+/32U934Y+DnwBCohP1rVRkQN07pj63Mmaz6b57Dn08t7lyeD8GkiEAVaRXDJE5ds4YjzO0bKZEVBVjWxAfANNBv1c0ZXEfwHcAz6d72UkKgIernkiUu2SHWnfmIw1wAb5nnLLQ2TGt4dqQ4FR6PTeAFaCrQWMXJ+FamfRZSBgb6By4CHc4zCqR3LO2yV2ogyM9y/VTelbleEC9DycJv3Oq7zQKo7lV68bHFDnteDZA1wC7A38BFGl3ufCXwH6EWzYlMr4F9ElRGfGX80K9kjsPr9DKY91Z06PCC3apoolVwgyWXJjetj9X8wmD2tl6Igq7rYEjgZ+Caw+SjvG65pT6MzSkJLV3fXnq5xb0EzcTZvYTg70ZT4XoXdivDQ1de1q+u6/yG3lPPxqa9OffecPeeM1PAediZ8j9YwHd0d+zvGyTnkMJg/xJvjnwjKpwjo7O7cyRizhJHmbxkedXBOaG1qfaCijo2P3YFT0DL3xlHetxK4CfgWEI3wmGCkelLfBL5rmVe6uB9ub25/KAifapUo0CqAoQGUd+Ivx6rGIGsqOm1+UzSrI8AqVHnpFfIrMNUC4xW46ER7tUJNqjf1DYTvAvV5Xn4Al6MTMxLLK+1XRC6pntQdwKdzjIbPJJoSvwnGo7JSD2yLBlrft157F5pldivsU+Cku9M/FSOHem3GmEPjTfGfBeVThMphd/V2He3ifttg8mV9BLg6szpzVnJWcmWl/RsH4xXO6EB7umqVrdCh0JsM/W8G3QutRHvfQle5UiwjiPOskKzs1Taz7eVAnKpBokBrnCQfTdY3TGn4KcJnrZfeNpjPxJvj9wTimLI98HG0hOCdaOnbaFkc0PkbTwCPAX8C7h6yhZWxzh15GbgCuAz4XwX8KiuLnl40eWBw4HIR+VqelwcROlqaWy6abWaHQoa+luns7TzUiPmp12bE3B5viX8+KJ9KSCPwIWA/4H3ATmhmtW6Un+lHT9SfRDd6dwP/psaDr9SKVDNZHiU369CTMZlZyaZkNHw2YDr6OraJSewyETlshLf0iSNfbpvedn9FHRs/GwHHAXOBHdfz3vtQ4afRShDDwE7oXuhDwK5Dzzce5f2C9uk9CTyCrkF/onYPogFYvGxxw6rYqruxWikM5g/NTc2fivYLpSEKtMZB8u5kXcOODUsx2BuijINzcGtzqz2noBK0AEcDR7L+4bpjQdD+gZvQIbzPlOCa5cZBA6s24APreW83mi6/mho5werq6drZxb0VzQ7kYDC9IvLFREtitGbpiAqRfDRZ3zC54RFyB6dmYhLbbV7LvGVB+VUkk4FD0XVoP0YvVxorr6K9lNejG56aDLpSPamLgXavzWDa483xzoBcirBI96QPFuQqNCtrM4jQkWnOXJg0yWr/Nzr8PTkf7ekajeXoIeT3gNVl9qtUvBctlzwCPXQulix68PND4GZqdBB0R0/HVg7Og1j/zUTkwraWtvMDcqumiMQwxoiImMZpjVflCbKywNEVDrIMqpx3D7AMHVRYiiBr+Nq7o71KK4DfoJunamRDcgUuRguyQi9wkY/Ons7PCfIAeYIsDL+un1S/ZxRkVQ8NUxrOJDfIArgkpEHWNPTQ4gX0YOYzlCbIAi15Pha4C23ePwc9ma8p6qbUpbAOswRJdD3VlW9THxEA8eb4LzImMwsNOuwsTx2G8xt7G38bgr+ZC/wCeD/rF86YgQpn9KHCGeurjAmKSej3+n+Af6IqwaUIskDViD+MVr48B/wIeHeJrl01tDe3v+g67iFYeyJjzPyunq4DAnKrpogyWmMk1ZO6ELBllwXhhERL4v8q6MphaGC1ewXvCXD/0H2DyNrZbAWchPZgjaacNFx7noIR56WEkqQkncaexpQYOQf/5zgLXBBvil9sjAlz+UdNsXD5wi0HncEn0R4BRXi2boO6nc/Z+pxVwXk2bnZEN19HkX/4dbl4FVgMXIL2VtQEqe7UlzDclGM0/F+iKXF8QC5FjECqN/VpXK7BsF2el180xhwTb4rfWXHHCmc3VChqfcIZq9DMTrUIZ8SAE4AEeuBTKQTdU5yHBnY1Q2d359eMMfZe9gUX9z3tze1hbicJnCjQGgNDAgNLLLMYY06KN8Vte7mYiZ4ef2o97xO03+rPaO/Vf9HTmLfQemOD1ipvDOyAZnh2RXubdhqDH7cCZxJMSeFM9EthrAIXHeh/g5oi2ZtsrHfrrzPGHJnn5VeMMV8J2Zf9hCDVnboSw4k5RuGoREvihwG5NF4moX0e56LCOqPxKppxfwhdg3pR8Z1VaCnSRkPXeAe67uwE7In2CqwvK/YUenL90/W8LxSIiEn3pu9FT8+HcV3j7tHe1P7voPyKyM8lT1yyxZpJa64FPpfnZQG+m1mdOSc5KzlQYdeKYbyHl53o4WsQvB/NMq1vtlkWDYYeQNegJ9Hs+yrgdVSkZwM0W7cjugbNQhUn8wXS9rWvRMswa6aPK9WT+gEadK/FiLm7ubn5gKhfq3CiQGs9dHZ3HmSM+SlWM7fBnBtvjl9cARcMurnpAEaa4ZEF/gDcCPyWwsUstkfLf45GA6+R/n2sRJWMrivwPuNlrAIXL6GL33fRTV3N0flM5+ZmwNxOvgHEhn8ARySaEn2V9itidIbk3B/Cs44I8rdEU+IDIck6vgv4MboRGYkVwA1oAFSomMVktIH9i+gw8U1Hee/twPHUQO9EV0/X3i7uA+Sub39MNCeqtWx7QjOkTPhNQRaQ52BAkAdd3CPmN89fEYB7xbAh+pk6k/Vniu5Dy/B/gu5Byk09KtRxGiO3vWTQ8sib0LLjQoOgnYCD0KBjtHLBF9DSxd8WeJ+qYuELCzcYfHvw7+jh+1qivtHiiAKtURj68rsL6/TWYK6ON8e/UQEXpqJyyAeP8PqbwOVDj2dLfO8m9NT4BGDKCO+5Hp1RVY4ynuHG3Xb0BGs0wti4O266+rp2FVd+JUiT/ZoYuXLg7YEzQnaKOmHIJ+cuyL5tzW33BuTSeDgOPbwYaR34HboBupvSilY0ArOBONYXv4cVqBBQ6GWpUz2pG4GveG2CHNLW3PbzgFyKWA+dKzrfabLmx2gJns0rGI5INCX+WGG3SkG1ff82oYIUe43w+vPAIuBaSj+X7z3AWeg6k0891UXbE5LAYInvXXE6lne8y3Gcv5FbNZTF5ZOJGYm7gvIrzESB1ggsWLagJRvLPgBsYb3085amlsMqkEadAdwJNOd5rR9YiC4sr5fZjy3ROuhvkn+ReQjdQD5fovtV84laYHQu7/ywcczt+P89Zg3mzHhz3B48GFEl5BtOC9ySaE7MDsShseOgPVGnjPD6n1CRinKLrTjA4Wgw5ztkANagweCNZfajrHQt69rejblPkBvQPjH11am71fAQ69CT7E02NkjDcKbFZgDhpAr3cZeaoCtKPoxmqfLNAnsN7R2/Gt0XlZNm4GLgSyO8fieqeFjNs9XGRKo3dSLClZb5mYzJ7JZsSpZ7z1lzRKqDeUguS26crcv+DGtTK8jf6qbUfbkCQdb70B6rfEHWnejp2XmUP8gCXTzPBPYgv6DE7mjAM5b+rtHYCj0RWoGqHY0UZLmoWtKH0C+AW6j1IKu38xjjmLvwB1mrDObzUZBVvSyVpTHHON+2zAO4tAXi0NipR8tv8gVZL6PlMvtR/iAL9DN/C1q+2IEGVl4modn1uRXwpWy0zmx9RpDvWOadX5v62ol5fyCiKkg2JfsTzYnTgdkGY38n12O4Nt2d/vZSWRoLwr8S8Ge0qmZn9OBlpIBmSzToeRb9PJZCCfkgtCwvX5D1A2AXNLCrxNy5HuDLwP7k7/3+FNrCYX9Ph45EU+IqMXKTZd6+URovCcShkBMFWhZLHlwyqSHWcBvi60V4gno+WwF1sD3QEpytLPsAupH4DJqqrzQPAfsCF+EvD2oC7sUvWz0WdkKzUn3oIj1SI+4qtDThneiiX+1DIktCqid1lhFzHbrx9fI8Lh+NN8d/EYRfEWOjp6/nBPxlRZcmZiSC+AyPlUnAbWiflM3daM/C9VR+oOnbaPP5R9ADGS8GVUSr9gB2VAZWD6SwqgMEuWDR04tGEyiIqAISzYlbxJW9EB6zXxMjZ3T3dv862Zscreew2lkGnI4OIL+AkXsjG9A+78fQTNSHCrzfIWi/p12y/Abav3ksehBcae5C92nfz/Pa3sAfqV45/DHjuM7JBtPrtQlyfLo7fWBQPoWVKNCyeGXzVy4FPmGbcTmobfu2cgsszEQVfewJ5i+iQc63CXZaexbNpH0Gf5Pplmi2bZsxXmsfdBH+L1pyMZLS2Evooj4NmEMNqgiORKo31YpKWdvlGo8YMR9IzEj8IwC3IsZI13+7NhKRpNcmyKsNkxrSAbk0FgxwDXqSbNMFHEDpyoQL5a/ocNJ8DegXo6qkoSQ5K7kSQ9Iyb5ZZkwl1ADlRSMxILHfWOB9Av9tsPtkgDX/v7O3cpdJ+lZgX0eqT4e/kkWYAOug6ch+aFfsCKss+FvZFxXfsdoXH0IqfW8flcelZBXwNbXOwM+yz0KqbkXpaQ0G8Jf6G67pHYx2si5GrO5/pDH0gWUmiQMtDZ3fnXCPmJMu8xnXcL1TgBHoLNFDZ0rL3oEHJX8t8//HwW1QC9QXL3gTcwcjSzw6ajXoAzYAdxMg138tRMY7p6KJekyqCI5HqTbUi+DbkgvzOiNkn3hJ/Kgi/IsaO2+AmgK29NoM5f+4Oc6tZJa8DS94X/aI9AxWlqJYy3dfQteRHlt2gfSKhPXVtmd5yLfCwZT51QfeCQioGIipM6y6tb7U0tXzeYBbleXkGwn1dvV0fqLhjpWclWmWyCyp1P5ogzYfRkStPoFmx0caz7Ar8HP/h6wNoANZToL/l4P/QzJtd6fQBVLwj1Hvsthlt9xkxiy3zNmaNidoVxkGo/xGUknRf+jPGmAWWWQzmhPbp7XeX+fYGrTe2G7270SCrGsuM/o2W8NjB1rtRBSIvG6KLay+6gI6mYnQfqjS2C1pSWLMqgiOR7kkvzBdkATcOrBj4bLwlXjNzO2qVrmVd2yOcbpmfmPrq1ErN3SuEA9FgyosA30A/i9XGADo02R4zEUNLG3esuEclYLaZncXlDMtcnzXZVCAORYyb2WZ2Nt4cP8uI+QpW/5DBTHXF/UO6L/2ZgNwrNS6awfsguif4JSNX3rSgPdgr0ANUu59pChqQbWLZ70MrjarxwHVYUdbeqxwEzKu8O6Wl3+lPYHg0xyh8Kd2TrnYxp6ohCrSAVG9qurhyA/60dme8OX59BVw4G/isZXsBba4MukxnNJaTv4zwWLRZfmt0MX0KXVxH2vgMC1x8kAkicJEPETGp7tR3BDk7z8vXZJoyxyT3S4ZePnYi4Na5C7BKR0Tk7CpWj9sBPeyxM8wJVDK5WnHRQNCWQJ+Klh5NqrhHJSAxI3EXhl9b5sPTPel9A3EooiDiLfGbxJVP4J9tOUVc+Vm6O/3lIPwqI7ZwxkgHpe9Ae7KfQQ9FhsspF6OiN14eHrpmufvji+HP6AGx/f18EflmXoaIZFOyX5DjsfZkgly+cPlCuwIrIg8TXt49+Whyw4bJDffj/3DflmnKzE6aZCnnwuRjF+A/5IodZNAP54Nlvnep+BTaW+YN3Ic3lKNtdFaiqfdvo2IYExYRMene9FXoptHminhT/JshGWw74ck3fNaIuTveEv94gG6tj1/hP+z5HtqDEQYmo6qo77HscbS3LHR09nbuYsQ8RO4a+q9MU2bPCnwvRZSQi3sunhYj9hvWBRTDuCJyQltLm52VrRW2AU4FTiS/cuAwWVTgyg5KXkZ7sp4p0o8t0Z6yTYYe/egB8YvogXGpvltPwV/RsxwVRKqEMmLZ6Ozp7DAYu1f0hkRzwi41j7CY0BktETH1k+uvxR9k/SfTmPlqhb7MLsevKHc2hQVZMbQWej7aLPo4qgwkwFvA08A96MZjf0oXaN+JDuzzMomRgyyvwMXpTPAgC6Crt2sBeYIsQS6Pgqxw4eJ+i9zPlisi5wTlzxg4HH+Q9TD4ytdKydbAb9Bhx97HFQVebzXabP+mZT8P7fMMHW1Nbf9F5wN5eW99b/1X8r0/onqZ3zx/RcOkhg/jV8t1jDHXprpTJwfhVwV4HlUC3RE9tHlyhPfF8AdZLqpeON4gaxP08DeJ7k1eRwOqv6FrzK1oBc29Q/68hq5FRzF679hYuBwdi+FlBjqHLNQMrB64AFWf9nJUanmqmg8Qq4IJndFKd6fjYiQnQBDkVRMzeyWmJSrRcHkk/mbuX6Jp8rESQ2uXj0Z7LMYjH/skmtouxaDPGFpHPVr/1TJ0ISrnBPnQke5JnyKIfQoGQleiJWH3zERUMeme9GxBbvbaDObaeHP8hKB8Wg+T0Qb1HTy2DJoZ+m8Z73sbcFge+z+APYu47jH4ZZdvRYOw0NG5rPMdJmaW4e1ZEZ6t26Bu5wqMGokoMclHkxvWT67/icEcYL3kCnJiW3ObHVjXGg66T2lDBSNGYxFw1jivvzvwLwpPIjyLZuB+WuDPg35WHwG299j60dE0vXl/IiR0dXft6Rr3AXLbbJZlTGb3ZFMy1Bm7cjJhM1qp7tQnxMjFljkbI3ZkhYIsB808eVlN/unyo/En9DTmK4wvyAKdYXUDcDuw0Th/1iaLyirn6yG6D1Ul2pkJKnAxEqnu1FcE8Sv4GJJRkBUuFi9b3CCIndldaerMeYE4NDZOIDfIAs14lzPIOpL8QVYpuB6d9eXlcLT8KHS0zWx7WYx05hgN22Xfzp4ZkEsRRZCclVw5sHrgIOBn1kuOwSxJ9aaODcCtSmILZ4zUj/0M2sM1Xuoobl+7HfATiis3fgP/8PRG/EJDoaO1pfVBQa6yzDMbaQy96Ec5mZCBVkdPx1YY8olfJFqbW39XITcOA99Q5A7Gf+Jhq/N4WYOm7h9Cs0kjBTiHoItfsXMfHgbyqaolhq4flb95SC1PfRzDtfgzy1ckmhIXBOFTROGsrFt5EtBsmRe07tj6XBD+jIF6tEzZSy/kVbwsFVtQXgVDQQ+rvJs3Q4jVvwbeHvgOlvKsIK0dPR32UPuIEJCclRzIrM7MNmJut14yCN/r7O7MN8OuFhkWkLg8z2tnoj3cpeBptGTwSjSASgz9742MPJdzHsWtGbcAv7dsx5Kb5QolA9mBNoRnvTYRaevq6do5KJ+qnQkXaC2VpbEYsR9iz7cx5ifxpvglFXTF/hC/Smk2IP9GF5IPo7Lq26KS6zuhGa9P4l8AAD4KdOaxj5dO/E2fod3klIuu7q49cfgZ0OC1i8iPM02ZUwNyK6JAksuSGxvxNQo/k2nMfCsQh8bGkfiVQFOUN+O8mHWzAjOUJ3P2CDrDxssR+IPgUJCclRwwmHbLvKEjTiIQhyKKJjkrOdDf3/9F/JmtScaYpenudKiV6sZBA/Aly/YQWlpcKC+hIltfRAObHdF9z8loVik99L9Ho+Ike6KVQTYXoHL0hWJXMtRT3r7XipCcmXwTx1fS2eDiXiUiE7odaSQmXKC1vHf5hYLsn2MUHotNjn21goID7wT2smyLKfwEZw0qwfw+4L3oQvIXdM6MlwH0ZOeTQL7m/FPwqyKNl+fQRc7Lp7EC24lMakWq2TXur9FAeC2C/G6gf+CYSFEsfNTH6s9BJYvXIZyf3Db5djAejYmvWc+fRUvvysVB5G6qOijfjMCL0TKlYWL4BzGHhtam1luwB8IaTlqwbEExG8GIABnObKGl/14mu8b9WWd3505B+FVhDsJeN/Wwp9C92MOo0uHx6DyuZ0d/O6B9ofuj4yC8NKKl1YVyP/AHy3Y0IR054SXRlLhZkDst88fSPekjA3GoyplQgVZHd8f+BmPXyfYbY75c4cbiY6znayhcbesmtPfpBLQJdCwIcAkqq+6lDv/pUiHYmbk6oNbmhRREsjfZaAbNLdhfLoZ/xAZihydnJe3gOKLKWbh84ZYG4xtOnHkqU4kZfIUyDbBnMl2JZpnKwSaAt7b/Ecoru/44/sz90YRUAMoYI2LE/u6qz8ayySD8iSgNyVnJgUxj5nC0jG4tBjPVGHNH57JOOwipNY62nj+Hlt0VyhpyD1jGShaVoH/Vshdbxvkd6/mW6EF36DHGnIh/ttnChS8s3CAIf6qZCRNodS3r2t4xzo+xfmcxcmK8Of6fCrtjR/13oPMiCiFF4Uo2SfxZtFJMq38S+/S1NAFc6Gmk8QoxktOYL0a6XXEPbN2l9a2g/IoonDWxNUksMRljTGuVD5f+IrlroUtp1EdH4jtoo/nwvebgz7iXGjvQbQb2LvM9y0ZbU9uf8pwifznVl3pvIA5FlITktsm3GyY1HILwmPVSs4mZ25Y8uCT0GZAR2AT/fuOH5BfHqARvoPMEvUwr8pp3oqWMXmpiL5RoSvRhj/WJhHryMiECraQkHbfO/QHaiL0WY8x1bU1tP6iwOzvj74u4ocI+DPMm8EfLZvtWKPam7X3A1BJdO5SkelMniohdrvVa3WDdp9qb218MxKmIorio76ImI+Z4y/zX1umtPw/EobHzCev5vcCKMt1rf3Kz+Jehpc3l5nb8J662rHaoEGQeuSf2DoKtnhsRMubuMPdVHA5E5z15+cirU19dFIRPFWBf/DNEg9oLDWMrTm+EVuQUyhr8/aKlnGEaKJnVmYVipNtrE6S166mubYPyqRqZEIFWfV/92Qi5Q9UMj/Y39H8zAHfs4W5Z8otTVIo+6/nmJbquXXfuAB8r0bVDR1dP196Ir4zAFZGj5s2c1533hyKqnjq3LoW9WTDEq3zAdD3wIeJNs3EAACAASURBVMt2R5nutTHaszm8sXgK/1iLcrEKHdDuZb8K3bsstDe3P4SxNm7CZzv6OkL9e0VohsAR5yDA7uv8Zo3Kvtt7oefQHqsgsQcWv07+kTXjwd4LbY326Yee5KzkgCOOLQK1oZt1LwrEoSql5gOtjr6O9xkx9h/9bcc4XwioUf0j1vN/oB/moNjYev5Kia7bjf+E3O4JmRAsenrRVBf3ZiyFQYSL2lrafh2MVxHFku5Jvxv/INxfJpoSfwzAnfGwF2DX0duzp0pFF7lZ8lOBSpbI3mU9/xAhb0YfNIPtWGWXxjXpSPEr/LS2tD5ojC9DDsLlHcs73hWAS+XE3g/Yn9UgsEXKHizBNe9BM1tePlqC61YF8eb4UrQiYh3CsanlqT2C8aj6qOlAK/lccorjOj/EOnE2Ys5und76eEBu2ScZf877rsphz/J6qoTXvtd6bt+r5hERk1mT+REwPfcF7sg0Zy4MxKmIkiDIJVh9Tq5xzw3Kn3Fgr0Fvowc+peZjaC/WMDcBlS6ptNfXRkIq8z7MudPP7RXkaq/NYPZOr0gfGpRPEaUj3hT/sRFjVz9McWLOjxc9vcjOuIQVB9jVstn7hUrzAfwBUClKGVeiY3e81ERGaxjXcc/ALml2qOS4pKqmpgOthtUN38KWKxfuaG1utSdbVwoHmGnZggr4QPvF3mfZ7DR3MdgzciaCXG0O6b70HPwqQysmDU76aiTjHl66eroOwN/ndEN7U7v9hVqN2IMll1H6BvQpwNWsKxl8FR1CWmnyra/hX4eyXICdGXRJJe9OFtNPElEl9D/Vfw72bCdhVmYw0xGMRyVnB/xlekHuhfYEfkrunvhB9HCoFNT0Xqh9evs/gR9Z5o+le9OHBOFPtVGzgVZXT9cBmJzTVICXXON+LcD+ie3QDYiXJ4NwZIgEuU2ZAtxawuvbU9fzLa41S7o7vSPCAsvcj8vhZ+989v8CcSqiaETEuMZNWeYBYoQlQ2kf9pRjDeoAZniez8WvvlUJ3sAvMBD6TU7bzLaX8UtH71w/rd4eHRIRQpL7JQedOufL2KX8whkd3R375/+pUGGvQVC5vZABNkMP4b+Eysn/ldxZn93A4RTfnzWMvRcK/Rpkk52UbcUSHxKkIynJmo0zxkpN/gdId6c3cXGvxQoiROT4gNXdtshje7riXigfwz/A82Z0KnupsH83wwRSHhQjS/DLfscTMxLlKNOKqBBdfV1fRMitPzdcnpiWsBWrqhVb8KaU5cKgJTinep7fTXkHIa8P+/fLtw6HjszqzAKsINJgLkw+l7QP8yJCSOuOrc+JEXtgrnGMc00NzCqyP4MD+A9ESsU30UPk4YeLZtgfRzNWR7BuL+yi5YJ7U9p1sSbXIC/zd5j/rCC5s1mFWQ09DTUhZ18MNRloiZHFaPbEy+VtLW2/DMIfDxvlsb1ZcS/05OZ6cgPRlUB7ie+T73fL99+g5ujs6Twe+LRlfqB5evNlQfgTURqWPLhkkojY4jpv1WXr0oE4VBi2AE4pxSkagGuB2NDzt4GvoxucoLB/v5pYg5KzkiuNMXZmddv6TH0QaroRZaCtqe124BrLPD37dugHVdufwaBnSAoadO2KHkDbg4uLxf79prBujawZBlYPdGHPhDWcP9FLmmsu0Er3pA/GytQYTG9mdSYRkEte8n3B2wODy80kNHNlB6Kn4p8hUSz5Fs+a2OSMRtdTXdsazELLnJGYHD/bzA5qGGNECXht6mvfILckDqDrnBnnBFEWVygbWs9LuQYlyW30Pg8twwmSmgy0APrf7r8Sa912xEksenrRhKkcqHUy2cxZwDNemyBnDKmehpVqC7QM8GV09t5xlH5vbP9+Br/ya+hJzkquxPAtyzyzfsf6owNxqEqoqUAr2ZvcVESutMwucGxyVrLSAc1YqeRJrwNch19W9Ubg+xXyoeYliN1B9yq0BnwtYuS8tmltjwXkUkQJWPjCwg0EsWdAveQMOIsDcahw7M9gqdag9wBneZ7/B7i0RNcuhmqeaVYUyVnJAUGSXpsgm2YGMvMCcimixCRnJt8UEbvfvE6QyyJJ/zHxNDqr1Pu4l/wHQLuiGfm7gHeU2a+a/NvVTa67DLuk2Zjzk48m7eHUE4aaCrQapOG7GLbLMRoujTfH7aGVQRFkhscAlwNfsex3oaU95aBaSiUrRro7fSBwsNcmyN9mTJ9hn/JEhIzBtwfnktswjRGTbN2lNejT2PFSjgxPHTqYeHhG1SB6MlyqZvJiKGepZOAMNA380GBy1S4Np1+8/GK7aiEipLS1tP1akKWWeZ9UT+rzgThUPPZn0M6yl5KfAQdYj33RyoQtgFOAXutnPgr8Af/aUSj2GitUvpqpIpyz9TmrELos87TGKY1fC8ShKqBmAq2Ovo798AcRTzTUNZS676gYggy0FgAnWrYHgEOA/jLdsxpKJSvGkgeXTBIjdkA14GSd46KSwXAzVIp1lmVevtlrm9n9E2HAPuwoxRo0H3iv5/klwD9LcN1SYG/iairQSpqki2BnWhsd41TTd19Ekbiuezba87gWY8zCJQ8uCeMA7mop530FuALYDfix9dpuwKIS3cf+/d6m9CM1qoaMk7kSu9xV5LwamgM3Lmoi0BoaTHwNualYVxz52twd5q4Oyq88vJLHVolTxxRwtmV7CDiQ8gY+9u8mlL7JtGp4ZfNXTsE/o+jS+Mz4o0H4E1E6+tf0nwVs4rUZTPucPeesCcilYrA/g8WuQbuhoyKGWQZVJXVv/3751uFQE2+J/8qIudtrM8Ycd1HfRU1B+RRRWubPmP80gt372/zq5q+GMVNgfwYbgC2DcGSIVcBRaBbLy7GUZsB5za9BXpJNyX4jptMyb5tZkzk+EIcCpiYCrcbVjefj/zBc1ja97f4g/BmFZwA78Cv3PIUkELdsj6PDVssd9NhBxzNYJ3K1QvLR5IZGjC248pIRUysDJicsnc90bm4wXrlyDObfrU2ttwTlU5Ess54XuwZdAgzX3wswB/86FxQbAdtYtiBnF5YNV9xzLdOkOreuLRBnIspCZnJmAfBcjlFoX7xscUMwHhWMvQZB8LOlsvirFmLA7BJc2/7danIN8rLZa5tdYzB2Sea8kGZgiyL0gVa6O72bGDnTMj9nxJwXiEOj4+JfYHYt4/3OBM63bMvQIOtl/9tLjh1o2UP7aoaGyQ1nYp3ICdIWb4m/EZBLESXCZMw5WKUfLu75AQ4+L5Z8wzOLkRq2Z8Lchh7ijOXxKetn35PnPcVI5+dbX2tyk9M2o+0+QX5nmY9JrUiV4kQ+ogpIbpt8G61Q8bLjqrpVYctqPYX/MKace6Gx8h/86svvL8F1d7Ge1+xeaJg5e85Z4+La/1Z3eG3qa18MxKEACXWglZSk4xr3GtY1YAMgyElVvMF93Hq+T5nucwb++uIe4OPYJ2Ll4yPW85pU3UutSG1mMHMt8+MzmmZ8Pwh/IkrHJU9csgWGk702I+afiabEL4LyqQTYn8Mp5PZXFYNBFTfH+rBPN2N53lNMXb+9vmYIXm6+fDi+g7VJuCWfjxgRIBtkN7gaq/8FYW5SkmHaz7n4g41y7YXGix1obZ33XWNnA/zra03uhWw2zG54PdZ+U5B5E00tM0wfTB8NfQ0nGMzeXpvB3NrW3PbzgFwaC/daz/egdMo2w3wdf5D1NJrJesb/9rIwDbD7A6pF/bG0uJwmyKaW9fxIACP8rJm0Zh7+RubzQpzNAvg7/tPkjwfhSAXY33r+FyCMfXVjom162/0Id+QYhWM6uzuDLsuKKBGnzTwtAz5Vt5mNvY0HBuFPEdj7gf0C8WL9FKuc+hHWlVYPU5t7IYvTZp6WESP2iI/dUj2pzwTiUECENtDqfKZzcwS72e4NU2dOD8ShsWM3W9ahAVCpOA64ilxhkGfQRcyuly0nn7Weu8AfK3j/ipB8LjkF4RSvzWD+HW+K3xqUTxGl4ZInLtkCOCnHaPhHa3Prr4PxqGRkgPss26eLuN6z6ClwIQ+7ZzOT5z2FNo5PwT8z8O58b6wphHPJnR0WM8ZEvVo1RN2UuusEye2xFuwWimrnLuv5DsCsIByx2NF6/mLed40de219EZgwAlkDgwNXGczrXpsxpjUof4IgtIEWGTqAzb0mgzm3dcfWSpXFFcp/8WeVjirRtY8Frib37/oi8EkqXy5j/07/ogaVdhr7G4/HGmwoSGfIMx4RwOCkwQR+afD5NfK3tQ98Pkrh6oOfA1oKfNibrUfyvKdQBcND8P/9fl/gtUJDYkbiH8CvLPNRXT1dds9sREg5Z+tzVmH4ntcmRj6WWp6aEZRPBfAn/Nnlo4NwxMNO+IUrlhdxvTrgSMt2FzU8RN1maOD2Esu8b3p5+kOBOBQAoQy0Ovo63meMOcEyP9K/ov/KQBwaP/a8hgOxgsYC+Co60dz7N30JLQmy+8LKTQvwQctm/86hR0SMIKdZ5uUtTS0/CcShiJKxoHfB1oLYc+fuTzQlfhOIQ6Xnx+R+2TuU7sCnWrA3bX3o7MCax3Xc87GyWi6uPWsrIsS4WfcKcmcxGXHkmKD8KYDXAXs9/QrB7kvzqQTbhxbj4QBgK8v2oyKuF0pc434be16r4ZxgvKk8oQu0kpJ0HNe5glyVLBHk5OR+yWJraSvF963n9fiHCY+HL+APsv6H9icE0XR5OtZMM2pwcUn3pg9Ap8uvw/CtqDcr/GTdbBwtPVuLg2MLDYSZPvz9oifh7yUIK7vgVzS8nglyktw+vf2fgN2r/KWuvq5qUHaLKAHzZ8x/WpCcDK3BHBMyUYwbrOfbA4cVeK1paBa7EKEFA3QCR1j2HrSvs1DOsJ6/jD+4rHnam9tfNJicv7UY+dxEybKH6QMJQGNv41H45TZvbGtuszcN1cyjwD8s2xmoOs14ORy4CU1RD/M6Whf8SEHeFcfWgJ1tvBPt46gtxBccv1E3uc7+4ogIGR19Hdtg+EaO0fCX1uZWWzo77Hzfer4DwZfulIo2cr/fXPybuprGiDkX/b2HieWZtRURYowx11mmHeq76+1qkmrm5/hbCtopLFh6B3A7KtF+Nn4xrnzE0KzTn8kduj7M6eRmDcfD3mjbhpcfUsNiPKPhZJ0ucv9bOi7uySO9v5YIVaC16OlFkwW5yDK/5dQ59kDeMGBPeN8CODXfG0dhCpopqrPsw7Nnflfgo5i5Kwn8cswLirheVbKgd8HWGA722gS5/pytz1kVlE8RpcFxnTbsf8NZanGDehP+ftEE0BiAL6XkncCXLNtPKK7XInTEW+IPG2NuzzEKX0x3p3cLyKWIEpMh8zPgLa/NcZxCM0JBkAEWW7b3AJ8v4pq7ofurHnRe1y9QFebzgVZ0jbtkyP4s8FsgX7/Qd4BfFuGH3Vu6Bvh2EdcLNfNmzuvG/9/zmOSjSbuPtuawN+hVzcCagbPwK8JcFAIBjHzcis6R8KZOz0V7J/rGeI0Y/jk0oIFSMcGSLWc9Vt6FrdIG91ODaoODDB5pMDmfHyfr2A2fESGj66mubd1B93jLfF9iRsIWbagFMsC3yP3ybwHmUbgARTXwXfzfbcUMPQ4vg5xHjENZd6jquMadD0y4oaG1SLIp2d/Z3fkrY8xawQVBDhORs0Mk2nMpMBfYxGP7NloJU+zB5Q6MX+RH0HVxXhH3PYz8pctPFXHN8CNchuEQj2WThikNR6FK2TVLaDJaHT0dWwmS8w/fYHo3yG5gn4aEhSxgZ+emoKcoYSSGfljswK8WMwEYjC0c8Nf4zPiEkWytVbLZ7HzsbJahlkUEvoe/rDcBzAzAl1LwFfwzwX6Kv1R7QjC0Jt3mtRnMFzp6OnYPyKWIEmOMscWXpqeeSoWpF+8NNNjysiNw3jiv8xbam14MD6MKrOdQeD/nRvgzVxkgVYRfNUG8Of4HVHl7HS6n1voA49AEWjFiHViZFkFah4b3hZWb8M91OYTihDGC4lzgw5ZtKX4Z6dDT2du5C8IeXpvB/DAofyJKQ7o7vaMRc1yO0XBXoinxx2A8qghvo6fJXhrRjLtdAlwO/o1Krg8//lbEtVqAyy3bauCsIq4ZeiQmSXJ7tUyMWE0egE1EnAHnN1h9P47rHBCQO4WSRkv9vJyN9k+NlSfQHvGPouqBfwRWrudnXOBJ4Bp0sPDu+EWCxssV+CuvFlD5ETtVhzFGxEiuOrjhnam+lD3vsKYIRaCVXpaeJcixlvn+GhgKK2hflt0c+W3gvWP4+ZXA1DI8xiuisT/4Tv3fwr+Bqwkc8dXAD2bJLg3EmYiSIUbagQavzYi5ICB3KslStE/By+5oH0O5ORfdTA0/Cm2ObgRuJrf8COBiKjuovepom9b2mIjkrE+CHJ7uSb87KJ8iSkfrLq1vAX/12kRk/4DcKZTVqPCEFwcVsNlmHNfJAveg+5H90MP5HdDg6/Noxns2utbsBWyKtm98HRXEKJY5+Mdk9BBls9YSy8SuA9702oyYUwJypyKEItCSmHRiy7m7ck6IapBH41Egadka0UbNaev5WQFeK8NjPCo77wJuIffvA3Amtag0CAhii2Dc3d7cXuz0+IgASXend0QHfq9FkN/Fm+P3BONRxfkG+tn3cjJ+eeJqJAbcCLlZZjQ7VolgMQycD3jHnxgRqaVxBRMb4xvEvW/IZN5BhRKut2xbAb8GNi7ius+gwdftaBXRLWj2/EEsIZEi+RTaH+olCxyHBpIRDB0MCDdZ5s9f/PTF2wXiUAWo+g9iV2/XB4DPWeZb22a03ReEP2Uijf9EeTu0GXTLyrszZqahMyE2s+w/ROd61RwdPR1bobKta3FwfhaQOxElwsVtw5oh5bhOMhhvAmEFuiGwD68WUd2DjA1wGTrmwsvrwJHAQMU9qkLaWtqexB4abzg0ymrVDH+ynm9S31e/UyCeFMfJwOOW7T3Az6hMKXOhvB8tt7Z71C/A/7eZ8Lji2iXedbGB2NcDcaYCVH2gJa50WqasGBlvk2S146Lza2xFmp3Rk5j1ZbaCYBaaardPIR4lnD1mY8IY82lyPzeC8Iug/Ikono6+jm2MMcd4bYLcGZ8RL2ZQZRi5HVXb8mLQeVtzKu7N+qlDD3Ts9cZFs5MTumTQJiaxC7GzWki+2UERISPzduZBrEoUR5y9R3h7NbMKLe1707J/DD3U3bTSDo2B/dGxOLZM+W/QXrEIi/YZ7Y/gD0BPWCpL7cqomqCqA61Ub+rTYmQ/y3xtW1Pbf/P+QLh5CR0ybA/v2xm4j7H1bFWKj6EB4PaW/Wn0d1hfA2poMWJsRbNH4i3xiS3ZGnJi2dg87NlRrk8RdKLQip35WKcoegHV852xMaom+LU8r52GnoBHeJjXMm8Z2sfm5Yiunq6d870/IjwkZyVXYmWCBLFLacPCI6g8ui10ti+6ObeFJoLkKOBX+EfiPAh8gVwRmggvBlsUY7vu3u6w9RaOiWr50vQxJPd4sWVenZ2UDfN8l/XxOHAg/tkR2wF/ofBG8VLhoFPbf4eKZnh5BQ2y7AGotUZOoGXE1Jyq4kSi85nOzcXICV6bEXN3jZUmjwcXOAZ/KTOo3PKvCb6c+X2oXPtBeV67CL/yYMQQjuN0kLv5i4mR1qD8iSgp/7KevzMQL0rDH4Cv4u8X3x39Pe12kkrTiB4+3YAloISqGB5IDR84l4LM25mfYsnxi5FjRnh7qKnaQKurr+tzPgltMd+dv8P8mhRY8PBXNBVtZ7Ya0Q3EzwmmlHBX4C40+LWHgT6Dqvo8VmmnKklqeWoGdhbPqb1hzBOKAeZilXxkyU70co8BdMzE7Xle+xQqyR7EwNsGVKXwL8AM6zVBlcZqray8pLROb33ciPm51yYiR6V6U9MDcimidDxhPd8lEC9Kx1K0jNDObE1F16bv4T/wrQT7oBmrfOXU/0L3Qi9V1KMQkpyVHBAkJ8NuxByW7E1WY3loUVRloCUiRhB7zseb9fX1XYE4VHn+is50yFeSdjAa0LTjrwkuB1OBLuA/6AJi8xjwIbQ3q7YxvN+yuP30R42uISXdnd7EwcnJEgvyt/aW9ihLCf3AEeh8GZtt0PLC3wKVEFMw6An2w8CF+E+QB1F55okeII8JicmF5IqeTEIm9qyxWkBEbBGJ7br+22WXtIWNn6CVMm9YdoN+5v+LKqbaIhTlYEe0X/UetEfd5i60reKFCvhSExgxP7BMjY00BnGIV1aqMtBK9aYOtrNZgiyeu8PcV4PyKQAeB/ZElQdtpqCZpT70BPcdZbj/dmiA1QfMI/9CdjPwQbQ3q/Yx2M3FTySbkq8H4ktE8RhOFSTn9MzBscuVJzJZdDNzMhp42RyAnuD+HP+w8lIwCT3R/ifaczUzz3ueRct5a1LltBwkpif+hZZ/ezmho69jPPOKIqoMB2eZZTISk2rqZyqUP6Izr/6d57V3AEuA5cAplOfweWf0wGkZWlZtrNcFHSPxafwiHhGjkGhJ/B14yGsT8c3MDT1VGWgZjD38dmX9mvpLA3EmWF4GPotmr+yhxgCbow3qz6IbkS9QnCrPO1D1wztRued5+Js8Ad5GN19HMrEWlr2s5w8G4kVE0Sx8YeEGgpyWYzQ82t/U/6uAXKpmrkSz1nZpEuim42BUgfS/6FpVTG9IHVqasxhd125G5Z3zcQcqEnRvEfebkIjxqfk2xrKxMMxMixiBfqfff+BZ5xOsCivL0EPdK/GPoADNNl2GZpNuAD6DHkgXyo5o4HY/uq4djzX+Y4j/odn2c8i/R4tYD4LcYJk+0NXXtWsgzpQJOzIPnM7uzoOMMTly2YJ0tjW3tQflU5XwTnQhsVUYbbLoKfPwxudJtARxJTqcz0FPfTZCe712Grr2R4DdWP+/iVfQTFtfAb9DaElK0mnobXgT2MBjPjXRnLgsKJ8iCifVnToTwyKvzRjzpXhT3Fbci1hHI3r4ksBWafTzPHA3elr55NDjraHHSnQN2gwtTd4J7SfZA1UWW9+p9EuosqCtoBcxDlI9qXvRoHaYtxomNUyfYJUjNUWqJ/UWns+PICe0NbfVWrb3I2i/+m7red8A8AAaLD2JHhQ9jx4Or0QDp42GHs3oGvROtEXC7gG1EeBqdC2MPi9F0NHTsZWD8zTeqimhK9GSiAfnVWmxRQ0Cxxhj/8ddRZbvBOJMdfEYKpLxDeAKRs5GxtBAaM8y+bE5uiD1len6VUlsRWwauUEWBvPQCG+PqGKWPLhk0qvm1ZzTezHS3TK95ZagfAoJ/WiP1AfRMpnR2Ab48tCj1NxDFGQVjYikjDHeDO5GA2sGTsav9hsRHp7HU2JrxAStEFoO7gXOBH6/nvfVowc3+5bBhzWowmkUZBVJe3P7i6me1G/QygjFcPRSWdo+28y2VSdDSVWVDnb2dr4fu9ZfuKxtZtvLwXhUdQjwLir3dxP0RMhW/UlX0IeqIJaN+Zpf69bU1bTKYq3y2tTXvoQ1i8W4prNWFvUysyeqPOil3P/d7PXncKprrmAoSTQn7sCSBBfkm8ne5PqylRFViiCvWaZNAnGk/NhjfvKVE5YSex5WPZHCaekQbFGMbbu7u/OJr4WSqtosO+KcbZkybsydiL1ZIzENbU738jAqWrGihPd5Eu39momeXtvlce9GNzsTB+OTyn3p7J3P/l/e90ZULUOKpnNzjTyb6c/cGJBLYSNFbnlxFl0jjkNLBQdLdJ+VqLLhgcAHyN3oGJiwA6VLhjFGEBZa5q0aaazJWTYTBFudrxYDrc+hPaNeLh+yXYn2tpcCQYV45qLiYHdbrx9H+CX0q4JMf+YXQM4hgRiZHZA7JadqAq2L+i5qEuTzXpvB3Ng+vf35oHyqQvJJG58BxIHpaGA0B/ghukCMZWDem8DfgR8AX0ODuZ2BJNA99J4UftGLfPO0apnp1vMng3Aioji6ero+iy1Jbrg0OSs5EIxHoWJf4BOW7QZ0/bgOVf+big4S/hYqdzyWAeaDqGrYr9BT4n3Q/q0voQOS/42/VPBA8o+biBgHLc0tS8VIt9cmIvOWytJYUD5FFI6DkxNoiSMbB+VLmXDwZ7NWoaMd7kdFurZC1/gzgVuBR/BnxfPxP3RG31Wo0NfWaN/ot1GRjTi5mbMYuk+KKJLkrOSAMSZnbqMx5vDk3cma2GNWzS9R59bNRf/hDiNk+XZQ/lQhO+Pvd/gtupkZZvnQ43se27bAFmiD7IboQjEsjPE/xjbz4RV043SBx7YTKnVaa422eTGY6ZaplBnEiAohRs6xTG8aMd/L++YIm7T1fAB/ZuktNGDy9v5siJ4IDwtgxNCNz1toj8OzQ9cajXY0i+5V/roYbYyPKJDZZnY23ZO+VJDFHnNzd0/3ocBtQfkVURiC5AQURkwl5ktVki/jn913Kbn7GEFFeB6Ctf39DroGbY6KX0xBs/HDwhgvsP5+q78Bv8TbS6TjJxagB9sRRSDIUvSwf5gtJk+bvB/+URShoyoCrdSK1GZkOTbHaLgjPjNe+0Nwx06a3L+XMLYa4eeGHsWyiHWnRcNcANwErC7B9audad4ngkSBVshIdaf2wsqCGDFXxVvidrlNhJ9D0RJBL1cBPWP42ZXkl4YfD71o1myOx7YPKuN8R5HXntD0N/Zf29DfcB56IKcYWokCrTCSW7or1bHHKxGT8GeQXkcPgdeHi877LHbmZxuaTR+uBjPogc9ni7zuhCfTl/l9w7SGl/HMhXVxZ1MDgVZ1lA5mORlb0jc7pg/PRGEv4BDLdhvw1wr6sBL/ifZ2wIkV9CFIvAEmjjhjKYmKqCKMMfMs05pBGYzk+dePg3+DswqwZzGVmwvQGX5eJpwwT6lJbpt8W0SusMx7pXvS5VBriygjBjNoGWop0Po60GLZ0lRW+e8R4EeW7TPAxyroQ02S3C85aMgtHwQOTz6azDe/LFQE/gW1eNniBnQwnJf/xFviduPhRCZfA3oQijdXoCfLXtqAWqsDz0duU7GDre4UAorOSgAAIABJREFUUcWkVqSafT2gxtw4f8b8Yk84JwJH4S/X+TbwYoX9eB5teveyOzqoPaIYGliMBs9rEXxlthFVjiA5m1JjTK0M0Z2M7jW8PA98NwBf5uMvdY5GIpQAg7FHrGzWMKXh44E4U0ICD7RWOau+gs5cWYsRs8AYU265zrDwCXR+lpcfAI8H4MsA/gVlC7TptGYZkjvOPVUxPnWniCpGsuLvAR2MsuZjYBJwvmV7DXKHPVeQFPgOOS7CO+wyYty0bd/2Cvgklg/sXNH5ziD8iSgMI2Yj73MReSsoX0rM6WgFjZeL8Ge4K0Ef/t70D6MlhVWI1IEcAfIFz2Of9f9c5WlqaroLHUi/FoMJvfpgoIGWiBgc5lrmZzZ7bbNocOg67GbzfMFOJckX5J0N1OJgRAA2yG7gy9i5g66twhhRpSR7k5sajC1Z/cuoB3RMzAGaLVsaf7BTKfIFeTPB6vGNGDeDzuAl5Pb4GAY5LSh/IgrAsJFlqYVAaxPAzq72EawQV74gL0UVJC/ycDZwC7DU84gH6tEIzDazswg/8dpE5NCwlw8G+o+iq6/rkwg5g2AFuXTOnnNqJd1dLJ9HZ8h4uRJ/+V4lyVe2uCHQGoAvFUGM+GaRCBJltEJCgzR8HbsH1ASWkQkTk/F/IT+Pf65epclXtphElcQiCuTc6ef2Aj/z2owxR3c+07l5QC5FjBPBknM3NRFozUPHRng5l/UrlZaTfOvgbqgKYRUhOxOywcpuzF1qmTabPHlyqEd5BJvRcmWOZXrLEefqQJypPmL4s1kr0VOToLkNlTr1cjKwQwC+DCHvAfmG9di2FFfOStaX0YqZWJTRCgFDczi+aZkfjk+P/ykIf0LGGfjLdS4kmHIdL6vwr4PbAicF4EtNYcR8xzJNcQYc+3s6onrJzWi5oQ+03gGcatkeQdWOgyZfGfOFVE0Zszho1m/ykMEd5c1Vw8xpM+/BGjqdJXvwCG8PBYEFWh19HdtgOMgyXxdJLa/lKMjN9hFMA3o+8knLN6KnTAEgG6DB3xLrUaqp7b6MFkQ9WmGgYXrD4cCOXpsY+VbUA7peNkVLTrz0Av8XgC/5yJfZTzAxhHnKRrwl/mdBcg7RBDkl7KU7EwaxAq3wZ7TOww4edaZeNQQN+aTlZ5I7CypITkN7xwCuR2eGVT2zzewsOqh+LQbzuYDcKQmBBVoxN/Y1rMjfdd0om6XU4w9kXoOqGuB8J2ArQx5H6YKb8bAAfx9JyRAjvpKk/qf6V+V7b0SVIZxuWV4aYODmQHwJF634y3XyqW0FRb5hyZuDr+c3Ypw4OLaS27aNjY1HBOJMxPgwuQcNYkIthjENlXT38jfgFwH4MhLfIXdYMqh4UMBlzDKddevj//AfmlU7v7KeT0svS9uJh9AQSKAlIkYQO+q/r31G+yNB+FOF5GtA7yS4BvSRiKPZrWFi+OftlBnZl3WzvLLluIMTc/zX3bIqm14jPHR1d+2JPWTXcEWyKdkfjEehYRv85ToPAz8OwJfRuB54zLKdRQ0L81SCzV7d7GYgZ06gOBIFsFVOuju9CVb2xzHOSyO8fQipA9kT5FSQ74P8GuQfIE+D9ID8C+RHIKeDbDP6tUrOhUCDZUuQu+cImnzzBLdFWykCQgzwPdb1JZ8O5uVRfqDqyGQzd2If6jmEtnwwkM1iujd9ADDDaxMjUTZL2QD/vIjn0BlW1cbfgF9attnA+ypze5kCXIP+Ox4ErirLXbKST5wlKqWpcrJkz7JMmRixJYE4Ey7mo+uQlzaqo1zHy0jCPFWpqBUW5uw5Z43B5H5OhD06l3d+eIQfiagCsk62yba54o4inCUnoiXwfwcWA8egw3ffB2wPNAHvAY5EMzdPgVwFsmmpfc/DzsCXLdtvgbsqcO/xsgTosWxtwGYB+AJ6UH/A0P+/A0w19LONi+TM5JvAPV6bGLFbjUJDIIGWwdjp4DcGGgYiSXflDGBry3YBwTegj4S9ATNUTn7+IrQmGuAS4F/luIkxxlcutfHGG0eBVhXTtaxre2PM4TlG4UfzmubZZR4RuUwHTrBsf8NfylEt/AT4q2U7CasvL2J81K2puwpY7bU5jnNGQO5EjAHHOHagJQMNA0+N8iNbM74Stzp0E/9XkHILX6WH7jdMvr7wamEAzb552YxA5ovKdqwTCloFnFJ5H0qDMcY+xP/gwuULQ1mtUPFAq6OnYytBDvHaBLk+uW2yhIGEbAfyOUuB7pD1/1zgbIqWvnhZBlwXgC9j5RHgR5btM8DHyntbeT+s7b9Zhn+hKxn5Aq3Va1ZXibJQRD6ysewcrB5QY3yKagEiMZBZIF+x1qndA3bsIvzZWrtEuJoQ/CI8jVTvpiwUnL3z2f8Dfui1CXJo17Ku7QNyKWI9SFbsQOuFMe6rVqKVKa3oSJn3A8Nl18eg3+/eqo6dgNt0DSsLewH2fu02/Acq1cSNgD2X8Uxgqwr7cTW6jwRoAxPkKKCiEEfsXjwn62Q/HYgzRVK3/reUlpjEviZGcjZAYqQIJSvZBPgIsAe6OOyB9hjY/AVrRkgVksCfbj6P3EWuGpkPfIHcDdrFQJmmj0sDKlsaQzdaJ4NZXa694KAZHHAk90xCXIkyWlVK8tFkvcHYWZk/xpvj/wnEIQCkGVWA2mPo8V785XmggzkfqqBjXt6Fv1znN/hFb6qN36ElRR/32I5FFcHs4eplQCahIkBbohurLYB+4NWh+/8XTFn6R8uJ67qXOo7j/RzVuTH3G0RBbFVijMkNtMx6523eCfwBeADMSHuMB4DrQdKoEtzwuIe90O/8cvRtptDKmGHylQhXG1n0wMc7bHdDdE9XoUywHIsecoNWIVxemfuWh8S0RE+qJ/U4sOuwTZCD0N7cUFHRjJaIGDFynGV+oL2p/d9FXPZgVIUmCRxE/iArDGxDnnk/6BTvaqcP/5T2DwMHlul+57FO+v5aML8v032UQb/SWr3UR4FWldI4pfEw7PJbCXzI7kL0C+J09AAiX5AVNB3kficIKqUcBvIJ81xQvtvJB0EWgNyL9rk8BPwezQJdivZt3IJm/F8EuXSorCc0DIlT2fPmvh5JvVctuYGWrC/QMveDuXeUIMv73ofQ7JaXw8bl3dj4BLC/ZfsBFTkwKZqfooGpl5Ow/y5lQbZmndT8IDAnjIc7ebDLBz+15MEloasmqmig1dnT+XHW9dQoQrlEMLKUSYWuTJyHv146TvU1oI/ERfj7yFKU/N+YvBs99QeVVZ1X2uv7yVc6KJOijFa1IiL24Nrnp7429eeBOLN+qiVbvTf4VJ1uAf4ZgC+F8HfA/hsfgf5e5eAb6Dq0D+sGgo7E5uhMm8e0VDQ8GIwtwrR1w5SGzwfiTMToiG9D31fiO9wFPO953lLi64N/ZMMAlev5LgW2EE89lZkvejnrxnGkwBSTvKgaDL4+rY1f2fSVcq3pZaOigZZjnGMt0xt1G9SVaqbN82j0ewHwOXSieHeJrl1umtAZVF7uwxraVuU8D9jzV3YDvli6W0gdOjB1+ETjNDBll7yvc+p8g/4kK5uX+74R46dzRec70VLidRiWzNlzTjUENGtQOfIb0HKSjwAfCNSjdeQr10kG40rBtJN7uGYoY++mxUvomn0bGqD+DFhuvWdjtAzLLs+sWvpX9P8E4dkcowQpXR2Rj0VPL5qMYSevTYwsK+1djJAbaNnS68VyGP718Ar8g8mrmT+h5Zhevgq8s3y3lCNZl118Er/cfGjpX9H/F6xBy07MsTOeVU/FAq1FTy+ajN3gKPzonK3PKXbw6y+BjcBsC+ZgMEkwv6jEBryEXIy/AX1+EI4USRrtS/DSQemk0NtYJx3/KzAVUaqcPn36y1jZUde4tjJkRBVgsuYUcgOGwWxd9pqg/PHwTWAKmFlgvgrmUjB/pjoGAH+S3P4mUAGeMJTreHkUsKWMPwXsV4Z7vYYGVScBM8BsBWYfMEeAmQ3mUDAz0RJnr2KjA1wGskUZfCo5yf2Sg4LYZeH7prvTuwXiUERe+rP978HquZes/KO0dxHDuh4tKG0AFMOfzVqJ7inCRr4y5jId+MgWaKkyQ/c8CUzNzIlM7pccBO712kQkCrRGIrMmcxDWMD2D+eEIbx8H5nUwK4u/TmDshs6p8PJr4I+Vd6VoXmddnfAwTYA9nLoAZFe0sRT0hOPEUd5cUmab2VkgZ+CfgxPWXsCaJflockPgKMv80/k7zH823/sri3kezGDQXuQhX9anP48tLJwHZCxbmtzguwSYuUNB1VVgRqmcMI+hFRa3eYybofMGQ4HE5CqsElfXce3y3IgAMa7ZwzK9vebpNf8t8W0+S66KXinFxY7Gn/VZBLxYwntUigeB2y3bYaiaY6n5LusGtC8BU41zxopDfBnCDwx914eGypUOCl+yLE/3N/f/pWL3r1468TeghzGbNcylaO+Ul/MZ37wOC3HQwcSNQ4Y4mGcKv15BeEsmcIkyWtVGQ2PDV9HyrLW4jntlQO6EhcPxbwCuAJ4OwJdS0IeuFV72RoWSAsK4rDskGuaDQXhSCO3T25/H6n8zYo5OLktuPMKPRFQaITfQMvx7KBtQqhvsC3zfY3gELYEuBfn6mF7Bf2gbJhKoKMUwBn/GrkjkINYd0j+Pf42pCYxf7Ky+cYPGj+R9c5VSkUAruSy5MQZb//7HSZMMi9BDudgbvzLfzZRp8G6FWIWWC3rZhuIG550NfGjo/z+AKnpVFskNHh0TZbSqDsMcy/J427S2PwbhSkjIV9KyEugKwJdSciHwlmWzD7QqjFmGqhMO846gPCkIF1sUY8NGpzFUwh41jckNtEQKKRuU94N8wfM4FiQJcg9aYTNc7voocCAYO3NcKCcCzZYtjdWbEzKewJpDBxyAv0S7QGQj4Hsewyla3VV7tDa3PoJ1eC9uuMoHK/LFU19X/3lsZSbHN+R2ImKXtGQpqyRxxVgC9Fi2fDPCxoDsxLqm/AHg+KET4sri5Ga0RCQKtKqIzt7O9wM5w34N5kpjTLUO2q0GvopnRskQ30KFHcLMS+CT838X+KoqKo13wOsbI76rCknMSNyFyR3IKo4cH5Q/EetIPpecgv05NhTSn3UqOk5m+HEdWo3yEXSfshxV+d0LzFNFuOxlA/yZmOfAF9iHkfPxlzHbokOFsh3rRhkNACeD/G70B95yu/dbr1etwI0xRsSIXRL5iUCcKZDKnPD5ywafSExPhDlrUwryNWn/H1DquuogWIM/YNwMmDu+y6wtGRwO0juGeh4qjojknKgYTFQ6WEU44tibvtUSkxsDcSYc5CvX+R/aF1ELLMAvzJNPdKhCyHvJ3ejY83aqH9ealSjskepLvTcgbyKGmJyZvDuWEIbjOqUeyyBoOV+G0u4bz8Ceeah7B3tUTBhZAb7xRfnGaBRLPRp4rO/h/TeyhfVajmJltWFcY/dp7d7R07FV3jdXIWUPtDqXdb7DYHLTfFKWaeJhYqQG9BLX8AbKjcB/LNuZ5DbTro9vsE6q+1ECVCBycHJT18j2QfkSkcvCFxZuIIg9RuDWxLREmJRHK02+QZopwl2u4+V1dEi0l+lAAFkYmQx8x2N4g9L1t1QMaZDrsU/oXd9YkogK44pr96u83f9UfyGKoX9DRxPcgoq3/AEYzlwZtJfzUuChoYODYtkUOMuyLUMzabVCvjJmezB8xHrI06dlYiZWDjXZslD2P7ZxzBewZUfrZGm571vl5BukeTnhbUDPh4t/Ds8GqET7WPEu5vXAr9aTGrczZgut9xR8AuKKa0vZbnHJE5eEQqK51hlcNfhFLBEMMT5J6oh1bEj+cp1aEw65FLAVJ8+lKGGe8SCboAcAfwX2HTIOAseB+V9lfCgdbdu3vSKIrTR39FDpWkRACL5+lb8UJoRhFg+NJpg9pKj5CTDTgF3QapvhMuxm4PcgxQ4sztdOcB7VM8S9FLyMf77ou4BiZ+k9iyqXjufhHaX0T+u17xfpT1mJt8SfQmeErSVMMu91639LkRhf2eC/2qa1BVL+VSXE8JfVrURLXWqN24H7yVXYOhHdANk9XOtj5tBjPLzPet6Y911jwNSZx3InacFg3eAuwJ8LvWZEiXA4ntxOrJ7E9MQ9beOK6ScU+TLL5wOrA/ClnKxGRTAu99i2QXtRSiz4IT8G9vIYNmadeMAwy4A5YO4u7b0rh8FcS640/SYNqxs+j7/xP6ICJB9N1hvMPjlGocQS3+YJ4HiQe1mXbZqK9lF9qsCLboPOFvTyMNobVmssQPc9Uz22i9DftcA5iuYtNPM4DsQrnvF8peaQlgox8gcjxlviuM+Ib64yyprRuvjpi7djnVocAGJkopcNHou/AX0h4W9AH4m49Txfb0jVk9kx04dVNy6OlHHae8RY6Ozt3AXJlco2Yq6ORDBGZDM00PKyjCo/0SyCqwF7zlWc3E1PKdgWPekffniDrCywGHh3mIMsgExT5vdo78laDCYSxQiI+sn170crRdbiGMfuZykR5vvArR7DJ0HGe/g5zHn4M8txtBKm1ngD/0H6dOCEyrsSXhzXsQ+1d+5c1hkK9dayBlqxwdjB1j3EFffmct6zymlEFxgv/yO3fr/WuAd8A+fyDSfMx/cYX2r8KuvnL7Ref5kCGRpF8ITXZlxjB8wRFcYR5+vkqjgNmknm+qD8CQFt+Mt15pM786WWyCfMsynjFuYpihhwGrAM5OgK3rfkJE3SxeT20IiRj6WWp2YE5dNExoixy6feaGpqKkRxcKzYAkOFzDNqAl9v333Ar/+fvfOOk6Os//j7mb2SJiGhGkhP6EUNIqAooKgUsQARpVlQrAERuBLKAuH2kmCUKChWBAFFUfjRBekoIAjSIblcSKRDqEnuLrfz/P747iYzzzN3t2VmZ/d23q/XvZJ9bnfm2b3ZeZ5v+3xLmlFtsBgwe3+ehmEkJwxMv+q/1xhSpGqjH2G0NVouh3gfavRDp0077bmBnl4HfBuYZIydy/ApQB+IVvAldwX17wlAPSTh7UJ/eNA4wJ3Gc8pSMtJK+wqMXeUmhlaMpJ9IN2l743p9y6SWF2KZUPUzATBlfB/F76UejlyGLcwTpHZWDkcA03M/2yHCAV9F0nvyNSdbAZeAruk+ZcpVv8MfeVDa0V+Naz51jbL6Mt05W83OBj43HMyU/1LqnoPUP08rbTo1Qz6N2Us+jTmhAHK2g0/HwNHOh2OaTlFEVqO18KWFo/vX9PtUQRzlXBvV+WqAMdhpdM8TR/PdyvMgUq/1ec/YF5DNyP2xzKgEHO08pT32okJFkjo4f/n87V3XPRCYpNFbKlSfQr3kKnelUuru1smtjySpcdA0qulAtL/xa66GJCGYM7HTdVoYnuk6XlzkvV/tGcsL88wJ5xQqyLh/ALgY9M6IMZuvLzgV9IO1ViORp3V664rMsszfgU/nxxTqqLROn56L/CdUgPQL6VH0+EW1lFIh12dZmBGY1YHPGpidEaeElxuQhsjDnV8jKoteEZEWJHPHbEWREMw/gfUKw1rpmjC0IotorVu7bn8M8QGtdD0bWj8ENjfGzmD4FaAPRBv+9CRFjcnZu9o1RVy2nv/0/PeEdfzO7s5PZZZl/uu67pPAecAchZoNHKXRJyutzsflP53dnS9mujPfDOu8tYrSylRueqnnuZ4bY5lM9TMTibB4uQe4KYa5xME1iDCPl+OReqqIUY8hvWq8Ms/ngA6jcWksKJQpwT2paXlTKWlkCSXStLZpH6DZN+iGLYRhsZvx2FT1HIoOjHIShn80K886xOHjZWPg5Aqd/w7g1tzPIxU6Z6golJk+uFu6O12yyFmliMzQUih/UzbN862TW2vyjxsCm2LXBDwL1FMtyTPYylT7g5X6UL04mIaW0g1653IPe9GDFzVmlmUu1lrfBOxSwEu2QFP2eWuZ9JL0RsDBvkHFFaXJGtcFZwONxli9bHDyBAnzmDWzEaFWgq/h77YUVqdalfSonv9TqDe9Y452joxrPvWIo5zPGUMvtUxreTy6M+pm/KluWeDOIg6wO3CQMfYn4OEyJ1ZLXIFt5JyIpBVHjPo8qP1zPzV57886WdPQam7KNs2KZTJFEImhldZpB82B3jHt6P+r43SnNow+P8Bchm8B+kCcidnwUpoQ14Rnt29531KMVAlXuXuUc8z07emGVeNXXQEc6z0scJ9GdyjUHBTfRpFGmkjWXP+dKGhKNR0OjPQNZhOJ6QHYGb8kN8B1FLdJGg7cBdxijB0F7Fih8//TeLxthc4bOump6R6t9F+9Yxo9e9HKRSMHek1CeKR12tFonzNba31N4XssPQv0IYVHVfV7EDlyr7z2tUX2gzPX+iy2UM1wx8V27ozE7muYEMDMyTP/i9kAOkXVpw9GUqM1cvnI3V1cX6FxHddnTUBEMLw8hGya643nkHxkr1fsg8AhSGpPuXTj7y0RqmR+et90f2ZZ5t/APvkxRzllqd6MmDwio9GH5h9r9ANa6ePnTp0bGP29Ul+Z6u7u3s/Vbl03S3a0c6T27ymebpvRFqXaVi3Tid+p5lKDLRZCog1J48tv+FJICvMXKnBus56lOfBZtUKWy3B86nFje/t6D6Q+17aK0tTd9GEMMRfHcf5WxCEmI3+npaCvAm4DHgX10oan6BGIE+IgpA/Uez2vX42UQxTKp4F9jbHfAE8XcYzhwrWI08Xb+uibwI+xW1EkeJitZmc7l3U+4G3SrbSqT0Mrq7OfUf4gxeoe3VPT/UPK4CxMz7sUYddrdO8cpJeYt7bpXMTDXqZakroNos1R1+h/KtQ+nsd7DfL0Qcl0ZT6o0d6eRnc1jmo88JQtTxmwwDinKGV65euK+SvmT3D73Y8aw6bscILwEfBnFwB/pEZz9EPgIeCvwKGesc8DewD3RXzuKcbjmu6d2Du9947m7uaVwETP8JEkhlbkKKU+b+wg3upZU9IeawYiyNAiD7WL9H1qwL9Ge1kDfAaUqUA44HSRNd5LD6I+WK+0IhH2PI1Ixs8x8UyndnC1e69SvrYGe2mtVTVnzEWSOhhQn/X39NR0TxTnqnJmIkaFl7uBv1d+KlXDq0hPCS87AqawQXWirYL6CfOWzZtc0rEUP0I86gBvZxuzXx7MyEoQstnsUWz43AB0v9N/eVzzqXJMwZmggux6ox07bbuzAuc9zPN/F3iiAueMjLRKu0qrK3yDioM6/texSUxTqhuUVv76LMUN6R3TfUUcYg3BaqMO0mdvICPrDmBWkY23Dwc+YIz9DEOqu864G7jZGDsS2DWGudQUSlmCGJt2dnVOD3xylRB6RCu36TQL9a8L+zx+9NcRwQmT8Z7/bwW6JeA5K0FFtUk7B/szNguy65GFSCqCd0E+G8kBN2u4qotm/kUfGk+ueUqn9kLSIgum47mOHchuaPaotf7JaRNPK1bBqS5RWplF9/ecPuX07lgmUzB6E+C4gF+YfWj2BZ0KeN4/QJl94obiQDxprjl+Aywt8jjDjWeBS/GrMH4M+Dh2c/UA9FhQbxV3Sn0c/tSp+/xpWjWKyyWkONUz0uSscw6jPtqWxEJmeeb92tVTfYOaYtIGAXUT6AnIPeKjSAr/DOx01l7gcaSh8KUl3INSQNoYexfZA9Q7c4FPsmEv4SCf1ecHekECKK3u10r79mCkmEUVr2uhG1opUvsbQ66r3OvDPo/BDxi6oHkywV7Lu4AoDK1dEE+Ol3xubr3zFrAA8DbunIJsRC+IY0KF0r51++uZZZkleIqCtdJ7ImpCBaOy6iueh1o5llxyQgC5HmM+ZUalVC2IYGxGYVGTA7FT/UCUqYrZ5AS1T+jBTuGpV9JIFN27sexElNGGSkH5KugjkMj8daAGaTivN0ecaycYvxgWUcXWma1PZJZlHsWjlqq1PoLE0IoO19qI9zp9TgltGtTLwO9yP+QcPJsjPT+zyDr9Nqh1AxygEL4CbG+MLaTG02ZDIl+r7410fw7YE7sVRUKO1umtb3V0dyxTWm2IYmlmIQqWVUnoqYNaa2/uJEqrR+ZOm/ty2OepAYIK0CskJVwTLAb+Z4ydwcApC9WEeRMsRRBjQ42R5qm2qW3LzSekdTqy9gu1iuu6hxlD65oammqy8WvEzMZO1/kp9neuXlkB/MIY2w3Z6BTCh5B2Fa+AfgD0L0HPk6wJfTron4K+A+kz9AP8a8GPQN1a5vyrBq206ejYe+HShWbPyITwMO+Bt7Rs1/JO4DOLQmVBvQhqidRfqdfLNLJGYO95XgN+UsYxhxtB6tOVSGOuaZQ2IquKqpZ4DzWipbVWnd2d+/jGlC4gFaNsbgarx1GhPBXmRHJ8BDjAGAvqn1DP5L3rP/eMbQ58D8jEMqMCUUr9S2u9Xo5doXZNP5Eek94x/W4hr08/kW4C3rd+wJEi/HR3ekSz2/xlFMcAO9LNppllmbXI5vh2rfTl7VPb602S28TcZPzjpIknrYplJsXxDn5FzGJZUsRzg9J13sIfQU6QYvyv4XfuZJDMg0JbbzQjaVcfLOC563LnrKlG7UORVdk/N+gG77WV6k/1fw5RmE0Ikfld83dzcX0RIq39MvtVxLeBScbYucAgEeC6I99P1ave+VFEGXXYOGMi4CHgi+sfaWZVsyBGqIZWR1fHjo7j+CRHUZF3KgdUMTKjlcBU00kK0IP5DSIRO8MzdiqSdlK1m+dsNnuv4/iCTY1NI5r2ocBaxBFNI2Zqaf6Y59nOZZ27avRlKCsFdiQiqjJTafXNzLLMTTqrj2mf2f5qOe+hFlnQtWBmlqzZ0LlGFM7U89i9rKLia8B2xth5wOsVOn+tkPeue6Xut0WK0n8/yOtuQr6TByEp6UPxLnKdLgBVqkOwajl9yundmWWZh4H3rx/UHEpiaIWOq9yjjaGePqevyPqsijAGux79eZKU0iDOAL6EX506g9SLVqXhEDsuDxn5eGNzghhVWacVamqSSqn9jKF+p9cxFULyINOCAAAgAElEQVSGOwchhdVefk3SHyGIIAN0Y+CUGOZSMO3T258AXvCOKaU+Vejr3ZQ73jegeY+LexvaZ2S9gagymWqdn1Ypdd/8FfMnFDfr2idL1oxmZXVWh9F/bTgxArtH1qvA+THMpRYIqhc5i0F7XKmnQX0X1BTEY38YsqlcjNzrf4lED09AvNNbgvrKcDSyPJgOj/3Oe+a8uu71Fzbp29MNeL34gEb/X3pq+s2YpjQYP0QyVLycAayNYS7VTpABuhuJKMaAKKUewjRCU9WbPhiuoaUtQ+u+cHKHa4agAvS1QEcMc6kVgnr6nABsFcNcCkIppdGWRP+nCz6AZqwxcqpC5Y2vSxzH2aFtWtv4tmltk3rX9o5VSn0OeMbz/Gluv/vHK/WVQep0wxfH1/sIhbqjHiN7Q/Bd/H2NQCLs9XQfLoZ3sBXQJiMNRAtArQR1Faj5oE4A9Q1Qx4NqBbUY1N2ghn3LBq21mRbb0NfQ95nAJyeUxIhJIz6FqVKquTSe2QzKpsBJxlg+RS4hmKCUyg4i6nVb67ROb30LM3ih68DQym36fE1EtdYVSBusKo7Amz4hBIk+JGzAxfbAj0R63VQtylFmD4wZC5YsKKiXg6OcUcZQY+7f09qmtR3bMqVlfd1gesd0X+vU1mtIsSfwX89r9l7WvcxneAxnMt2ZKWhD3EHXStpgxRgDPqltkLYDSbrO4AT19Dmd2hDmqQrap7c/i/L3BVNK1c39qRJopc20wVc3eWMTcx2qBtqAjYyxINGHhA0EiYRsCxwVw1xqhYe8D5RWu8U1kaEIzdDq7u6ehTS68x69ngytBuw0uLyMecLgXAeYIg/fAKq2CZ129M2IBO56sk62oPRBV7trAob/1Tq1dcDIZ9vktjdc3GPwhMs1+vsFTrfm0VofhrdvBriqUSVpg35OwU7XSVPtvenipwe7rnYzYE4Mc6lZtKtNx8cnM89lxgU+OaEo0kvSGwG+CKFG//H43Y4vRxUwCiYgIhhe8jLmCYMTlMacZtA05rrGZ2hppT+gtVYDPTlOQjO0tNZm2uCaMf1j7gvr+DXA1xEPhJcFVLGoQ5VxmvG4EVs5rWpom9z2BmZfI0VBhpZylKVOqFAXDqWYM3fa3EdR3O4Z+lD6ifSYQs5Z6yjUZ42hf7ZMankh8Mn1yaZIry0vzwB/iGEutchvgKeNsVPwN1VPGAStLEOrkf4iUqoTBqTZaT4M8GVCKK2qMW3wLPyiDiDZKYmow9C8i+2YnwwcH8Ncqh7Xcc2+knlBjKojPENL6X19j9H3zpk5p148qSOwDYVXkb41CYVxD6Lm5eXLwK4xzKUwtDXfjy9esnho75PiLXOo3+0vVLb9Ls//G0eOHLlDga+rWXJe8T18g5pqlTSOi7nY6TrtJOk6hZIFzjbGxmKnYiYMwNxpcx/FVP1Sgc23E4pFcawx8kzb9LZ/xzKXgZmJNCj2cjdY9cwJA3MBdhrzaSRpzBbr9LqHMQx47ej3DfD0WAnF0MrVZ+3lO7B26ilt8PvA1sbY2SQF6MXSitRs5XEQD1lVolPazI8f/U7jO3sFPtlD7+reJfhvELp/ev/zBZ3U9d+Eszo77JW9lKs+hVEUrNHXxzSdamQrbK/ng0A1yj5XM38EHjbG5mDf2xMG5gbj8aeTxuvlMX/5/O0x6t8Vqhoj1fOwxRtMifeEwenBFlTbDBEIS/CQU9v07YeUUmZ7nKoglBtg13NduyCF2OtxU269NFYdC7QYY88Bv4phLrXOf7FzuT8L7BnDXIZkxuQZD2i0LzU0lU0dMtTrco2Nn/MMZc/kzIJSK7TSvpx8x3EaB3rucEFrbTb/XtY+vf3ZWCZTnZyNna7TRpKuUywaW5hnBFUuzFNVKG40RjZtWt5USDPnhAFws67pROnvb+z/XSyTGZhdsJvJ/x/wzxjmUuv8FjuN+WSSNGYbxeO+x9rqQ1oVhONpylob4b4+3Wd6BocrQV+AM0gK0EvlNOx0p844JjIUs9XsrFJ+9UGt9KGFFGQqpf7jedhw1vKzTMn3YDSbeR+6Wfe1gl5Xo+S84WbtW0GNoeuEbYBjjLG7gFtjmMtw4HpsYZ7jqGJhnmpidP/o25Fak/UoV5mOkoQCWbRy0UhTbVBpdd1pE08rLAOicszHv590kX1QQvFksevTgxz6dY9y1ePG0E6xTGQIwjG0lGVoPZyemjYbrQ5HgkK6zwCXxzCX4cKzwO+NsY8C+8cwlyHRrv6LMTQx81xmj8An+16ITzGv0W0srAeE4+sVobWjq7ITeliM7B65G2bvGNtrXs+cS5KuEzZBwjxVm8JcTeTqsu/wDSZ1WiXT098z29NjEQDt6Gpr1/AR7D6Sl+NvR5JQHFcC/zHGgkpU6hujpQQws6A6+QoTiqGllfYZWkqrf4Vx3BogqEixlaQAvVzOxO4gn8Ev710VjGgacSOGB9dxHTOFwkKn9LVAX/5xykkdOdRrOrs6xyqtvBK/T8ydNvflIqZbc7jaNTdpa5sbmuslLXkoZgFmr6KrgXq5/0bFPWAZ818CqrLQutpQWpl1WrPOXXbuFoFPThgUpZWZNrhi+pTpt8QymYExWyOso4oVg2sEje3wCRJdq2u0q01Dq2GNs2abWCYzCGUbWuc9c96mSqtpvkFVFwv9JIIL0JPePuXzPPALY2wW8IUY5jIoJ008aS2SbrQejT58qPTBnDz8+veotT66o7vjQ4O9Riu9AI9hr5T6dUmTriG0suqz/pH7zBMkmuXrLUaywQmLIGEeU5UwIYB+3W+m9jpKqUTmvUjOXXbuLpj1yZpfzFazs8GviIWDgY8ZY78CumKYy3DjRszocHAbobqlubn5SYx+phpddemDZRtafY19e2FEGvrd/nowtNLYjeRaSQrQw6IDeDtgzEyTih1NQPrg8szuQ74uq+cp1Ju5hw1obujo6rDSbBa+tHB0x7KOxcA382MK1d3U0PTLsiZe5WSey4xTGN3edZI2mGNv7Nq1y0jSdcLiUeDPxthnMNR1E2xOm3HayoCUnk/GMpkaxsExHbnr3JR7cRxzGQCF7XxYi2SfJISDKc7TQFL7tp6c07XbN6iqTxCj/E2rYk/DtHjhtBmnmX0AhhvbAEcbY3cC/4hhLsOV14AfI2mEefKfe1UpLvWN6Luhuad5NTA6P6a0Ohy4f7DXtc9sfzXTlTkcxQ1Ao0KNR3F9ZlnmUTT/VEq962p36/41/fsrlFdwZY129KHDPbKjs3pvhfI5g1JuypTUr1dMgZh1JHVEYXM6EkX3Knt2YkhtJ9gorW7W6PUbHuUqM+qRMAi5RvRmOvnVc6fMfTGO+QzAEcD7jbHFwP9imMtw5R6kZYLXAXsEsBB4JJYZVR9PADM8j0OLaC1esrh5ddPqUUM/c3DKjmgprfz1Wah6kPMMiqwkBejhcx7wijGWxo4kxkp6QnoNdv+Y2YWoD7ZNb7tVKz3bkInfBcW3NPpkpdQReFUtNc872vlY25S2Ya/qqZTaxxj636kzT01SUoIjK78kSdcJmyXAxcZYUCQxwcDFaO+i2GrBkgWJcmOBNI9s/gqiNLcB10qnj5MG/E5QgLeABTHMZbgTlMZs9tqqZ0zlwdAiWmsa1nyFLKvK/SnL0Erfnm4AnwoaGn1fWe+s+gmqFfobMNzfdxy8i8jGepkEfCuGuQyFmWY0cUH3goL6x7RPbb861ZDaWWt9EfD6AE9bqZQ6s9ft3aFlesuDZc20Rgjwgt8RxzyqjKBaoSRdJzrSwBpjzJSyTjBQKXU3/s0hWSebRLUKINfS4vvG8NOt01tvj2M+AxBUK7QAWBXw3ITyeAxRIfQSVBtXn2grTXnawpcWjg58bjysLmuxaJzYuB1Go2LlDnvFQVP9LoudR5sQHhdidP8mWO0xVnpH9F4PrPaOudr9cqGvb5nU8kL79PZvjV81/r04fEBr/Rk0RylHHai13rZtWtuk1qmtZ6dnps26tWFJZ1fnWK30rsZwojYYrH73E0RAJiF8XgB+boztiq32mOAhJ/bzmG9QJRvDQhjRPeIgJE1+PUqrxUqpaqn/DlK/exX4aQxzqRfOQNLDvZhqj3WJwuql5fT19FWNYIhS6sqyarSUo8z83GzPqJ7hnDca1M/pD2BZ1Anh0YN48H/lGdsUOJEqCp+nJ6TXdCzruF6hZq8fVByZfiJ9anrHdN8gL/Vx/G7HrwMezv3ULS7u3gqV8o6ldKreDa2gfk5vIim2CdGRAb4BbOQZm4dkMiStPAbmTsQozZPUthWG2ZvzjdTo1CWxzCSYoH5OZwPvxDCXemEJUpv+Tc9Yvn/ZTbHMqEoY5Y56dnVqdRZYv19wtDMTuw9Z8cfuH3Xx6qbVZjRxcPpZhOIr6x+7/LY8QwtlepyfzdWrDFfMAvQ+EsnfSvA74CRge8/YKcBF2DVcseFo5xKt9GzP0KbNo5sPQjZkCUXgKOdj2q+y8+Kp009dEtd8qoTjALPOZT5Juk7UvA78CL+Ruw1wLPCbWGZUA2it71RKzfEMTZm3bN7k06ad9lxsk6pyzl167k4avZ9vUPPLU7Y8ZfUAL6k0Y4EWY+w5/I7QhGg4CzgK8IozzAf+jpGmW0/MmTmnN7Ms8z9g8vpB7RPHKOvYQG+hz093p0c0q+bPeoaebZnWcm95eebKrzijlR7OXvjPYva0kI3+shjmUm9ksfsDvQcxtqqGnhU9NwN+VSjX49lIKBgX1+f91lrXezRrJNBujL1Ikq5TKRYBZnPws5C/S0IATf1Nd2G0O2mgIUkfHISUkzoRf2lCv0JdGNd8AjgZrziTcAZFbEYTSuYFpJTCyy7AYTHMpapQqKXG0Mw45tHsNh8KjFs/oPitUkqXZ2hpdjFGhmsPFwc7ZWc1oj6YUBn+jB0K/h4wMYa5BJLeN92PpJJ6OXBB94It45hPrbJ4yeJmhfLVISlH1buhNQc7XWceRl1gQmS8i53RsBXVKcxTFZy87cmvoXjSO6bRgzZlr2c6lnRsptFmXe9VrdNbV8QyIZvNsNManwEuj2Eu9UoH8IYxNo8q7C9aSbTWZrZLKBGtonH4mudRv5NyLpXhEuns6pyE1MqsR2k1XOuzjsSfaw5SgP5SDHOpVzQw1xgLKsqNFQfHTCVq6Hf7CxbFSIC1ztqdgSbfoBq8J9kwZyxwqjG2HPh15adS11yI2RxToowbBTw3QTC/tx+IZRY1gEqpb2FESLWjz49pOkEEiVC1ktQpVpI3kP6iXmZCnWfOKGKPaGW6M1PQ7OMZuqFlUssLUIah5SrXVL7CxR2OEa1G7H4RbyI5+wmV5SbAlLj9OrBdDHMJpGVayzMa/YB3TCn11bjmU4toR88yhvp6V/fWs+DMqcB4Y+x0pEY0oXL0YSt9bQr8IIa51ARKKzML4X0XPXhRY+CT65jFSxY3A9/2jmn0A+1T2qtFxXkycLwx9m/gmhjmUu/8mCSN2YfGimht3tnVOTbwydFN4mt4bCql1G/z/y/Z0HKU4ze0NM/PnTbX/OMPB76JXYCewQ7fJlSGVvx5/ylsQzhWHO1cbAztdO7ycxNPboForf2fleKxYpQbhxmbYffUeZwkXScufg88ZYydDGwew1yqHtdxTUNrxBvj3qgax1i1sLph9bHAe71jSqufxDSdINJAszHWhlGDl1AR3sXumzgBw1CvJ1JOyhLK0lpXLH0w1/vuWM/Qy+NeH3dD/kHJhpZ2td/QUgzHtMGRyM3Ey4vAz2KYS4LwAHC9MfZFwGw1ECeXI01k16Nc9ZV4plJ7aAxDS/NQTFOpBs7ATteZSx2rTMVMFvmbeBmDrcSWAPQ19f0XI7UsIGJd16R12kHzQ2N4xfg3xv8llgnZbIuo3Xm5FfhHDHNJEH6OncbcRp2mMY9cN3IZcm/eQKpydVrNy5s/CUzKP1Za/T7Xqgcoo4BOO3oXpTeI42j0cEwbPBEpePZyNjCcJexrgXbgQDY4ChSS0nNQbDPy0Dq99a2Oro5rlFJHeIaPXLRyUctJE09aO+ALE7jowYsaV7FqJ9+gqtueYpOR/k1eHgCujWEuCRu4Cvk77O4Z+w5St2s2V69r0hPSazLLMk8D67/TuYj1xbHM54X0qOa1zQeg+DywDZoJKFLAyyie165+EM3f+1b23Z8TN4qc5u7mQzEbFCu1yLtRi5kO7L3i6XFMJGE9fUgf0d96xjZF2uCk45hQnARJvCutKlenpX0iGKiUutj7uKSI1qKVi0Yqrab6DqzUo6Ucq4rZGEkJ8dKN/8JOiIfHgD8ZYwcCVSMd7Dj+9EGFGt/b3/vFmKZTM6zadNVOiMjJBty6jWidTZKuU41o7KjWCJLN50D4vr9KqYpHtLTWqqO749jmnublKP6CCFx9EMVWwJbArmgOVEqdoRx1T/Pk5n9WcHq+aJZGr+pZ01Mt/dl2Az5vjP0VuC+GuST4uQT8qp7ItVSvacy+9EFNZVIHO/7XsQlwiGfo3pYpLb708pIMrbX9a7c1X6tcZf7Ba52kAL26mYv9tzAL1WOjZ0rPLZg91jRzgp+dkEe5amdjqL+5qfnxWCYTL9sCplrl34HbYphLgs3N2MI8X6OKhHmqBa20v05Ls4vWWg3w9NC56MGLGucvn/8XpdXFSM1jIYwY+inlk+nKfALwSd4rpRand0y/W4nzF0AH/r5eWRKHQrWQxa5Pr9s0Zq0MQQxVIeXBdRyN1yGq7WBMSamDjutsj/826fY4PVYxWkxsioThpyMX3Ua5n3eRnjOrkfSOZ4EVBNc6bI70aPLyOHBFNFNOKIFu4Hf4lZA+AhwA3OgZSwFTkGtiK2A0cl2MQK6JN3P/LkWuiVBETtIq7WaWZX4BLPAMv3/+svm7t0xreWCg19U7AV6orhpMt2xC+nhsg9xLxuR+HOT+8xZynS1BrrmgVOQM/vtzUBQlIV5aEc9+fjVMIWk7RwQ8dxPkepiB3IPGIrV3a9iwNv0PuR6eYxjV4ClXPWnsF8YsXL5wCyrQHiV9e7ph1fhVV6D5gmd4DXA5mpvQLE+p1Nv9Tv94hdoezSeAz0U9r/Uoa1O8prGv8YKQjj4BcdhMZsO6tzEb1rzVyLX2DNIM1+SjwP7G2B+woygJ8XEV0kLBa6wPlMbchOyLtwG2QK6H0cg6sxq5Lt5kw16oWoz9glCuWuK7z2imVeS82ld/v9pZ5/zZfE5JhpZGb6c870gr3Z2emu4p5VhlooD3AfsB+wJ7YkehBqMHeAS4A/EU34OIGAQVoLczjBa/YcJZwNHAKM/YAmTDsy+wD1Ib0GS9cmBeBv6FXA+3AaXLiqf4NVnS3vm5uN9F6jsSglDMMBLjzP4Y1cjmyPW2H5K+OgO5BgtBI5udu9lwzW2Ovdn7K3ZPooR4eQC4DviMZ2w2sBARgPCuS5tarx6YXuC/bFiX7qaG64KzKrskZXwdXNedQQUMraYpTSehOdQzdJfS6ugBmgDfD1y88KWFo9017l5Rz+3c7nPfh+bj3jGl1S9P3vbk10o4XAOy2d4v9zMLew8zGO8ADyJR2tuQz8Js0N2HpDMnVA95B9zNnrERubE2/OvSTIrb869A7j23I8Iny8ufbnRo9HPK79HZIt2dHhGlbZLpynwQb49dxZ9atmt5x3xeaWIYyp8eobR6uqTjlM5MZIN9FDB1iOcOxghgj9xPK/A2csGam5z8gppQXbwIXACc4hnbifLEArZA/v75a+Ap4FLEk1dUoXvb5LY3Ml2ZK1G+ZoJfXLh04SmnzDjllTLmOHzRfqUgpVW1RMpNRiG1C0cDn6Bww8pEIRHXKbljgUS8zHSdJJpVnbQjIjxeYZ678Dt/iqUZEdrYHUlhfxcxtC9BNj015fDrn9q/MtWd6sGTjqcdPQNxbEbGgq4FM7M6m/YM3d8wquHAU7Y8ZfVgr8v9/pYo5wbgaKcd//d8Xb/qL1bS/QPAMcCXKK825z3IpnxfxJh6C4m6erkIMx0+oRrIp5Tv5xn7OvBVSl+XQFT0jsz9gBjflwJ/BF4v47iRkCK1wvXfGlVztnlronXW+kUwsiqwtrKkGi2F2t54bPYViYoPI5voZ5A84XKMrCA2Ag5HmhR7MXs3JVQHTUgKYZQbj+2RPPXlyLX3wWJe7KbcnxpDzVmV/VrgkxMAv6Glle6KayIDsAmSHrYSMb4/RXmLWRDmBudqknSdauVxpJG6l3KMrCDGIJvpW5FNwwlUqIYoDNIq7WJIUWutzd6UoZN1sq1saOLap1P6a0MZWZViQdeCmeBLZwS44rRppz1X4CE+gWRePIRcD2ELIJj3oF7s3k0J1cM8/HtURfjr0oeQ1kYvIE6fyqn6FYCjnaAo9aSAsVBYtHLRSENZ+tmW6S2BDcaLNrRyjbl8H7BGP1PscYpkVzak9h0MFFJI24tEPJYiN6OnkFBosXmnazEaCSbEjgN8C0m5upDCr+O3c695ErkmliGpgoXI6DrItXc/8H8UeJOZO2Xuf4B/e8e00t++Ul8Z9k2w5smp94zzDaqqSR0chSxmy5EC5EJTlN9ArrPHkDTlZcBrFOccOAT4BcWlnyVEz06IN/nAIl7Th6TMdeFfl6x0kwGYitRfPIs0yKyYqESZ+L/HOtoeN51dnWPRbFB5VVzVPrm9apwVWbLt+DfC2nXdhQW89BNIWuktSCZOIawFnkeumYdy/z6P0etxCJqR2uePD/XEhIoyAlmPrqG4e8GbbFiXHs79/1XMXlTBNCHZF08Cv0GygGLn5Oknv4p5TaeiM7R61vUcptEbe4Z+rZQKDMgUnTo4aumoqdlU1udN066OKqK1EVKH8z0Gn2sPUpR8G7KpzRcUD3TRbI4Uie7MhlqegTYxI4HLgOOA7yILY0J8zEKa9Q0VWXoRSbO5E7mZPMvA4e4GZAOzLeK12RdJ2zEjmyA3s88AnwTmI3nsgy5YWukLcopXeSYt7V56EGKwJeRZF7D5ylaFoXUIcD6S3jcYS5Br7i7kPjFYQXEzUpS8LRKp3w+5HwUtlo2I6MthSHT9t9RY+tgwYwyyuTmB4HtEnl7EMXMbkn7+LGKoD7QubYqoFu6MrEn7MHCkYiLSi+rrSPF7dStzKpYa/vZopZcVn0UK/fNcEun5iuCc5edMxV2fjpXn2rkz5g72N5wA/IhgoRUv7yD3n9sRx86zDJ7yPhG5D+Vr3fdm4NquXZGo6hWIjPiLQ8wlIVoOAH6KCFwMRhcb1qUnkWtiIMdOM5JVsh2wF3JN7EKwM7sBSZ37AqICfRGFGWqRoJTSuV5aG5zgmomRnQ/lzUzqdx33DwM9t2hDq9/p316Ze4ERRFGjtTvSK2nKAL/PIl/6S5DUmmKKhV/J/dyNREQUcoM5BtnMmGFzkM33w0hDuAuLOFdCODhIcedZDBwSXwVcjuQRFyM40Y9skpewoRZvLHAock18FHsD3IzUzRyKFMAP6C0d0z/mj6tTqxfg2TQp1HdJDC0fCjXFGOof/+b4QlNpomA0kirxlUGesxy53i7F6OMxBL2IA+Ax4C+5sQmIpPsxyGbbZBPgV0h689HIPSyhsnwAWZcGMhRcxLC6BKmrKiZV7TUka+MexJmkkM3O0cg9ZlzAa/ZGohQtiDOgOlPcXZYad9BIFcG01nt6ztffMLLh7vyDK/WVqa6urqk6pbdyss46jX6ldXpr10De6LBpdBvbNNpvoOtBW5N8FlHYDfr7gziar0WuuZspLEMjz8rczz8QQ64RSYc+GnEwBaWofin3nK+SrGFxMBKJan9zkOesYMO6VEzGWS8iAPYEomgI0mfuCGRden/AazZGauUPQ+q5YjPAFWqFRs/0PI7E0Mo5S7x9W6+fO2XugO+76NRBhdVt+dX2rdvDLoz7AWIETQn4XS/wS8QL82lkY12uIpNGrP3jgK2RRsVBH1ozckFdiUTbEirDZkgdxDyCjazliFd3K+D7hKPq9xYSOdgHSRH6A2KQmeyYO98xAx1ozsw5vUZEC2D/ziWdO4Ywz+GDtnrcvHr8bscXs2kIkx2Qv+tXBvj9Q4iRPR0xuMMQ7XgBOA/xIO7LwAX5n0ScPlXToLtO+C7wT4KNrD7kfrEdIol9KcUZWUFo4F4kTXprZF18PuB5TcCPEcNu44Dfx0/Kv54q1Lj07enSxLgKwfFlPDxxypanrO7s6hyb6cp0dnV3rcRhidLqDu3oe3FY0tnd+WJmWeaic5afE3bdt495S+dN1OhjfYOaG9umt/074OmNwCLgbwQbWW8hdVOTEUP8OoozsoJYlzvOF3PHzSAp9ybjEQd33jhLqAzbINlbAxlZjyB/u6nAaRRnZA3ES4hh9wGkhc6NAzwvH4yIL71U4avTcnEjSR1scBu+jsf5rggWwchTtKGllTZvRGEWq6cQT94igiW5r0QWueOJTv3mXeTmMQ25UIOMuMORlLQtI5pDwgamIZsbs58HyELzfSRU/HPEsxcFTyIevh2wC99BIh+/ZxDp23Wpdb/AH1ZXukGfFOYkax2llFn3FJey0UeRDe4OAb97DlEb3A3Z2EaVwncHYlDtDTwa8PsJSH3QlyI6f8IGHGSj8TO8jSk38DdkA/R1wjG4g1iTm8N0JHoVZMR9DnFQbhXRHEpGucr8LqsxE8ZEZxRqXw3t8kxX5oNa6UdzfauCaq63AL7Z4DY8nenODBYpKAvHcVow9jYK1RHw1NFI3c0PsLMpssBixBHdTnSR7Vdyx5+CpKiZaWEKyfC5mvAFYBJs9kAEUHYJ+N3/kH3pB5B9clTr0r1ITeoeiLPRZAtkj/SViM4/KNrVPkNLoUI3tHI6FV7H+svjVo0L2heup5SIli/kr5UOy+BpRmQjvxXwuxXIpuOLyAVVCXqAc5GIxd8Dfv8+5KKLNte8vtkVSaMJ+oz/gniPf0ZwpCkKliB50YcjhaMmpyN5ylbU7fQpp3cr1N98g5oj56+YPyGCedYkWutNjKFVMYwwBwgAACAASURBVEzjs8hCYW4CXaQ/0g7IxqJS3IPUJf4Q25HQhERaT6jgfOqNRiQlK+gzfh6Rdv8CYoBXgl6kV+D2wA0Bv98JWZe2C/hdfGRtp0m2MWt+30MhJzS0vs5Io0eguJ4NCmSrNfpm4DIUNyjUm56XN6G5qLO7M/Tv1ILuBVsadR0At7ZObzVl7scj0ewDAg7zCFKffAIiaFAJ3gDm5M77SMDvD0TKOIrpYZpQHAcQ/BlrJGV4e2RPVKnU4XyT5O9jByMakOj+qRWay3q00mY94uSwz9G0rOnT4Kn90lw8VOZNKfLuPkNLadU90BOLIIWkWhwW8LtrESs98r4WA7AcSVE8ETssPw3xPIf+x0xgJmLgmt7HXuRvcTgVaHg5AH9BamiCrslvItE1C631AmOo2c263wt5brWMfxHRFY9ofQLxBo40xl9DFCdPJZ7Gsf1IlH8v7IhJPtpyYqUnVQcopCbOFC4AqWnZjWBjpxKsRK7JE5G0RS+TkflFmgZXDE6DY32XtWs5VkKha0XXRvjTej6FpJ+jUIt61/Zu2T6t/dNt09qOapvadtCo7KgtFep0PFEArfXC+V3zdwtzXq7rtmDcWxTqHONpo5DUvT0DDnEpIpzzcJjzKoKHkUjG4oDf7YnUhxXTJDmhMPZFouajjfHXkTq6EyleTTsMsoijezek1tiLQsTCTq7khFKkTIn3UYtWLgrVAaCU31miHX3xUK8pytDSWisMo0KjwzC0LkA2zibnIB7muJuj5b0GB2DnK2+FLLaJNyc8tka8N6bi1qtIKtX5FZ+RzcvI9RBkVH0DuXZ95PLw7/YNar41/+n5yeIEoPFtvJRSlYxo7YFEqsyU5ceR6PVAeemV5GFkUbs14Hc/Quo0EsLjJ4iEuskCJJU5LkdPnvy6tD92dGMCEpk16x5jYezrY1/H9LY7RGJoNaxrCKwZ0uiO1mmtP0zvmPZtSufMnNPbOq11nkJ5nRWNrnJDaxJ+7rJzt9BKmymJd7ZOa73L87gJuQeZRlY/IjxxDPE4erz0ItG0r2FnkuyGCCgElX0klMYsJIXUTFl+Ove766xXVJ6nkGs2yOm0ACm7qBRWL621/WtDSx/MtaA52DN0T/vU9iHFAIsytBasXPBejFxc7ZSdOngKUnPlxUVCkmdQXSpK/0C8Cy8b4zsg9RrRFffWDyORm4f55ViOFGIGFQ3HRRYR4Tgr4HenEbBJ0+jzjKFx2ebs1yOYW+2hYotoTUQi56bH8F6kXitIgCAu3kbS1f5kjDtIituHKj6j4cn3kXQpLxrx0LZQXevSXYgwygvG+DbIxj12sYJcao3PwNEqmojW6lGrg7z7KzZZtUl6sNe1TG35GcpXd3LwwqULQ2kE7OD8ELOOyVYavBC7FnktUhN6cRjzCJHfISmzpuG3P+I4Tyif9wLXY0cJH0AcznEq8pqsRmpELzXGFdJra+9KTGLtiLUrMe7NKZ0KT3mwl2PxGL1a698W8rKiDK1sX9ZKRdBuWYbWh5A6KJOTkJBkNfIfRFXF9LZ/DOmtklAe5yO1WV5eQAzcZys/nYJIE2xsXYAhqNA2te1ajF5sSquTLnrwotg3Q1WAX8lT8VYFztmA9IUx++j9G0kZfqMCcyiWPkQG/s/GeDOS+phE18tjN6Qez+QUJHJYjTyKrEGmMMJeBK+xceD7LilUUBuVsjnzvWeuxRRuUFw2VB2FUkqjfQaNWues+2i58+lY0rEZdu35fW3T27yR6SMQMRUv6xBl02qIWgRxLZK61muMH0dloxjDkbzjzGwG/Bgirf9axWc0NOsQ5/LvjfFGRB18oF61oZGekF6DEd3XSgeJ35RE+/T2RW3T2lT+p316++8KeV1xNVqO1fti3TZTtylVnGIcslEwN5hnUR2pYYPxBHKDMRvVtiN1Hgml8UUk7c7LG8iGd3nFZ1Mcaew0wtEYNT9KKa3RPzaeN3HVuFVB9Yl1hUL5G6GjzQU8Cs5B6h68PI0UeMeR914oLrKZud0Yn4TUFSWUxkZItNBM1ZlP9RpZeZYizdRNRcKTkes5bnzfZxc3SMGxbHL9sPyOUFVYJoRW2vc8pa12NkXjOE47RlRCOcqrULsN9nfWRdIFqyFleTD+gRiIZoT3QuR9JZTG6dh7yS5EFK5SIiiloJE93M3G+NbYBlhU+FK6FSp2dfCiUt2UUtO8Xyet9IrZanapnaAzYHVtvpbgyEA1ci9yg7ncM+YAv0aiGHHnUtca47CLbDUiE2oWWhbDVogc6pTc/0ciTRjfRmr/ngEeJJwme99DFhdvH4kdkVSjdH5gTHbMJatTq8/CK/ShaNFa/7FSTTOrEa11kyFkHLWhtTN2se67SKpOuR7DLZD2D+MQtcA3kchsUE+aUulFPN7/wd9z8Au5n7+GeK564WzsZro3IM3SS2UEUuc3A/k7jUEaoq9GWlS8gNyDnqB8BdUHkPoZb2qpAn6BrEvxOQ8Ufd79g6OdKGt5nsJTn6ayKkgl1kKhXjYGykpvnL9i/gS33zVLI/7dMrnlplZa849/jlwTXjqBy8o5dwW5DFG7bPWMjUGMx32orjTbWmAHYK4x1oPc68utC90c2XeMQ9aP/LoUZvbIOkRz4d/Atp7xA5FMjMuDXhQiLyMqjABorc2oYMUpztBCTdae70wZioO7YUcunkM21VF9KccgkpRBYcQZlCYlfQUSxvXW4kxGNtZJGmFxZLDFLxZQfOf5fLPXPXM/hRZCPohIkv6W0jf4LqJQ9gj+HmutyLXyDEjxdeeyzgs12iuYsWtnV+e+wG0lnrv2Uf4oQsQRLQeR4jfvgd9GIlrFsjNyL/g4opIaVNvhIpvp25Gm60+UcB6TN5BI8N34i9B/gqh2VnNUrtrYGWlK7OV/yP29mHVpYyQKv0fu5wMUVie1Ctm0/ozy0qSvRK5Dr/jCRKRutDXwFRVAuapXe/xIUUW0ANA8iWJD2p8qMHsnS4P3mVrpshoAu+vc01F+pUFXu20eh9pRwH7Gy+7D45iLgClIb8oRxvhrlB6FOh2pof6IZ+yjSErkFSUes175Kfb94jvAf0s41o7416Wg6I6LOCZuR4zjoL6NxfIOIs50P/7rbBHiuIosKqe1fkkpn8c2dkOrqNRBV7tbex8rVKnFeBcY59ZIGkyUKmOdiKdgXMCP2RCwGL6HSO16OZXCN/gJ4u01De/HkJt3sSxCNplfpLi/wW5IusMjBEvrFsrLyDXhpRkxGtfT1Nh0IUaKj3Z0bJugKsG38XJwojS0jsL+O/8R6UtVKFMRwZ5nkcVpIbLBHqiA3kE283MQNcNrCHb8FMsDSGqbl4nE0MekxvkZtuH9VYqPbu6LbC5PQOqQC62/HI+IcDyKbLStfnxFcCJg1k//AGl4HAta+R0nUUa0tNL/NB4X9D3TjvG8MgR55i2bNxmF2Tfr7rnT5/4j9/9RGOsCkglzBHYrmbBQiIMpH9Xw/pTTQLof2cOZmTznYbfLSBiY2diG99WI+EihTEIiYk8j68yPkGjSQCl0DmKQfQ8x5q5HUv3K5VFs9eUtkBKbyHBwzKhf7KmDRRlaCuX78LXWpdRnfQrY3Ri7GFP2Olz2QjzVUfAusoB5GYEUTicUxlxsw3sO0S02g7Ed4tn5bBnHuApb6vQQxKMEwEkTT1ql8SvWKNT+nUs79yrjvLWOf+OlrP5AYeEgUWcv71B8z4+fIanOpdZxHIIshB8Z6okFkMHeWJ9AeZuneuJjgCl8cAXBUvpR04xkRFxB6YqBa7Gjc03Y130l8X2fNTqyiJZKqevwrh+6sO+Y0uojxuOgBr0FkdKpNMY9TaFO8zw8DtvRcg7Rqskdh9T5RMFyoMMYmwCWsZkQjMKOOK/B3l8OxY+BefjT9orhQMTRbRp8pXAedobId4i27YQv/VcrXVuGFoaVq5UuRfbYtGbfJdp0hpGIIVdKc+ZCuQo75es4qsCSrgG2Q+pJvFyBNIIul7eQXjLzkb/HAchmah9kkzsXMfDNtKDm3BxmlHHuH2DXW/iuc0c752FuPhxdTi1IrePblJge8BA5FEMNEtnghCXj3gPcg+Sin49ESq8nOF1ifO53s8o851psQ3Ej7M12QjDmuhT0eZZCFvHs/jJ3vM8jEth7Ihve4xFvddC1cThSM1YqN2GLKRyLXRtdKXzfZ+3oyCJabZPb3kDev5wL/eX0kvRGg7yE9BPpJu1or/Lfmt6Rvf8c8AWDsKBrwUwUR/kGNTd6+mY1YV9fXUhGRlRMwI58h815gFlScipJb61COBh4vzHWSXhCYL1IyujlSD38hYguQpCy7sZI2cYeZZ6zD1ER9zIau3VGaLi4PkNL6fjFMAo2PnI3KZ9yjuM4xUa0ZmF7DS/ClqQNk3PY4HHWSLpOFJgL4giii6INJ+bgvw5dypMjfhxJu5mFbGIPQAyc3yAL793AncgNpgO5HvfErokYiXiGSuVZJBXNy6F4Gn63Tm9dgbJS1Q6e3zV/tzLOW5Okn0g3YaTw6mxkhpbpIXwdWXTK4S3kXrYPskjtjdTrnYgYOwcjjpfvYAtibIRstsvtw3c1dn7990M47nBnR2wv/2+x+1IVytuIwtahyD1oV8Sg+hHyN7oVqcO5BTHAvobcF34RcKwfUnrEFOx1qYn4jG+fU0mhoqvRAlzHTSPrCQo1vjnVfGFapwfc84wYOSKjtFqfWqm0+mVOLrpo+uk/G+N75ygn7Xl4GLbBOx8ii+KD3OPG5f6/Bo8hGiK9iHHgZRLyXUgYHHNdegtbIKxY3kYE2vZD1qUPI+vSCch94BBkXfoGtsE1GglSlGsk34jd//Q72MquoeCkrNTB0ekn0qbYTEUp2NBqamqycjazbrZYD/AxxuNeopXM/RCy0clzEdGJDdyJeLG9HE159V/DnWYkH93L1cCTZRzzRCSd6z/kFtkCuB/ZGJvKgwcwcL1NIWSMOTjITW49qf5UB0bkqx6jWiNTI62bbkRiGNtg12YtxpbELpQViEPlvUivnDsZWEylF1EY2x276fnOYHjAi0djp+5sgdSNJQyMuS6tI7iPVqH8AxF2+iuFq0y+jVxHZmuTRkSpq1Tuy83Hy1FEm+ExEP7vhY5mo5Vn7pS5/1Eob23Lkc3dzVd3dHds533evGXzJmeWZS7RaK/n/YWUTmVKOe+5S8/dSSk12zeo+VvLtJYHPCPmNfc80jcpKo7Gnw4/FzvVOCx+j50dYL7fBD+TkfRlLxdSuhrg80jN1XsRI+p2JNMiiD7EGPsg9t9tW8JJ/TSd5+ORVhSho9GWMmNzc3OsUa2Cb7YqqyxDa2TjyGIiWg2IQIGXqwlHVjuIZiSKkS8ofpHyJHoLweyjNJVw6i+GK59hg4ctT5BXtxK8gl24mcKOwBbDk8BdxphXoZJTZ57ahRH50lp/vrOrc+cyzltzpFTK2nQppaLw7pqNNLOU3ndqPhJt+AV2T73BeCZgHiBe7nK5CtuIK9eAG8442IbMDURbJzMY7djG2b5lHtNcl7YinPqLoghIBY48nWxUdtR3EedHns8orZ7KLMssz3RnHswsyyxJkerG/318Rzv6sFNmnFJSpk1Kpc7ByNJQSnnb1kzA7pH0G6JrZ7EZfof2vxFlu6joRSLCXvYnHOGf4crR2HXqpa5LP0bKHi6guDZDXcCXsEspwliXrsUWjYtkXXIcx1z/UCre9MHCvVraUiFZc9LEk4pRCfwYtszipUW8vlhOR1JC8nyX6Bu9XY29SNZ9I9pBMNMJnideefOg5pBblXlM00u5DdJzZD0pnTob2fDnUS5upMo81UZ2ZNbadKVIRbHxML+Pf6d0Z89dlJ7qcwt2OoXp0SyFfuyU1c8QUZrGMGBPbIWtKNeloViD3zAA2ZiXw3XYir4VX5eUazlOIr8m58yc0+v0OZ9BWRv/yWhmIRvSDVknmidd192rfUr7v0o5X2ZpZpZW2i+kpPhT67RWrzT35/ArSmqKUzstlp+zQXygH0ljLbX/aaFcin/DnkLed0Iw5l7oDuxat0K5h4GjV0NxN3Zm1t6UHwF3sftnHYCkJ4bK2q61r2BkM7nKrRFDS1kbzmLrs0wP2uvY3aPDYlf80sZXAX+L6Fxe1mD3faq457BGUNie2j8R/QIwGJYnhPK9rldhb8a9DY05dfqpS4C/eMeUUofPXza/VNWgmqPP7atE6uAERHzFS5w9XkynwhiMOtgSMd/TKMprWTCcMe/P7yCGSZyYkZRyDZJe7PWv8hEt+/tcEYGElu1a3mmb2vZ1jf5ozuAy7/PvAtdr9DHTp03fZe6MuY+XfDKHefjLBbIabdbJmZ/9g8CSks85OIfj38TPBx6O6FxeliDvy0uyFwpmE6T/p5dqWpeaoLzG3TnM99SEGHGhkt433Q/+tgyKeCNaBRdJa623MpqAFVsobH7J7sBWZQuDBiQMn5fFfQsp/KsUt+IPie6IFBuW29F7uLEjdoTzljgm4mHTgLFSC+LzvI30OfKmkO6LUYuhU/pslVWHs8H5kcpFtXyphsOVlLZTB13lhm1oBS30Zv1KJXk1YGw0stkvhweRwmZvWu6+hKPkOdwwr4m7iC6Fq1DMTU0Yapi3Al5FvZlIXUjFUiSVUr7PVeloxTBM2qe1302ujcziJYub3069vWV/tv+N9Mx0oXV0g9KxtOPDmPWQit+3T233yls72JHrqNa9TfCnCD6LyH5XiluRup88+yLvv9Da6Xoh/7l4iXMvNNC6FDReDI8iTg7vvm9fohBlUfxaadWstX5NK/1KykmVpB4aFgUbWgE5jsXc/Juw5YtvL+L1xdBmnOsUwpNtLgTTG6AQicyrKziHWsD0sK/DDllXGjNvXhNOf7fb8BtaHzaf0D65/clMV+YaFJ/3DB+5oGvBvFzEa1ijtW5Shm6MckKv0TKvuaco35AuB7NmIUv5i1n+OHfhL363rrkEHEQwyUtU61KhNGPXhYZ1D9L4oy17UElDC9WnPdlk2omuj9ZQzJk5p5eQ37ujnHO0v7xlHY4lArAttkMvqmvufDZsajUitlJqSlkp3Ia/Ln4T5P0/VcE51ALmutRNeJLupRBUSxeU7VMsGnH2ebUaIlmX2qa2VVXpRTF5lz5DS6GK+eCnYzdevL+I1xfKdvj7odyJqKlUkpXYhp2ZrpRgfyZPICkccTEOOMMYux5RlSsX81rflKBQfIpzMPLas2TrosbP0Y616co62bAjC2Yq5gOBz6ocZuP2LsJLnTWvubpJQy2CyUgbBy9RrEvFcDqiyJWnn9KL4r28gl3zUdF1yUwd1Dq6PlqVJtOV+YRW2kyF/1Xb5DZT2c/8HmqiuQ8dhF/h9pdUvv75AWxhheQ+ZGN+JnHfg8x1aQXFiT0NRl2uS4WrDqJ8MteucosxtII+TLNvUbk4SMrgiNzjXsSDY37RK4H53raJYQ7VjnlNPBPLLIT3I17FKZ6xd/C3BiiHoPdmXRNtU9oeRhTPAP6lUIe0TW8rSWK41gjqqTNizYiwDS1zY2l2rK8kM7EVScOsDTKvua2QGrCEDQStS3FdE6OBBYjstpcOYGlI54h1XTJVRKPuo1VRlNWvrMfJOkH3bvOae5HCWwAUylj86r0vEb3ichBvY5dMJHshG3NdinMvNAmjhpxo16Vgp/Mwo/AaLbTP0FKoYqRPpxmPXyD8m8sPgb08j88mvhD1M/iFHqYP9MQ6xrwmwja8veyH36kwAmkSux3yd/ow/pSa15C0q66Qzr8cMfy9G4sZgKVs5eK2p1Tqp61TW6MSiqlKtNZNZse5d513w0wdHIGt3hblNTcUGfzXnIv0nwkLc0FTyHfObGhcz5j3oFXIdz8K8jVReRTSQHQCsBuiDDnWeM1ipPl6WDyDv4aooutSXGIYUdPR1XEwRvqXQl3YMrMlSDCsEuvej/AraX4XuxltpXgGfypashfy4+C/L0C861IHtsz8xSEeP8iInI4hXjHcKMjQynVV9skwKlVU6uDGxuMw8j29bIM0qc3zGOU1nCwX8/2Z7z8h+mvCy3XYKUJBrAMuQ9JPw+zvlkVuJN6NvrmpAmDutLmPUoebYdWgmrXrDz6PZ3yYEa2NsJuHR3nNDcZnseV8LyPcv3vQewu85uqYSt6DvoHUCxfCo8BpSO+ZMIl3XXLpM76BNR/R0lqrzu5OM+V8dZbsggFeYn7mYYtk7Ye/wex1SOPsuEj2QoMzBnsfHte69Cn86aYgashmG5JyqMt1qaDUwVFNo0x1OLJki7kYTMniMGtxHCSHPb+RdpE+EetCPEexmKphYUg2DzfMz6RcpbVyWYHcZL5ONE20k2tiEFztWpuu78/4fpgRraDPO45rbgZ2M8/XCT+1J+i9JdecnyjXpVLIAj9DRHnCNrIg/nuQ6TipeUMr0505FL+yHhp9/txpcwfaH0V5zY1G9kJ5c/Zt4FshHr8U4r7mqp1qWZcmY/cPfJPCnUOFshpbdXLYXxMFGVr9jf2bm2MN6xqKSR00m5KFeXP5Ln6VpvMJSMmqMMnNZXAU0tvHy+o4JuJhEnAlUqMxO4Ljm6myyTXhIaBeo08pFWZ9ZVB9UqWvufcg/Yy8YgcaMe7DVkbtxe7fllxzfqJcl0ohBXwPcfpcQPie3njXJWVdjzWdOqi1Vgp1pjH81ojGET8a5GVRXnPz8acmVlpxOYhkLzQ41bAujUT6fW5mjH+H8FVJNfY1P+yvicJqtLTV70iPZGQxMsRmdMlUICyVyeCTT30OWzkuDswFJGyZ6lpHI2pa3s8prGsiiNPxX+sOUoC5PVKf5d3QzEQaJ38cEVMJq+eHZUiEdNxhQYC8e9hCGEER7iivOZORSIuHnYzxRcA1EZzPwb6/J9ecn6jWpSBuxK5DGI04eD6EvyB+BLLJORj4JOEVx8e6Lml0r/Edr+mIllJKn7v03C85jnMG0hgYpdSikyaetGqQl0V1ze2FrFd57iIctcpySfZCgxP3utSMpJaa7ZcuILqmyXW3FyrI0HJwtjD6Q7yZ60NRKFFYsAqRLPUe63sB54qDakuLq0bexe/Zj1IRbTAP40gkZbADv0fnm8jf7eSQ5pBcE4MQFNEK+RRBn3elVPiagD9jN8e9FDg1onOOxs5YSK45P5X0rN7O4P2SZiH3oE96xiYhDbXfTzj91WK9Bzk4vcY+oqYjWgBzZ8x9HJjduazzoxp9ak9/z0+GeEkU19xIRLAg/33vRVIG41BcNknWvcGJc11KIWvQp43xvwInRHTORmxDa9hfEwUZWlprM3WwmLRBsD/IjYp8fRDH4V+ULiVcGcpyMFM+hv2FVAJv4ze04iqIXIv0Wrse2dRs7/ndSchNJ4yu4uY1n1wTHhSq2diEhR3RCvq8K3HNpYA/IH1tvFyNFK2HFTE1CXpvyTXnJ4p1qVQeQorRz0GEMPJsBfwYOCqEc8S6LrnK7VPaF9Fq0lqrkFOEY6F1WutdSBRpKMwU8jDuQecgmRjex9XSFDjZCw1OnOvSpeQisR5uBr5MeP0cTepyXSqoRkujzdzNYg0tU1xgSqHnHoAJSD5yntcJL/IQBlONx1GIK9Q6ptqSKXtbaV5E1OC8G3xFOBGHsdi9IpJrwoPZvFShwja03gbWGGNRX3MDLWa3AEcg6bNREfTekmvOj/l5TKSyaTtBnI4thHEEtgR0KcS6LjnKsb7TP13605qPahWJue6Zf5Ni+RD+fo+PIf3YqoVkLzQ4PYjohJeo1yWF9Fn7kjF+G/A5wndyeqnLdamgiJZCjfcN6KJ7jZh9AUYii1qphXbfAsZ5Hv8YCbcWEnINao42Gb+lvQ5YWeLcoLqa8VYrzwJ7eB5XQyPDJcAfgWM9Y59EQt3l3HyCGqMm14QHM6IV0HOnXFyk8esunrEor7l8A3VzMbuD6BczsK+5tUBQX596xlyXGpCNQNzfzbORvlp5UkhE9MIyjxvruqT7da/pXl3FqnLvrbWGec3NRO4VpUa2f4hcH3kyyN6qEMwIbgp7I7yG0iXoHfyRNoj/u1WNPAvs7nkc5bqUN7KOM8b/iTiaeyI8N9j3oHVAd8TnjJ1CxTDGe2tYlVKDFXsGEfTl2pnSDS1z3vNyP6XykPH4afwpZMXQhP1FSW4uNuZnYooExMWt+A2tkcjNoZweRzsaj+vi5lIM2tHNRkVBFAWyz+A3tKK65hTwc/zXEcA9yAbajKxFgXnNPUt0aYq1SlBj0J2I/379ENJg1utMfF+Zx0zhF9yACr9PpVSfkR7MGHdMvUW0zM98JNKwdUmJxzP3QpeXeByQVP4uY+waxDFUCtOw+1fG2Yy3WnkGv6EV5V7oJ0j9uZcHgAOojL6BuS4tI95WTBWhsPQ95aulQaOLNbRewfaKfKzIY9QKu2PfXOquAW0BmJ/JlgRHfipNkBxuUBS0GPYxHj9FHdxcisFMHSQaL7d5zX2Y8FPFFLAYezG7H4lKVEqsZx/jcXIPsnkLWG6M7VP5aVho7HSa8UFPLIJZ2BGMil4TQVHqnlE9Na08WAKPYzs8huteaB/jsYu8/wQ/5vdwD0R5NGwywBxj7BHEyDJrB6NiH+NxXaxLhdZJmamDb5RwrjuMx/uWcIxa4OPG43eA/8QxkSrnbuyCS1OVLQ6C0k/LLdY039dtZR5v2KFQjcZAFIboHcbj9wC7hXyODKJ+6uUR4EAqt5htgmQMeBlM8a6eucN4XC3rktlvKex70BrEk10xXMe1vtOp/lTcNXGV5g3gv8ZYtVxzYWPuhR6GkvaOw507jMcjgD1DPsc8oNUYexRpjl5s4KRUxmLLyNfFulRY6qA/hQGckr4styFFvXnej+QSl1IL9W9E2r0UdkJ6Tni5BH9uaqk5yeDPrQe4k2iL3muVt4AHkWLePIcgKVdxYqbXQHnXwweArY2xuri5FINCpXxpRToS1aP7kYiS15g+hPAanHcALcZYpRczkPdkhx8naQAAIABJREFUOtES4z6Y24CveB7vAMxA6vniYiNE8MnLy2Ue01yX7qXCtVEpN5XVhsBgqj+VGuDpw5nbkP1Pnk9Teh3wLZQu/f8R5HrP04PshbyYRmGhNONXhYZk3RuIvAHq3WcfQnif19nAXGPsGUTl1OztFyUHYtscdbEuDWloaa1VZ3fnxt4xpYuu0QL4O5ISka/2chDJ2kwJx/pb7qcU5mAbWicRzgW3A7bFflMIxx2u3Izf0PoE8F7iU6FRwGxj7BXKm8/RxuMebA9W3aO1Tvl7mUZiaK1DbuyHeMaORBahcuuXzgHajLGnkc1GJRczsK+5pym9Hna4cwtyreU3/Ar5/M6MbUbwBeyU1lI3vCCGo+khr/i65Da4WZX1f8n7nf56NLRuQkQs8oxH0or/WsKxynFMXoDf0HoHOL6M43k5CDvdNdkLBZNF7kPevceXgFMo30l/CqJk6mUJEkUtx4FcCua61E389bAVYcjUwbOWnzUWwyDTuugaLZCF3uwzcQwY26vaxix+XwdcGcdEaoQ/GI8bCKdfTKkch51Kdh2lb/qbsFXnrqVyKWS1g/IpZ6FQUfXxMK+5idgpLsWSxt/7CKToez/Kj0QUyxTsmg/zPSds4CVEAMfL0ZTXfqQcNgXONcb6gBvKOKa5zmYRddWKksqmrO90KlWXEa3bsWuBzb1DrWO+n+dJHIyDYd6jt0Bqp8rhJGyp/6WIkVVpZ/YEYH9j7LIKzyE2hlxMUio1zhxzU26paTDmxbQdcHCJx6o2NsIugL+B0sP69cAS4D5jbA5ioJTCQiRaUazxroBvYMsnu4gUaqkci9wwvZipGQmAVtq34dLoqAyta7H7lpTTg+9k7OhHXIsZiKfce1/XlKdEVg+Y69JU4NASj7U/kqpTioDOdCTiaqYNXoakWpfCaKQdipdbgBdKPF7JuK5rfaf7++syopXF/k4ejK3IVqtsj72v+wPRNcEdDtyI7ZQrZ106EfiRMbYCuT8FCX5FzQ+wM+jqxgE4pKHV0N9gqR052inV0LoSe8Ewc0drlTnAxsbYb+KYSI3xK+Px1ogHthRmIXK0S5CG1ntjK215GYd4r+9Fav7MG8GvkHrAUmjArtdZiaRLJhgorfwbrmhqtEBSNy81xj4JfLCEY/0AMe695Bezim9kkbRbsz/K30laCQzFX7HTO9spLdtiIyRVZwXwF+DLDN6UtgFRGfsposhmipi8gV3EXgzfATYzxmJZl/ob+u2IllOXES2A3+JPV3aAU2OaS9i0499busDvYppLrdCP7YT9KFJHVyxzkN6yXlYizr/lJRyvXDbFdvbcSZ2kDUIBhpbjOJah1dvfW6qh9TaSF+zlQ8DnSzxetbA5Eqb18iiSdpYwOH/AFkQ5HVt1qximI4vWXUj0ogsRQrgFyRO/F7nhrEJubkEKP7cgG+lS+WZuHl7OI5F1D0RhGFoqUu/nj7D/DqXUipq57wCTEMNGl/hziH3IgjkTWxa4lPdVb6wBzjfG3odfvKlYRiFRscuQXjFvIuqzdyD3oNuQNeJdRIzle9h/u3eR6+GVEucwHnvz/jSl1QKVjepXVr1J1snWq6H1NHad+Zfx9/mrRd6HnS5/FXW0qS6DH2M3DM5QvMMnyDEzEdkHlboumbXrxXA6tppzXa1LQxparnJNQ6v/zBlnliM1uwi7l8z5BMtq1wrzMZUZRYFMBzw3wU8fYoB4mYRd81IqCmmcuDsitvEpRAxl8gDP14jx9VlgbYnn3ARJH/LyMvDrEo837NFamxuuKA2t57DTFj5OeRvruJmFpL96+RfiOUwYmsXYKaWLEEniMBiLKM19DLkH7YtErwbqI9WFRFrvKeOcHYg32UuGmBpXNzY1Wt9pJ+vUq6EFIrnt3SM0IOnrtVq3rpD5m3/T+THMpRZ5ETvy9xFqu35vJySq7uU/SKZF3VBIwa9pQLyhlCrHgHgd6U7tZSKi2lWL7If9RXgISRtJKIxfIV5fLych3rFi+BvSDLhU7kHSvo6ldCMLJA3IrNE4G/GcJwRhimGoyMQw8pyD/TdehL0xrQWake+QWZs1XFKRKsFb2A6fLSne8/oUUptreqYL5WXgLMQIK6ftwEewDe/HiLFer7/fTh10nLo2tB4B/myMfZjwlP8qzbews0P+iOyHEgqjAzsQsQC5F9UajYhz2SzJaKHOghBDyrsrrcZr/2cSRj+YDiRMPs0zdgKSVnFNCMcfjHuxQ6ulboA3R+o9vB4oF0kDSQo/C2ct8F2kIDRPE7IIzaJwlb6f5n5mIBuNDyMblhnYhs864H+Id+UhJL3h2dKm7+Pr2KkTDwEXhXDsYYtWOqX0hq+R1pGJYeTpRu5DXgfPe5Fo5sEU5vV/hMFrAEvBjKoUwkL8fXkAfk950ZB6ZCGieurtpfdtpLn6FQUe40lE2no0UmOxJ1L/NxOJoptr7irgCeQ+dBtipJUr6TwOidiahvecEI5dMs3rmrPrGv0Zu9ls3aYO5jkRiXB6I6c/RkSiHon43NcgtYR5ynEE7oQtvvAO5Qk61CP/Qxwt3trfzZD7zycobF/5KOHXCJfSO/dc/O17QAxvU+V12FNIw2IzohWGobUWuel7a5gUUiD6QezoRpg8RDgelkbEO2gqRP0KW0kvYWhuQiJS3nq9GYhH5IsU5wFZmvu52DPWhKSnNiGGWxTRpfchKUheskjoPDG8B8EUw6iAoQWymB2Jf2N9AFLMPa+A1+8XxaSKZDbi2PHyBrYQS8LQ9CGf5S34nWc/RwyhYupMViOOoxuN8Y2BkUjEq5TNy1A0IOuSmRr9e2KW1143Yl3WvAvWeUSL/2fvzMPkqsrE/Z6q7nQnnRUIEHYIq6CgoCCiIO67uKCMgjoqiCLz01EIiJoZgbCoI+467joujNsIKCoqi4IsAi5hJyEJECAb2bvT3ff8/viq7HvPuVVdy92q6nufp57k3qq691TXqXO+/UPCxf4D8aZXGUQE0qNozfDSKL8hmRCuOUgEz1Tn/MfIp8Jdp/MZpCBYuDDOccg8aSSl4qUpjKlZXoOvZG8g2j+uZ5g0dNAaG83RMoltDlfhl87eDvnhF91NapAqdW7/nfvRcJ12OA1/YX4j4qVql22IkeAx0lGy9kas0dOc858Abknhfl2FWwwjg9BBgBFEiXdDCP8T+NcM7t8uxyICtJvT8S5aL6DQ6/wOvzDGLMQKu3sC138SEa7TULIMsqe6gtaDiOckV0a2jXi/6R4uhhHmMkS5D3MAoqS3UxQqC6YinrEDnPN/IJl9uxcZQ6JiXDnlI8D7sx9O0xyJFAFy96X3kU813tyZVNEygYkqWjYRj1aVD+C7x+cjAmsrfUiywCCu/bc754eBN6DNaNthFeJhcDfk9xFf4a0o7IqUbZ/nnP8DjXlGep6Mi2GE+Ru+ccQgoZ5FroZ6BCLguJXqPktOVeW6iLPxjSO7IcbBHbMfTsNchIQuhxlBiry02ocrMYZmDGkxjHgCpM3IY875o5CQsVoFU/JmAPgR0kYlzEpEUdAojtZZjKTTuPwXxS7adCgiv7sGgq/SQ32zXBrxaKUROlhlGPFYuE19n46U5k7CgpgkfUhVGPcHYJFy3n/LfETdx3XEewX/E4kBL1pFpgOBG5EcjDAPIXmIutk0QvbFMMJ8Hr+3Vh/S96+Inq0XIUq8WxHvetSjngTbkJBMt+H0U5G8t3p9sfKgjAgycd/9GcBt2Q4nns1bN3u/aVMyqmgJjyNzzi2i8irEs5V0Lmi7zETG9Srn/DASJeA231Wa52v4fUbLiLfo9OyHMynHIfKbW6n8zxTAo54njVQddD1aSYc8PAC8HEmcDPMUpHDFsxK+X6vsgOSUxZXaXIAvqCmt82n8CmAglQh/AMzIdjg1eREieO3hnF+FJDi7FkqlNnnkaIV5J34+TR+y2X0Cv2RxXpyKrENuO4y/I3HxI5mPqDtZhuTruZ6g/ZB96TmZjyieOcDP8SsMguTIFKalxLw187zfdBAERfldFYEbiDfOPR8RYN2+jHmxL2LUeb5zfhwZ/w2Zj6h7OR2/QFwJKaN/MY3VWciCtyP7p2v8uxspLtVOFeeOZ3JFy0YVLVuySXq0qtwGvA4/JnV35Ef7QfL1ZDwXuAMRnl0uRcpvKslyFlIcxeVNyHxptvR7kvQhwvfV+CGuTyJVx5KoYNhL5K1ojSLedbdSn0ESkK/BL3yTJTORIgdfQQq6hHkAyctJM3G+F/kr0jDYLbc8DykssYDGjJVpcRSyL70y5rnPUrCWKSsPX+mHDhoNHXT4GZKr7FY9PQwp4vXGzEcU5cTKOA51zgdIeXe3CbPSHuNIGObvY547C1mH8oz8GkKKjn0TP4z9IWRfWpPtkIrH5JuEiSpaJVtKQ9ECEWSOB1Y756cgIWPXE63CkgWzkUTVPyAx+mEsUgVGQ3XSwSJJ/XHNDvdHcijyaHR9OGLRPg//97MSsfLdmvGYOh6LzTN0sMpmpIRuXI7TcUzEzWctHL4KCUt22waAeLKOpUeTjDOgarl3w9v7kB5bt5B91MUQko91A/GN1y8mPr8jVxaahQFO9djAqEcrhq8j+d5uGOEsJJz5CuK/9zTZBWl98SP8MMZtSG51YbynXcZWJOrr8pjnnoN4jc4me+/Wq5D9Jy7KazHioFge81zPUVfR+uz9nx3AKdkZ2CAtRQvgZkSgWRbz3DFIid3Pk74GPxUpP39v5V93MxhBQo0WpjyOXsciVuMP4YdT9CPfzWIkj6Y/5bHsh1R4qyVY3QUcTfq9T7oSY5zy7mTu0apSrUT49ZjnZiOld29GvJZpe9mPRqqw/oJ4werXyEarSla63IYIDQ/GPHc4kqP5ZWCvlMcxiLSKuJ94wWoU8Ya4fSKLRMRTYwLN0arBzxBBNs5L/UrgH4j84ebDJM12iEH5PqRgh8s6RAn4Ycrj6HVGkLDML8Y8VzW83IaEj6e9Lz0TKXjxC+LzVX+PyOsPpzyOjqGuorV1YKtf+a+cSlnaMIuBZyBfoksfUoHuAcR6krQlcTekh85SxFsSV2FqCTKJvpnwvZXafArJh3KT00Hyo76OzIkPkGxVMIM0Hf0+YjU6hfjfzHeQufhQgvfuLWzUmFGypTyLiIwh3tR3Et8K4HAkT+p2ZE4kWYJ5CrJZ/g7xnL4o5jXjwMeJz21V0uFe5Hv/ScxzZUTBuQ8Jo3l2wvfeBVGslgBfwK9uCmKcfB5SFKPIRH7XroFFiXANIgvFtQeZjqwBDyGpCwfGvKYdDkTSIh5Ccv3i1rhbKuP7XcL3VuIZR+Tfk/HDmUHCOX+OhDy/g2Rz2fsRBf9q5Ht/WcxrAqRJ8YvRMPYIdRWtcTs+2zuZfDGMONYCr0UqlWyOeX4KIgTdjAjACxHlpxWvxkHI5P0tslldAOwU8zqLlKd8BgWp4tRj/AGpRnlFjef3QIpoPFx5zbtpLXl4KtIfbREi2FyHhGzFCQTrkCTQtxE/T5XGiXq0TG4erTDfQBToWg3OD0O8nI8hyvabiV87JmM2olx9EfFO/ZzazZCXIOGN/4mfx6Gky3okpOt9xLfx6EfWghsRpesTiPLj5tQ1wgFIIvzVSPjNRcQrWCDhXM9AqnsVncjvOqatgxJlKeJNvRQxALnMAD6MyEE3V/7/DJrPHSwhhoQPMyFXfYh4YX0MUe6eixoX8+B7iFfp5hrPPxXZux5DKhS+hdprRz1mIl7VzzMhV8XVKQBZo16KpFQUYe8uFHVjOstBeWbg7OXW2Kz6cVjEq/RTpHfA62u87kDEsvNxRNj9K3APstGtRTTrjYhFZgYyefavPA6hsQl4F7K5XtvSJ1GS4nEkOf1VyNyIc1tXLS/VBPHliJf0XiT0ZwNiDdqKWAXnAHMRweYA4Gn4SZ0uFrFcn42fu6G0RjRHy+aSoxXHYqQB43uQnmi+8Unm0cmVh0XWnrsq/y5jYs6NI2vQDCT8eX9k/TqYyfO+hhHh5iJ6vIJTAfgiogx/ito9bfZDhI7zEK/o35jYl1YT3ZemI/vSfsgadAiNFV65F2lg6ja7LTJRRcvYolRNKzLbkFzw7yIeTbdvVZVnMRHlsw4JY7+v8qjOt/VIrtcMZO+rykKHVo4n4wZEFvp7C59DSY57kNDydwEXEt93dhoSbvgvlePwvvQQMh82Ir/J6hq0GzIfquvQZL/Pbcg6eD7x0R8Kk/wRAxt4vRumDE7JuiHvCsSKeCyyab2wzmuHkMl3dEL3fhARbL6NxL8rxeAKJKziNMTqtmud1+5RecS5upvFIiGt56NezUSx2LIJhZbbUiE8WlXGEQHncqQC6nup3dfGMKG0J8E2RKlfhFqPi8SjiKf788i+9NI6r52GVAg8KqF7P4Qo3V9H5kcnoaGDrVMtfHMC8BHEc1WLOUgRF7cEe6vcjux7P8cpaKLkRoCECv8Eif46g3hDYJWqUp0Eo4jivwhJ21DqUN+9bD238fiHdvpQXlrrdUi+QrVbelpWXYuUeD4ZsTZ/DVWyishWpDDBfEThSlPx2YDk5B2GhLSqkpUwXjGM7Mu7N8Iq4Byk6MF5pLvBrER6yVXn90Mp3ktpnT8hRpxnIiE9aYYQ34jkXuwPfInOU7LACX/T0MGmsUiUzxFIQZ6riA8pTIKxyvVfgYQV/gxVsorIGuCjyL50Dum2lnkCiTDbD0nfUSWrAep6tCx2pokWMNlkjMn7h3Yz4gqdiXi6TkDi4NvpnD6GlOT+FRLTuqTNMSrZMYJYdb6K5Nu9FbEuH0Z7PW7WIqGiP0GseOoWTxFjTdmG9vAChQ7GsQ7J5bwQ8Z6fhBiB2rUWPox4an+EhIMV+W+gRLkNMc5NR8LcX4d4H9wGns0wjuQH/gpR4rpBqFGPVjJYpPLbL5G80JOQkPqjmTz0vR7DiEJ/BWLQfry9YSoZsh6JwLoIcUichBSmaLdQyqPIvnQ5Uuk2LcW+a6kff1lihmO/KFKFqw1Iwt83kM9xODK5qmE7x+AnIa9HciUeQbT+e5Ccruso1mdTWuNuJKTiI0hZ2uchOVcHIkLwPCZikUEWjK1I/Ho1f+Ju4CYkvl2LDWSE20cL0xFKhkU8Gn+qHO+KtKc4GBG24xSvYWQNWo0Izvcic+4GtMl1N7AJCTX/NpJ39wxkXzoQ2Zf2QvKvwm1TAsSD+SjRfel6ZM/qJqKKlpZ3T4LHkeiOzyBK1rMRD2s112YPJCdrNhLabJF5uglJj7gXmXe3IUqW279L6Tz+zERxnF0Qo88hTIQP7oTIQtVqkqOILPQYsi/dU3ncUPlXaYO6ilbJlmaGrczY2EpLRWAM8XSFq7DcSbR7+X+gfa96ibWIJ+rnMc/tiSw2VcvMFmpX8FGywFIOO88NhfZo1eIRxCMOsrZ+OPTcTYjir9bA3mEciZRwG5gvQHIbqtxP8uW5i4pbDEMVrWQZRir0/iHmuachAnZ1DVqJCtG9wKOId9JlCuKQKDMxJ/5CfEVVpQ2ay9Ey6vVRugYVeIuEKWR596TROaf0Oho6mC+6BilhLDonUqeuohWUgoiiZbGq6SqKkgbRtchq2KaidBvW2IhQFwSBKlqKonQ1dRUtY02kwETJlNSjpShK4lhj3bVIFS1F6TKMdYppmbYKFimKohSeyRa5qEfLWlW0FEVRFEVRFEVRJmEyj5aGDiqKkjrGmkgfCWtt3m0kFEVRFEVR2qKuomVL1u1NpR4tRVEURVFaIWJAMUQNLIqiKN1GU1UHS1ZztBRFSR5X4DLk3hhdURRFURSlLSbL0Yp4tDR0UFEURVEURVEUZXKaK4ZhtBiGoijJY7HRHK2S5mgpShcS+V1bYzV0UFGUrqav1hOX28vLDy59cFrkZCINi+2uwJ7ADsBcYCqwHlgO3A5GlTlFURRFURRFUTqamorWXQ/dNWOAgWjexLhpIXTQzgHeDjy78titzosDsNcAnwVzVfP3UhRFURRFURRFyZ+ailbfeN8ML7CwNY/WfsCnG3xtCXixPOxPgFPAbGnhnoqidBYRo06JkoYOKkr3oVUHFUXpKWoqWqX+0gzGnXOmlEQxjBHgfmAZUFWi9gSeioQRVnk9sB3YF4FxRqIoiqIoiqIoilJcaipa5aA8MyBwX92KR2scuAW4pvK4EcyI/zI7A3g38Amgmhv2fOB04PMt3FdRlM4hYtkOCDLyaNkS8Cvgmc4TR4BZks0YFKVd7DeB1zgnXwvm+jxGU4eoRytQj5aiKN1NTUXL7aEFwJZWFC3zF+DIBl63Efg02H8AVzMheJ2GKlqKoqTDe5FwZZdy1gNRlNawr0DyoF36Mx6IoiiK4lBT0QpsMDNqYybYesDWzamPCPMbsL8DXlg5cQjYmdBKIQ5FSQM7FRgEpiDCTAnx3G4DNoIZznFwnUo0R8tmkaNl9wQuTP8+ipIWdhbw5bxHoSiKosRT26NVYkbUyc+mhWZhUOPVSXMbE4oWSBl4VbSUnLAlpFrmLKSJ95RJXr8JeARYCUaLOhQSa4D/ZqJX4J3AYfmNR1Fa4tNMVPLthDkc7aOF9tFSFKW7qaloWexMEzYy2yR6aDWMG7azPsN7K4pLHzC/iddPBw4A5kkobFxOouIQbVhM6g2L3wW8qPL/qxDjTtGFVEUJYV8AvKNycCtiOPhqfuNRksNOQ4xAA6GT42AeyWlAihLCDiDzc5rzxCNavM6npqJlAjMUEX0MmzIYT5VjQv9/AMzqDO+tKJMxjlTMHAZGESWhuvCE8yJmAk8HexuYscxHqdTA7gJcXDnYALwHUbwUpUOwQ4hSZYAxJJf56bkOSWkRWwa2R/aP6iNONhtBIiUUJUOswZ+ftaJ6VoJbr1ypHTpoGIocWpNBfhaAfRvS2LiKFsJQisB6YB2wFtgQHxJoDRLmuh8TC9FUYG+kpYFSDL4IzKn8/2wwDzsRTYpSdC4F9qn8/xIwd4AtvqJlsWEDrjFadRARXA/OexCKUoMS0n5JaZF6ita0sOxhjU1R0bJl4FDEKvfO0BO/RBUtJXfMNuD2Bl5ngScqOVpHMBECuzPYB8FklePYeVgikcomtdw2ezITZbBvQkOtlI7DHot4YUEMOOfnOBglPar7RSnXUShKPBaZo1qhdxJq52gFdlrY2GSxW2q9tjnsq4DPhE70AzsSjUUeqbzmYxrvqXQeZgvYx4FdKif6EM9WRl5hJR47F/hU5WAEeJcqv0pnYacBX0NCBi1wOpit+Y5JSQCLhKNvdB5PBbbLcVyKAvHzcxOwF7BHfsPqDGrnaBnjhg4mpGgxnYmQhzjuQASgBjwIilJY3JzGKaiiVRvjFMOwqRTD+BIS2glwPpi7UriHoqTJ+cC+lf9/Dczv8hxM05SwkUgZrToIkid6XY1w9MwHoygOAXBDvNND52cj1HRJG2Oi1URMZkLi04E/g/062DmTvlpRiokrQIzmMgqlgn0j8PrKwT+AS3IcjKK0gD0SOLNy8Bhwdo6DURLDBNoGRCkuxmpkWXvUa1g85BiZk/JoXUG0VPZsYB7wHOBkpCdIP/CvwDFgjwOzMqF7K0pWzA79fwz1Zk2GiR4kKXjY7YHPVQ4C4LRK3p2idAh2APg6E/kQZ4BZl+OAWsNGTeAGLYahKEp3U9ujRVoeLbMJzJLQ43YwV4E5F9gfqQhWZX/gf9DwAqWjsDswEaIG0ltCLZb5cRmwU+X/nwNzY56DUZQW+DgTlemuBPOTPAejKIqiNEZtRctGc7SwiXm06mC2gnkf8IPQyecDL0j/3orSDrYEdibY/YFDQk+sBx7KZ0wK2FcAb6kcLAc+muNgFKUF7GHAhyoHG4DTcxyMoiiK0gS1qw4aG/FoGRIrhtEIC4GTQsevBa7J8P6KUgc7E2lHEMb9LQVIc8klWtmuIaLFMEpJFMOws4Avh06cAWZj+9dVlKywfUjIYLUR+lnS961jsdEDjVZRFKW7qd1Hi2jD4gyLYQDmPrCrmAi/OjC7eyvKpBjq/3a2Avd2ZA5Fd/EpJOcT4PtgrshzMIrSAh8BnlH5/03Af+c4FkVRFKVJ6jXCi3i0Ahtk6dGCaPGNgZqvUpTiMRU4DOyhYKfmPZjexB6PFNQBWAN8IMfBKEoL2IOAcyoH2vdNURSlA2nYo1WypQw9WnYA2Dl04vHs7q0ok7IR+LNzbgowE5m30yvntgMOB3unFIFR6hCtRha0XY3sg0yEI/4GOHaSnh9PcY5fVsmNqY7of9scj6I0y/uZMDJeDxwM9uA6rz/COX4e2HCz26vINgXAx1ByfoaqOCqK0tXEKloL7cISS6NepKCUqUfrVUS9WP/I8N6KMgkmQMIDw2xFCl+sALs70sLAILkVh4C9Ra3RdbAEkSwtU9fb3gjl0P9PIprz2QiXRQ9tSStHKhkTnsMvqjya4WPO8V7AsnYG1DZBtDV5sm0cFEVRike8MHMX03CS08u23IJHyxqwM5p8zzzgUufkz5u/t6LkhVlBVKCZStRDq7gYTZJXlK7HNaBY9WgpitLdxHq0hgaGpo0xFjk3Hoy34tHqBx4C+wUkGf2e2i+1ZeA1wKeBPUNP/AzMnS3cW1HyZDmwBxPGjB2AR/MbTuGJCFzW2HY9WndQPzTaZZ/Ko8qNJNekXVFa4W6aq7a7C9EQ2NuBtaHj4SQG1SYpVBdVFEUpLrGCyHDf8FBfEH3Klm2rOVrbIb1rPgp2KSIAPQisQ4Sr2cC+wDH4Vv8lwBkt3ldRcsSMg90MVD260+q9WvEs220qWubc5l5vFyJNYau8Hcz97Y1BUdrBfAb4TOOvt/+KlIKvchaY3yU8qHZRj5aiKD1FrKLVP9o/zZajhiYTJJJEu3fl0Qg3AW8Go14ApVMJCxEaClcHg7E2FD2YQDEMRVGKR9SjhXq0FEXpbmKtxqZshtxzwZSgFY/WGHAW8MfK/xvhFuAdwDFglrdwT0UpCoOh/2/LbRQdQGCCqGW7/WIYiqIUD/UoUeHyAAAgAElEQVRoKYrSU8R6tAITTHMrIZeDcgseLRMghS0uBTsEPBM4CPFqzUJyuDYATwJ/B24Dk29VJEVJBDuLaOVMLe9eB2ONFsNQlC7HYEzEc61VBxVF6XJiFS0TmCFH7mFo21CboYNmM3Bt5aEoHYSdAqYJj5TtBw5wTq5OckRdiHq0FKXLsUSL3JiStrxQFKW7iVW0rLFu4v7YmfudOZLBeBSliBwBdi3wGLC+dj8la4DtkeIuU0NPrAezJu1BdjgRgasUlFTRUpTuI+KpDgjUo6UoSlcTX/7YMOSEDmqZY6WXMcC8ymMM7CakQfEooiD0IYrVDGCK894RpEyzUp+8QwcvA74TOl6R8f0VpV3+l2jEyMqcxlGPqEfLqkdLUZTuJl7RCpjm1EhTRUtRhD6kJcHsBl67EVgMZmu6Q+oCDEFE1co8dNCsQ1pOKEqHYjYia05xsZiwbKE5WlXsPOJbgITP9YGdH/OaUS0cpqSL3Q6YE/OEKwftBTbOeLK0UrOhJ4kPHcQOmqimpYqW0svcj4QEziFa4KIW6xFr8mO1wwyVCDbq0TJoeXdF6TpcA4pWHawyF9lj6lEG9og5PwyooqWkyWzi557LbjXOL8PvldkzxBfDMGZq5IQtREd5RckJ8wTwhPzfDgFDiKWxXHmMIWGEW4CNzRXOUCpoMQxF6X6ifbRK2kdLUZTuJj500LHaG4wqWooCVKpnttJTTqmDwQRhp5YNrCpaitJ9uL/rnrVyO6xF8nlbYTTJgShKDBuBR9t4f08bVGopWuFGq1hjteKgoiipYbF5F8NQFCV9oh4tqx4twTyc9wgUpTZmFbAq71F0KrFWY4uNKFrGqkdLUZQUMVHLtjGao6UoXYh6tBRF6SliFS2DiYQOWqwqWoqipEaMZVtDBxWly3A91cZqsSBFUbqbWsLMoHOsipaiKGkSsWxr6KCidB8lnEbkJfVoKYrS3TQUOmjRHC1FUdLDuD02tOqgonQdrgHFzc1UFEXpNhoKHdSqg4qipExE4CoFJVW0FKX70D5aiqL0FA15tNDQQUVR0iUaOmg0dFBRupBS9KCkHi1FUbqaWh6tQedYFS1FUdJEi2EoSvcTNaCoR0tRlC6noWIYmqOlKErKRMu7o+XdFaUL0WIYiqL0FA3laKGhg4qipEs0dNDacl4DURQlG7QYhqIo3U6jOVrq0VIUJT2Ml6OloYOK0n30hQ+steN5DURRFCULtI+Woij5YxkLHxqMerQUpfuI/K5NyaiipShKV9OQoqXFMBRFSRWLK3D1xb5OUZSO5HJ7eRmnGIYJzFiNlyuKonQFtRStSI6WFsNQFCVVTNSjhVFFS1G6ibvuusv3UpdQRUtRlK5GPVqKouSPEzqIRUMHFaWbmOMbT2ygOVqKonQ3DXm0AoJOVLTcz7YDsDswJ4exKMWgDzEiTMl7IEoUY7xcjW7zaE0DDgP2AbbHWWOVnmEImOGc66cX9qXNMb/pQD1aGdCHrDeu8UpbaPQmhglZyEWLUKWAt/AtXLxwCv4fu+iK1t7A8cBRwP7AgcCOzmveV3kAbALuqzzuAP4A3A5enojSmRhgJiK8DCFC7jRqLyLTgKOBLZXHBmAdWm0zMwIbjJlo66xOU7SGgGOAY4GDK/8Pcyiy1lQZB5Yha9DdwPXAdci8U7qDPYHnI2vL/sABwM4xr9sHWAtsZmJfuhPZl/4C3aGMDEwZKLs7rClpjlbCDCH73gwm9r1aa+nOiAG6uu9tRNafzekPU8mIMjALmM2ELDRIbVnoMETuqc6JJyuP0dRH2sX4P8ApvpZb0Bytw4BTgBOAvZp873TgGZXHmyvnngR+A3wXuJou2dx6iBKyaeyEbDTNhp4NVB5zgF0r57YAq4DHKv9XUsIY04lVB2cDJwL/ggjT/U28t4wI2PsALwU+gPQSux34IfB9YGWSg1Uy4anIvvQ65LtthiHg6ZXHmyrnNiD70veAX9LBAo/dZvtMOepECYJAjZvtMxtRmran+WiNPsQoOZMJI8A2YA2y7z2Z0BiV7OgD5iKy0Cya81IZRBEbBLYDdgMsonw/XnkUUR8oNJ6iNWSGBsfcvHRbmBytKcDJwL8hG1qSVIWmE4EngG8A/1X5v1JcBpGQ0J1J3gsyDbFK74kIPCsQxUubbCZPVOCyhfZoHQachRh54sIvWqUEHFF5XAz8Gvgk4tlQiks/omz/G6IkJclM4A2Vx2rgm8i+1HFKuO23fSZwotXKatBskTJiENyVZNcgEDlrXuUxDDxSeahSXGyGEFllLsmGABrEOTEdMR6tQ2ShtQneo6vxvoxxM+79aAtQDKMfeD/wAPA1kleyXHYEFgAPAZ8lPtxDyZdBJET0KMTqkrZgPhMJCTsSmQ8a354ghqhHyxpbRI/WM4ErEa/TSSQv4IQpAy8Hfg/cCLwkxXsprdEHnI6E+n2L5JUslx2ADwNLgC8y4XnvCPpG+7zftBnX0MEmKSOpEkcD80l3DaJy/fnAs5HIoSKuy73OdEQmfhbixUozz8ognq5DgcMRL6oyCZ5wWqI0MO4YLnL2aB0LfAERcusRAH8DbmUizv0JxOW5GbHSDCGb1d5IzPyhyAIytcY1pyIK3inAecCXUKtO3pQQD9aeTL7ojwPrkZy8rZXHWOW8rby/jMyNqYgHayYyT2oxFTgIEXLuQ+LalTbxcrSK5dHaDrgQeDeTb2JPAH9E8q7uBZYj8+9JZM7NRubZrkjOzkGI0LRHnWs+Gwln/hnw/yrXVPLlaGQ/eNokr7PA34FbmNiXHmdiX+pHBKXtmdiXnoZ857XWoUFEwXsr8HHgc3RAqHvQF/S5sQBBWUMHm2BHYF8mL6RjkQiMTUjI+1Yk5HQckZNKyL7Xz8S+Nx3Z+2oZEPuR+bkLYvDWSJ/86UO+k12Z3PA7ishCm5E5MYzMh+q6UZWFBpjI7ZtFfUV+JrJWrUHWtbwdMoXFE2YCAj9Hy+SSozUNuAx4J7Un0QhiYf4hEl6zpoX7DCBekRMQK7VbRANkwn0OeDsSInJfC/dR2mc6onBPq/OaESS2fA2y2bQS5jcFydXaCRGy4+bfTMSiswKxMGs4YRu4VQetsUVRtF6NeNHn1nnNYuA7wK+Af9DaXJiPeK3eggjxcZwAvBjxanyphXso7TMIfBp4D7X3pW1IPtUPkH1pVQv3mYJ4z1+L7DlxURUzKmN5G7J33d3CfbJjnD7PTDFefAWxAExBojfqeQ/GkHm2CjHqtKLAlhFD0NzKI24NHkD24J2R+daxOYMdzvbInKiXk7eFCVloU4v3qeZq7YTMjVpjORJ4EHi4xft0NX5fC6ynaJVsKWtN9SnA5dT2Yj0CfAr4Nu3HiY4g1b6uAz4EvALJv4gTdg4HbgNORZQ7JTt2Rax5tTwK6xBL/zraV3q2MZH4WY1X3x2/2IFBPBGzEWFbLTqtEy2GYXIvhjEFuAjxIMUJ1GPAj5B8mb8kcL8HkXCwLwL7AWcgHjTX2z5Uec1xlec3JHBvpTH2R/alQ2s8vxKZD9+gNaNfmG3ADZXHWcDLEAX7eTGvPRSJ5HgvovAXklKpVA4IIuf6+vrUo1Wf2Yg8VMuLtQWpXrqK9qNtxpF5W/VQ7IhEjsQZNrdHQqnvQgtmZIlB8qRqRUBYZC6sIJm9YRh4tPKYishhu+BHE5WQfWsOooCrASWEJ7Ra4yta5aCcpQD5ciTMIk7JegKxJM5HNrSkk/HGgP8DnoOU5b015jUzEEvlJWieThYYJLxqf+KVrHWIoHsnMh+S9ixtQzaym5CQibgFZCZSwGBWwvfuHUyhimHMBq5BKgG6v3GLVCY9AAndSkLJcrkfKaywF7LOxVmNTwRuRgQhJX1eiBjZ4pSsNYhivA9wKe0rWS7jSOTGsYiidVPMa4YQw+NlFLQXjh33vdSao1WXXZHCO3FK1lbEe34L4rVIWmENKte9pXKfrTGvGaiMr6NyBTuYPmT9qaVkPY7sCYtJxwC3FZGBbkKM2kHMa3ZAZKFa6Tg9ibcgG2u8H/WmsU1ZKVonAz/Hj00PgC8jrtKvkE15yWuRkML3Et/b5sNIAnQzJZ2V5ighSZ5xYTPbEGvanWRj1R9HrEQ3IwuaSz+yCGpyaCtYt9RpborWLkhPq+fGPLcY8SSdgoSLps0TwAeRNhQ3xDx/IFIoI+3iQL3Om4Cr8BsNW+DryPfwBbLxaN+AGALfRbxCdybSGqB4TdnL/m96pDyiilY8eyHGRdfQEyBFum4hmwq4VQ/JLZX7usK1Qca5V8rj6HWmIEptXGPzLYgcdBfxCnHSjCIRGLcSLxtPRfYsd73sWXyPVkzo4Lyt87JQbN6NWORcxWUV4uU6neybeQZILsRhxFsRT0FCSfIOc+pGStRWXNYiP/I4hSdtqgreYnzvVhkReuvl8yhxGOdvaXP5Te2CCLJxisuXEEvd9ZmOSPgHouCdhz/ndkHCnmuFsyntcQrxissa4DWIwrM64zFVFbxDiVfA34QUTimUEXC8NO79pvvH+zV00Gc+UuTAZSviQV9KvDchTYLKff9CvDC/NzJuJXmmIBVN4xSXlYinPY9G91UF70F8hb+qGM7MelBFxA8xMF6VkfHTjjgt7YTH1yGCjGu9uRn5sn6d8v0nYzkStvG5mOdei3jZNIwwOQwSOhqXfLkUqS65LdMR+TyBLHBuI2ODxNTXShxVYrDWOqVOM/dozUYq+7lNZoeRQgPvJd8cvAC4AHgBvidjDlKIo9kGuUp9XoEoNO4+eTtisb0i8xFFeQQ4Hum15vJyJFesMPtSX9Dnhw72a+igwx7Eh4atRoyLrRY1SIpNyL4XZ1yoNXaldfqQyn5unlyA5ELdQ/6VsJcDd+DLZLXG3nN4ilaJkqtopS1cHAP8D75X6NeIUPFoyvdvlFEkLGMBvvb+TmBh1gPqYvZHYn3DWGRReYjiVPjbighdbuhiNeSxXpl4JYSxJs8crX7gF/ierPXASylW4ZvrkTXTLfE+D1kzt8t8RN3Js5BoBXce/h7J3y1Kif0xJIz9/+F7Od6KFHQpBDbwc7RK20qqaE2wE/FeoUcRr3beAnWVMWQ8cbLZfORzKO1jgEPwPVnjiLH5scxHVJv1iCzkejurKRXFC2XOEE/RCggiOVqWVEu7z0WEGFe5+wVSVnlzivdulYuB98WcPw8RypT22BkJhwpjkXC9ldkPZ1JGEfe5W3mpD1kkNay0AYzxLNtZ/t0uws/JWo94C67LcByNcg+ibD3knN8X+CYF8mJ0KHOQipKuJfZqxFNUxEqPlyFhjK4R6sNIiGPuBKXA+02PDo4WRXnImyGkwI7LCqQfX1GMi1UsMq4VMc8dgBoZk2Bv/JyscUTeyCNUcDKqhmdX2RpECpr17L40qUfLkFqzYoP0p3Er1tyEhOrkHRpWjy8BH3POlZDSulqBp3WmIt4sl6I3SBxHmpK6YR3TkER5ZTLcYhjxPVzS4BVIdcEww4ih5/aMxtAKK4AX4ecpvhrxbiitYZBwwb2c87cAbySbQkyt8k3gbOecQYo2xeX8ZIotxXi0RtSjhcgOT8E3Lj2O5L8UmQfwDaBlJPRfjYytMwc/DNMinsQiGnqqbEPCCN11cjt6OKy0kWIYaSlapyJCQZgHEIuhm/fSLEPID/3ZSFne5yDehbiKLa3yCWRjCzMXqY6oNE81t8ldnJfRGU3wxojPHdsRPwxScbDGupbtLDbpOcTnsbyd9otelJGN5XDEM3Z85f97xNyvVR5AckTdObeIeOu4MjmnII2hwyxD9qW882Ma4VL8Ztazga/mMJYIcTla5aGyKlqSWzndOfckkoNTNE9WHPfiR3QMUQDlvkPpI94DdA/JtzRKgxHE8OyGMu9Nj3o646zGUUXLpqJobQ+c75wbQaolNdv8bghpnHc0olg9k/oxwg8iQtS3kIpN7Sxk70WSosMVv16JhGr8XxvX7UV2wa9Q8yRS/CItphJfQGAjreVgjCAhjocSXST3Q1z9GiZTC8tY+C9mMFl4tC5EFOEwX0TCxprBIJ7Lo5hYh/andtW3DUhC+f8iPfnWN3m/MH8GzkEauFcZAD6PeLyUxpmD9EcMMwr8C631xupHjHzPQdaEAxHL7jTkO1+HVC/9M1JYI6m8rw8ARyJ7U5UXIvtrs3M7MQITlN3ddvPWzb2+Jg4BuznnqpVtW5FNyohiPQtR3qYicp5BjIGjiMFgPVLQIglF1yLz+JlEc3F2Q/KIOsFAUST2wu+dtpJ0c7JmEx+NtZrWqjtvRAyB4QglgxgAixwpkgpxwkz0CzaphEpcgm/lX0DzX8CrgZ/QXJjR/MrjHUhVw1MRT0QrDCOb8O1E/26XAb8hm54G3UA/vvVrjNY3m0aoCsdx1QHb8TisQwSmcCPZwcpxFr2XOhJjzJgNfdUWP8woYY5AfvthFgP/3sK1HqC5in8zmfByXYLkd36B1hXx/0IE6ZeFzr0QCXX73xav2Yucj694fwzpVdYoA4j36y3Ai6ndS2YOIlA9HSlacRmibJ2LeDLaYQR4M/BXoo1DP430A8tF8LWB7TPGWVofSkTQ72QOIN5z0Yzc1YdE0+yEKFi1Glb3I/NhJmLYHEeE6KW0n6qxDRn300LnDGJkvKPNa/cScYr3FqSJfVqUEVkorslwOzLsI8g6F253MwvJwy9SIY/UaaRhcdIerX2Q8IwwtxNfOn0yhmgvl+NIxLJ8YhvXuIuoNRlEqP7XNq7Za+yOb/1fSrr5ELuSXgn2h/B/N7tRsL42RcIr755+6OBCouufBc6gtfWunV4hMxAh+xf4RYEaxSJ9Bt2Q6/+gttClRNkdKSYRZjH+2l6P7ZFKbD8FXk9zDTvLSBjo7cg8bJf7kRDSMLsApyVw7dYo+Xv1x4/7eC97tLZDBM8wT9Cc93QW4jE9EBFqm/m9l5E58Szi+1U2yxqk72mY2SSbstHt7I2veD9AutEw+xCvZCXB/fhj34seK4zh52iZaI6WsYkXw1hAVDmySAheUhNpNfBHpPnxFxCL8VcRD1NcfGs/0pCynYqBF+CHfZxFj5e0bJA+fJf1JsQakhaDpNtzKMC3QJXxLVVKlVKmxTAORbwOYb4HXJvQ9bchScs/RdaeS5FQvh8ghpk4Xk57ZeSX4Ye9HYSfb6TE82H89fq9SKhVowwweXn9JxHPdq0Q+UHE6PjhJu5bi0sQIS3Mv9O6Qt8WJjCu8WTcGNMJOUhpsadzPI7/fU1GH5MrV6OIAamWjNWP5LAnoWzVEqyVyZmGH+nVrOLdLLNIt4DbCH513Kn4kQNdzaQ5WhabpKI1D3ibc+5KJISvVVYDv0QUqRupn9PTj4QbXkJU0C4jTYefQmsl5bcg+R7hQhh7INUTv93C9XqJXfDn4VLSTQI+kAmPyRiiGCWtFK9GcnHC3o7dEIW8l624sVhrx03IyJVy6OBZRC1qY7TXB28cyfu8CjHy/IX63tj9gY8iIWNhXoOEfLWqcH0a6fUXFvYXIOHVSm22x/dm/Zb2C6JY4E9ISOANSPREWHHbDXgD8CF8YecSZB79vo37jyDhkN8KnavuwV9p47qtUaLPWdV7OWxwFn5ExSO0H8VhEcF8LaLMbyG6l05DQrd2Ixo1UEIKiN3c5hhGEK/u7qFzs5F9sMjV8oqAWyjJkm6OegmRhar33IrI/0l7mx5G5kNYxtqT1nK/OpI4S0gkdNBgkgzfegu+QNtOQ8UrkEXjbUjT48km5SgidByBbGJh9qA96+838b1a72jjer3CPOd4M+lacHYhGsrwIM1ZrZthmXNcjaVXXExU6DJ41u+kmIX/O/8h7eXPHQAciwjHNzK5oHIfcDLwwZjn/q2NcWzED8E+gmjehOJzEn7ozIVtXG8TMhcOQPqzXYK0LXHXmYeBzyCNsn8bc53P0r7Q8338fTGffSnwDGq9bHDa2TkOaK+67jZkL7sRqfj2CLKXugbLLchadyt+qHGZ+IbJzbIcv+Kcu88rUcr4Xp7VtF+Bux57E+0VmFa/tgC/39oQ7YXcdxS+omWiilBggiT7WZ3sHP+Z5hKNXTbR2mK9DlHO3Enllptvhm34pd2fh7rN6zETvynoCtLzZg0Q3UjWk24T5NX4yaT1KmL2LCbwGhan5dF6E75Q/Zk2r9lq1cD/wvdYHEl7c+Rz+Int7rqrRHH/PnfSWhjpKKJU7Y30s2o0gX0dkp+12Dl/MJI/0w6j+Mr3keRQ/t8Yz3jSqx6tEr5Q/QSteZLGkHDDmxAFp1F5bStSLMX9DubS/tq7Db/v5Y5ovmg95uLnJcc1g06KmUS9jitJtwnyo/iyes/IQv7Et1FFy2CSsvYfgm9ZzTOsbjF+lcN2+z58j6glxyDWUiUed7MJ8JNpk2R/JjaRgPQsOGHc6jpz0Nw9D1My7iKclqLl/h4X43u3s+S7zrGhvXVoDRLCGObN9FjycRPsi6/MtLovrUIUrNUtvHcLUnHQxc0lbIXv4wvUme9L1kbDgS22VxWt7fHXt1arsK1HBHLXg9QIw/jCfIlkile4n6ePZHLAuhVX6dhKe20/6uGGDG6j+dzAZhnDXxd3okf2pTgLQ6QymjEmKY/WC53jbcDlCV27VdzJ1e4CswK4zjnnfm5lAjdxPKm+HnHsTDTRdBmt5eM1ixuHbNAqTB6BDdzvPY3QwSGkz1UYV9HJmjivR7vz43vO8W5IYQzFx12fx5GiJXlwNX6o0F4JXPdx/NDE7HusOVUHDZ5xpVdwf98jNN8/NCniDJtJFEt5Et9Dp/tePCX86pNplj/fk2jj4PvJxrvsykL9+I26u5I4j1ZE0bLWJuXROt45von8u1y7noUkxvMr5/ho0iud2clMwe8SnlZuVj9iua6ymeSag07GVnzhSTccF5NJ1cFj8H/zv0zhPs0Q591sN4Tjt/j5QO76qwju3+U28kvS3oZfbdWtQtYq7r70LDLOkYipOtirHi13/V9D+pEVtYgrdpZEGxKLL0/pvhfPTHzDYlqy8XSi1S7X4Id5psU6fM9rT8wJv4+WMe7Gn4RHyyD5SmGuTeC67WCAZzjnkuhY/TvneBCJiVeixPWwSitG+AAmNg+LhAy2EmrRKu7nSqt/V8dSsiXPun25vTxpr5a7Bq1CyrDnyeHO8SitN1CvshFRGMIc2+Y1uxX373JtHoMI4RqfkhK43H2pD9+7my7GM570oqI1BT8vOS9vFsRHNSX1vbj73jQ0bD4OVx4YQ9bwpDFEQwbHEFkoKwL8cMiekIX8PlrYqDXDJFKRbTd812i7pXPb5fX4fSy+n8B178T/kRycwHW7DVegGCadBsU7Eq309wjpxT7Xwt1IB9HE4Ch+Hy0ee+CxpL1a7u/wj+RnSQbxdJ/unPs5yVSactdXXYN85uLniea5L83Gr87WTiW6MHfj50hkOifcHC16s+qgu+9BvopW3HiSaukT97ni7tfruH+T9aSzL+1JtIn6EtKRuerhzomemA+T52gFiRTDiKtwVKtxZxYcB3zNOXcFyWyy1SILYfZP4LrdhmvVSyNfqh/YL3Q8THtlvFvFFZwNGk4awQbWE7rWDq1N2qPlrkN5rkGzgB8T7ee3FfhYQte/xzmeT7pNoDuRou1LJ+Enh/8moWtb/DmR6b5kjdXQQX/fGyN7YTeMW4TBklxkyQj+d+x+fsX/m6RR0n0aUcfCBqQSYNa4n22AHjA6Z1XefT/neAPpJvu5lJGk4jcAP0VKKoc9bHeSbG8RVbQmx1U00lhc9iMaqhDXtT4L3BLvoBtOlCBG6NqcqGJQJqrUgPSzypLZSLjWJyr3DleUGwVOwReGW8Vdg6agrSZc3H1pmOxyN10GkGbTYR5DvK5Jkeu+VLIlDR301/24vSErBvAVrSdJtq+k+/l03/NJWxaqhgxW5f0A2WfyiOboSaOzL8g4xTBKppTEj85t0prmZnYx8O7QcVzRhSrjwDeQxqGbEhyD26hWm9T6uAm3SYUrVNme6CbyOK2VXU6CcSTXMaz0aax6CFMyY9ZZ92cOzkxS0ZqF/zdPax3aF7gl5v61LHd3A+9ECgQlhbsGgRRWSLuMbyfhrssPk23uZphzEGEozH+SrNCb777k5miZngwdTHvfa4b98IswuM2t22WYaLhaEoU2uoky/neQ9JzYnahjIauKy3HEfbYp5DeeTIgTZCLCiA0SqTo4wzlOI9GvylQmr2QyBnwJ+CzpCB7u53M/v+LPvSQ33T6iYUGj5C9gup8vjfLlHUsQBOOmFI2aGh4eTvJvFPcb3JDg9cOUaaya0u2IMH0FyQv4cZ9N16EoWe5L9XgOomiFWYwf3t4uue5L1thyxJZie9Kj5a5peSmb8/AV7SdIPn9Z9736xP09kpwTg0QjGbaQn9ce4j9b188JvxiG8YphJBE6WJQNrUofkoT+OdKpxqWK1uSkueHsi4RFVHmAZKpntoNuOPUo+0KX7feS59sh7jeY9zr0DOAzwL+TfEjNZnzlTdehKEXYl3ZB+kmGDZzDwFtJ1psFee9L1lnz/JYOvUARwidn4YeNjpBOKHUWbTs6mTg5IKk5YZD+idV7VPM08/LaV8fg3r/rZSFv0htron20TCIeLTfBN83Y0HuBa0LH05A+Ba77tA94aeXxFeD9JLexuUJ11yf7JUBSc2IO0cpd68g2H7AW7ufriY7ojWLGzZj7K7FjiSpacX/vtNahzUTXoDKyBu2IrENh9gIuAd4FnAj8NaExVDe08F+16ze0JslyX4pjNnAlomyF+SCSN5w0+e5Lhr7wX9havwCOkjpTgUOIfvcWKQKTtGJfvbaSD7sSLZ+eR8XlOHpOFpo0dLAUJJKj5eY/pdkN+guVh4sBngK8BTiDqDXvNCSn50SSWRjcRpB5W86LyDjR+ZeEEFgmGjI4TrZ9IupRBEtmYQnKwXjJRuW+crmcpGIQl4OZ1jr0MPCiGs/NAV4B/BtwRFfz36cAACAASURBVOj8/kiRnuchYWPtMg1/zqUVKtmpZLkvuUwDfgE83Tl/GRLWngb57ks2Oh+NMb24BuYZ2TAIHIafq3ov6ZWY15L+9UkrlG6QaPGnvCouuxj8z9f168Ck5d0TqjpYhFA6iwgw5yIWHbeh5xuA9yV0ryKEpBQdd4FJwnsxn2gFm6XkW9UpTFFi8wtJeazsGXTKQTnJxOm432Ae69A64HvAUcB5RA072wE/IJlCKUUMlSwaee1LUxFP1nOd898EPpDiffPdl9xiGL2Zo+V+5qwUrQFEyRp0zj8ArEzxvrrv1Sfu75GELHQA0b99XhWXXdLOSSskcYpW1KNFIh4tt7v9rglcsx2WIxZntwrTR/EXolZwP9+aBK7ZbbjzaiD2VY0zi2gIzgaSa/bZLiX8aktphGl0LEEp8P4eY31jSSpa6/EX9N0SvH6zjAMXABc6558KvCmB68d9Nncd7nXcv8cupB/GMgXpn/Z85/yPkWq5aYZa5bsvORWNDYn06Ow0XEWr3X2vEaYgSpZbRnsJsCLle7ufrxe/83qM4+cstWtom4cY7arkWXHZJW6+d/2c8BUtywrkB7gcWGetTaLs4v3O8faVR548iViUw+xI7ZCfZnAbYbqfX0m+v8YBTAhJFgmHKEp8+FR8Aa4onrZCYErG85zbwCZZAn8U37BShP525yMbYZi3JnBddw0aJ/nSzZ2Ouy5Px8+XSpJ+RKF6uXP+/5BmxWlbdvPel6IVja3Nu0BRHmTdV6ofUbLc+ywjvgVE0mTRjLfTSXJOlJFiYFWKUHE5TNxn63pZyHNRnjP/nINTuE9cnsyBwJ9SuFcz/B9iYQr/HY5Gyi23g7uhZd0YtRNwF9x2N5zwJm6R8NBGca0s2yGhXWGW0XqIRdxn0w0nxHjf+Gh5NBpVUApKSfdcuZdo3LrbtygPhpEwsneGzj07geu6a9BSpLKYMkGtfemRFO5VBr4DvMo5/xvgzWSTp5DrvmSM6bc2ZPsy3W/JjsFd9/srjzT+Fn3Aofh9RKvG9LSpfrYwXS9Ut8AWot9Rrb6vjVAiKs+WkOq2zbw/zK6IAyLMvUgIfCu4stAIPRo6mAbL8BeY52R073psxK9INy/uhU1wEL637u42r9mNuPNhKsk1MyxVrtfow/0dlGNe087YZjnHPbG4NENpuOR7tGyiHi3wf4dFWIPA9yzMoP3CDO5n0zXIZyV+Fa405kQZ+DaiUIX5DfAasmlauzd+6GCmc8L9PRt8L3YPEBch5O4PSVBVsty8vIfJzsMR97m6ujFti7iykFu0ph3iZJl6D5e+mNe0oze4c6InDM5ZKVrjwI3OOTdGPS+SFniPd47HgJsSvkc34FY5MkRLkXYTbvPatCo8dSzb+rZ5Fl1TNkl7tG5wjncnGmaRF0mvQYOIZz6M+9kV8Xy7f5ek9yUDfBGpdhvmj8DryEbJAn9fCsg4osRi02gd02kM43/njTQ3b4YykuvpCuwryTZc1P1cI2Q33zsJVx6YQnteraJi8BWtnpCFsuyj8Qfn+LmkH588Gf3Azs65dnsuvcQ5vhUtqxzHMH4YQTt5e2NtPNxcLhvzmlab/MUtmq263buXgZiG0jaR6nthrsNXatzfax7s4RxvIr4cfaM8D986+fs2rtfNuPvSs0nOw1BVsk51zt8IvIxsrfvuPP8rGSfIuz06sT0ZOgi+cLld7Ktaoww8Dd9o+RjZtzpxP5fue/Gsx5cv2pkT7chCLkHMa1rNfZ+Fn67UE3Miyy7d1yBVtqoMAScA/5PhGFyOx8/PaSd2eXv8De13bVyv21lHVCCci+QNtKLU/LmNcTyLqDK0GvhHG9cLsxN+IYyeWFyaYd6j80bXbhctAucJZu2zDrgdeGbo3FuJ77uXFQZpmh6m3fwJ13uyhnQa4HYD1zjHA0irj6+3eV0DfB54j3P+z8j33Y4i3SyzgFc657Lfl4xnOOnF0EGQdShs4J2GhPi1W26/hHiyXCXrCeAesi0ONRPfkK5VT+MZR4zx4e9tZ1qrCDlKe9ELxxJ1wCSZz7eTczxOj7QcydKjdSv+F3ZKhvd3KQEfc85Z4JdtXPPN+KU5f9TG9bqdJ5zjPmCHPAaSIq7HdAMaPuFx2hGnjeIIAraceI4WwA+d46PIt/rgycB+zrmr2rjeEBKSFubHaE5gLf6Gn6vU7r5kgM8C73XO34FUHMxauHgDvoczj31JQweF1fjGRHefaJYSUgDKDddbBdxF9hV4XaE6QNvc1MOVhabTXeGDJfyiGqtoPVKoo8hS0bJIo84wLwRarXL4tMqjFcrAV/DzGK6n9d5LJeB059ztJOcZ6UaexK+Elmdvo6SZhV/UoN3Q1G4mIniZIHGPFogH3Q2ROKON672J1gulHIt4PcIEtCcEvxN/zn2njev1At91jp9Hc5W6XBbhz6m/Ivtd1t5sg6/w3Q3clvE4wDFCmqAni2GArD9u2ObOtB5hVFWy3ND7VcBisley+vAVx1VkU1mzU3kCX+noJlkobn73jCyUpaIFsuGHJ1MJOKvFaz0VCYe5AvEkNVKppYT0yboZeJfz3DjwgRbHAmI1dJXGb7dxvV7A4v/YZtE9RTH2co4DfMuVMoFr4U7Do/U4Uu0tzLvwLbCN8kUk3PUjSMXRRtgd+C/gt/hVwb6BCOWtMAX4sHPuPrQYz2R8D18IPLvFa10Y896/I0pWHqFTr8JXGr+VwzjAMUgY05MNi6u4rUL68KtCNkIJkTtcJWsN+XiyQBSEnhWqW2QU3+O3M9k0tE4bg5+HPEyPFMKAbHO0AB4Efga8PnTuX5DNqZVETYPEnr8Siff+OyKkLGUiRGsQSSw8GCndW6t8+7lIaEcr9AEfdc6tQYQmpT4PI4JnWOnfi87PKZmFn9C6kh7ogt4G2wiFSwQ2SMOjBXAp0aaxUxHh+IMtXm8vpPHw+Uij9zsRIWctE+XD5yAltp8OHEG8ketvtG54Ang3vhX0kxSncXdRWYF4EcO5ba9HjHl/b+I6LwXOiTk/BvygxbGtA05s8b0l4OPOufXAV1u8XrtEGxb3buggyNqwiaj3eXekh1sznp/diQ+3rxbFaIWNiKzWCn34a9AmND+rEZYjeepVSsCedH4f1p3xQ5dX0EP7UtaKFogw8jomCgT0IeEzL2rzulOAwyuPZhgHzgMuaePeZ+A3yL2MbBOeO5VtiAIStubNQeJ5O9X7Y/DzfiyykCq1iQhepVIpDY8WwLVIee1jQufej3igW/UmVdmj8nh1k++7HRHUWw0t2x74D+fcw6hXvVEuAE5iQgEuI97K59G4QFArp+LpbYzr8Tbeeyq+N+vz5GRJNph+G/5TBj1bDKPKMqJRMP2IMaaZEuy1opLaiQppRwDeBz+Uelkb1+slNiDrfzjPbhfEG9iplav7kDkRpirz9QxZhw7CRLhfmBfiN3OcjCdpv2rR7Ug534vauMYu+ALOGuBzbVyz11iOH5+8L/kYApJgV/w8mZVoEYzJiP6eky/vHuY/neM+5Dfb7Jq4qs1xbAYWIEU52rnWJ/HDhy6gdyu7NcvdSNGQMMcAb8thLEmwI9EqvyDerM/kMBYArIn20cL0vHd/FX6J/13xw4k7hZmIPBRmM+2vkb3EQ86xQYoluZWLO4V98FMAltNjxZnyULQAzsRfYL6EWHMa5SpkMzkJKcV7N41ZYpYhIX3PRUJ4bm3ini4lxGLs5octoIfiTxNgGN/qNQAcmNH9NyPhEtWH29+rGYaA+c65UZIrkdq1GKI5GymGDoLkR/3EOfdc4kO/6nEQcCTwCaQnUyO9kTYhXrVTEcHkYtoLKX0j8Hbn3O3Af7dxzV7k3/GjED5HdutQUpSQfGg3dPk8Mu6dFcFq6KCDxQ8LM4iXq9OMjGVkLXQVgvvpoRCxBHgS34s9Ez/fOy02EZWF2jHUbY+fd7iZ1gvOdSx5/ZiXIV6kT4TOzUZKLz+Xxr/c9ZX3VEs2T0c8IXsh7td+RIDZwERvpCRLjC5EvHFh/ozmZrXCcqQgQbj3xlzkh/pIyvdenNB1ykgIqWvAWILmZk2KxUZ+9yVSCx2s8gGk713Y+7gQCSu8rsFrWOCWygNkDuyBrENzQ9fegKxXS2m9V1wc+wJfc84FSKW5nrIaJsDDSHTCpaFz04HvIxVqJ/NIX41vZGmXVr7DBfj9HO9AjJl54hpO1Ns6IViHi/FMBQ6gsX1pBckXmmhlzh2I3zfrMbRnZCs8gCgpYfl8T2T/SDvX7S8JXWeQ+OJQ99GDinee7sgB4E/4OVU/RJKSi15f/62I1TD8N9yCNL9NSnDvNWYh+Qzhv6lFEtKL3oOj2izStSKvRYoc9Nzi0iwL7184c2DKQHlk08jowoMXZpXfeBrwZefcGsTg4/ZXKhpzEaXQzQf8FPCh7IfTFfQjSvaznfM/QzyHRVde34jsoWFjzzDyeXIvMLTQLiwNLB+YBTC0bWjLmfud6bb36EX6kSbqboW5hxDDTNHZG9/jMoJEC6mBsTV2xldUxpBIhUaiJvKkH5Hj3JzVR+j8wh4tkXfc53xEg57lnP8cEl5YVF4G/B++he5fgW9mP5yuYk/85MlxpEjBev/lhcAgi6JbInwbstmo5bbY/BDphxVmOZKjsyL74TTEDOB3iIAW5mZESVQBp3X2QDxArtHkq8B7KK7R5AVISL0rsL+X/L1ZSn1mA4fhy2T3kX5ERzvsSnzhpzvR9Il2OQi/H9kIomwVNd+7DByKL9NvQmT9ojtQUqGc8/3XITG8byS6wByJTLCrKd6m9nrgcvwEv28hYUdKe2xAhMhwGEK1q3i1ZH+RKAFPwVeyqp64olufFOmrdQLRMsmzKud+RfG8qXOBX+MrWauBF6OllNtlPVKe/01EPUOHI0rYVRRPYHgV8FMkZCfMD2g+71DJnmFkz5jjnN8emWtFNDLugRRqcFlKe9UyFWEdsieFZc0+RBZaS/GMaf1ISwFXyRpDFO+ijTcz8la0QMJzNuDHlB+BJIVeSXG+oPcj+RCuJ+s3SD+wooeVdAqrkQ0nbJmtKlvDFEd56UPCBeP6mNxN8QR0JZ4RRHg+kWjFr9lINdQbKE4C776IJ8vtkbMZWUPvynxE3cl9SLW0Vzjnn155XElxPNWnImHsrifrWjoj3FER1iNCtVtcaztkrylKvpNhIhfe5RG08FNSWEQWmks0X6uqbBXJ8DyIeGTdipkBEo3U062OiqBogRSQmEq0rw2Ip+AEJGY+zxKhMxCP1Vn4hQ5uRTbjdirVKVGqC8wORJVagyw6U5BNJ09v50ziFxaQZNZHsx2O0iZPIpUI30y0ueIQcAqiyNycw7jCnIgI+G4J5VEm1kklOW5D1pzjnPMHIJENN5Cv5X4IqSx5Hv6+dCfSl60oRimlMdYi36ub3zILMfysI1/FeQAxLroRHCAy2j3ZDqfrGUfmxE5E5fUyEvVVBG/nDki4oOtNt0i9gqIYCHKjKIoWwDXIeI51zs9FShcPI0pN1iEbL0T6frnjArgeUbI6tZlckQmQhXs7/DDNmciPexPijciSEmLJOwjfs2kRJaso3g+lOZ5AwpVfQ1SBLiPeomcBN5J97sFc4AtIXyTXa7EZUbKuznhMvcK1SOjL84mGt2+P7EtjSMXJrPel44Bf4Fe9BbgJUbI0R6YzWYUYe9xejIOIwL2NfBTonRAlK64x9+OIN71oqR7dwCgSHbMDUc+WQeSj2YiyNZbxuPqR0NF98Q09AaJk5ddOokAUSdEC6UOzGtkkwptaPyLonADcSzaVePZCqpFdjJ8UDRIPfwI97hJNmXFE+J1J1MsAonzNQwTPjWRj5dsB2Wjm4ictB0i4YE91PO9CHkcqzL0C/3e/HxKmVUYSe9MOHZtSud9P8avggayVL0E9WWlzAxIS9QqiAkUfoui8Hniw8kibPYDPI5Ul58Y8fyXwatT41+msQuaXm+9SRr732YjskUX46nQkjWN34mXGFYhcpqTHKDIntsc38E5FohxKyJxI2+hjENnrqcg8dBlDwgV73pNVJe+qg7V4MfBdJA41jj8BFyJW3KQn1UFIiOBb8Cc0iED/H4h1uWgJ0d1KCalEuHuN5wNEwVlO8jHL1XDFPfEtjFWGEeuNCjfdw3ZIBdFX13h+DXAZ4mlKuvjENOAdyDq0R43X3IwUa3AbfSvp8XzgfxAhI46bkX3pSpLfG/ZH5sPJ+B5+Kve7ECnIpDlZ3cM8xMBTyyi+Gtn30ggfm4WsP3E5yCDz7D6S7+Ol1KYP6VkWZ2QBUXIerjySrm1QQsIV98QPE6yyEZGFNJUmRFEVLRAN/X/w4+PDLAe+B/wIqfDWqtt6HuKdOhk4qs7rHkUUsGtbvI/SHjsgi0ycAgzy/a9HFv7VtL7QGCR0bEckXKJe49zVSFx6UQq2KMlhkKbGi6g9B6qFNL6LhD+36uEeQMqyvxV4HfG5fyAC9WVIU9qiFGPoJXZCvusX1XnNw0iT4x8iuVKt7ks7I2GsJyMNk2vt149XXvPbFu+jFJshxKMUF7JXZTOy762iPSF3KhP73mT3W4zmAObFbkh7JDdkr0qAGAAfq/zbqvGlhCjcO+EX5XB5BEmdUAeEQ5EVLZDxvQMJ36tlVanyBBJ6eCvixr6/cm4DE5NsFhKGth9iIXwqYqWM62AdZgz4IvAx8k887HX6kZhgt79EHJuQPIVNyOazFfkuq/PBIJbCAcSLMBWZH7OprcxVGUHmWJ5FWpRsOADxXL1gkteNIvk61yP5CvchTUc3Ic3MQSyB0xHv7P7I2nM08ByiLQ3iuBPpiXRTsx9ASRSDKMSXEl8UIMwqxDBX3ZfuQxSjjUzkVMysPPZF5tohyL508CTXHge+ghTD0DCd7qaErBl7MnnKxzCy721E1p0tyFwL5/D0IXvcVGTdmYHse7U8FVXGES/6ClSgzpupyB4Sl9oSxiJy8HpEMd6CzJFxJmShElFZaBoiL89i8vm2GVnXNCe0BkVXtKpshyg5p+Ln6jTCNup7JerxK+BcRMhRisMsJJwwLka4EQJqW4PqMYZYbpahITq9xolIaNZkhpk4gsqjnkWwFo8gYWFfQedckZiNKDnvob71vxbt7Eu/Rfpj/aXF9+fKBQ9dMK9sy68OCJ5nrNkVw3QCnsCwwmCuHN46/OuFBy9Uj63PILLv7Uhr8lur+55FDNdLKE5JcUWYC+xNa2sQyHfbylzahshBj6BFUOrSKYpWlZ2ADwKn4SeJJsk4UtHpAjp0I+shZiNWvjmkO59HmYh9zrq6j1IcSkjxgwXAM1K+14OI1+RbZF9dU2mcucD/A07HbzibJONImOqF5N9qoCUWLl04OGgHL7DY06lvNH3UWnvaufPPvTKrsXUYU5F9bydaU5waJUA8sMvQvJsiY5Corz3w+7AlzTCStrMS9Wo2RF3B9PwV5+9aHi2fYI19irFmV4sdMJjHsaykxB22315z7m7n5tGUdRBJUj8FKZwxWZhXo9yJxN//AK0e12lUS9/uzOQhWI0SIEUPHqv8q1YbJczTgLchvbfc3latsg74MdKA9k/onOskBoBXIvvSS/BL8bfK35Bc5O8j1uOO5JP3fnKH0f7RK4EjG32PwXxkwT4LLkxxWJ1OH6Lo74wYn5MwNoZznVehhsVOYwiZDzuR3Bo0hsyFx5C5oftSE8T+KC958JL9xhm/DMNLqG8tCTBce87e50yWu5AmQ0ij4+ORBfwgalcrDLMJiSv9K/B74HeoctUtDCKW5dlIPsw0GrP6jSDxy+uReOP1qMVGmRyDNFd/AfA8JMdmHyY3AAWIpfhuJK/r98DtaHhgNzANybs7HimwdCCN5ZVW8x3+xsS+1LHKVZWFf1jYN7jn4NUWG5YVVgCXBEFwbd+UvrVj42M7loPyS62xH0LKWANYY81bF8xf8P3sR91x9CN73mzEqzGNxkKVx5B9bwOy7z2JFnfqFoaYmBPTEdloMlnIMiELPYkY/zaiylXLeIrWhUsufLfBfJbJkyIBsNi15+5z7vaTvzJTZiN9sOYwMbmqHbQ3INUDtalsbzGI5EOUkc3HMJEMOoaERaiAqyRFH6Js7YCsQdVQ502Vx1qkQpOGBPYOs5B9aTvi96XHkJCcruOiBy863Rr7xdCpm4w1L1swf4FXXKoSSfN7JNEfi127zWybv3DvhZps3zz9yDzrQ/a+EjLnqvveMKpU9RIGXxaCCVloFJGF1MCcIBFFa9HSRadi+XLkvOUug7mSEkustWuNMdtZa/dDKmUdabFPFlDRUhQlYxYtW9R2fsrQtqEtZ+53piofitIlLLQLSwNLB5Yy0RNuw3j/+FPO2/28mp66i5ZcdKjF/oVKxTNr7EXn7n3uORkMV1EUJVH+6Va+cMmFz8XyJSaUrFUW+95z55/741pvvmTpJTuPMfbmtAepKEoHMM4TtFZV759sKW85AymlrihKF9C/rP8wQo23Lfbb9ZQsgAX7LPjroiWLrkT6iGGsecfl9vLzTjQnatSBoigdRR9IJSBjzdeYiN1cEwTB8R/Z9yP/qPfms/Y+6zHgMymPUVGU3mFj3gNQFCU5SkHp6MixLf26kfcZa35tjX1N5XCnBx564BjguqTHpyiKkiYlgEEGT6MSDw1gMB+cTMlSFEVJmK22bK/IexCKoiSHxUaaOhtjHmjwffdH3mfNi5Icl6IoShb0AVjsu/95xrD47L3O/u4CFuQ2KEVROo9z9jmnqTYLFzxwwSGlUunvoVM/OWfPc9YlPCxFUXLEWDMjnA0+GoxuaeR9tmw3myD0RstTkh6boihK2pQWPbjomVgO/ucZyzeNMVrGUVGUVCmZ0rvCx0Ep+EZeY1EUJTUi1QL7bF9DDVVLlGZHThhVtBRF6TxKSA+qf2Ks+Y37oksfu3To0scuHcpsVIqidDULFy+cguFfqscGs3R0z1HNv1CUbqPEo+FD22cPaeRtNvBet2tiY1IURcmIPot9lpnw628c3md4McCFD1748hKld2M4bmzL2GyARUsWbQVuAa4obSt99ewDz9bEdUVRmmZg6sBrgLmhU99YaBZq7w5F6TLMmPmTLYeCZCwnAD9q4K2vc46HFtqFJV0nFEXpJErGmH+GDRrMgwPLB2YtWrLox8aYq6yxr7XYsPt+KnAs8MlgSvDghUsufHXmI1YUpeOx2HeGDgMs38ltMIqipMbZ+559F3BP6NTrFz2w6PB676nIFkc5pw0PMD3p8SmKoqRJCQg3GV3DOD8HXh85B/8AVjjvnWswP1u0dNHb0x2ioijdxMX3X7ybwbzwnycsv14wf8HyHIekKEpKGGMshotDp/oo8fMLllzwtLjXL3pg0fEGE2t4mTk4s60+fYqiKFnTB/zTY2Wxx1HtxI69BfjQOXuf88dqcYyLHrxoDwznhaoUlrB8adFDi/56zl7n3JHx2BVF6UCCcvAOKusMgDVWi2AoShezYK8F375o6UWvpdKAGNitROkvi5Yu+l9r7R8M5kmD2dFiXwK8EjDAOHAfcFDlPXa33XZbn/3oFUVRWqcEDIaOq8LPdYP9g8edu8+5N4QrEC6Yv2D5gn0WnAqcF3rPIONckMFYFUXpcKy1xhr7ttCpNdPHp2vvLEXpYowxtrStdDJwZeh0H5aTDOarwOUW+3ngVYiSFRjMvwF3hV6/8URz4nh2o1YURWmfErDZOTc6zvjbPrj7B7fWetPI3iOLKh4vwfDSRcsW7ZPSGBVF6RIuXHLh8caa+f88YfjOmfudOZLjkBRFyYCzDzx748jeI6/BcBpQL1T47/+/vfsLbasMwwD+vCdp03Zu1elW6uxqMwfeTPwv240OES+cYCcMBHUwh94Vb1Z6uokfsuakVHIno6KwP4gXwyHCdlFQZBtD1AleVKYuSUs3yuwcFcdc0uS8XjSxJ23jYk3OWdrnB7l43/N94bkKHHLO9yr0mb5o3wcK7fD0L9c4IhFR1YWhuA5Bq6d36kD0wPi/bTJiXCftHILiyUJLNK/bAaRqlpSI6p5YsgeeA8hE5UhwaYjIT4UTAz80aj5qGG94OJQPPaXQNgjCAplUS88VX0MY/n644RqubfFsPx9MaiKipQvDwjS0ZD7FmYp25nEW1lwpKlvKLyailS6ejLeq6kvFWqHf2lH7xyAzEZH/CjdcPxQ+i5peO/0IZk86BgCo6Hc+RCMiqioLLn4p6WjpcMFyMqHMpZKG4O7qxSKiZUfwKoCWYmmJxUMwiGhRqvqKtwznwicDC0NEtESWio56GyJS0cum7b+3l6xTVR67SkTlKfZ4qr9u4mYlQ0uJaIVxxp27VPS1Yq3Qkd7NvckgMxERLYUFWfDc87pKNk61Ta331iJytWqpiGhZGUgNPKSij3pax02XmQ4sEBHdtiQvg8DcUzICSQQYh4hoyaxsJDsC4EaxoaqPV7QxZz0xr/VzNYMR0fIhkL3emrOziFaWxESi+VZrVFWclPOeZ1YnIPjUjtojNQ1HRFQjlrnX3BDIqX86gu7BC4Orb7VRXHndW2tIv6pBPiKqc2bUNArE+75Fyr7fPh1YICLyXXYme85JOidiydhOkzZ3eq8lJhLN8VT8xfhY/CyAdzyXxsL58Nv+JiUiqp4wAKilMbjYidm5Wq1uo/s+gLfKbYon4y+ozJ0eBuBMf2f/T+XWE9HK1dTStFNV7ynWAvnYOwidiJY/hUYg6BZId0QjcFLOJIA/oFidmcmsA9CI0l+FcQi273tg32+BBCYiqgILAApzKz7x9N90ks4h86tZ412sqhJLx3ar6HHMTm8HgLyo9PsTl4jqjauu9xCMfM7NHQssDBHdLtoBPAjBBgCN86595lruVrvLHvM/FhFR9RRvlmBGzR2RlsjXUDzmuX4dgtNQXFLVNSKyDcDGed/Ra0ftIV/SElFdOXjxYEfICqUBhAqtk3bU3hFkJiLyXywde9pSa4dCnwewHe2s/QAAAQ5JREFUYO6mQKYV+oWr7tH9m/Z/GUBEIqKqE28xkBpoE8gxgTxXwd6siPT0dfUN1ygbEdU5J+kYCN4t1qr6cv+m/hNBZiKiYJm0aQpruC3shjcoNGM1WpNdHV1XdsmuisbLEBHVC5nfUFWJj8V3Q7EXwLZF1kwJ5HMVjfFvfSIqx6ixIulICkBnoXV1VX7VfT2bezJB5iIiIiLyw4Ihw4WX1A8DODx0cWh9TnKdKtqulv4pIleyG7MXjBjX96REVFea083PunA7Pa0jvMkiIiKilWLBjZZX4bQfnvhDRP+ZC/eNefXRoLIQERER+W3Bo4NERP9XYiKxNjOTuQygqdD6xo7aW4PMREREROSnvwFzu+nn7+UKTAAAAABJRU5ErkJggg=="
+ }
+ },
+ "cell_type": "markdown",
+ "id": "f6329f5d",
+ "metadata": {},
+ "source": [
+ "
\n",
- "Question: The previous example described positive search overhead. There is also negative search overhead, resulting in superlinear speedups. Can there be negative search overhead in this parallel TSP algorithm? (Provided the workers communicate the minimum distance with each other)\n",
- "
\n",
- "\n",
- " a) No, because we use the nearest city first heuristic. \n",
- " b) No, because each worker has to search the whole subtree before the algorithm completes. \n",
- " c) Yes, because the parallel algorithm does not need to search the whole search tree.\n",
- " d) Yes, because the global minimum distance can be found more quickly, enabling the parallel version to do more pruning."
+ "In order to minimize search overhead, workers need to collaboratively keep track of a global minimum distance. However, this needs to be done carefully to avoid race conditions. We show how to do this later in the notebook."
]
},
{
- "cell_type": "code",
- "execution_count": null,
- "id": "fff58498",
+ "cell_type": "markdown",
+ "id": "5e4cee1a",
"metadata": {},
- "outputs": [],
"source": [
- "answer = \"x\" # Replace x with a,b,c or d\n",
- "tsp_check_3(answer)"
+ "### Negative search overhead\n",
+ "\n",
+ "The parallel algorithm might search more branches than the sequential one when we parallelize the pruning process. However, it is also possible that parallel algorithm searches less branches that the sequential one for particular cases. Imagine that the optimal route is on the right side of the tree (or the last route in the tree in the limit case). The parallel algorithm will need less work than the sequential one in this case. The last workers might find the optimal route very quickly and inform the other workers about the optimal minimum, which can then prune branches very effectively. Whereas the sequential algorithm will need to traverse many branches in order to reach the optimal one. If the parallel code does less searches than the sequential one, we way that the search overhead is negative. \n",
+ "\n",
+ "Negative search overhead is very good for parallel speedups, but it depends on the input values. We cannot rely on it to speed up the parallel execution of the algorithm. \n"
]
},
{
@@ -717,7 +743,16 @@
"metadata": {},
"source": [
"### Option 3: Dynamic load balancing with replicated workers model\n",
- "In our parallel implementation, we will use a coordinator process and several worker processes. The coordinator process (or _master_) searches the tree up to a certain maximum depth _maxhops_. When _maxhops_ is reached, the coordinator creates a job and delegates it to a worker. The workers repeatedly get work from the master and execute it. This is an example of **dynamic load balancing**: the load is distributed among the workers during runtime."
+ "\n",
+ "In this third option, we explain a strategy to improve load balance based using the [*replicated workers model*](https://en.wikipedia.org/wiki/Thread_pool) also known as *worker pool* or *thread pool*. In this model, the main processes (aka master or coordinator process) sends jobs to a job queue. Then, workers take one available job from the queue, run it, and take a new job when they are done. In this process, workers never wait for other workers thus fixing the load balance problem. It does not matter if there are some jobs that are larger than others as long as there are enough jobs to keep the workers busy. The main limiting factor of this model is the number of jobs and speed in which the main process is able to generate jobs and send them to the queue. This is an example of **dynamic load balancing**: the load is distributed among the workers at runtime.\n",
+ "\n",
+ "\n",
+ "In our parallel implementation, we will use a coordinator process and several worker processes. The coordinator process will search the tree up to a certain maximum depth given by a number of hops/levels _maxhops_. When _maxhops_ is reached, the coordinator will stop searching the tree and will let any available worker to continue searching in the subtree. In the figure below, the master process will only visit the nodes in the top green box. The worker processes will search in parallel the subtrees below.\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n"
]
},
{
@@ -745,28 +780,19 @@
},
{
"cell_type": "markdown",
- "id": "81219a27",
+ "id": "9e345393",
"metadata": {},
"source": [
- "
\n",
- " Question: To find the right maxhops level is a tradeoff between...\n",
- "
\n",
- " \n",
- " a) Communication overhead (large maxhops) and load imbalance (small maxhops). \n",
- " b) Search overhead (large maxhops) and load imbalance (small maxhops). \n",
- " c) the number of workers (large maxhops) and the job size (small maxhops). \n",
- " d) buffer for the job queue (large maxhops) and idle time of the coordinator process (small maxhops)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "0cf1ec88",
- "metadata": {},
- "outputs": [],
- "source": [
- "answer=\"x\" #Replace x with a,b,c, or d\n",
- "tsp_check_4(answer)"
+ "### Performance impact of maxhops\n",
+ "\n",
+ "We introduced a new parameter `maxhops`. Which is then the optimal value for it? When choosing `maxhops`, there is a trade off between load balance and communication overhead.\n",
+ "\n",
+ "- A small `maxhops` will reduce the number of jobs communicated to the workers (less communication), but reducing the number of jobs is bad for load balance. In the limit, we might generate even less jobs than the number of workers.\n",
+ "\n",
+ "- A large `maxhops` will increase the number of parallel jobs, improving dynamic load balance, but it will lead to more communication.\n",
+ "\n",
+ "\n",
+ "The optimal value of `maxhops` will depend on the given system, the number of workers, problem size, and also the particular input values. It is not possible to determine it in advance."
]
},
{
@@ -776,6 +802,8 @@
"source": [
"## Implementation of the parallel algorithm \n",
"\n",
+ "We will implement this algorithm using the task-based programming model provided by Distributed.jl as it is convenient to implement the replicated workers model.\n",
+ "\n",
"First, let's add our worker processes. "
]
},
@@ -937,7 +965,7 @@
"\n",
"### Simplified example\n",
"\n",
- "We will demonstrate how the workers communicate the minimum distance with each other with a short example. Each worker generates a random value and updates a globally shared minimum. The variable for the global minimum is stored in a `RemoteChannel`. The buffer size of the channel is 1, such that only one channel can take and put new values to the channel at a time."
+ "We will demonstrate how the workers communicate the minimum distance with each other with a short example. Each worker generates a random value and updates a globally shared minimum. The variable for the global minimum is stored in a `RemoteChannel`. The buffer size of the channel is 1, such that only one worker can take and put new values to the channel at a time, thus solving the race condition problem."
]
},
{
@@ -1087,7 +1115,7 @@
"metadata": {},
"source": [
"## Testing the parallel implementation\n",
- "Next, we will test the correctness and performance of our parallel implementation by comparing the results of the parallel algorithm to the results of the serial algorithm for multiple problem instances. "
+ "Next, we will test the correctness and performance of our parallel implementation by comparing the results of the parallel algorithm to the results of the serial algorithm for multiple problem instances. Run it for different values of `n` and `max_hops`. Try to explain the impact of these values on the parallel efficiency."
]
},
{
@@ -1097,12 +1125,13 @@
"metadata": {},
"outputs": [],
"source": [
- "n = 18 # Safe to run up to 18\n",
+ "n = 18 # Safe to run up to 18 on a laptop\n",
"using Random\n",
"Random.seed!(1)\n",
"C = rand(1:10,n,n)\n",
"C_sorted = sort_neighbors(C)\n",
"city = 1\n",
+ "verbose = false\n",
"T1 = @elapsed min_serial = tsp_serial(C_sorted,city)\n",
"max_hops = 2\n",
"P = nworkers()\n",
@@ -1115,6 +1144,50 @@
"@test min_serial == min_dist"
]
},
+ {
+ "cell_type": "markdown",
+ "id": "92e68978",
+ "metadata": {},
+ "source": [
+ "### Super-linear speedup\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9a724509",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "Question: For some values of `n` and `max_hops` the parallel efficiency can be above 100% (super-linear speedup). For example with `n=18` and `max_hops=2`, I get super-linear speedup on my laptop for some runs. Explain a possible cause for super-linear speedup in this algorithm."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "eebe7e9a",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "uncover = false\n",
+ "q_superlinear_answer(uncover)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "19835531",
+ "metadata": {},
+ "source": [
+ "## Summary\n",
+ "\n",
+ "- We studied the solution of the TSP problem using a branch and bound strategy\n",
+ "- The problem is $O(N!)$ complex in the worse case scenario, where $N$ is the number of cities.\n",
+ "- Luckily, the compute time can be drastically reduced in practice using the nearest city first heuristic and branch pruning.\n",
+ "- Pruning, however, introduces load imbalance in the parallel code. To this fix this, one needs a dynamic load balancing strategy as the actual work per worker depends on the input matrix (runtime values).\n",
+ "- A replicated workers model is useful to distribute work dynamically. However, it introduces a trade-off between load balance and communication depending on the value of `maxhops`.\n",
+ "- The parallel code might suffer from positive search overhead (if the optimal route is on the left of the tree) or it can benefit from negative search overhead (if the optimal route is on the right of the tree).\n",
+ "- In some cases, it is possible to observe super-linear speedup thanks to negative search overhead.\n"
+ ]
+ },
{
"cell_type": "markdown",
"id": "c789dc7a",
@@ -1130,15 +1203,15 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Julia 1.9.1",
+ "display_name": "Julia 1.10.0",
"language": "julia",
- "name": "julia-1.9"
+ "name": "julia-1.10"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
- "version": "1.9.1"
+ "version": "1.10.0"
}
},
"nbformat": 4,
diff --git a/dev/tsp/index.html b/dev/tsp/index.html
index ec79fe1..0ec9318 100644
--- a/dev/tsp/index.html
+++ b/dev/tsp/index.html
@@ -1,5 +1,5 @@
-- · XM_40017
How to parallelize the solution of the traveling sales person problem
-
How to fix dynamic load imbalance
The concept of search overhead
+
A dynamic load balancing method
@@ -7568,9 +7568,17 @@ a.anchor-link {
"It's not correct. Keep trying! 💪"end|>printlnend
-tsp_check_2(answer)=answer_checker(answer,2)
+tsp_check_2(answer)=answer_checker(answer,4)tsp_check_3(answer)=answer_checker(answer,"d")tsp_check_4(answer)=answer_checker(answer,"a")
+functionq_superlinear_answer(bool)
+bool||return
+msg="""
+ Negative search overhead can explain the superlinear speedup in this algorithm. The optimal speedup (speedup equal to the numer of processors) assumes that the work done in the sequental and parallel algorithm is the same. If the parallel code does less work, it is possible to go beyond the optimal speedup. Cache effects are not likely to have a positive impact here. Even large search spaces can be represented with rather small distance matrices. Moreover, we are not partitioning the distance matrix.
+ """
+println(msg)
+end
+println("🥳 Well done!")
Given a graph $G$ with a distance table $C$ and an initial node (i.e. a city) in the graph, compute the shortest route that visits all cities exactly once, without returning to the initial city.
In this notebook we will study another algorithm that works with graphs, the traveling sales person (TSP) problem. The classical formulation of this problem is as follows (quoted from Wikipedia) "Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once?" This problem as applications in combinatorial optimization, theoretical computer science, and operations research. It is very expensive problem to solve (NP-hard problem) which often needs parallel computing.
+
+Note: There are two key variations of this problem. One in which the sales person returns to the initial city, and another in which the sales person does not return to the initial city. We will consider the second variant for simplicity.
+
Our version of the TSP problem can be formalized as follows. Given a graph $G$ with a distance table $C$ and an initial node in the graph, compute the shortest route that visits all nodes exactly once, without returning to the initial node. The nodes on the graph can be interpreted as the "cities", and the solution is the optimal route for the traveling salesperson to visit all cities. The following figure shows a simple TSP problem and its solution.
The sequential algorithm finds a shortest path by traversing the paths tree of the problem. The root of this tree is the initial city. The children of each node in the graph are the neighbour cities that have not been visited on the path so far. When all neighbour cities are already visited, the city becomes a leaf node in the tree.
-
The possile solutions of the problem are the paths from the root of the tree to a leaf node. Note that we assume the children are sorted using the nearest city first heuristic. This allows to quickly find a minimum bound for the distance which will be used to prune the remaining paths (see next section).
A well known method to solve this problem is based on a branch and bound strategy. It consisting in organizing all possible routes in a tree-like structure (this is the "branch" part). The root of this tree is the initial city. The children of each node in the graph are the neighbor cities that have not been visited in the path so far. When all neighbor cities are already visited, the city becomes a leaf node in the tree. See figure below for the tree associated with our TSP problem example. The TSP problem consists now in finding which is the "shortest" branch in this tree. The tree data structure is just a convenient way of organizing all possible routes in order to search for the shortest one. We refer to it as the search tree or the search space.
@@ -7626,13 +7637,13 @@ a.anchor-link {
-
+
-
Of course, visiting all paths in the tree is impractical for moderate and large numbers of cities. The number of possible paths might be up to $O(N!)$. Therefore, an essential part of the algorithm is to bound the search by remembering the current minimum distance.
When building the search tree we are free to choose any order when defining the children of a node. A clever order is using the nearest city first heuristic. I.e., we sort the children according to how far they are from the current node, in ascending order. This allows to quickly find a minimum bound for the distance which will be used to prune the remaining paths (see next section). The figure above used the nearest city first heuristic. In blue you can see the distance between cities. The first child is always the one with the shortest distance.
The algorithm keeps track of the best solution of all paths visited so far. This allows to skip searching paths that already exceed this value.
-
For example, in the following graph only 3 out of 6 possible routes need to be visted when we cut off the search after the minimum distance is exceeded. (The grey nodes are the ones we don't visit because the minimum distance had been exceeded at the previous node already.)
The basic idea of the algorithm is to loop over all possible routes (all branches in the search tree) and find find the one with the shortest distance. One can optimize this process by "pruning" the search tree. We keep track of the best solution of all paths visited so far, which allows us to skip searching paths that already exceed this value. This is the "bound" part of the branch and bound strategy.
+
For example, in the following graph only 3 out of 6 possible routes need to be fully traversed to find the shortest route. In particular, we do not need to fully traverse the second branch/route (figure below left). when visiting the third city in this branch the current distance is already equal to the full previous route. It means that the solution will not be in this part of the tree for sure. In figure below (right), the gray nodes are the ones we do not visit because the minimum distance had been exceeded before completing the route.
@@ -7662,40 +7673,14 @@ a.anchor-link {
-
+
-
Note that it is not necessary that the graph be fully connected. Variations of this algorithm work for sparse graphs or directed graphs as well.
-
In the previous example, the shortest route was also the leftmost path in the graph. Although it is more likely that the shortest route be found in the left part of the graph when using the nearest city first heuristic, the solution can be anywhere in the search tree.
-
-
-
-
-
-
-
-
-
-
-
- Example: Look at the following graph and its corresponding search tree. If $x\leq 15$, the shortest route is the leftmost branch of the search tree. If $x > 16$, the route is situated on the right side of the search tree.
-
The total number of routes we need to traverse is $O(N!)$, where $N$ is the number of cities. This comes from the fact that the number of possible routes is equal to the number of possible permutations of $N$ cities. Thus the cost of the algorithm is $O(N!)$, which becomes expensive very quickly when $N$ grows.
+
In practice, however, we will not need to traverse all $O(N!)$ possible routes to find the shortest one since we consider pruning. The nearest city first heuristic also makes more likely that the shortest route is among the first routes to be traversed (left part of the tree), thus speeding the process. However, the solution can be anywhere in the search tree, and the number of routes to be traversed is $O(N!)$ in the worse case scenario.
+Note: The implementation of this algorithm is rather challenging. Try to understand the key ideas (the explanations) instead of all code details. Having the complete functional implementation is useful to analyze the actual performance of our parallel implementation at the end of the notebook, and to check if it is consistent with the theory.
+
The first step is preprocessing the distance table to create a new data structure that takes into account the nearest city first heuristic. This is done in the following function.
@@ -7743,7 +7731,18 @@ a.anchor-link {
-
+
+
+
+
+
+
+
+
Execute the next cell to understand the output of sort_neighbors.
The data structure we will use for the connections table is a matrix of tuples of the form (destination, distance). The tuples are sorted by their distance in ascending order (per start city).
-
-
-
-
-
-
-
-
-
In [ ]:
-
-
-
C_sorted=sort_neighbors(C)
-
-
-
-
-
-
-
-
-
-
-
-
-
The connections matrix can be indexed by a city. This returns a Vector{Tuple}} of all the destinations and their corresponding distances.
+
The output is a vector of vector of tuples. The outer vector is indexed by a city id, for instance:
@@ -7814,6 +7789,17 @@ a.anchor-link {
+
+
+
+
+
+
+
This returns a vector of tuples that contains information about the connections to this city of the form (destination, distance). In this case, city 3 is connected to city 3 at distance 0 (itself), then with city 1 at distance 3, then with city 4 at distance 3, and finally with city 2 at distance 4. Note that the connections are sorted by their distance in ascending order (here is where the nearest city first heuristic is used).
+
+
+
+
@@ -7831,7 +7817,7 @@ a.anchor-link {
-
Next, we write an algorithm that traverses the whole search tree and prints all the possible paths. The tree is traversed in depth-first order. Before we go to a neighbouring city, we also have to verify that it has not been visited on this path yet. If we reach a leaf node, we print the complete path and continue searching.
+
Next, we write an algorithm that traverses the whole search tree and prints all the possible paths. To this end, the tree is traversed in depth-first order using a recursive function call. Before we go to a neighbouring city, we also have to verify that it has not been visited on this path yet. If we reach a leaf node, we print the complete path and continue searching.
Now, we add the computation of the minimum distance. At each leaf node, we update the minimum distance. Furthermore, as we add another node to our path, we update the distance of the current path. That makes it necessary to include two more parameters in our recursive algorithm: distance, the distance of the current path, and min_distance, the best minimum distance found so far.
Now, we know how to traverse all possible routes. We just need a minor modification of the code below to solve the TSP problem (without pruning). We add a new variable called min_distance that keeps track of the distance of the shortest route so-far. This variable is updated at the end of each route, i.e., when a leaf node is visited. After traversing all routes, min_distance will contain the distance of the shortest route (the solution of the ASP problem).
@@ -7928,6 +7914,19 @@ a.anchor-link {
+
+
+
+
+
+
+
+
+Note: We could further modify the function so that we also return a vector containing the cities in the shortest route. However, in this notebook, we will only return the distance of the shortest route (a single value) for simplicity.
+
+
+
+
@@ -7936,7 +7935,8 @@ a.anchor-link {
In [ ]:
-
functiontsp_serial_no_prune(C_sorted,city)
+
verbose::Bool=true
+functiontsp_serial_no_prune(C_sorted,city)num_cities=length(C_sorted)path=zeros(Int,num_cities)hops=1
@@ -7965,7 +7965,7 @@ a.anchor-link {
else# Set new minimum distance in leaf nodesmin_distance=min(distance,min_distance)
-#@show path, distance, min_distance
+verbose&&println("I just completed route $path. Min distance so far is $min_distance")returnmin_distanceendend
@@ -7983,6 +7983,7 @@ a.anchor-link {
Finally, we add the pruning to our algorithm. Anytime the current distance exceeds the minimum distance, the search in this path is aborted and continued with another path.
Finally, we add the pruning to our algorithm. Anytime the current distance exceeds the minimum distance, the search in this path is aborted and continued with another path. By running the function below, you will see that only three routes will be traversed thanks to pruning as shown in next figure.
@@ -8033,6 +8034,7 @@ a.anchor-link {
functiontsp_serial_recursive!(C_sorted,hops,path,distance,min_distance)# Prune this path if its distance is too high alreadyifdistance>=min_distance
+verbose&&println("I am pruning at $(view(path,1:hops))")returnmin_distanceendnum_cities=length(C_sorted)
@@ -8054,7 +8056,7 @@ a.anchor-link {
else# Set new minimum distance in leaf nodesmin_distance=min(distance,min_distance)
-#@show path, distance, min_distance
+verbose&&println("I just completed route $path. Min distance so far is $min_distance")returnmin_distanceendend
@@ -8072,6 +8074,7 @@ a.anchor-link {
n=11# It is safe to test up to n=11 on a laptopusingRandomusingTestRandom.seed!(1)C=rand(1:10,n,n)C_sorted=sort_neighbors(C)city=1
+verbose=false@timemin_no_prune=tsp_serial_no_prune(C_sorted,city)@timemin_prune=tsp_serial(C_sorted,city)@testmin_no_prune==min_prune
@@ -8119,7 +8123,7 @@ a.anchor-link {
-
You can observe that, especially for larger numbers of cities (n=11 or n=12), the performance of the algorithm with pruning is much better than the performance of the algorithm without pruning.
+
You can observe that, especially for larger numbers of cities (n=11), the performance of the algorithm with pruning is much better than the performance of the algorithm without pruning.
Unlike the previous algorithms we studied, in this problem we don't know beforehand how much work is performed since we don't know where the pruning cuts off part of the search tree. Still, we want to divide the workload among multiple processes to enhance the performance.
The first idea how to parallelize the TSP algorithm is to assign a branch of our search tree to each process. However, as mentioned in an earlier section, the number of branches in the search tree can be up to $O(N!)$. This would require an unfeasibly large amount of proecesses which each do only very little work.
We can (at least in theory) assign a branch of our search tree to each process. However, as mentioned in an earlier section, the number of branches in the search tree can be up to $O(N!)$. This would require an unfeasibly large amount of processors which each do only very little work. Thus, we skip this option as it is impractical.
Instead of assigning one branch per worker, we can assign a fixed number of branches to each worker. This way, each worker can perform the pruning within their own subtree and less workers are needed.
Instead of assigning one branch per worker, we can assign a fixed number of branches to each worker. This would be a good strategy if we do not consider pruning. However, it is not efficient if we include pruning (which is essential in this algorithm).
However, this approach has a problem with load balancing. Since we don't know beforehand how much pruning can be done in each subtree, some workers might end up doing less work than others. This uneven distribution of workload leads to some workers being idle, which impairs the speedup.
Another disadvantage of this kind of parallel search is that the pruning is now less effective. The workers each run their own version of the search algorithm and keep track of their local minimum distances. This means that less nodes will be pruned in the parallel version than in the serial version. This is called search overhead.
Pruning is essential in this algorithm but makes challenging to evenly distribute the work over available processors. Image that we assign the same number of branches per worker and that the workers use pruning locally to speed up the solution process. It is not possible to know in advance how many branches will be fully traversed by each worker since pruning depends on the actual values in the input distance matrix (runtime values). It might happen that a worker can prune many branches and finishes fast, whereas other workers are not able to prune so many branches and they need more time to finish. This is a clear example of bad load balance. We will explain later a strategy to fix it.
Another disadvantage of this kind of parallel search is that the pruning is now less effective. The workers each run their own version of the search algorithm and keep track of their local minimum distances. This means that less nodes will be pruned in the parallel version than in the serial version. The parallel code might search more routes than the sequential ones. This is called search overhead.
@@ -8226,7 +8230,7 @@ a.anchor-link {
-Question: How many nodes are pruned in total when we assign two branches to each worker? Look at the illustration below.
+Question: How routes are fully traversed in total when we assign two branches to each worker? Look at the illustration below. Assume that each worker does pruning locally and independently of the other workers.
@@ -8266,42 +8270,46 @@ a.anchor-link {
-
In this example, the parallel algorithm prunes less nodes than the serial version because not all workers are able to use the global minimum distance as a pruning bound.
+
In this example, the parallel algorithm traverses more routes (1 more) then the serial version because not all workers are able to use the global minimum distance as a pruning bound. Remember that the sequential code only traverses 3 routes completely. See figure:
-
+
-
-Question: The previous example described positive search overhead. There is also negative search overhead, resulting in superlinear speedups. Can there be negative search overhead in this parallel TSP algorithm? (Provided the workers communicate the minimum distance with each other)
-
-
a) No, because we use the nearest city first heuristic.
-b) No, because each worker has to search the whole subtree before the algorithm completes.
-c) Yes, because the parallel algorithm does not need to search the whole search tree.
-d) Yes, because the global minimum distance can be found more quickly, enabling the parallel version to do more pruning.
+
+
-
+
+
+
-
-
In [ ]:
-
-
-
answer="x"# Replace x with a,b,c or d
-tsp_check_3(answer)
-
+
+
+
In order to minimize search overhead, workers need to collaboratively keep track of a global minimum distance. However, this needs to be done carefully to avoid race conditions. We show how to do this later in the notebook.
The parallel algorithm might search more branches than the sequential one when we parallelize the pruning process. However, it is also possible that parallel algorithm searches less branches that the sequential one for particular cases. Imagine that the optimal route is on the right side of the tree (or the last route in the tree in the limit case). The parallel algorithm will need less work than the sequential one in this case. The last workers might find the optimal route very quickly and inform the other workers about the optimal minimum, which can then prune branches very effectively. Whereas the sequential algorithm will need to traverse many branches in order to reach the optimal one. If the parallel code does less searches than the sequential one, we way that the search overhead is negative.
+
Negative search overhead is very good for parallel speedups, but it depends on the input values. We cannot rely on it to speed up the parallel execution of the algorithm.
+
+
+
@@ -8309,7 +8317,8 @@ d) Yes, because the global minimum distance can be found more quickly, enabling
-
Option 3: Dynamic load balancing with replicated workers model¶
In our parallel implementation, we will use a coordinator process and several worker processes. The coordinator process (or master) searches the tree up to a certain maximum depth maxhops. When maxhops is reached, the coordinator creates a job and delegates it to a worker. The workers repeatedly get work from the master and execute it. This is an example of dynamic load balancing: the load is distributed among the workers during runtime.
+
Option 3: Dynamic load balancing with replicated workers model¶
In this third option, we explain a strategy to improve load balance based using the replicated workers model also known as worker pool or thread pool. In this model, the main processes (aka master or coordinator process) sends jobs to a job queue. Then, workers take one available job from the queue, run it, and take a new job when they are done. In this process, workers never wait for other workers thus fixing the load balance problem. It does not matter if there are some jobs that are larger than others as long as there are enough jobs to keep the workers busy. The main limiting factor of this model is the number of jobs and speed in which the main process is able to generate jobs and send them to the queue. This is an example of dynamic load balancing: the load is distributed among the workers at runtime.
+
In our parallel implementation, we will use a coordinator process and several worker processes. The coordinator process will search the tree up to a certain maximum depth given by a number of hops/levels maxhops. When maxhops is reached, the coordinator will stop searching the tree and will let any available worker to continue searching in the subtree. In the figure below, the master process will only visit the nodes in the top green box. The worker processes will search in parallel the subtrees below.
@@ -8338,34 +8347,20 @@ d) Yes, because the global minimum distance can be found more quickly, enabling
-
+
-
-Question: To find the right maxhops level is a tradeoff between...
-
-
a) Communication overhead (large maxhops) and load imbalance (small maxhops).
-b) Search overhead (large maxhops) and load imbalance (small maxhops).
-c) the number of workers (large maxhops) and the job size (small maxhops).
-d) buffer for the job queue (large maxhops) and idle time of the coordinator process (small maxhops).
-
-
-
-
-
-
-
-
-
In [ ]:
-
-
-
answer="x"#Replace x with a,b,c, or d
-tsp_check_4(answer)
-
We introduced a new parameter maxhops. Which is then the optimal value for it? When choosing maxhops, there is a trade off between load balance and communication overhead.
+
+
A small maxhops will reduce the number of jobs communicated to the workers (less communication), but reducing the number of jobs is bad for load balance. In the limit, we might generate even less jobs than the number of workers.
+
+
A large maxhops will increase the number of parallel jobs, improving dynamic load balance, but it will lead to more communication.
+
+
+
The optimal value of maxhops will depend on the given system, the number of workers, problem size, and also the particular input values. It is not possible to determine it in advance.
@@ -8376,7 +8371,8 @@ d) buffer for the job queue (large maxhops) and idle time of the coordinator pro
We will implement this algorithm using the task-based programming model provided by Distributed.jl as it is convenient to implement the replicated workers model.
+
First, let's add our worker processes.
@@ -8555,7 +8551,7 @@ d) buffer for the job queue (large maxhops) and idle time of the coordinator pro
We will demonstrate how the workers communicate the minimum distance with each other with a short example. Each worker generates a random value and updates a globally shared minimum. The variable for the global minimum is stored in a RemoteChannel. The buffer size of the channel is 1, such that only one channel can take and put new values to the channel at a time.
We will demonstrate how the workers communicate the minimum distance with each other with a short example. Each worker generates a random value and updates a globally shared minimum. The variable for the global minimum is stored in a RemoteChannel. The buffer size of the channel is 1, such that only one worker can take and put new values to the channel at a time, thus solving the race condition problem.
@@ -8721,7 +8717,7 @@ d) buffer for the job queue (large maxhops) and idle time of the coordinator pro
Next, we will test the correctness and performance of our parallel implementation by comparing the results of the parallel algorithm to the results of the serial algorithm for multiple problem instances.
Next, we will test the correctness and performance of our parallel implementation by comparing the results of the parallel algorithm to the results of the serial algorithm for multiple problem instances. Run it for different values of n and max_hops. Try to explain the impact of these values on the parallel efficiency.
@@ -8733,12 +8729,13 @@ d) buffer for the job queue (large maxhops) and idle time of the coordinator pro
In [ ]:
-
n=18# Safe to run up to 18
+
n=18# Safe to run up to 18 on a laptopusingRandomRandom.seed!(1)C=rand(1:10,n,n)C_sorted=sort_neighbors(C)city=1
+verbose=falseT1=@elapsedmin_serial=tsp_serial(C_sorted,city)max_hops=2P=nworkers()
@@ -8755,6 +8752,65 @@ d) buffer for the job queue (large maxhops) and idle time of the coordinator pro
+Question: For some values of `n` and `max_hops` the parallel efficiency can be above 100% (super-linear speedup). For example with `n=18` and `max_hops=2`, I get super-linear speedup on my laptop for some runs. Explain a possible cause for super-linear speedup in this algorithm.
+
+
+
We studied the solution of the TSP problem using a branch and bound strategy
+
The problem is $O(N!)$ complex in the worse case scenario, where $N$ is the number of cities.
+
Luckily, the compute time can be drastically reduced in practice using the nearest city first heuristic and branch pruning.
+
Pruning, however, introduces load imbalance in the parallel code. To this fix this, one needs a dynamic load balancing strategy as the actual work per worker depends on the input matrix (runtime values).
+
A replicated workers model is useful to distribute work dynamically. However, it introduces a trade-off between load balance and communication depending on the value of maxhops.
+
The parallel code might suffer from positive search overhead (if the optimal route is on the left of the tree) or it can benefit from negative search overhead (if the optimal route is on the right of the tree).
+
In some cases, it is possible to observe super-linear speedup thanks to negative search overhead.
+
+
+
+
+
@@ -8766,6 +8822,6 @@ d) buffer for the job queue (large maxhops) and idle time of the coordinator pro