Commit a7e1871c authored by Nitin Shukla's avatar Nitin Shukla
Browse files

Final stuff

parent 7e7bc69f
%% Cell type:markdown id: tags:
# A breif introduction
![A breif introduction](../pic/par1.jpg)
%% Cell type:markdown id: tags:
# Soul principle of Julia's parallel computing
### Julia’s message passing principle
* Julia provides a multiprocessing environment based on message passing to allow programs to run on multiple processors in shared or distributed memory.
### Communication in Julia is generally “one-sided”
* Programmer needs to explicitly manage only one processor in a two-processor operation
* These operations typically do not look like “message send” and “message receive”
* Resemble higher-level operation like calls to user functions
%% Cell type:markdown id: tags:
## Why Julia is fast?
![A breif introduction](../pic/par2.jpg)
%% Cell type:markdown id: tags:
## In Julia functions are generic and dynamics
%% Cell type:code id: tags:
``` julia
function mandel(z)
c=z
maxiter = 80
for n = 1:maxiter
if abs(z) > 2
return n-1
end
z = z^2 + c
end
return maxiter
end
```
%%%% Output: execute_result
mandel (generic function with 1 method)
%% Cell type:markdown id: tags:
## Everything is a virtual function calls?
%% Cell type:code id: tags:
``` julia
methods(abs)
```
%%%% Output: execute_result
# 13 methods for generic function "abs":
[1] abs(x::Unsigned) in Base at int.jl:173
[2] abs(x::Signed) in Base at int.jl:174
[3] abs(z::Complex) in Base at complex.jl:277
[4] abs(x::Rational) in Base at rational.jl:259
[5] abs(a::T) where T<:Dates.Period in Dates at /Users/nitshuklayahoo.com/opt/julia/usr/share/julia/stdlib/v1.8/Dates/src/periods.jl:102
[6] abs(x::Bool) in Base at bool.jl:149
[7] abs(x::Float16) in Base at float.jl:522
[8] abs(x::Float32) in Base at float.jl:523
[9] abs(x::Float64) in Base at float.jl:524
[10] abs(x::Real) in Base at number.jl:164
[11] abs(::Missing) in Base at missing.jl:101
[12] abs(a::Pkg.Resolve.VersionWeight) in Pkg.Resolve at /Users/nitshuklayahoo.com/opt/julia/usr/share/julia/stdlib/v1.8/Pkg/src/Resolve/versionweights.jl:32
[13] abs(a::Pkg.Resolve.FieldValue) in Pkg.Resolve at /Users/nitshuklayahoo.com/opt/julia/usr/share/julia/stdlib/v1.8/Pkg/src/Resolve/fieldvalues.jl:51
%% Cell type:code id: tags:
``` julia
function mandel!(z::UInt32)
c::UInt32=z
maxiter = 80
for n:Int = 1:maxiter
if abs(z)::UInt32 > 2
return n-1
end
z = (z^2 + c)::UInt32
end
return maxiter::Int
end
```
%%%% Output: execute_result
mandel! (generic function with 1 method)
%% Cell type:markdown id: tags:
# Avoid Runtime uncertainty
* Sophisticated type system
* Type interference
* Multiple dispatch
%% Cell type:code id: tags:
``` julia
@code_native mandel!(UInt32(1))
```
%% Cell type:markdown id: tags:
## Different flavours of parallelism
%% Cell type:markdown id: tags:
![Diff](../pic/par1.jpg)
%% Cell type:markdown id: tags:
# Creating a “kernel function”
%% Cell type:markdown id: tags:
import Pkg; Pkg.instantiate()
%% Cell type:code id: tags:
``` julia
using .Threads, BenchmarkTools, Test, InteractiveUtils
```
%% Cell type:code id: tags:
``` julia
l = 2^20;
a = fill(1.0f0, len) ;
b = fill(1.0f0, len) ;
```
%% Cell type:code id: tags:
``` julia
function serial_mult(a::Array, b::Array)
c = similar(a)
for i in eachindex(a,b)
@inbounds c[i] = a[i] * b[i]
end
end
```
%% Cell type:code id: tags:
``` julia
res = serial_mult(a,b);
```
%% Cell type:code id: tags:
``` julia
@test all(res .== 2.0f0)
```
{
"cells": [],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 4
}
......@@ -9,14 +9,26 @@
2. when multiple threads are executed simultaniously we have multithreaded parallelism.
#### It has some cost:
1. Synchronisation issue: when multiple threads are accessed by the same variable.
2. All threads must live in the same physical machine
Julia's multi-threading provides the ability to schedule Tasks simultaneously on more than one thread or CPU core, sharing memory. 
### What will NOT be covered
MPI See the example from M. Creel “https://github.com/mcreel/JuliaMPIMonteCarlo.jl”
%% Cell type:markdown id: tags:
# Julia implements multithreading using task parallelism
%% Cell type:markdown id: tags:
![Multithread](../pic/threaded.jpg)
%% Cell type:code id: tags:
``` julia
using BenchmarkTools, Test
using Base.Threads:@threads, @spawn
......@@ -59,25 +71,14 @@
%% Cell type:code id: tags:
``` julia
# The number of real threads that Julia can run is fixed at startup
julia -t 16
export JULIA_NUM_THREADS = 16
# julia -t 16
# export JULIA_NUM_THREADS = 16
```
%%%% Output: error
syntax: invalid assignment location "export JULIA_NUM_THREADS" around In[31]:2
Stacktrace:
[1] top-level scope
@ In[31]:2
[2] eval
@ ./boot.jl:373 [inlined]
[3] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
@ Base ./loading.jl:1196
%% Cell type:code id: tags:
``` julia
# squential
......
......@@ -24,10 +24,32 @@
3. Julia code can be run using a slurm batch script by including the julia command with the file name of the code
`````
This adds 4 worker processes for the master to control.
%% Cell type:markdown id: tags:
----
# Julia uses the master-worker model for distributed computing
---
 processes do not share any memory, not heap or data
%% Cell type:markdown id: tags:
![Distribue](../pic/dis1.jpg)
%% Cell type:markdown id: tags:
----
# Distributed (or multi-core or multi-process) parallelism
----
%% Cell type:markdown id: tags:
![Distribue](../pic/Dis2.jpg)
%% Cell type:code id: tags:
``` julia
using Distributed, BenchmarkTools, Test, InteractiveUtils, Hwloc
```
......
......@@ -12,10 +12,32 @@
Topics that we cover here:
1. Array programming
2. Data-Level Parallelism
%% Cell type:markdown id: tags:
## The GPUs and CPUs were designed with different goals in mind
%% Cell type:code id: tags:
``` julia
```
%% Cell type:code id: tags:
``` julia
```
%% Cell type:code id: tags:
``` julia
```
%% Cell type:code id: tags:
``` julia
using Pkg
Pkg.instantiate()
......
%% Cell type:markdown id: tags:
# A breif introduction
![A breif introduction](../pic/par1.jpg)
%% Cell type:markdown id: tags:
# Soul principle of Julia's parallel computing
### Julia’s message passing principle
* Julia provides a multiprocessing environment based on message passing to allow programs to run on multiple processors in shared or distributed memory.
### Communication in Julia is generally “one-sided”
* Programmer needs to explicitly manage only one processor in a two-processor operation
* These operations typically do not look like “message send” and “message receive”
* Resemble higher-level operation like calls to user functions
%% Cell type:markdown id: tags:
## Why Julia is fast?
![A breif introduction](../pic/par2.jpg)
%% Cell type:markdown id: tags:
## In Julia functions are generic and dynamics
%% Cell type:code id: tags:
``` julia
function mandel(z)
c=z
maxiter = 80
for n = 1:maxiter
if abs(z) > 2
return n-1
end
z = z^2 + c
end
return maxiter
end
```
%%%% Output: execute_result
mandel (generic function with 1 method)
%% Cell type:markdown id: tags:
## Everything is a virtual function calls?
%% Cell type:code id: tags:
``` julia
methods(abs)
```
%%%% Output: execute_result
# 13 methods for generic function "abs":
[1] abs(x::Unsigned) in Base at int.jl:173
[2] abs(x::Signed) in Base at int.jl:174
[3] abs(z::Complex) in Base at complex.jl:277
[4] abs(x::Rational) in Base at rational.jl:259
[5] abs(a::T) where T<:Dates.Period in Dates at /Users/nitshuklayahoo.com/opt/julia/usr/share/julia/stdlib/v1.8/Dates/src/periods.jl:102
[6] abs(x::Bool) in Base at bool.jl:149
[7] abs(x::Float16) in Base at float.jl:522
[8] abs(x::Float32) in Base at float.jl:523
[9] abs(x::Float64) in Base at float.jl:524
[10] abs(x::Real) in Base at number.jl:164
[11] abs(::Missing) in Base at missing.jl:101
[12] abs(a::Pkg.Resolve.VersionWeight) in Pkg.Resolve at /Users/nitshuklayahoo.com/opt/julia/usr/share/julia/stdlib/v1.8/Pkg/src/Resolve/versionweights.jl:32
[13] abs(a::Pkg.Resolve.FieldValue) in Pkg.Resolve at /Users/nitshuklayahoo.com/opt/julia/usr/share/julia/stdlib/v1.8/Pkg/src/Resolve/fieldvalues.jl:51
%% Cell type:code id: tags:
``` julia
function mandel!(z::UInt32)
c::UInt32=z
maxiter = 80
for n:Int = 1:maxiter
if abs(z)::UInt32 > 2
return n-1
end
z = (z^2 + c)::UInt32
end
return maxiter::Int
end
```
%%%% Output: execute_result
mandel! (generic function with 1 method)
%% Cell type:markdown id: tags:
# Avoid Runtime uncertainty
* Sophisticated type system
* Type interference
* Multiple dispatch
%% Cell type:code id: tags:
``` julia
@code_native mandel!(UInt32(1))
```
%% Cell type:markdown id: tags:
## Different flavours of parallelism
%% Cell type:markdown id: tags:
![Diff](../pic/par1.jpg)
%% Cell type:markdown id: tags:
# Creating a “kernel function”
%% Cell type:markdown id: tags:
import Pkg; Pkg.instantiate()
%% Cell type:code id: tags:
``` julia
using .Threads, BenchmarkTools, Test, InteractiveUtils
```
%% Cell type:code id: tags:
``` julia
l = 2^20;
a = fill(1.0f0, len) ;
b = fill(1.0f0, len) ;
```
%% Cell type:code id: tags:
``` julia
function serial_mult(a::Array, b::Array)
c = similar(a)
for i in eachindex(a,b)
@inbounds c[i] = a[i] * b[i]
end
end
```
%% Cell type:code id: tags:
``` julia
res = serial_mult(a,b);
```
%% Cell type:code id: tags:
``` julia
@test all(res .== 2.0f0)
```
{
"cells": [],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 4
}
......@@ -106,11 +106,11 @@
* Julia is NOT a interactive langauge, is compiled
### A simple [benchmark](https://julialang.org/benchmarks/) relative to C:
![benchmarks](pic/benchmarks.svg)
![benchmarks](../pic/benchmarks.svg)
-----------------------------------
%% Cell type:markdown id: tags:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment