"2. Implementations & benchmarking of __sum function__ in: \n",
" * Julia (built-in) \n",
" * Julia (hand-written) \n",
" * C (hand-written) \n",
" * python (built-in) \n",
" * python (numpy) \n",
" * python (hand-written) \n",
" \n",
"Consider the __sum__ function `sum(a)`, which computes\n",
"$$\n",
"\\mathrm{sum}(a) = \\sum_{i=1}^n a_i,\n",
"$$\n",
"where $n$ is the length of `a`."
]
},
{
"cell_type": "markdown",
"id": "5812ea46",
"metadata": {},
"source": [
"## 1. Julia built-in `sum` function"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a17adb59",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import Pkg; Pkg.instantiate()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "12536e98",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"a = rand(10^7) # 1D vector of random numbers, uniform on [0,1]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1c220406",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"@which sum(a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8a7ab7a3",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"sum(a)"
]
},
{
"cell_type": "markdown",
"id": "4fcbcdbe",
"metadata": {},
"source": [
"The expected result is ~ $ 0.5 * 10^7 $, since the mean of each entry is 0.5. \n",
"So let's try to time the execution time of this function by using `@time` macro:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "edd112c1",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"?@time"
]
},
{
"cell_type": "markdown",
"id": "320efc31",
"metadata": {},
"source": [
"So what is the performance of Julia's built-in sum? "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3b17f7df",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"@time sum(a) # try to repeat the execution of this cell!"
]
},
{
"cell_type": "markdown",
"id": "69d7b634",
"metadata": {},
"source": [
"The `@time` macro can yield noisy results, so it's not our best choice for benchmarking!\n",
"\n",
"Luckily, Julia has a `BenchmarkTools.jl` package to make benchmarking easy and accurate:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "40217f68",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import Pkg; Pkg.add(\"BenchmarkTools\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a916508a",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"using BenchmarkTools"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4efb5fcf",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"@benchmark sum(a)"
]
},
{
"cell_type": "markdown",
"id": "3bfc362f",
"metadata": {},
"source": [
"If the expression to benchmark depends on external variables, one should use `$` to \"interpolate\" them into the benchmark expression to avoid the problems of benchmarking with globals. Essentially, any interpolated variable `$x` or expression `$(...)` is \"pre-computed\" before benchmarking begins. So in short with `@btime` `$` is used to \"interpolate\" them into the benchmarked expression in order to get a correct benchmark results."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bcc80f47",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"@benchmark sum($a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5b8abfde",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x = 1\n",
"@btime (y = 0; for _ in 1:10^6 y += x end; y)\n",
"@btime (y = 0; for _ in 1:10^6 y += $x end; y)"
]
},
{
"cell_type": "markdown",
"id": "777708f1",
"metadata": {},
"source": [
"We have seen before the performances of Julia built-in sum function. Let's save them in a dictionary:"
"But that could be doing any number of tricks to be fast, including not using Julia at all in the first place! Of course, it is indeed written in Julia, but would it perform if we write a naive implementation ourselves? \n",
"So that's about 2x slower than the builtin definition. We'll see why later on.\n",
"\n",
"But first: is this fast? How would we know? Let's compare it to some other languages..."
]
},
{
"cell_type": "markdown",
"id": "7c591bf6",
"metadata": {},
"source": [
"## 3. C `sum` function\n",
"\n",
"C is often considered the gold standard: difficult on the human, nice for the machine. Getting within a factor of 2 of C is often satisfying. Nonetheless, even within C, there are many kinds of optimizations possible that a naive C writer may or may not get the advantage of.\n",
"\n",
"If you do not speak C, do not read the cell below, but one could be happy to know that it is possible to put C code in a Julia session, compile it, and run it. Note that the `\"\"\"` wrap a multi-line string."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6c1e8f7d",
"metadata": {},
"outputs": [],
"source": [
"using Libdl\n",
"C_code = \"\"\"\n",
" #include <stddef.h>\n",
" double c_sum(size_t n, double *X) {\n",
" double s = 0.0;\n",
" for (size_t i = 0; i < n; ++i) {\n",
" s += X[i];\n",
" }\n",
" return s;\n",
" }\n",
"\"\"\"\n",
"\n",
"const Clib = tempname() # make a temporary file\n",
"\n",
"\n",
"# compile to a shared library by piping C_code to gcc\n",
2. Implementations & benchmarking of __sum function__ in:
* Julia (built-in)
* Julia (hand-written)
* C (hand-written)
* python (built-in)
* python (numpy)
* python (hand-written)
Consider the __sum__ function `sum(a)`, which computes
$$
\mathrm{sum}(a) = \sum_{i=1}^n a_i,
$$
where $n$ is the length of `a`.
%% Cell type:markdown id:5812ea46 tags:
## 1. Julia built-in `sum` function
%% Cell type:code id:a17adb59 tags:
``` julia
importPkg;Pkg.instantiate()
```
%% Cell type:code id:12536e98 tags:
``` julia
a=rand(10^7)# 1D vector of random numbers, uniform on [0,1]
```
%% Cell type:code id:1c220406 tags:
``` julia
@whichsum(a)
```
%% Cell type:code id:8a7ab7a3 tags:
``` julia
sum(a)
```
%% Cell type:markdown id:4fcbcdbe tags:
The expected result is ~ $ 0.5 * 10^7 $, since the mean of each entry is 0.5.
So let's try to time the execution time of this function by using `@time` macro:
%% Cell type:code id:edd112c1 tags:
``` julia
?@time
```
%% Cell type:markdown id:320efc31 tags:
So what is the performance of Julia's built-in sum?
%% Cell type:code id:3b17f7df tags:
``` julia
@timesum(a)# try to repeat the execution of this cell!
```
%% Cell type:markdown id:69d7b634 tags:
The `@time` macro can yield noisy results, so it's not our best choice for benchmarking!
Luckily, Julia has a `BenchmarkTools.jl` package to make benchmarking easy and accurate:
%% Cell type:code id:40217f68 tags:
``` julia
importPkg;Pkg.add("BenchmarkTools")
```
%% Cell type:code id:a916508a tags:
``` julia
usingBenchmarkTools
```
%% Cell type:code id:4efb5fcf tags:
``` julia
@benchmarksum(a)
```
%% Cell type:markdown id:3bfc362f tags:
If the expression to benchmark depends on external variables, one should use `$` to "interpolate" them into the benchmark expression to avoid the problems of benchmarking with globals. Essentially, any interpolated variable `$x` or expression `$(...)` is "pre-computed" before benchmarking begins. So in short with `@btime``$` is used to "interpolate" them into the benchmarked expression in order to get a correct benchmark results.
%% Cell type:code id:bcc80f47 tags:
``` julia
@benchmarksum($a)
```
%% Cell type:code id:5b8abfde tags:
``` julia
x=1
@btime(y=0;for_in1:10^6y+=xend;y)
@btime(y=0;for_in1:10^6y+=$xend;y)
```
%% Cell type:markdown id:777708f1 tags:
We have seen before the performances of Julia built-in sum function. Let's save them in a dictionary:
%% Cell type:code id:ac55aa78 tags:
``` julia
j_bench=@benchmarksum($a)
```
%% Cell type:code id:ddf52068 tags:
``` julia
d=Dict()
d["Julia built-in"]=minimum(j_bench.times)/1e6
d
```
%% Cell type:markdown id:2af9911a tags:
But that could be doing any number of tricks to be fast, including not using Julia at all in the first place! Of course, it is indeed written in Julia, but would it perform if we write a naive implementation ourselves?
So that's about 2x slower than the builtin definition. We'll see why later on.
But first: is this fast? How would we know? Let's compare it to some other languages...
%% Cell type:markdown id:7c591bf6 tags:
## 3. C `sum` function
C is often considered the gold standard: difficult on the human, nice for the machine. Getting within a factor of 2 of C is often satisfying. Nonetheless, even within C, there are many kinds of optimizations possible that a naive C writer may or may not get the advantage of.
If you do not speak C, do not read the cell below, but one could be happy to know that it is possible to put C code in a Julia session, compile it, and run it. Note that the `"""` wrap a multi-line string.
%% Cell type:code id:6c1e8f7d tags:
``` julia
usingLibdl
C_code="""
#include <stddef.h>
double c_sum(size_t n, double *X) {
double s = 0.0;
for (size_t i = 0; i < n; ++i) {
s += X[i];
}
return s;
}
"""
constClib=tempname()# make a temporary file
# compile to a shared library by piping C_code to gcc
