julia-course-slides.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# # Check multithreading config:\n",
    "Base.Threads.nthreads()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# # Instantiate package environment for this notebook\n",
    "# using Pkg; pkg\"instantiate\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# # Check active package versions:\n",
    "# using Pkg; pkg\"status\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "nbpresent": {
     "id": "7b7b849a-81ae-4985-92ec-78b66da75f5f"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<h1 style=\"text-align: center;\">\n",
    "    <span style=\"display: block; text-align: center;\">\n",
    "        Introduction to\n",
    "    </span>\n",
    "    <span style=\"display: block; text-align: center;\">\n",
    "        <img alt=\"Julia\" src=\"images/logos/julia-logo.svg\" style=\"height: 2em; display: inline-block; margin: 1em;\"/>\n",
    "    </span>\n",
    "</h1>\n",
    "\n",
    "<div style=\"text-align: center;\">\n",
    "    <p style=\"text-align: center; display: inline-block; vertical-align: middle;\">\n",
    "        Oliver Schulz<br>\n",
    "        <small>\n",
    "            Max Planck Institute for Physics <br/>\n",
    "            <a href=\"mailto:oschulz@mpp.mpg.de\" target=\"_blank\">oschulz@mpp.mpg.de</a>\n",
    "        </small>\n",
    "    </p>\n",
    "    <p style=\"text-align: center; display: inline-block; vertical-align: middle;\">\n",
    "        <img src=\"images/logos/mpg-logo.svg\" style=\"height: 5em; display: inline-block; vertical-align: middle; margin: 1em;\"/>\n",
    "        <img src=\"images/logos/mpp-logo.svg\" style=\"height: 5em; display: inline-block; vertical-align: middle; margin: 1em;\"/>\n",
    "    </p>\n",
    "</div>\n",
    "\n",
    "<p style=\"text-align: center;\">\n",
    "     March 2023\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Course material\n",
    "\n",
    "**Git-clone or download the course material:**\n",
    "\n",
    "## https://github.com/oschulz/julia-course"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Why Julia?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Science needs code - but how to write it?\n",
    "\n",
    "* Choice of programming language(s) matter!\n",
    "\n",
    "* Need to balance:\n",
    "    * Learning time\n",
    "    * Productivity\n",
    "    * Performance\n",
    "\n",
    "* Usually involves compromises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Programming Language Options\n",
    "\n",
    "* C++:\n",
    "    * Pro: Very fast (in expert hands)\n",
    "    * Pro: Really cool new concepts (even literally) in C++11/14/17/...\n",
    "    * Con: Complex, takes long time to learn and much longer to master\n",
    "    * Con: Straightforward tasks often result in lengthy code\n",
    "    * Con: No memory management (General protection faults)  \n",
    "    * Con: No universal package management\n",
    "    * Con: Composability isn't great"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Programming Language Options\n",
    "\n",
    "* Python:\n",
    "    * Pro: Broad user base, popular first programming language\n",
    "    * Pro: Easy to learn, good standard library\n",
    "    * Con: Can't write time-critical loops in Python,  \n",
    "      workarounds like Numba/Cython have\n",
    "      [many limitations](http://www.stochasticlifestyle.com/why-numba-and-cython-are-not-substitutes-for-julia/),  \n",
    "      don't compose well\n",
    "    * Con: Language itself fairly primitive, not very expressive\n",
    "    * Con: Duck-Typing necessitates lots of test code\n",
    "    * Con: No effective multi-threading\n",
    "    * Con: Composability isn't great"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### What else is there?\n",
    "\n",
    "* Fortran:\n",
    "    * Pro: Math can be really fast\n",
    "    * Con: Old language, few modern concepts\n",
    "    * Con: Shrinking user base\n",
    "    * Con: Composability isn't great\n",
    "    * Do you *really* want to ...?\n",
    "\n",
    "\n",
    "* Scala, Go, Kotlin etc.:\n",
    "    * Pro: Lots of individual strengths\n",
    "    * Con: Math either fast *or* generic *or* or complicated\n",
    "    * Con: Calling C, Fortran or Phython code often difficult\n",
    "    * Con: Composability isn't great"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### The 97 and the 3 Percent\n",
    "\n",
    "> We should forget about small efficiencies, say about 97% of the time: *premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%*.\n",
    "\n",
    "Donald E. Knuth\n",
    "\n",
    "* Some programming languages (e.g. Python) great for the 97% - but can't make the 3% fast.\n",
    "* Some other languages (e.g. C/C++, Fortran) can handle the 3% - but makes the 97% complicated."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### The Two-language Problem\n",
    "\n",
    "* Common approach nowadays:  \n",
    "  Write time critical code in C/C++, rest in Python\n",
    "\n",
    "* Pro: End-user can code comfortably in Python, with good performance\n",
    "\n",
    "* Con: Complexity of C/C++ **plus** complexity of Python\n",
    "\n",
    "* Con: Need proficiency in **two** languages, barrier that prevents  \n",
    "  non-expert users from contributing to important parts of code\n",
    "\n",
    "* Con: Limits generic implementation of algorithms\n",
    "\n",
    "* Con: Severely limits metaprogramming, automatic differentiation, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The Expression Problem\n",
    "\n",
    "> The expression problem is a new name for an old problem. The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts).\n",
    "\n",
    "Philip Wadler\n",
    "\n",
    "* In other words: The capability to add both new subtypes and new functionality for a type defined in a package you don't own\n",
    "* Object oriented languages typically can't do this  \n",
    "  (Ruby has a dirty way, Scala a clean workaround)\n",
    "* If you have programming experience, you have felt this, even if you didn't name it\n",
    "* Result: Packages tend not to compose well\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### We were looking for a language ...\n",
    "\n",
    "* as fast as C/C++/Fortran\n",
    "* as easy to learn and productive as Python\n",
    "* with a solution for the expression problem\n",
    "* with first class math support (vectors, matrices, etc.)\n",
    "* with true functional programming\n",
    "* with great Fortran/C/C++/Python integration\n",
    "* with true metaprogramming (like Lisp or Scala)\n",
    "* good at parallel and distributed programming\n",
    "* suitable for for interactive, small and large applications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Julia\n",
    "\n",
    "* Designed for scientific/technical computing\n",
    "* Originated at MIT, first public version 2012\n",
    "* Covers the whole wish-list\n",
    "* Clear focus on user productivity and software quality\n",
    "* Rapid growth of user base and software packages\n",
    "* Current version: Julia v1.8 (v1.9 release candidate available)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "b6763142-7446-43bf-a8a3-e7af798914d0"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Julia Language Properties\n",
    "\n",
    "* Fast: JAOT compilation to native CPU and GPU code\n",
    "* Multiple-dispatch (more powerful than object-oriented):  \n",
    "  solves the expression problem\n",
    "* Dynamically typed\n",
    "* Very powerful type system, types are first-class values\n",
    "* Functional programming and metaprogramming\n",
    "* First-class math support (like Fortran or Matlab)\n",
    "* ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Julia Language Properties, cont.\n",
    "\n",
    "* ...\n",
    "* Local and distributed code execution\n",
    "* State-of-the-art multi-threading: parallel code  \n",
    "  can call parallel code that can call parallel code, ...,  \n",
    "  without oversubscribing threads\n",
    "* Software package management:  \n",
    "  Trivial to create and install packages\n",
    "* Excellent REPL (console)\n",
    "* Easy to call Fortran, C/C++ and Python code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Julia large-scale use case examples\n",
    "\n",
    "* Celeste: Variational Bayesian inference for astronomical images (doi:10.1214/19-AOAS1258), 1.54 petaflops using 1.3 million threads on 9,300 Knights Landing (KNL) nodes on Cori at NERSC\n",
    "\n",
    "* Clima: Full-earth climate simulation, https://clima.caltech.edu, large team, uses everything from MPI to GPUs\n",
    "\n",
    "* ...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### When (not) to use Julia\n",
    "\n",
    "* *Do* use Julia for computations, visualization, data processing ... pretty much anything scientific/technical\n",
    "\n",
    "* *Do not* use Julia for scripts what will only run for a second (code gen overhead), use Python or shell scripts\n",
    "\n",
    "* *Do not* use Julia for non-computing web apps, etc. (*at least not yet*), use Go or Node.js"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Julia 101"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Verbs and nouns - functions and types\n",
    "\n",
    "* Julia is not Java: Verbs aren't owned by nouns\n",
    "\n",
    "* Julia has: types, functions and methods\n",
    "\n",
    "* Methods belong to *functions*, not to types!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Functions\n",
    "\n",
    "Short one-liner function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = x^2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Function that needs more than one line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function f(x)\n",
    "    # ... something ...\n",
    "    x^2\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "is equivalent to"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function f(x)\n",
    "    # ... something ...\n",
    "    return x^2\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** `return` is optional, and often not used explicitly. Last expression in a function, block, etc. is automatically returned (like in Mathematica)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Types\n",
    "\n",
    "An abstract type, must be empty:\n",
    "\n",
    "```julia\n",
    "abstract type MySuperType end\n",
    "```\n",
    "\n",
    "An immutable type, value of `i` can't change:\n",
    "\n",
    "```julia\n",
    "struct MySubType <: MySuperType\n",
    "    i::Int\n",
    "end\n",
    "```\n",
    "\n",
    "A mutable type, value of `i` can change:\n",
    "\n",
    "```julia\n",
    "mutable struct MyMutableSubType <: MySuperType\n",
    "    i::Int\n",
    "end\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Type parameters\n",
    "\n",
    "Julia has a powerful parametric type system, based on set theory:\n",
    "\n",
    "```julia\n",
    "struct MyRealArray{T<:Real,N} <: AbstractArray{T,N}\n",
    "    # ...\n",
    "end\n",
    "```\n",
    "\n",
    "defines an array type with real-valued elements.\n",
    "\n",
    "```julia\n",
    "foo(A::AbstractArray{<:Real}) = do_something_with(A)\n",
    "```\n",
    "\n",
    "is equivalent to\n",
    "\n",
    "```julia\n",
    "bar(A::AbstractArray{T,N}) where {T<:Real,N} = do_something_with(A)\n",
    "```\n",
    "\n",
    "defines a function covariant in the element type of `A`. Can also be contravariant:\n",
    "\n",
    "```julia\n",
    "baz(A::AbstractArray{>:Real}) = do_something_with(A)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Type aliases and union types\n",
    "\n",
    "Type aliases are just const values:\n",
    "\n",
    "```julia\n",
    "const Abstract2DArray{T} = AbstractArray{T,2}\n",
    "rand(2, 2) isa Abstract2DArray == true\n",
    "```\n",
    "\n",
    "Type unions are unions of set of types.\n",
    "\n",
    "```julia\n",
    "const RealVecOrMat{T} where {T<:Real} = Union{AbstractArray{T,1}, AbstractArray{T,2}}\n",
    "```\n",
    "\n",
    "is the union of a 1D and 2D array types with real-valued elements."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Syntax: Variables\n",
    "\n",
    "```julia\n",
    "# Global variables:\n",
    "const a = 42\n",
    "b = 24\n",
    "\n",
    "function foo(x)\n",
    "    # Local variables:\n",
    "\n",
    "    c = a * x\n",
    "    d = b * x # Avoid, type of b can change!\n",
    "    #...\n",
    "end\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Loops\n",
    "\n",
    "For loop:\n",
    "\n",
    "```julia\n",
    "for i in something_iterable\n",
    "    # ...\n",
    "end\n",
    "```\n",
    "\n",
    "`something_iterable` can be a range, an array, anything that implements the Julia [iterator API](https://docs.julialang.org/en/v1/manual/interfaces/).\n",
    "\n",
    "While loop:\n",
    "\n",
    "```julia\n",
    "while condition\n",
    "    # do something\n",
    "end\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Control flow\n",
    "\n",
    "If-else, evaluate only one branch:\n",
    "\n",
    "```julia\n",
    "if condition\n",
    "    # do something\n",
    "elseif condition\n",
    "    # do something else\n",
    "else\n",
    "    # or something different\n",
    "end\n",
    "```\n",
    "\n",
    "Ternary operator, evaluate only one branch:\n",
    "\n",
    "```julia\n",
    "condition ? result_if_true : result_if_false\n",
    "```\n",
    "\n",
    "`ifelse`, evaluate both results but return only one:\n",
    "\n",
    "```julia\n",
    "ifelse(condition, result_if_true, result_if_false)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Blocks and scoping\n",
    "\n",
    "Begin/end-block:\n",
    "\n",
    "```julia\n",
    "begin\n",
    "    # *Not* a new scope in here\n",
    "    # ...\n",
    "end\n",
    "```\n",
    "\n",
    "Let-block:\n",
    "\n",
    "```julia\n",
    "b = 24\n",
    "\n",
    "let my_b = b\n",
    "    # New scope in here.\n",
    "    # If b is bound to new value, my_b won't change.\n",
    "    # ...\n",
    "end\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Arrays\n",
    "\n",
    "Vectors:\n",
    "\n",
    "```julia\n",
    "v = [1, 2, 3]\n",
    "\n",
    "v = rand(5)\n",
    "```\n",
    "\n",
    "Matrices:\n",
    "\n",
    "```julia\n",
    "A = [1 2; 3 4]\n",
    "\n",
    "A = rand(4, 5)\n",
    "```\n",
    "\n",
    "* Column-first memory layout!\n",
    "\n",
    "* Almost anything array-like is subtype of `AbstractArray`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Array indexing\n",
    "\n",
    "Get `i`-th element of vector `v`:\n",
    "\n",
    "```julia\n",
    "v[i]\n",
    "```\n",
    "\n",
    "Most higher-dimensional array types support cartesian and linear indexing (usually faster):\n",
    "\n",
    "```julia\n",
    "A[i, j]\n",
    "A[lin_idx] \n",
    "```\n",
    "\n",
    "Use `eachindex(A)` to get indices of best type for given `A` (usually linear).\n",
    "\n",
    "\n",
    "In Julia, anything array-like can usually be an index as well\n",
    "\n",
    "```julia\n",
    "A[2:3, [1, 4, 5]]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Array comprehension and generators\n",
    "\n",
    "Returns an array:\n",
    "\n",
    "```\n",
    "[f(x) for x in some_collection]\n",
    "```\n",
    "\n",
    "Returns an iterable generator:\n",
    "\n",
    "```\n",
    "(f(x) for x in some_collection)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Hello World (and more) in Julia"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "println(\"Hello, World!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let's define a function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x, y) = x * y\n",
    "f(20, 2.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiplication is also defined for vectors, so this works, too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(4.2, [1, 2, 3, 4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Let's Look Under the Hood"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@code_llvm debuginfo=:none f(20, 2.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@code_native debuginfo=:none f(20, 2.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Multiple Dispatch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "foo(x::Integer, y::Number) = x * y\n",
    "foo(x::Integer, y::AbstractString) = join(fill(y, x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "foo(3, 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "foo(3, \"abc\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "foo(4.5, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Functional Programming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = rand(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "idxs = findall(x -> 0.2 < x < 0.6, A)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A[idxs]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Even types are first-class values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mytype = Number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subtypes(mytype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Julia type hierarchy extends all the way down to primitive types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Float64 <: AbstractFloat <: Real <: Number <: Any"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Broadcasting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = [1.1, 2.2, 3.3]\n",
    "B = [4.4, 5.5, 6.6]\n",
    "broadcast((x, y) -> (x + y)^2, A, B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Shorter broadcast syntax:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(A .+ B) .^ 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Loop Fusion and SIMD Vectorization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "foo(X, Y) = (X .+ Y) .^ 2\n",
    "@code_llvm raw=false debuginfo=:none foo(A, B)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "@code_native debuginfo=:none foo(A, B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Package management\n",
    "\n",
    "* Julia probably has the best package management to date\n",
    "\n",
    "* Press \"]\" to enter package management console\n",
    "\n",
    "* Typically `add PACKAGE_NAME` is sufficient, can also do `add PACKAGE_NAME@VERSION`\n",
    "\n",
    "* To get an unreleased version, use `add PACKAGE_NAME#BRANCH_NAME`\n",
    "\n",
    "* Easy to start modifying a package via `dev PACKAGE_NAME`\n",
    "\n",
    "* Multiple package versions can be installed, selection via [Pkg.jl environments](https://julialang.github.io/Pkg.jl/v1/environments).\n",
    "\n",
    "* Also useful: `julia> using Pkg; pkg\"<Pkg console command>\"`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Package creation\n",
    "\n",
    "* A Julia package needs:\n",
    "\n",
    "    * A \"Project.toml\" file\n",
    "    * A \"src/PackageName.jl\" file\n",
    "\n",
    "* That's it: Push to GitHub, and package is installable via `add PACKAGE_URL`\n",
    "\n",
    "* Use [Documenter.jl](https://github.com/JuliaDocs/Documenter.jl) to document your package\n",
    "\n",
    "* To enable `add PACKAGE_NAME`, package must be [registered](https://github.com/JuliaRegistries/Registrator.jl), there are [some rules](https://github.com/JuliaRegistries/RegistryCI.jl#automatic-merging-guidelines)\n",
    "\n",
    "* Use [PkgTemplates.jl](https://github.com/invenia/PkgTemplates.jl) to generate new package with CI config (Travis, Appveyor, ...), docs generation, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### No free lunch\n",
    "\n",
    "* Package loading and code-gen can sometime take a long time, \n",
    "  but mitigations available:\n",
    "\n",
    "* [Revise.jl](https://github.com/timholy/Revise.jl): Hot code-reloading at runtime\n",
    "\n",
    "* [PackageCompiler.jl](https://github.com/JuliaLang/PackageCompiler.jl]: Ahead-of-time compilation, producing custom Julia system images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Performance tips\n",
    "\n",
    "* Read the [official Julia performance tips](https://docs.julialang.org/en/v1/manual/performance-tips/)!\n",
    "* Do *not* call on (non-const) global variables from time-critical code\n",
    "* Type-stable code is fast code. Use [`@code_warntype`](https://docs.julialang.org/en/v1/manual/performance-tips/#man-code-warntype-1) and [`Test.@inferred`](https://docs.julialang.org/en/v1/stdlib/Test/#Test.@inferred) to check!\n",
    "* In some situations, closures [can be troublesome](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured-1), using `let` can help the compiler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "This is efficient (not runtime reflection):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "half_dynrange(T::Type{<:Number}) = (Int(typemax(T)) - Int(typemin(T))) / 2\n",
    "half_dynrange(Int16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "@code_llvm half_dynrange(Int16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### SIMD\n",
    "\n",
    "Demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Shared-memory parallelism\n",
    "\n",
    "* Julia has native multithreading support\n",
    "\n",
    "* Simple cases: Use `@threads` macro\n",
    "\n",
    "* Since Julia v1.3: Cache-efficient [composable multi-threaded parallelism](https://julialang.org/blog/2019/07/multithreading/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Processes, Clusters, MPI\n",
    "\n",
    "* Julia brings a full API for remote processes and compute clusters\n",
    "\n",
    "* Native support for local processes and remote processes via SSH\n",
    "\n",
    "* MPI support via [MPI.jl](https://github.com/JuliaParallel/MPI.jl) and [MPIClusterManagers.jl](https://github.com/JuliaParallel/MPIClusterManagers.jl)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Benchmarking and profiling, digging deeper\n",
    "\n",
    "Demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Docs and help\n",
    "\n",
    "* [Official Julia docs](https://docs.julialang.org/en/v1/)\n",
    "\n",
    "* [Julia Cheat Sheet](https://juliadocs.github.io/Julia-Cheat-Sheet/)\n",
    "\n",
    "* [ThinkJulia](https://benlauwens.github.io/ThinkJulia.jl/latest/book.html)\n",
    "\n",
    "* https://julialang.org/learning/\n",
    "\n",
    "* [Julia Discourse](https://discourse.julialang.org/)\n",
    "\n",
    "* [Julia Slack](https://slackinvite.julialang.org/)\n",
    "\n",
    "* [Julia Gitter](https://gitter.im/JuliaLang/julia)\n",
    "\n",
    "* [Julia on Youtube](https://www.youtube.com/user/JuliaLanguage)\n",
    "\n",
    "* [JuliaCon 2020](https://juliacon.org/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Statistics\n",
    "\n",
    "Demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Visualization/Plotting: Plots, Makie, plotting recipes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Let's Make a Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Plots\n",
    "range = -π:0.01:π\n",
    "plot(range, sin.(range) + rand(length(range)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Histograms are easy, too"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Distributions\n",
    "dist = Normal(0.0, 5.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stephist(rand(dist, 10000))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Automatic differentiation\n",
    "\n",
    "Let's define a simple neural network layer and loss function and auto-differentiate through it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "struct DenseLayer{M<:AbstractMatrix{<:Real},V<:AbstractVector{<:Real},F<:Function} <: Function\n",
    "    A::M\n",
    "    b::V\n",
    "    f::F\n",
    "end \n",
    "\n",
    "(l::DenseLayer)(x::AbstractVector{<:Real}) = (l.f).(l.A * x + l.b)\n",
    "\n",
    "f_loss(y) = sum(y .^ 2);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mylayer = DenseLayer(rand(5,5), rand(5), x -> ifelse(x > zero(x), x, zero(x)));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = rand(5)\n",
    "mylayer(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f_loss(mylayer(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "using Zygote\n",
    "g = Zygote.gradient((mylayer, x) -> f_loss(mylayer(x)), mylayer, x)\n",
    "g[1].A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g[1].b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Calling code written in other language, REPL modes\n",
    "\n",
    "### Shell REPL mode\n",
    "\n",
    "* The Julia shell is multi-language\n",
    "\n",
    "* Press \",\" for a system shell"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### PyCall, RCall\n",
    "\n",
    "Calling Python from Julia is easy, can even use inline Python code:\n",
    "\n",
    "```julia\n",
    "using PyCall\n",
    "numpy = pyimport(\"numpy\")\n",
    "numpy.zeros(5) isa Array\n",
    "\n",
    "A = rand(5)\n",
    "py\"\"\"type($A)\"\"\" isa PyObject\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## An incomplete tour of the Julia package ecosystem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Math\n",
    "\n",
    "* [ApproxFun.jl](https://github.com/JuliaApproximation/ApproxFun.jl): Powerful function approximations\n",
    "\n",
    "* [FFTW.jl](https://github.com/JuliaMath/FFTW.jl): Fast fourier transforms via [FFTW](http://www.fftw.org/)\n",
    "\n",
    "* [DifferentialEquations.jl](https://github.com/JuliaDiffEq/DifferentialEquations.jl): A suite for numerically solving differential equations\n",
    "\n",
    "* ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Optimization\n",
    "\n",
    "* [JuMP.jl](https://github.com/JuliaOpt/JuMP.jl): Modeling language for Mathematical Optimization\n",
    "\n",
    "* [NLopt.jl](https://github.com/JuliaOpt/NLopt.jl): Optimization via [NLopt](https://github.com/stevengj/nlopt)\n",
    "\n",
    "* [Optim](https://github.com/JuliaNLSolvers/Optim.jl): Julia native nonlinear optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### TypedTables and DataFrames\n",
    "\n",
    "* [Tables.jl](https://github.com/JuliaData/Tables.jl): Abstract API for tabular data\n",
    "\n",
    "* [DataFrames.jl](https://github.com/JuliaData/DataFrames.jl): Python/R-like dataframes\n",
    "\n",
    "* [TypedTables.jl](https://github.com/JuliaData/TypedTables.jl): Type-stable tables\n",
    "\n",
    "* [Query.jl](https://github.com/queryverse/Query.jl) LINQ-inspired data query and transformation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Plotting and Visualization\n",
    "\n",
    "* [IJulia.jl](https://github.com/JuliaLang/IJulia.jl): Julia Jupyter kernel\n",
    "\n",
    "* [Images.jl](https://github.com/JuliaImages/Images.jl): Image processing\n",
    "\n",
    "* [PyPlot.jl](https://github.com/JuliaPy/PyPlot.jl): Use matplotlib/PyPlot from Julia\n",
    "\n",
    "* [Makie.jl](https://github.com/JuliaPlots/Makie.jl): Hardware-accelerated plotting\n",
    "\n",
    "* [Plots.jl](https://github.com/JuliaPlots/Plots.jl): Plotting with generic recipes and multiple backends"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Statistics\n",
    "\n",
    "* [Distributions.jl](https://github.com/JuliaStats/Distributions.jl): Probability distributions and associated functions\n",
    "\n",
    "* [StatsBase.jl](https://github.com/JuliaStats/StatsBase.jl): Statistics, histograms, etc.\n",
    "\n",
    "* Many specialized packages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Automatic Differentiation\n",
    "\n",
    "* [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl): Forward-mode automatic differentiation\n",
    "* [Zyote.jl](https://github.com/FluxML/Zygote.jl): Source-level reverse-mode automatic differentiation\n",
    "* [Enzyme.jl](https://github.com/wsmoses/Enzyme.jl): LLVM-level reverse-mode automatic differentiation\n",
    "* Several other packages available ([ReverseDiff.jl](https://github.com/JuliaDiff/ReverseDiff.jl), [Nabla.jl](https://github.com/invenia/Nabla.jl), [Yota.jl](https://github.com/dfdx/Yota.jl), ...)\n",
    "* Exciting developements to come with new Julia Compiler features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Bayesian analysis and probabilistic programming\n",
    "\n",
    "\n",
    "* [BAT.jl](https://github.com/bat/BAT.jl): Bayesian analysis toolkit\n",
    "\n",
    "* [Gen.jl](https://github.com/probcomp/Gen): General-Purpose Probabilistic Programming System\n",
    "\n",
    "* [Mamba.jl](https://github.com/brian-j-smith/Mamba.jl): MCMC for Bayesian analysis\n",
    "\n",
    "* [Turing.jl](https://github.com/TuringLang/Turing.jl): Probabilistic machine learning and Bayesian statistics\n",
    "\n",
    "* ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Machine learning\n",
    "\n",
    "* [Flux.jl](https://github.com/FluxML/Flux.jl): Julia native deep learning library\n",
    "\n",
    "* [Knet.jl](https://github.com/denizyuret/Knet.jl):Koc University deep learning framework\n",
    "\n",
    "* [MXNet.jl](https://github.com/dmlc/MXNet.jl): [MXNet](https://mxnet.apache.org/) Julia API\n",
    "\n",
    "* ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Calling code in other languages\n",
    "\n",
    "* [Cxx.jl](https://github.com/JuliaInterop/Cxx.jl): Call C++ from Julia\n",
    "\n",
    "* [PyCall.jl](https://github.com/JuliaPy/PyCall.jl): Call Python from Julia\n",
    "\n",
    "* [RCall.jl](https://github.com/JuliaInterop/RCall.jl): Call R from Julia\n",
    "\n",
    "* ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Efficient memory layout\n",
    "\n",
    "* [ArraysOfArrays.jl](https://github.com/oschulz/ArraysOfArrays.jl): Duality of flat and nested arrays\n",
    "\n",
    "* [StructArrays.jl](https://github.com/JuliaArrays/StructArrays.jl), [TypedTables.jl](https://github.com/JuliaData/TypedTables.jl): AoS and SoA duality\n",
    "\n",
    "* [ValueShapes.jl](https://github.com/oschulz/ValueShapes.jl): Duality of flat and nested structures\n",
    "\n",
    "* ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### GPU Programming\n",
    "\n",
    "* [AMDGPU.jl](https://github.com/JuliaGPU/AMDGPU.jl): Julia on AMD GPUs (WIP)\n",
    "\n",
    "* [CUDA.jl](https://github.com/JuliaGPU/CUDA.jl): Julia on NVIDIA GPUs\n",
    "\n",
    "* [GPUArrays.jl](https://github.com/JuliaGPU/GPUArrays.jl): Generic CPU programming API\n",
    "\n",
    "* [XLA.jl](https://github.com/JuliaTPU/XLA.jl): Julia on Google TPUs\n",
    "\n",
    "* Maybe more architectures in the future?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### IDEs\n",
    "\n",
    "* [julia-vscode](https://github.com/julia-vscode/julia-vscode): Visual Studio Code based Julia IDE\n",
    "\n",
    "* [Juno](https://junolab.org/): Atom based Julia IDE (now deprecated in favor of VS code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Final Remarks\n",
    "\n",
    "* Julia is productive, fast and fun - give it a chance!\n",
    "\n",
    "* Multiple dispatch opens up powerful ways of combining code"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "celltoolbar": "Slideshow",
  "hide_input": false,
  "kernelspec": {
   "display_name": "Julia 1.9.0",
   "language": "julia",
   "name": "julia-1.9"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.9.0"
  },
  "livereveal": {
   "start_slideshow_at": "selected",
   "theme": "simple",
   "transition": "none"
  },
  "nbpresent": {
   "slides": {},
   "themes": {}
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}