Introduction to Python

This workshop will provide an introduction to the Python programming language. It will cover the basics of Python 3 syntax, and will also introduce programming aspects such as NumPy and Matplotlib that are especially relevant for AM205. The workshop will be run using Google Colab, a free cloud-based Python programming environment.

There will be several short programming exercises for students to complete, covering probability, cellular automata, and fractals.

Xiaoxiao (Catherine) Ding, Danyun He, and Michael Emanuel

Introduction to compiled languages

In AM205 we'll primarly use interpreted languages like Python and MATLAB. But why are compiled languages the go-to tool in coding applications as small as apps in our phones and as big as simulations on supercomputers? The reason is greater control, better performance, and a 100% legit reason to have a long break.

In this workshop, we will take a critical look at the differences between compiled languages and interpreted ones. We will also work on some coding examples together to get familiar with C++, a widely-used language in scientific computing.

Michael Emanuel and Chris Rycroft

Introduction to the command line

The Linux command line is an extremely powerful tool, allowing the user to remotely access servers around the world, run simulations, and perform complex file manipulations and transfers. The command line is also built into Mac OS (via the Terminal app) and is available on Windows (e.g. via Cygwin).

The workshop will begin by covering basic commands for copying and editing files. It will discuss regular expressions for manipulating text, remote computer access via SSH, and using the “screen” and “tmux” commands for launching programs remotely. The workshop will be interspersed with several exercises drawn from codebreaking and game theory.

Danyun He, Xiaoxiao (Catherine) Ding, and Yue Sun

Introduction to multithreading

In high performance computing, we often see the word “multithreading” come up in the discussions (in addition to GPU computing). Multithreading is a parallelization model for CPUs to execute one job simultaneously on multiple threads. Modern laptops are equipped with processors of at least 2 cores (that is 4 threads), but oftentimes we only use 1 thread when running our Python or C++ programs. We can benefit from this performance boost if we do multithreading correctly, spending less time waiting in front of our laptops.

In this workshop, we will discuss two frameworks for parallelization: distributed and shared memory. We will then focus on the latter and introduce concepts for implementing shared memory parallelization in the code examples using Python (with joblib) and C++ (with OpenMP). We will learn about some gotchas of multithreading in memory management and reduced performance, and finally explore how these concepts are used by default in Python libraries such as NumPy, SciPy or JAX.

Either Python or C++ experience required.

Yue Sun and Michael Emanuel

Introduction to POV-Ray

POV-Ray is a free ray-tracing program for making photo-realistic renderings and has a large following on the internet. On the POV-Ray website, you can browse some amazing images that people have created. POV-Ray uses a simple markup language for describing elements of a scene, such as the camera position, lights, and objects. As such, it is excellent for scientific visualization, since you can write functions in your programs to output your results in the POV-Ray syntax. This workshop will cover the basics of the POV-Ray language, and demonstrate how to create beautiful visualizations of program output.

Chris Rycroft

Introduction to the Kalman filter

Do you wonder how a navigation system on a car estimates its position, even when GPS signals are lost? How do we estimate quantities like position and temperature, when we don't have direct measurement, or when the measured data is noisy? The Kalman filter is an optimization algorithm that combines a physical model of the system and noisy measurements to give a best estimate of the system states (e.g. position or temperature). In this workshop, we will discuss the mathematical basis of Kalman filter, and use it in an example application.

Michael Emanuel

Data-driven dynamic mode decomposition

According to the internet, the best estimates suggest that at least 2.5 quintillion bytes of data are produced every day. That is a LOT of data (2.5 followed by 18 zeros)! Researchers now find themselves in a new dilemma: We have the data of the phenomena we want to study, but the phenomena have not yet (or only ideally) be modeled with governing equations; hence, very few insights can be concluded. With access to these massive data sets, e.g. lab experiments, numerical simulations, weather recordings, financial tradings, disease trackers, or any time-series of your interest, the natural question to ask is: Can we discover some underlying physics or principles purely from these data?

In this workshop, we will learn about a new method that is purely data-driven and equation-free to extract spatiotemporal coherent structures from high-dimensional nonlinear data. Dynamic mode decomposition, or DMD, is a matrix decomposition technique that computes time-and-space features from experimental/measuring/simulation data, without any prior knowledge of the underlying physics or dynamical system. We will implement the core DMD algorithm in Python, and study some domain-specific applications of DMD (e.g. fluid dynamics and epidemiology). We will also briefly explore the integration of DMD with machine learning algorithms.

No domain-specific knowledge required; simulation data will be provided for this workshop. (You are also welcome to bring your own data set!) Python knowledge required.

Yue Sun

Using the SVD for image processing

This tutorial will delve deeper into practical applications of the singular value decomposition (SVD) of a matrix. Applications of SVD are found across disciplines and include data compression, pattern or outlier detection, and unsupervised learning. This activity will focus on three image-based applications: compression, denoising, and 3D reconstruction, which are useful for building intuition about this versatile matrix factorization.

Xiaoxiao (Catherine) Ding and Jovana Andrejevic

Data fitting for imaging Comet NEOWISE

Chris lives in Central Square, an urbanized neighborhood in the middle of Cambridge that has high levels of light pollution. Even on a Moonless night the sky appears as a dim gray, and only the brightest stars can be seen with the naked eye. In late July 2020, Comet NEOWISE was visible from Earth, and Chris took a sequence of images from Central Square, where the comet is washed out by light pollution and only appears as a tiny dot.

In this workshop, we'll use the power of data fitting to eliminate the spatially-varying light pollution, revealing much more structure in the comet image. This workshop will not be dependent on any programming language, although it will involve reading and writing image files.

Chris Rycroft and Michael Emanuel

ENO schemes for simulating traffic and reference map advection

Many interesting phenomena, like wave propagation and fluid motion, can be simulated using hyperbolic and convection–diffusion equations. Since these phenomena typically have huge jumps in density or other physical fields, solutions to these equations are thus characterized by discontinuities or sharp gradients, i.e. “shocks”. However, the numerical methods we have learned in Unit 3 often fail to produce satisfactory results. Generally, this failure takes the form of spurious oscillations in the simulated solution near the discontinuity or sharp gradient which experiences unstable growth.

Essentially non-oscillatory (ENO) schemes are a class of methods developed for the simulation of systems containing shocks. In this workshop, we will first investigate the cause of vanilla methods numerical failure, then introduce the family of ENO methods. We will focus on two examples (ENO2 and WENO4) and apply them to simulating physical systems which experience discontinuities or sharp gradients, such as traffic trajectory and droplet contours.

Either Python or C++ experience required.

Jovana Andrejevic and Yue Sun

Numerically integrating the solar system

This is a short and practical introduction to the gravitational n-body problem as applied to the solar system. We will touch briefly on the history and theory of this problem. We will spend most of our time with a hands-on demonstration showing how to solve this problem numerically for the solar system using freely available open source software and public data. Our practical solution will use the Rebound package in Python as a numerical integrator. We will obtain real initial conditions for solar system bodies from the Nasa Horizons system. Our learning goal for the session is that everyone leaves knowing how to numerically integrate the motion of the solar system, with a working solution they can refer back to later.

Michael Emanuel

Implementing a DAE solver to simulate the chaotic double pendulum

Unlike an ODE system, a differential-algebraic system of equations (DAE) contains both differential and algebraic (constraint) equations, and thus requires special considerations to integrate. Examples of DAEs arise in complex mechanical systems, electronic circuits, and chemical kinetics problems. In this activity we will implement a DAE solver, which builds on concepts of implicit integration and Newton's method covered in lecture, and explore its application to a fun (and chaotic!) mechanical system: the double pendulum.

The structure of code in this tutorial will highlight object-oriented programming, and some basics of Python classes will be covered.

Xiaoxiao (Catherine) Ding and Jovana Andrejevic

Neural networks and their connection to numerical methods

In the last few years, the term 'neural network' has increasingly become a buzzword. Given enough data, the black-box modeling of deep learning is presumed to be able to tackle many of the hard problems we face. But how does it work? Is it really that hard? And how does it relate to the traditional numerical methods as covered in AM205?

This workshop is designed to unravel some of the mystery that is often associated with neural networks and aims to connect them to some of the techniques taught throughout the course. We'll cover the basic mathematical formulation of a NN (layers, weights, backpropagation) and briefly touch upon some of their many applications (such as curve-fitting, classification, CNNs, RNNs, …). We could expand on how optimization plays a decisive role, how curve-fitting from AM205 compares to NN predictions, and on how some recent research is using NNs to solve ODEs and PDEs.

The workshop would be supported by Jupyter notebook, running in an environment where Pytorch should be installed. It could be cool to use the NeuroDiffEq package to solve some basic ODEs and PDEs (link).

Xiaoxiao (Catherine) Ding

Further optimization methods

In AM205, we introduce the derivative-based optimization method: Newton's method. What if you don't know the derivative? This workshop is going to introduce two no-derivative-based optimization methods: the Golden section search method and the Brent method. It will start with the basic concepts and algorithms. We will compare different optimization methods and discuss their limitations and advantages. There will be several exercises about implementing the algorithms.

Danyun He

Fluid instability in coffee

We see fluid dynamics in action every day, but we know very little about it. The existence and smoothness of the governing equations, the Navier–Stokes equations, remain one of the seven Millennium Prize problems yet to be solved. Understanding these equations would be the first step to understand fluid instability, and eventually to turbulence. However, we do not need to take a full course in fluid dynamics to make a morning cup of coffee—where we exploit two fluid instability concepts! Boiling water in a kettle is an example of the Rayleigh–Bénard instability, and the Rayleigh–Taylor instability leads to the mesmerizing fluid motion when adding cream into the coffee.

This workshop will be a brief survey of the basic techniques in finding the onset of fluid instability. We will first introduce the two instability setups, and show how to use nondimensionalization and linear perturbation to scale the famous Navier–Stokes equations for analyzing fluid instability. Then we will focus on the Rayleigh–Bénard instability and show that its governing equations can be reduced to a single ODE, which can be further simplified to an eigenvalue problem via a differential operator. Finally, we will apply a Krylov subspace method to calculate the differential operator's eigenspectrum and look at some visualizations. So the next time you enjoy a cup of coffee, do not forget you can apply AM205 materials to find the modes of convection (eigenfunctions) and the corresponding temperatures (eigenvalues)!

No fluid dynamics knowledge required. No particular programming language required.

Yue Sun

A one-dimensional implementation of multigrid method

AM205 will introduce the multigrid method, a highly efficient iterative approach for solving a class of sparse matrix problems. This workshop will go into more detail about the theory and application of multigrid methods. We will implement the multigrid method in one dimension, and compare its efficiency to other approches such as the Jacobi and Gauss–Seidel methods. We will also discuss contemporary developments in multigrid research, as well as available software libraries that implement the method.

Danyun He

Final project presentations

In this special group activity, students will give a talk on their final project. This will be organized into several 2 h sessions.

For a project team of n students, the talk length should be (3n+6) min in total. Each student in the group should speak for part of the talk. Each talks will be followed by a short Q&A session. Students are expected to attend the full 2 h session and listen to the talks given by other students.

For grading, this will count as two group activities. Sign-up for presentations will open later in the semester.