differentiable-programming-experiments

This repository contains implementations of various differentiable programming models and algorithms. The goal is to explore and experiment with different architectures and techniques implemented from scratch (just using JAX for automatic differentiation and numerical computing).

A general tip for building differentiable programming is first to mock the data as a function. This will help you to pass the data through each step of the algorithm.

First-order differentiation

• multi-layer-perceptron
• convolutional-neural-network
• [] recurrent-neural-network
• [] long-short-term-memory
• [] transformer
• [] graph-neural-network
• [] neural-ordinary-differential-equation
• [] physics-informed-neural-network

Higher-order differentiation

• [] neural-tangent-kernel