Natural gradient descent is an optimization method that accounts for the geometry of the parameter space when updating model parameters. Unlike standard gradient descent, which treats parameter spa...

My notes taken from this Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution. import torch from matplotlib import pyplot as plt Diffusion Language Models Motivation G...

A follow up to my earlier posts on time dilation and length contraction. This post will talk about how velocities add under Special Relativity and how it differs that Newton’s laws of motion. Expe...

The following is a solution to TTM book 4, lecture 5, exercise 1: Given the metric $g_{\mu \nu}(X)$, show that the Euler-Lagrange equation, to minimize the action along a trajectory in space-ti...

This is a second post on electrical circuits and their similarity to mechanical oscillators. The previous post covers the mechanical harmonic oscillator and the LC circuit. This post focuses on the...

A weight on a spring has a mathematical similarity with an electrical circuit consisting of a capacitor and inductor (LC circuit). They are both examples of harmonic oscillators which exhibit a sin...

Here is a summary of how to link code in jekyll websites. Markdown Code Snippet For very fast code blocks that do not need to be synced to an code repo, we can simply wrap code in backticks: Code...

Consider a nonnegative random variable $X$ (i.e. annual income, home value, stock price). Here is the famous Markov Inequality 1: [\begin{flalign} \mathbb E [X] &= \int_{-\infty}^{\infty} x ...

A post on how to combine an LLM with the Monte Carlo Tree Search (MCTS) algorithm. There are numerous versions of MCTS, I follow Planning with Large Language Models for Code Generation’s implementa...

Details of an experiment originally explained in Albert Einstein’s 1905 paper on Special Relativity entitled On The Electrodynamics Of Moving Bodies. The experiment consists of an electric charge a...