Optimal Transport 🕷️ and Wasserstein distance
To qualify as a distance, a measure must satisfy the following properties: Symmetry: $ d(P, Q) = d(Q, P) )$ Triangle inequality: $ d(P, Q) + d(Q, R) \geq d(P, R) $ However, in practice, we often deal with weaker notions of distances, commonly referred to as divergences. Example: KL Divergence The Kullback-Leibler (KL) divergence is defined as: $$ D_{\text{KL}}(P || Q) = \int p(x) \log \frac{p(x)}{q(x)} dx $$ Properties of KL Divergence Not Symmetric: $$ D_{\text{KL}}(P || Q) \neq D_{\text{KL}}(Q || P) $$ Infinite for Different Supports: $$ D_{\text{KL}}(P || Q) \to \infty \quad \text{if } P \text{ and } Q \text{ have different supports.} $$ ...