From Correlations to Grammar: How Neural Networks Acquire Hierarchical Language Structures

About these Francesco Cagnetta lecture notes

These notes trace how hierarchical compositional structure can be learned from data. They introduce random tree-like grammars, analyze what examples reveal about latent variables, and connect sample complexity to neural representations and scaling laws.

• Random hierarchies and probabilistic context-free grammars
• Hierarchical compositionality and latent variables
• Method of moments and sample complexity
• Last-token prediction and data-limited learning
• Neural representations and scaling laws