MADE: Masked Autoencoder for Distribution Estimation

This is an in-depth explanation of a technique for doing density estimation with the help of masked autoencoders.
Why should you master it?
Masked Autoencoder for Distribution Estimation is now being used as a building block in modern Normalizing Flows algorithms such as Inverse Autoregressive Normalizing Flows & Masked Autoregressive Normalizing Flows.
To learn more about Normalizing Flows check out my very comprehensive tutorial ( • Normalizing Flows - Mo... )
Sections:
00:00 Introduction
01:27 Goals
03:39 But what is an Autoencoder? (Essential Background)
10:16 Autoencoder for Density Estimation - Formulation & Associated Challenges!
19:12 The Big Idea to address the challenges
21:54 How it works (Step by Step algorithm explanation)
26:56 Summary of Key Insights
28:43 Experiments & Results
31:04 Conclusions
Link to the paper:
Masked Autoencoder for Density Estimation - arxiv.org/abs/1502.03509
Links to the key papers that make use of MADE
1) Improving Variational Inference with Inverse Autoregressive Flow (arxiv.org/abs/1606.04934)
2) Masked Autoregressive Flow for Density Estimation (arxiv.org/abs/1705.07057)
Multiple implementations are available for this paper:
paperswithcode.com/paper/mask...
#densityestimation
#autoencoders
#normalzingflows