Having established the context for representation learning, we now focus on the autoencoder. This chapter introduces the standard autoencoder architecture, a neural network used for unsupervised learning. The main idea is to first compress input data into a lower-dimensional representation (encoding) and then reconstruct the original data from that compression (decoding). This process helps the network learn significant features of the data.

In this chapter, you will examine the essential parts:

• The Encoder Network: How it maps high-dimensional input $x$ to a latent code $z$.
• The Bottleneck Layer: Its function in holding the compressed data representation $z$.
• The Decoder Network: How it reconstructs the data $\hat{x}$ from the code $z$.
• Reconstruction Loss Functions: Measuring the discrepancy between $x$ and $\hat{x}$, using functions like Mean Squared Error (MSE) $L_{MSE} = \frac{1}{N}\sum_{i=1}^{N}(x_i - \hat{x}_i)^2$ or Binary Cross-Entropy (BCE), depending on the input data characteristics.

We will also cover the mathematical definition of the basic autoencoder's objective function and discuss practical aspects related to implementation using current deep learning libraries. Finally, a hands-on practical section demonstrates building a simple autoencoder.