The Idea of Data Generation Models

Synthetic data serves multiple purposes, including filling data gaps, protecting privacy, and augmenting limited datasets. Creating this data involves specific generation techniques. Rather than immediately presenting complex algorithms, the focus is on the fundamental idea behind many generation methods: employing a defined procedure or "model" to produce artificial data points.

Think of a data generation model not necessarily as a sophisticated machine learning model (like the ones we might train using synthetic data later), but more like a recipe or a set of instructions. This recipe dictates how new, artificial data points should be constructed. The goal is to follow these instructions to create data that shares important characteristics with the kind of data we actually want or need, even though it wasn't collected from reality.

At its core, a data generation model provides a mechanism to systematically produce outputs (our synthetic data) based on some specified inputs or rules. For the basic methods we're covering in this chapter, these "models" often fall into two simple categories:

Statistical Descriptions: We can analyze existing real data (if available) or define desired properties and describe them using statistics. For example, we might want to generate synthetic customer ages that follow a pattern similar to real customers. We could observe that real customer ages often cluster around a mean value with a certain spread. Our "model" then becomes a statistical distribution (like the normal distribution you might remember, often called a bell curve) defined by that mean ( $\mu$ ) and spread ( $\sigma$ , the standard deviation). Generating data means sampling random values from this defined distribution. The distribution itself acts as the model guiding the creation of plausible age values.
Explicit Rules: Sometimes, we know specific constraints or logic that the data must follow. For instance, in a dataset about online orders, a rule might be "If the country column is 'Canada', the currency column must be 'CAD'." Or, "User age must always be 18 or greater." A rule-based system uses these predefined conditions to generate data points that adhere strictly to this logic. The set of rules is the generation model in this case.

Consider this simple flow:

A simple view of data generation: A defined model or set of rules guides a process that produces synthetic data.

Whether we use statistical properties or explicit rules, the fundamental idea is the same: we need a blueprint to guide the creation of artificial data. This "model" or procedure is our tool for moving from needing synthetic data to actually producing it. In the following sections, we'll look at how to implement these basic ideas using statistical distributions and simple rule-based approaches to generate elementary numerical and categorical data.

Was this section helpful?

References

Synthetic Data for Machine Learning: Principles and Practice, Maksim Khakimov, Altynbek Seitov, Aziza Alimzhanova, and Zhamilya Baizakova, 2021 (Springer) DOI: 10.1007/978-3-030-74697-3 - Provides a foundational introduction to synthetic data, defining its purpose and outlining basic generation principles, including statistical methods.
A Survey on Synthetic Data Generation, Jingge Li, Xiangyu Han, Yaliang Li, Chengyu Song, Jun Ma, 2022 Journal of Big Data Analytics in Transportation, Vol. 4 (Springer) DOI: 10.1007/s42493-022-00078-0 - Offers a comprehensive overview of synthetic data generation methods, from basic statistical models to more complex machine learning approaches.
Monte Carlo Statistical Methods, Christian P. Robert, George Casella, 2004 (Springer) DOI: 10.1007/978-0-387-21239-5 - A classic text detailing methods for generating random variates from various statistical distributions, fundamental to the "statistical descriptions" approach for synthetic data. (2nd edition)
Privacy-Preserving Data Publishing, Raymond Wong, Min Wu, and Chengqi Zhang, 2011 (Springer) DOI: 10.1007/978-1-4419-5867-0 - Discusses techniques for publishing data while preserving privacy, including methods that involve generating synthetic or perturbed data based on statistical properties.