Row Sampling Techniques

Okay, let's get into one of the most straightforward ways to create synthetic tabular data: sampling rows directly from an existing dataset. Imagine you have a table of real data, like customer information or sensor readings. Row sampling techniques use this existing table as a blueprint to build a new, synthetic one.

This approach is particularly useful when you need a dataset that closely mirrors the overall structure and statistical properties of your original data, perhaps for testing software, creating anonymized versions for sharing, or simply generating a larger dataset with similar characteristics.

Basic Row Sampling: Sampling with Replacement

The simplest method is sampling with replacement. Think of your original dataset as a large bag filled with numbered balls, where each ball represents a row.

1. You reach into the bag and randomly pick out a ball (a row).
2. You record its number (copy the row to your new synthetic dataset).
3. Crucially, you put the ball back into the bag.
4. You repeat this process until your synthetic dataset reaches the desired size.

Because you put the ball back each time (sampling "with replacement"), the same row from the original dataset can appear multiple times in your synthetic dataset. If your original dataset has $N$ rows and you want to create a synthetic dataset with $M$ rows, you just perform this random draw-and-replace action $M$ times.

Here's a small illustration:

Original Data (3 rows):

ID	FeatureA	FeatureB
1	10	Red
2	15	Blue
3	12	Red

Possible Synthetic Data (sampled with replacement, size 4):

ID	FeatureA	FeatureB
2	15	Blue
1	10	Red
2	15	Blue
3	12	Red

Notice row 2 was selected twice.

Advantages:

• Simplicity: It's very easy to understand and implement.
• Preserves Row Integrity: Each row in the synthetic dataset is a real row that existed in the original data. This means complex relationships within a single row are perfectly preserved.
• Preserves Distributions: The distribution of values within each individual column (the marginal distribution) in the synthetic dataset will closely resemble the original, especially if the synthetic dataset is large.

Disadvantages:

• No New Data: You are not creating truly new data points or combinations of features. You are just reusing existing ones.
• Limited Privacy: Since entire rows are copied, if the original data contains sensitive information, this method offers minimal privacy protection on its own.
• Doesn't Improve Diversity: If your original dataset is small or lacks variety, simply resampling rows won't add any new patterns.

Sampling Without Replacement

Another option is sampling without replacement. This time, when you pick a ball (a row) from the bag, you don't put it back. This means each original row can appear at most once in your synthetic dataset.

This technique is typically used to create a smaller, random subset of your original data. For example, you might use it to create a training or testing split for a machine learning model. It's generally not used to create a synthetic dataset that's larger than the original, because you'd run out of unique rows to sample.

Stratified Sampling

What if your dataset has specific subgroups that are important, but some are much smaller than others? For instance, imagine a customer dataset where 95% are standard customers and 5% are high-value customers. If you use simple random sampling (with replacement), a small synthetic sample might accidentally miss the high-value customers or include very few, not reflecting their true proportion or importance.

Stratified sampling addresses this. The process involves:

1. Divide: Split the original dataset into distinct subgroups (strata) based on the values in one or more columns (e.g., based on the 'customer type' column).
2. Sample within Strata: Perform random sampling (usually with replacement if you want to maintain or increase size) independently within each subgroup. You decide how many samples to draw from each stratum. Often, you'll draw samples proportionally to maintain the original group sizes, but you could also oversample rare groups if needed.
3. Combine: Combine the samples drawn from each stratum to form the final synthetic dataset.

This ensures that the synthetic dataset preserves the relative proportions of the defined subgroups from the original data.

Process flow for stratified sampling, ensuring representation of different customer types.

Row Sampling with Modifications (A Glimpse Ahead)

While the methods above copy entire rows, a slightly more sophisticated approach involves sampling a row and then modifying some of its values slightly. For example, you could:

• Sample a row.
• Keep most categorical features the same.
• Add a small amount of random noise (e.g., drawn from a normal distribution with mean 0) to numerical features.

This starts to create data points that didn't exist in the original dataset, potentially offering better privacy and novelty. However, making these modifications requires careful consideration to ensure the resulting data remains realistic and useful. This technique acts as a bridge towards more complex generative models that learn the underlying patterns in the data rather than just copying rows.

Limitations of Pure Row Sampling

It's important to remember the main limitation: basic row sampling techniques (without modification) primarily replicate the data you already have. They copy existing rows, either exactly or by ensuring group proportions are maintained. They don't generate fundamentally new patterns or relationships that weren't present in the original dataset. If your goal is to generate data that explores possibilities that aren't explicitly in your source data, you'll need the more advanced methods discussed later.

However, for tasks like creating balanced test sets, quickly scaling up a dataset while preserving its basic structure, or generating simple anonymized versions, row sampling provides an accessible and effective starting point.