Comparing Data Distributions

While comparing basic statistics like the mean ($\mu$) and standard deviation ($\sigma$) gives us a quick check, it doesn't tell the whole story about our synthetic data. Two datasets could have the same mean and standard deviation for a feature but look completely different in terms of how the values are spread out. That's where comparing the distributions of features becomes important. We want to see if the overall shape and spread of data points in our synthetic dataset mirror those in the real dataset.

Visualizing Distributions with Histograms

One of the most straightforward ways to compare distributions visually is by using histograms. A histogram groups numerical data into bins (intervals) and shows the frequency (count) of data points falling into each bin. By plotting histograms for the same feature from both the real and synthetic datasets, we can directly compare their shapes.

Let's say we have a real dataset with customer ages and we generated a synthetic version. We can plot histograms for the 'Age' feature from both datasets.

Comparison of age distributions using histograms. The blue bars represent the real data, and the orange bars represent the synthetic data.

When looking at these histograms, ask yourself:

• Do the shapes generally match? Are they both skewed, bell-shaped, or uniform?
• Are the peaks (most frequent bins) in roughly the same place?
• Is the range or spread of the data similar?

Minor differences are expected, but large deviations (e.g., the synthetic data showing a peak where the real data has a valley) indicate that the generation process didn't capture the distribution well for this feature.

Smoother Comparisons with Density Plots

Histograms are great, but the choice of bin size can sometimes affect how the distribution looks. A smoother way to visualize distributions, especially for continuous data, is using Kernel Density Estimates (KDE), often shown as density plots. A density plot tries to estimate the underlying probability distribution from which the data points were drawn, resulting in a smooth curve.

Comparing density plots can make it easier to see subtle differences in shape and peaks.

Comparison of age distributions using density plots (represented here using violin plots split side-by-side). The blue area shows the density for real data, and the orange area shows the density for synthetic data.

Again, look for similarities in shape, the location of peaks, and the overall spread. Density plots can be particularly useful for identifying modes (peaks) in the data that might be obscured by binning choices in a histogram.

Checking Relationships: Scatter Plots

Data generation isn't just about getting individual feature distributions right; it's also about preserving relationships between features. For example, in a real dataset, height and weight might be positively correlated. Does our synthetic data show the same trend?

A simple way to check the relationship between two numerical features is with a scatter plot. Create one scatter plot for the real data and another for the synthetic data, plotting the same two features.

Scatter plot showing the relationship between height and weight in the real dataset.

Scatter plot showing the relationship between height and weight in the synthetic dataset.

Compare the patterns in the two plots. Does the synthetic data show a similar trend (e.g., positive correlation, negative correlation, no clear pattern) as the real data? Is the spread or density of points roughly comparable? Significant differences here suggest the generation method failed to capture the interaction between these features.

Comparing distributions visually using histograms, density plots, and scatter plots provides a much richer understanding of synthetic data quality than basic statistics alone. It helps us assess whether the synthetic data truly mimics the structure and relationships present in the real data, moving us closer to understanding its potential fidelity.