Data Visualization with Plots.jl and Makie.jl

Data visualization is an indispensable step in the data preparation pipeline. Visualizations offer intuitive understanding of data distributions, relationships between variables, the presence of outliers, and the effectiveness of preprocessing steps. Julia provides powerful and flexible plotting libraries, with Plots.jl and Makie.jl being prominent choices for creating a wide array of static and interactive visualizations.

Exploring Data with Plots.jl

Plots.jl is a versatile plotting meta-package in Julia. It acts as a unified interface to various plotting backends (like GR, Plotly, PyPlot), allowing you to switch between them easily. This makes it a great general-purpose tool for quick explorations and generating publication-quality figures.

To start using Plots.jl, you typically need to install it and a backend, for example, GR, which is a fast and popular choice:

# Run this in the Julia REPL if not already installed
# using Pkg
# Pkg.add("Plots")
# Pkg.add("GR")

using Plots, DataFrames
gr() # Sets GR as the backend

# Assume 'df' is a DataFrame from previous steps
# For example:
df = DataFrame(
    Age = rand(25:65, 100),
    Income = rand(30000:120000, 100),
    Category = rand(["A", "B", "C"], 100)
)

Histograms and Density Plots

Histograms and density plots are fundamental for understanding the distribution of a single continuous variable. They help identify skewness, modes, and potential outliers, which can inform decisions about data transformations or outlier treatment.

histogram(df.Age,
          bins=10,
          label="Age Distribution",
          xlabel="Age",
          ylabel="Frequency",
          title="Customer Age Distribution",
          fillalpha=0.7,
          linecolor=:auto)

This code generates a histogram for the Age column. Adjusting the bins argument can help reveal different aspects of the distribution.

Here's an example of how such a distribution might look, represented directly:

Distribution of ages. Such a plot quickly shows the age spread and concentration.

Density plots provide a smoothed version of the histogram:

density(df.Income,
        label="Income Density",
        xlabel="Income",
        ylabel="Density",
        title="Income Distribution",
        linewidth=2,
        fill=(0, 0.3, :blue))

Scatter Plots

Scatter plots are excellent for visualizing the relationship between two continuous variables. They can reveal correlations, clusters, and patterns that might suggest feature interactions or guide model selection.

scatter(df.Age, df.Income,
        xlabel="Age",
        ylabel="Income",
        title="Income vs. Age",
        label="Customers",
        markersize=5,
        markerstrokewidth=0,
        alpha=0.6)

This plot would show if there's a discernible trend between age and income in your dataset.

Box Plots

Box plots are effective for summarizing the distribution of a numerical variable, potentially grouped by a categorical variable. They clearly display medians, quartiles, and potential outliers.

boxplot(df.Category, df.Income,
        xlabel="Category",
        ylabel="Income",
        title="Income Distribution by Category",
        legend=false,
        linewidth=2)

This helps compare income distributions across different categories, highlighting variations and outliers within each group.

Interactive and Advanced Visualizations with Makie.jl

Makie.jl is another powerful plotting ecosystem in Julia, designed for high-performance and interactive visualizations. It excels with large datasets, 3D plotting, and creating complex, publication-quality figures. Makie.jl has several backends, such as GLMakie.jl for interactive desktop windows, CairoMakie.jl for static vector graphics (SVG, PDF) and raster images (PNG), and WGLMakie.jl for WebGL-based plots in browsers.

A simple scatter plot with CairoMakie.jl (for static output) might look like this:

# Run this in the Julia REPL if not already installed
# using Pkg
# Pkg.add("CairoMakie") # Or GLMakie for interactive plots

using CairoMakie, DataFrames

# Reusing the DataFrame 'df'
f = Figure(resolution = (600, 400))
ax = Axis(f[1, 1],
          xlabel="Age",
          ylabel="Income",
          title="Income vs. Age with Makie")

scatter!(ax, df.Age, df.Income,
         markersize=10,
         color=(:teal, 0.7), # Using a color from the palette with transparency
         strokecolor=:black,
         strokewidth=1)

# To save the figure:
# save("income_vs_age_makie.png", f)
# To display in a capable environment (like VS Code Julia extension plot pane):
# f

Makie.jl offers a more scene-graph-based approach, providing fine-grained control over plot elements. While Plots.jl is often quicker for standard exploratory plots, Makie.jl becomes particularly useful when you need highly customized or interactive outputs, or when dealing with very large datasets where its performance benefits shine.

Applying Visualization in the Data Preparation Workflow

Visualization plays a direct role in various data preparation tasks:

1. Understanding Feature Distributions: Before applying transformations like scaling or normalization, plot histograms or density plots of your numerical features. A highly skewed distribution might benefit from a log transformation, for example. Visualizing before and after such transformations confirms their effect.

2. Detecting Anomalies and Outliers: Box plots are a standard tool for outlier detection. Scatter plots can also reveal unusual data points that deviate significantly from general patterns. For instance, plotting age vs. income might show an individual with an exceptionally high income for their age group, prompting further investigation.

3. Assessing Missing Data: While discussed in data cleaning, visualizing missingness can be insightful. A heatmap showing missing values across features and samples can reveal patterns, such as a feature that is mostly empty or samples with many missing entries. Plots.jl can create heatmaps:

   # Example: create a matrix indicating missing values
   # Assume 'raw_data' is a DataFrame possibly with missing values
   # M = ismissing.(Matrix(raw_data))
   # heatmap(M, title="Missing Data Pattern", xlabel="Features", ylabel="Samples")
   

4. Guiding Feature Engineering: Visualizing relationships between variables can inspire new feature creation. If a scatter plot of two variables X and Y shows a clear non-linear relationship, you might consider creating polynomial features (e.g., $X^2$, $Y^2$) or interaction terms (e.g., $X \times Y$).

5. Evaluating Categorical Data: Bar charts are useful for understanding the frequency of different categories within a feature. This can be important for deciding on encoding strategies or identifying imbalanced classes.

   # Using Plots.jl and a DataFrame 'df' with a 'Category' column
   category_counts = combine(groupby(df, :Category), nrow => :Count)
   bar(category_counts.Category, category_counts.Count,
       xlabel="Category",
       ylabel="Frequency",
       title="Distribution of Categories",
       legend=false)
   

Choosing the Right Tool and Plot

• For quick, everyday exploratory plots: Plots.jl with its simple syntax and multiple backend support is often the most convenient.
• For high-performance needs, large datasets, interactivity, or complex 2D/3D scenes: Makie.jl is a strong candidate.
• Choosing the plot type:
  • Distribution of one variable: Histogram, density plot, box plot.
  • Relationship between two continuous variables: Scatter plot.
  • Relationship between a continuous and a categorical variable: Box plots (one for each category), violin plots.
  • Relationship between two categorical variables: Stacked bar chart, grouped bar chart, or a heatmap of counts.

Effective data visualization is more than just generating plots; it's about asking the right questions and choosing visualizations that clearly answer them. Always label your axes, provide titles, and use legends when necessary. The goal is to gain understanding that leads to better data preparation and, ultimately, more effective machine learning models. As you work through your datasets, consider visualization an iterative partner in your analysis, helping you refine your data at each step.