Hands-on: Scripting File Operations

The following exercises involve writing scripts that read data from files, perform simple processing, and write results to new files. These practical examples demonstrate how to make Julia programs interact with the file system.

Exercise 1: Reading and Summarizing Numerical Data

In this exercise, you'll write a Julia script to read a list of numbers from a text file, calculate their sum and average, and then display these results. This task is common in data analysis preliminaries.

1. Prepare Your Data File

First, create a text file named student_scores.txt in the same directory where you will save your Julia script. Populate this file with the following scores, each on a new line:

85
92
78
95
88
70

2. Write the Julia Script

Now, create a Julia script (e.g., analyze_scores.jl) with the following code:

# analyze_scores.jl

function process_scores(filepath::String)
    scores = Float64[] # Initialize an empty array to store scores as floating-point numbers
    total_sum = 0.0
    num_scores = 0

    try
        open(filepath, "r") do file_stream
            println("Successfully opened $filepath for reading.")
            for line in eachline(file_stream)
                try
                    score = parse(Float64, line) # Convert the string line to a Float64
                    push!(scores, score)         # Add the score to our array
                    total_sum += score
                    num_scores += 1
                catch e
                    println("Warning: Could not parse '$line' as a number. Skipping.")
                end
            end
        end # File is automatically closed here

        if num_scores > 0
            average_score = total_sum / num_scores
            println("\n--- Score Analysis ---")
            println("Scores read: $scores")
            println("Total sum of scores: $total_sum")
            println("Number of scores: $num_scores")
            println("Average score: $(round(average_score, digits=2))") # Display average rounded to 2 decimal places
        else
            println("No valid scores found in the file.")
        end

    catch e
        println("Error processing file $filepath: $e")
    end
end

# Define the path to your data file
data_file_path = "student_scores.txt"
process_scores(data_file_path)

3. Code Breakdown

• function process_scores(filepath::String): Defines a function that takes the file path as input.
• scores = Float64[]: Initializes an empty array scores that will hold numbers of type Float64.
• open(filepath, "r") do file_stream ... end: This is the recommended way to open a file. It takes the file path and the mode ("r" for read) as arguments. The do file_stream block ensures that the file (file_stream) is automatically closed when the block finishes, even if errors occur.
• for line in eachline(file_stream): This loop iterates over each line in the opened file.
• score = parse(Float64, line): The parse function attempts to convert the line (which is a string) into a Float64 number. If the line cannot be converted (e.g., it contains text), it will throw an error.
• try ... catch e ... end inside the loop: This handles potential errors during parsing. If a line isn't a valid number, it prints a warning and skips that line, allowing the script to continue with other valid numbers.
• push!(scores, score): Adds the successfully parsed score to the end of the scores array.
• The script then calculates the sum and average and prints them. The round(average_score, digits=2) function is used to present the average nicely.
• The outer try ... catch e ... end block handles potential errors with opening or reading the file itself, like if the file doesn't exist.

4. Running and Verifying

Save the script as analyze_scores.jl. Open your Julia REPL, navigate to the directory where you saved the file, and run the script:

include("analyze_scores.jl")

You should see output similar to this:

Successfully opened student_scores.txt for reading.

--- Score Analysis ---
Scores read: [85.0, 92.0, 78.0, 95.0, 88.0, 70.0]
Total sum of scores: 508.0
Number of scores: 6
Average score: 84.67

If you had a non-numeric line in student_scores.txt, you would also see the warning message for that line.

Exercise 2: Transforming Text Data and Writing to a New File

This exercise demonstrates reading text from one file, performing a simple transformation (converting to uppercase), and writing the modified text to a new file.

1. Prepare Your Input File

Create a file named meeting_notes.txt with the following content:

Agenda: Project Phoenix Review
Attendees: Alice, Bob, Charlie

Discussion Points:
- progress update
- budget allocation
- next milestones

Action items to be finalized.

2. Write the Julia Script

Create a Julia script, for example transform_text.jl, with this code:

# transform_text.jl

function transform_and_save(input_filepath::String, output_filepath::String)
    println("Starting text transformation...")
    line_count = 0

    try
        open(input_filepath, "r") do input_file
            open(output_filepath, "w") do output_file # "w" for write mode
                println("Reading from $input_filepath and writing to $output_filepath")
                for line in eachline(input_file)
                    uppercase_line = uppercase(line) # Convert line to uppercase
                    write(output_file, uppercase_line * "\n") # Write the modified line, add newline
                    line_count += 1
                end
            end # output_file is automatically closed
        end # input_file is automatically closed
        println("Transformation complete. Processed $line_count lines.")
        println("Output saved to $output_filepath")
    catch e
        println("An error occurred: $e")
    end
end

input_file = "meeting_notes.txt"
output_file = "processed_notes.txt"
transform_and_save(input_file, output_file)

3. Code Breakdown

• open(output_filepath, "w") do output_file: Opens the output file in write mode ("w"). If the file doesn't exist, it's created. If it does exist, its content is overwritten.
• uppercase_line = uppercase(line): The uppercase() function converts all characters in the string line to their uppercase equivalents.
• write(output_file, uppercase_line * "\n"): Writes the uppercase_line to the output_file. We append "\n" because eachline typically strips the newline character, and write does not add it automatically like println does.

The general flow involves reading from an input file, processing the content, and writing to an output file. This can be visualized as:

Data flow from an input file, through the transformation script, to an output file.

4. Running and Verifying

Save and run transform_text.jl from your Julia REPL:

include("transform_text.jl")

The script will print messages indicating its progress:

Starting text transformation...
Reading from meeting_notes.txt and writing to processed_notes.txt
Transformation complete. Processed 7 lines.
Output saved to processed_notes.txt

After running, open processed_notes.txt. Its content should be:

AGENDA: PROJECT PHOENIX REVIEW
ATTENDEES: ALICE, BOB, CHARLIE

DISCUSSION POINTS:
- PROGRESS UPDATE
- BUDGET ALLOCATION
- NEXT MILESTONES

ACTION ITEMS TO BE FINALIZED.

All text from meeting_notes.txt should now be in uppercase.

Exercise 3: Maintaining a Simple Activity Log

For this exercise, you'll create a script that appends a timestamped message to a log file each time it's executed. This is useful for tracking when certain automated tasks or scripts are run.

1. Write the Julia Script

Create a Julia file, say log_activity.jl, with the following content. This script uses the Dates module from Julia's standard library to get the current date and time.

# log_activity.jl
using Dates # Import the Dates module for timestamp functionality

function append_to_log(log_filepath::String, message::String)
    try
        timestamp = Dates.format(now(), "yyyy-mm-dd HH:MM:SS") # Get current time formatted
        log_entry = "$timestamp : $message\n"                 # Create the log entry

        open(log_filepath, "a") do log_file # "a" for append mode
            write(log_file, log_entry)
        end # log_file is automatically closed

        println("Successfully wrote to log: $log_filepath")
    catch e
        println("Error writing to log $log_filepath: $e")
    end
end

log_file_path = "activity.log"
activity_message = "Script execution started."
append_to_log(log_file_path, activity_message)

# You can add another message for completion
activity_message_done = "Script execution finished."
append_to_log(log_file_path, activity_message_done)

2. Code Breakdown

• using Dates: This line makes functions from the Dates module available, such as now() and Dates.format().
• timestamp = Dates.format(now(), "yyyy-mm-dd HH:MM:SS"): now() gets the current date and time, and Dates.format() converts it into a human-readable string based on the provided format.
• open(log_filepath, "a") do log_file: The file is opened in append mode ("a"). If the file exists, new data is added to the end. If it doesn't exist, it's created.
• write(log_file, log_entry): Appends the formatted log_entry to the file.

3. Running and Verifying

Save log_activity.jl. Run it from the Julia REPL:

include("log_activity.jl")

Output:

Successfully wrote to log: activity.log
Successfully wrote to log: activity.log

Now, check the contents of activity.log. It should have two new lines similar to this (the timestamp will reflect when you ran it):

2023-10-27 10:30:05 : Script execution started.
2023-10-27 10:30:05 : Script execution finished.

Run the script include("log_activity.jl") a few more times. Each time you run it, two new timestamped entries will be appended to activity.log, demonstrating the append functionality. For example, after a second run:

2023-10-27 10:30:05 : Script execution started.
2023-10-27 10:30:05 : Script execution finished.
2023-10-27 10:32:15 : Script execution started.
2023-10-27 10:32:15 : Script execution finished.

These exercises provide a foundation for interacting with files in Julia. You've learned to read data, process it, and write it out, which are common operations in many programming tasks, from simple utilities to complex data processing pipelines. As you continue, you'll find these skills indispensable.