Introduction to Concurrency

Why Concurrent Programming Matters

Until now, every program you've written has been sequential - instructions execute one after another, in a single flow of execution. This is straightforward to reason about, but it leaves a lot of computing power on the table.

Modern CPUs have multiple cores. A typical laptop might have 8 cores, and servers often have 64 or more. A sequential program can only use one core at a time, leaving the others idle. To fully utilize modern hardware, we need concurrent programming.

What is Concurrency?

Concurrency means dealing with multiple things at once. A concurrent program has multiple threads of execution that can make progress independently.

There's a related but distinct concept: parallelism means doing multiple things at once. The distinction matters:

• Concurrency is about structure - designing your program to handle multiple tasks
• Parallelism is about execution - actually running tasks simultaneously on multiple cores

You can have concurrency without parallelism (a single-core CPU switching between tasks) and parallelism without concurrency (a GPU running the same operation on thousands of data points). In practice, we often use concurrent programming to achieve parallelism.

// Sequential: one thing at a time
download_file("file1.txt");  // Wait for this to complete...
download_file("file2.txt");  // ...then start this
download_file("file3.txt");  // ...then this

// Concurrent: multiple things in progress
// All three downloads can happen simultaneously
start_download("file1.txt");
start_download("file2.txt");
start_download("file3.txt");
wait_for_all_downloads();

Why Use Concurrency?

There are two main reasons to write concurrent programs:

Performance Through Parallelism

Some tasks are CPU-bound - they need lots of computation. If you can split the work across multiple cores, you can finish faster.

// Processing a large dataset sequentially
for (const auto& item : million_items) {
    process(item);  // Takes 1ms each = 1000 seconds total
}

// Processing in parallel across 8 cores
// Ideally: ~125 seconds (8x speedup)

Real-world speedups are rarely perfect (there's overhead in managing threads), but 4-6x improvements on 8 cores are common.

Responsiveness Through Asynchrony

Some tasks are I/O-bound - they spend most of their time waiting for external resources like disk, network, or user input.

Consider a GUI application. If you download a file on the main thread:

void on_download_button_clicked() {
    download_large_file();  // Takes 30 seconds
    // UI is frozen for 30 seconds!
    // User can't click anything, window shows "Not Responding"
}

With concurrency, the download happens in the background:

void on_download_button_clicked() {
    std::thread download_thread(download_large_file);
    download_thread.detach();
    // UI remains responsive immediately
    // User can continue using the application
}

Threads: The Unit of Concurrency

A thread (or "thread of execution") is the basic unit of concurrency in C++. Each thread has:

• Its own instruction pointer - where it is in the code
• Its own stack - for local variables and function calls
• Shared access to heap memory, global variables, and static variables

A program starts with one thread (the main thread). You can create additional threads that run concurrently with the main thread and with each other.

#include <iostream>
#include <thread>

void say_hello() {
    std::cout << "Hello from a new thread!\n";
}

int main() {
    std::cout << "Main thread starting\n";

    std::thread t(say_hello);  // Create and start a new thread

    std::cout << "Main thread continuing\n";

    t.join();  // Wait for the thread to finish

    std::cout << "Main thread done\n";
}

The output might be:

Main thread starting
Main thread continuing
Hello from a new thread!
Main thread done

Or it might be:

Main thread starting
Hello from a new thread!
Main thread continuing
Main thread done

Or even:

Main thread starting
Main threadHello from a new thread!
 continuing
Main thread done

This unpredictability is a key characteristic of concurrent programs. The operating system's scheduler decides when each thread runs, and we generally can't control or predict it.

The Challenges of Concurrency

Concurrent programming is powerful but introduces significant challenges:

Race Conditions

When multiple threads access shared data and at least one modifies it, you have a race condition. The result depends on the unpredictable order of execution.

int counter = 0;

void increment_counter() {
    for (int i = 0; i < 100000; ++i) {
        ++counter;  // Not a single operation!
    }
}

int main() {
    std::thread t1(increment_counter);
    std::thread t2(increment_counter);

    t1.join();
    t2.join();

    // Expected: 200000
    // Actual: Some random smaller number (e.g., 143267)
    std::cout << "Counter: " << counter << '\n';
}

Why? Because ++counter is actually three operations: read the value, add one, write the value. Two threads can interleave these operations and lose updates.

Deadlocks

When threads wait for each other in a cycle, they can get stuck forever:

// Thread 1: Lock A, then try to lock B
// Thread 2: Lock B, then try to lock A
// Both threads wait forever for the other to release their lock

Data Races and Undefined Behavior

In C++, a data race (unsynchronized access where at least one is a write) is undefined behavior. Your program might crash, produce wrong results, or appear to work until it doesn't.

C++ Concurrency Support

C++11 introduced a comprehensive threading library in the <thread> header. Before this, C++ programmers had to use platform-specific APIs (POSIX threads on Linux/macOS, Windows threads on Windows) or third-party libraries.

The standard library provides:

When to Use Concurrency

Concurrency adds complexity. Use it when:

• You have CPU-bound work that can be parallelized
• You need responsive UIs that don't freeze during long operations
• You're writing servers that handle multiple clients
• You're doing I/O-bound work where you can overlap waiting

Don't use concurrency when:

• Your program is fast enough already
• The work can't be meaningfully parallelized
• The added complexity isn't worth the benefit