Read my new blog post on my work on Test Cases Strategies for CNCF - LitmusChaos! Read more

Read my new blog post on my work with Kubernetes during the Google Summer of code! Read more

GoGolangConcurrencyBackend

Writing Concurrent Programs in Go

2025-02-14

Goroutines and channels are elegant, but concurrency bugs are subtle. A practical guide to Go's concurrency model, common patterns, and the mistakes that will bite you.

Go was designed for concurrency from the ground up. Goroutines are cheap (a few kilobytes of stack, not an OS thread), channels are first-class, and the go keyword is all it takes to spin up concurrent work. But easy to start does not mean easy to get right. This article covers how Go's concurrency model works and the patterns that actually hold up in production.

Goroutines Are Not Threads

When you write go f(), you are not creating an OS thread. You are scheduling a goroutine on Go's runtime scheduler, which multiplexes goroutines onto a pool of OS threads (GOMAXPROCS threads by default, one per CPU core).

package main

import (
    "fmt"
    "time"
)

func main() {
    for i := 0; i < 5; i++ {
        go func(n int) {
            fmt.Println("goroutine", n)
        }(i)
    }
    time.Sleep(100 * time.Millisecond) // bad practice, just for illustration
}

The time.Sleep at the end is a code smell — the main goroutine exits and takes all other goroutines with it. The right tool for waiting is a sync.WaitGroup.

WaitGroups: The Right Way to Wait

import "sync"

func main() {
    var wg sync.WaitGroup

    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(n int) {
            defer wg.Done()
            process(n)
        }(i)
    }

    wg.Wait() // blocks until all goroutines call Done()
}

defer wg.Done() is important — it ensures Done is called even if the goroutine panics.

Channels: Communication Over Shared Memory

Go's philosophy is: do not communicate by sharing memory; share memory by communicating. Channels are typed pipes between goroutines.

func producer(ch chan<- int) {
    for i := 0; i < 10; i++ {
        ch <- i
    }
    close(ch)
}

func main() {
    ch := make(chan int, 5) // buffered channel, capacity 5
    go producer(ch)

    for val := range ch { // reads until channel is closed
        fmt.Println(val)
    }
}

Unbuffered channel (make(chan int)) — sends block until a receiver is ready. Provides synchronisation.

Buffered channel (make(chan int, N)) — sends block only when the buffer is full. Decouples sender and receiver speed.

The Select Statement

select lets a goroutine wait on multiple channel operations at once, taking whichever is ready first:

func main() {
    ch1 := make(chan string)
    ch2 := make(chan string)

    go func() { ch1 <- "from ch1" }()
    go func() { ch2 <- "from ch2" }()

    for i := 0; i < 2; i++ {
        select {
        case msg := <-ch1:
            fmt.Println(msg)
        case msg := <-ch2:
            fmt.Println(msg)
        }
    }
}

select with a default case becomes non-blocking — it falls through to default if no channel is ready. Useful for polling without blocking.

Context: Cancellation and Deadlines

Real services need timeouts and cancellation. context.Context is Go's standard mechanism:

import (
    "context"
    "fmt"
    "time"
)

func fetchData(ctx context.Context, id int) (string, error) {
    select {
    case <-time.After(200 * time.Millisecond): // simulated work
        return fmt.Sprintf("data for %d", id), nil
    case <-ctx.Done():
        return "", ctx.Err()
    }
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
    defer cancel()

    result, err := fetchData(ctx, 42)
    if err != nil {
        fmt.Println("timed out:", err)
        return
    }
    fmt.Println(result)
}

Always call cancel() — even if the context already expired. Failing to call it leaks resources. defer cancel() right after creation is the idiomatic pattern.

Pass context as the first argument to every function that does I/O. This is a Go convention enforced by linters and makes cancellation propagate correctly through your call stack.

Worker Pool Pattern

Spinning up one goroutine per task is fine for small workloads. For thousands of tasks, a worker pool bounds concurrency:

func workerPool(jobs <-chan int, results chan<- int, numWorkers int) {
    var wg sync.WaitGroup
    for i := 0; i < numWorkers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for job := range jobs {
                results <- heavyComputation(job)
            }
        }()
    }
    wg.Wait()
    close(results)
}

func main() {
    jobs := make(chan int, 100)
    results := make(chan int, 100)

    go workerPool(jobs, results, 8) // 8 concurrent workers

    for i := 0; i < 50; i++ {
        jobs <- i
    }
    close(jobs)

    for result := range results {
        fmt.Println(result)
    }
}

This pattern shows up everywhere in Go backend code — HTTP clients, database query batchers, file processors.

The Race Detector

The Go toolchain ships with a built-in race detector. Always run tests with it enabled:

go test -race ./...
go run -race main.go

A data race is when two goroutines access the same memory without synchronisation and at least one is a write. They are non-deterministic and can corrupt data silently in production. The race detector catches them at runtime with a small performance overhead.

When to Use a Mutex vs. a Channel

A common point of confusion:

  • Use a channel when goroutines need to communicate — passing data or signalling events.
  • Use a mutex (sync.Mutex) when multiple goroutines share state and you need to protect reads and writes to it.
type SafeCounter struct {
    mu    sync.Mutex
    count int
}

func (c *SafeCounter) Increment() {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.count++
}

func (c *SafeCounter) Value() int {
    c.mu.Lock()
    defer c.mu.Unlock()
    return c.count
}

Trying to use channels for everything leads to overcomplicated code. A shared counter protected by a mutex is simpler and more obvious than passing increment messages through a channel.

A Note on Goroutine Leaks

A goroutine leak happens when a goroutine is blocked forever — usually waiting on a channel that nobody will ever send to or close.

// BUG: if the HTTP request times out and nobody reads from ch,
// this goroutine is leaked forever
func leaky() {
    ch := make(chan result)
    go func() {
        ch <- doExpensiveWork() // blocks if caller already returned
    }()
    // ... caller times out and returns
}

Fix: always use buffered channels or context cancellation so goroutines have a way out.

Go's concurrency primitives are genuinely elegant once the mental model settles. The runtime handles the scheduling; your job is to design clean communication between goroutines and ensure every goroutine has a clear exit condition.