Study Projects

The demos in this roadmap are small on purpose: one idea, one file, checked against an oracle. Real projects are the opposite: the same ideas, surrounded by years of decisions about memory, concurrency, testing and compatibility. Reading them is the fastest way to see what changes between a lesson and production. This page lists open-source Go projects worth studying, tells you what to look for in each and which lesson prepares you for it, and explains how to make a first contribution. Every project below is public on GitHub or in the Go source tree; open the repository, since projects change and this page describes what to look for, not a snapshot of the code.

How to Read a Project

Do not start at the top of the file tree. A large repository read from the first file teaches nothing. Use a method:

  1. Pick one question. "How does it evict?" "What happens on a hash collision?" "How is a lookup kept O(log n)?" One question gives you a path through the code.
  2. Read the tests first. Tests show the public behavior and the edge cases the authors were afraid of. The _test.go file next to a structure is the best documentation it has.
  3. Read the API with go doc. go doc github.com/hashicorp/golang-lru/v2 prints the exported names and comments without opening an editor.
  4. Find the core type and its invariants. Every structure has one: "the list is ordered by recency", "keys in a node are sorted", "every level has half the nodes". Find where each method preserves it.
  5. Run it. Clone it, run the tests (go test ./...) and the benchmarks (go test -bench . -benchmem). Change one line and see which test fails.
  6. Compare with your own version. After a lesson, write the structure from the lesson's demo, then read the library's. The differences (locks, generics, pooling, corner cases) are the lesson.
  7. Read the history. git log -p and git blame on a tricky line lead to the issue and the reasoning behind it; bug-fix commits show what went wrong in practice.

Algorithm Collections

The best places to start: many structures in one repository, small files, readable code.

ProjectWhat to studyPrepare with
TheAlgorithms/GoA large community collection of algorithms in Go: sorting, searching, graphs, strings, math, dynamic programming, data structures. Compare its versions with the demos here; it is also the friendliest place for a first contribution.Sorting, Graph Algorithms, Dynamic Programming
emirpasic/gods"Go Data Structures": lists, stacks, maps, sets, trees (AVL, red-black, B-tree), heaps, all behind common interfaces. Study how one container interface serves many implementations.Data Structures, Trees
Workiva/go-datastructuresProduction-oriented structures: augmented and interval trees, queues, a Fibonacci heap, futures. Notice how concurrent access is handled.Advanced Trees, Production Structures
zyedidia/genericGeneric (type-parameter) containers for Go: a compact codebase to learn how generics change the shape of a data-structure library.Arrays and Slices, Hash Tables
gonum/gonumNumerical computing for Go. Read the graph packages for shortest paths, spanning trees and flow written for real use, then compare with your own Dijkstra.Graphs, Graph Algorithms
dominikbraun/graphA small, generic graph library: directed and undirected graphs, topological sort, shortest paths, cycle detection. A good size for reading a whole library in an evening.Graphs

The Go Standard Library

The standard library is the most carefully reviewed Go code there is, and it is part of the golang/go repository, so it costs nothing to read. Browse it on pkg.go.dev (each function has a "source" link) or in the src directory of the repository.

PackageWhat to studyPrepare with
container/heap, container/list, container/ringShort and complete: a heap over any sort.Interface-like type, a doubly linked list with a sentinel (the design used in the LRU demo), a circular list.Heaps, Linked Lists
sort and slicesA real sort is a hybrid: pattern-defeating quicksort with insertion sort for small ranges and a heap-sort fallback to keep the worst case O(n log n). Also generic binary search.Sorting, Searching
the built-in map (runtime/map*.go)The runtime's hash table: how buckets or groups are laid out, how it grows, how it randomizes iteration order. Read after writing your own chained table.Hash Tables
sync and sync/atomicsync.Map, sync.Pool, sync.Once: small pieces of code with detailed comments on why they are designed as they are.Concurrent Structures
strings, bytes, regexpSubstring search that picks a strategy by pattern length (including Rabin-Karp), and a regular-expression engine built on automata rather than backtracking.String Algorithms
golang.org/x/time/rateThe standard token-bucket limiter. It computes tokens from elapsed time, like the demo, and takes a clock so that it can be tested.Rate Limiters
golang.org/x/syncsingleflight (call collapsing), errgroup and semaphore: the production versions of the patterns in the concurrency demo.Call Collapsing

Caches

ProjectWhat to studyPrepare with
hashicorp/golang-lruLRU, 2Q and ARC caches. The simplelru package is the map-plus-list design from the lesson; the outer package adds locking. A short path from the demo to production code.LRU Cache
dgraph-io/ristrettoA high-throughput cache with a TinyLFU admission policy that uses a Count-Min sketch and a Bloom filter: the fix for the scan weakness, built from structures in the Advanced Techniques lesson. The TinyLFU paper is short and readable.Advanced Techniques
maypok86/otterA modern, high-performance cache with state-of-the-art eviction and lock-free-style reads. Read its README benchmarks and design notes to see how caches are compared.Production Structures
allegro/bigcacheA sharded cache that keeps entries in large byte slices so that the garbage collector has almost no pointers to scan. Shows how memory layout, not the algorithm, can be the bottleneck.Arrays and Slices

Storage Engines and Indexes

Databases are where data-structure theory meets disks. These engines are written in Go, are used in serious systems, and are small enough to read.

ProjectWhat to studyPrepare with
etcd-io/bboltAn embedded key-value store built on a copy-on-write B+tree in a memory-mapped file. One of the most readable database cores in Go: pages, cursors, transactions.Trees, Advanced Trees
cockroachdb/pebbleA log-structured merge-tree (LSM) engine in the LevelDB/RocksDB family: a skip list in memory, sorted immutable files on disk, compaction, Bloom filters to skip files.Sorting, Bloom Filters
dgraph-io/badgerAnother LSM store that keeps large values in a separate log so that compaction moves only keys. Compare the design choices with Pebble's.Hash Tables, Heaps
etcd-io/etcdThe consistent store behind Kubernetes: a replicated log (Raft), a versioned key-value index and a B-tree of revisions. Large; read one package, such as the in-memory index, not the whole thing.Advanced Trees
google/btree and tidwall/btreeIn-memory B-trees with iteration, ranges and cheap copy-on-write clones. Compare the two: generics, API, and how each splits and merges nodes.Trees
blevesearch/bleveFull-text search: text analysis, inverted indexes, scoring. Shows how tries and sorted term lists become a search engine.String Algorithms

Compact and Probabilistic Structures

ProjectWhat to studyPrepare with
bits-and-blooms/bitset and bloomA bitset over uint64 words and a Bloom filter on top of it: the same design as the demos, with sizing helpers and serialization.Bit Manipulation
RoaringBitmap/roaringCompressed bitmaps that pick a representation (sorted array, bitmap, or runs) per block of 65,536 values. A lesson in adapting the structure to the data.A Bitset
axiomhq/hyperloglogA HyperLogLog with a sparse form for small counts, so that a sketch with a few items does not cost 16 KB. Compare its estimator with the demo. The original Flajolet et al. paper is the reference.HyperLogLog
cespare/xxhashA fast non-cryptographic hash in Go with an assembly version. Hashes like this feed Bloom filters, sketches and hash rings; see why quality and speed both matter.Hash Tables

Distributed Systems and Infrastructure

ProjectWhat to studyPrepare with
buraksezer/consistentConsistent hashing with bounded loads: a ring that also caps how much any server receives. Compare with the ring in the demo. For a different idea, read the jump consistent hash paper (a dozen lines of code).Consistent Hashing
kubernetes/kubernetesVery large, but its work queue (client-go's util/workqueue) is a self-contained study: de-duplicating queue, delayed queue and rate-limited retries with exponential backoff.Stacks and Queues, Heaps
prometheus/prometheusIts time-series database (tsdb) combines an inverted index over labels with compressed sample blocks. A study in indexing and compact encoding under a heavy write load.Hash Tables, Advanced Techniques

A Reading Plan

Pair each lesson with one reading task and one small experiment:

After lessonReadThen
Heapscontainer/heapRewrite your heap to satisfy heap.Interface and use it for Dijkstra.
Sortingthe slices sort in the Go sourceFind where it switches to insertion sort and benchmark that threshold on your machine.
Hash Tablesthe built-in map's runtime codeExplain why map iteration order is randomized, and test it.
Graph Algorithmsgonum/graph or dominikbraun/graphCompare its shortest path with your own on the same random graph.
Advanced Treesgoogle/btree, then bboltFind the node split, and check how the tree stays balanced.
Advanced Techniquesbits-and-blooms/bloom, roaringMeasure the false-positive rate of the library against its formula.
Production Structuresgolang-lru, x/time/rate, singleflightReplace your demo's structure with the library and rerun the demo's checks against it.

Contributing

Contributing is how reading becomes skill: someone who knows the project reviews your code. Start small; the first goal is a merged change, not an impressive one.

A Ladder of Contributions

  1. Use the project. Build something with it. Every confusing moment is a documentation bug.
  2. Read the contribution guide. Look for CONTRIBUTING.md, the issue templates and the code-of-conduct file before writing any code.
  3. Fix the small things. Typos and unclear comments, a missing example in the docs, a missing test case for an edge case.
  4. Add tests and benchmarks. A test that compares a structure with a slow oracle on random input (the technique used throughout this roadmap) is a valuable contribution to almost any library. So is a fuzz test (Go fuzzing tutorial) or a benchmark for a hot path.
  5. Reproduce a bug. Open the issue list, filter by labels such as good first issue or help wanted where the project uses them, and comment before you start so that two people do not do the same work. A failing test that reproduces the bug is already half of the fix.
  6. Add or improve an algorithm. In a collection such as TheAlgorithms/Go, check first that the algorithm is not already there, then follow the repository's conventions for layout, documentation and tests.
  7. Take on a larger change. Discuss the design in an issue first. Large unannounced pull requests are the ones most often declined.

Before You Send a Change

  • Run gofmt -l ., go vet ./... and go test ./..., and fix what they report. Add -race for anything concurrent.
  • Keep the change small and about one thing, with a clear title and a description that says what was wrong and how you checked the fix.
  • Match the surrounding style, comment density and naming, even where you would have chosen differently.
  • Expect review comments. Answer each one, push follow-up commits, and be patient: maintainers are often volunteers.
  • Do not paste code you cannot explain, and do not submit code whose license you do not know.

Contributing to Go Itself

The Go project has its own process: changes are reviewed on Gerrit, not through GitHub pull requests, and contributors sign a contributor license agreement. The contribution guide explains the steps. Documentation, examples and tests in the standard library are a realistic place to begin; the x/ repositories, such as golang.org/x/sync and golang.org/x/time, follow the same process.


Practice Lab

  1. Diff of two LRUs. Read golang-lru/simplelru and 73_lru_cache.go. List every difference, and say for each whether it is about correctness, speed, or convenience.
  2. Oracle for a library. Pick a container from emirpasic/gods and write a random test that checks it against a plain slice or map. Report what you find, even if the answer is that everything agrees.
  3. Limiter comparison. Replay the traffic from 74_rate_limiters.go through golang.org/x/time/rate. How many requests does it accept in the boundary test, and why?
  4. Read a B-tree. In google/btree, find where a full node is split. Draw the before and after for a node of 3 items, and compare with your notes from Advanced Trees.
  5. First contribution. Choose one project from this page and write down: the issue or gap you will address, the file you will change, and the test that will prove it. Submit it.

More material is on the References page. The Demo Examples list has every runnable program in this roadmap.