Elixir: Collections & Enumeration

Enum and Stream are the workhorses of Elixir data processing. This lesson covers eager and lazy pipelines, comprehensions, sets, and — importantly — which structure to pick when the data gets large.

Enum: Eager Pipelines

Enum functions consume the whole collection and return a new one. Each step is a full pass — perfect for everyday sizes, wasteful for huge or infinite ones.

orders = [
  %{id: 1, user: "ada", total: 120, status: :paid},
  %{id: 2, user: "grace", total: 80, status: :pending},
  %{id: 3, user: "ada", total: 200, status: :paid}
]

orders
|> Enum.filter(&(&1.status == :paid))        # keep paid orders
|> Enum.map(&(&1.total))                     # project the totals
|> Enum.sum()                                #=> 320

# Reduce is the universal fold — every other Enum function can be
# expressed with it. Grouping paid orders by user:
orders
|> Enum.filter(&(&1.status == :paid))
|> Enum.reduce(%{}, fn o, acc ->
      Map.update(acc, o.user, o.total, &(&1 + o.total))
    end)                                       #=> %{"ada" => 320}

Stream: Lazy Pipelines

Stream builds the pipeline as a description — nothing runs until you call a terminal function like Enum.to_list/1. Elements flow through the whole chain one at a time, which means constant memory and support for infinite sources.

# Infinite generator: even numbers, forever.
Stream.iterate(0, &(&1 + 2))
|> Stream.map(&(&1 * &1))               # squares
|> Stream.take(5)                       # only 5 elements are pulled
|> Enum.to_list()                       #=> [0, 4, 16, 36, 64]

# File lines are streamed — the file never loads fully into memory.
File.stream!("access.log")
|> Stream.map(&String.trim_trailing/1)
|> Stream.filter(&String.contains?(&1, "ERROR"))
|> Stream.take(10)                      # stop reading after 10 matches
|> Enum.to_list()

Choose	When	Why
`Enum`	in-memory data, few steps, you want intermediate values	each step is a simple full pass
`Stream`	large or unbounded data, many chained steps, files/network	single pass, constant memory, early termination

Comprehensions

The for special form maps, filters, and flattens in one readable expression, over any number of generators. It is often clearer than the equivalent Enum chain — especially when combining two collections.

# One generator with a filter, collecting into a list (default).
for {:ok, v} <- results, v > 10, do: v

# Two generators — a Cartesian product with a filter.
for x <- [1, 2, 3], y <- [:a, :b], y != :a or x > 1, do: {x, y}
#=> [{1, :b}, {2, :a}, {2, :b}, {3, :a}, {3, :b}]

# `into:` picks the destination structure — here a map of squares.
for i <- 1..4, into: %{}, do: {i, i * i}
#=> %{1 => 1, 2 => 4, 3 => 9, 4 => 16}

Sets and Other Structures

MapSet is the standard set: unique elements, fast membership. Date.Range, maps, and even binaries all work with Enum because they implement the Enumerable protocol (next lesson).

s = MapSet.new([1, 2, 3])
MapSet.union(s, MapSet.new([3, 4]))    #=> #MapSet<[1, 2, 3, 4]>
MapSet.subset?(MapSet.new([1]), s)     #=> true
3 in s                                 #=> true — membership is cheap

Choosing the Structure

Cost chart comparing list prepend, list append, tuple access, and map lookup — Fig. 1 — Lists are cheap at the head and expensive at the tail; maps and tuples buy different guarantees.

Structure	Cheap operation	Expensive operation	Reach for it when…
List	prepend, head/tail walk	index, append, length	enumerating in order
Tuple	index access	insert/remove	fixed shape (returns, coordinates)
Map	key access, update	ordered traversal	keyed lookups and pattern matching
MapSet	membership, union	indexing	uniqueness and set algebra
Keyword list	reading options	lookup by key (O(n))	function options, ordered small pairs

Practice

Rewrite the orders example with a comprehension instead of Enum.filter + Enum.map.
Stream the words of a large text file and find the 10 most frequent words (Enum.frequencies/1 then Enum.sort/1).
Convert a list pipeline into a Stream pipeline and explain which version terminates early on File.stream! input.

Next: Structs, Protocols & Behaviours