Eve Generators

Generator, function and closure

A generator is a subprogram, declared with the keyword generator, that produces its values one at a time. It gives a value to its caller, waits, and continues from the same place when the caller asks for the next value. Its local variables keep their values between two requests. A generator is useful for long or infinite sequences, for reading data step by step, and for tasks that take turns on one core.

Declaration

A generator is declared with the keyword generator, like a function or a method: generator name(parameters) => (@x: T) is … return;. It declares one result, the value it produces. yield is allowed only in the body of a generator: in a function or a method it is a compile error, so a reader always knows from the header that a call creates a generator.

** produce the numbers 1 to n, one at a time
generator count_to(n: Integer) => (@x: Integer) is
  for i in (1..n) do
    let x := i;           ** set the result
    yield;            ** give x to the caller, wait here
  done;
return;               ** no more values

• yield; gives the current value of the result to the caller and suspends the generator. This is the explicit form;
• yield expression; is the short form of let x := expression; yield;. Both forms are valid;
• return; ends the generator: the caller sees that there are no more values;
• A generator can be infinite, with loop do … yield; repeat;: the caller decides when to stop;
• A generator can be declared where a function or a method can: in a class, with public or private (public generator walk(@self) => (@node: Node) is … return;); in a module, exported with export (count_to); or private to it; or in a driver or an aspect, private to that script;
• Calling a generator is deterministic: for the same arguments it returns an equivalent generator object and changes nothing, so a generator has no !. The generator object is what moves on, step by step.

Use

A call of a generator does not run its body. It creates a generator object of the core class Generator(:T), where T is the type of the result. The body runs only when the caller asks for a value. There are three ways to ask:

** 1. a for loop takes the values one by one
for v in count_to(5) do
  print v;                         ** 1 2 3 4 5
done;

** 2. a comprehension collects all the values
new squares := (v * v | v in count_to(5));   ** (1, 4, 9, 16, 25)

** 3. a generator object driven step by step
new g := count_to(3);              ** nothing runs yet
while g.next() do                  ** run to the next yield
  print g.value;                   ** the value given by yield
done;

A generator object is created by calling the generator, like any value: new g := count_to(3);. A generator called as a statement, count_to(5);, is an error, because its values would be lost.

Rules

1. One level: yield is allowed only in the body of the generator itself, not in a method or function it calls, not in a lambda and not in a process. A method called by the generator runs to completion. A generator that uses the values of another one loops over it and gives them again: for c in child.walk() do yield c; done;. This rule makes a generator simple: it is not a coroutine, and Eve has no coroutines;
2. Lazy: the arguments are evaluated when the generator object is created; the body begins at the first request;
3. State: the parameters and the local variables live in the generator object between two requests. They are dropped when the body returns, at close(), or when nobody refers to the object any more. A for loop left with break closes its generator;
4. Parameters: a generator receives inputs only, by value. It has no "@" parameters other than its result;
5. Errors: an error raised in the body ends the generator and is raised in the caller, at the next() or at the for line, where the recover region of the process can handle it;
6. Where it is declared: a generator in a driver or an aspect is private to it and needs no thread safety; a generator exported by a module must be thread safe (see Generators and threads);
7. One owner: a generator object is never copied. Passing it to a method shares the same object. It can't be an argument of apply or start: it belongs to one process and runs on the core of its caller.

Cooperative tasks

Generators are also the cooperative tasks of Eve. Each generator is a task, and each yield is a point where the task lets the others run. A loop in the caller gives the turns. Everything runs on one core, in a fixed order, so the result is the same on every run and no data needs protection.

# two tasks take turns on one core
driver ping_pong is
  generator ping(n: Integer) => (@step: Integer) is
    for i in (1..n) do
      print "ping";
      yield i;                     ** let the other task run
    done;
  return;

  generator pong(n: Integer) => (@step: Integer) is
    for i in (1..n) do
      print "pong";
      yield i;
    done;
  return;

  process main is
    new a := ping(3);
    new b := pong(3);
    while not (a.done and b.done) do
      a.next() if not a.done;
      b.next() if not b.done;
    done;                          ** ping pong ping pong ping pong
  return;
end ping_pong;

The library module task (version 2) gives the turns for you:

  task.round_robin((ping(3), pong(3)));   ** same output as the loop above

Generators and channels

A generator runs on the core of its caller, so it can't feed a started aspect directly. To send the values of a generator to parallel workers, a started producer aspect creates the generator and sends its values into a channel: for v in source() do jobs.send(v); done;. Channels are explained below.

More examples

An infinite generator

A generator can be infinite. The caller decides when to stop, with break; the generator is closed and its state is dropped.

** the Fibonacci numbers, without end
generator fibonacci() => (@x: Integer) is
  new a := 0;
  new b := 1;
  loop do
    yield a;
    let t := a + b;
    let a := b;
    let b := t;
  repeat;
return;

process main is
  for n in fibonacci() do
    if n > 100 do break; done;
    print n;                          ** 0 1 1 2 3 5 8 13 21 34 55 89
  done;
return;

A generator reads step by step

A generator does not read everything at once: each request reads one more piece, so a large file never has to fit in memory. This is the base of the streams of Eve (see the plan of the data client).

** one line at a time (the module fs is planned: see System Library)
generator lines(path: String) => (@line: String) is
  new f := fs.open(path);
  defer f.close();                    ** also when the caller leaves the loop with break
  for text in f.lines() do
    yield text;
  done;
return;

Read next: Closures