Subprograms & the Stack

call and ret

A subroutine is just a jump that can come back. The CPU provides two instructions that make this work, and together they are the entire mechanism behind functions, methods, and recursion.

    call my_func        ; 1. push the address of the NEXT instruction
                        ; 2. jump to my_func
    ; ... execution resumes here after my_func returns ...

my_func:
    ; ... body ...
    ret                 ; pop the saved address back into rip

call is therefore push + jmp, and ret is pop + jmp. This is why the stack must stay balanced: if your function pushes three values and pops two, ret will pop a data value instead of the return address, and the program will jump into garbage.

A minimal, complete function that takes a value in rdi and returns its double in rax:

; long double_it(long x)   — x in rdi, result in rax
double_it:
    mov  rax, rdi          ; copy the argument
    add  rax, rax          ; rax = rax * 2
    ret                    ; return to the caller

Note what is absent: no prologue, no saved registers, no push/pop. A leaf function that touches nothing but scratch registers needs no frame at all. Frames exist to protect values, not to decorate code.

The Stack Frame

The Prologue

At the top of a function you establish a frame: a region of the stack reserved for this call's locals, addressed through rbp so that the offsets stay constant even as rsp moves.

my_func:
    push  rbp               ; 1. save the caller's frame pointer
    mov   rbp, rsp          ; 2. rbp now marks the base of our frame
    sub   rsp, 32           ; 3. reserve 32 bytes for local variables
                            ;    (keep rsp 16-byte aligned for calls)
    ; --- body ---
    mov   qword [rbp-8], 0  ; local variable at rbp-8
    mov   qword [rbp-16], 1 ; local variable at rbp-16

    ; ... use rax as the result ...

    leave                   ; = mov rsp, rbp ; pop rbp   (the epilogue)
    ret

After that prologue the frame looks like this — and the layout is identical for every conforming function, which is why debuggers can walk a stack trace at all:

The Epilogue

leave is shorthand for mov rsp, rbp followed by pop rbp — it discards all locals in one step and restores the caller's frame pointer. The ret that follows pops the return address into rip. Together they undo the prologue exactly, which is the property that makes balanced frames work.

Arguments and Return Values

The ABI is not arbitrary — it is a promise that lets code compiled by different compilers call one another. For integers and pointers the first six arguments travel in registers:

Before a call, rsp must be 16-byte aligned. Because call pushes 8 bytes, the convention is that at the moment control enters a function, rsp + 8 is a multiple of 16. If you call a library function such as printf with a misaligned stack, it may crash inside an SSE instruction — a confusing symptom for a simple mistake.

Recursion

Recursion works in assembly exactly as it does anywhere else, provided each call has its own frame. Because arguments live in registers — which one call would clobber — the recursive case must save them on the stack before calling itself.

; long fact(long n)  — recursive factorial
; n arrives in rdi, the result returns in rax
fact:
    cmp  rdi, 1
    jle  .base              ; n <= 1 -> return 1

    push rdi                ; SAVE n: the recursive call will destroy rdi
    dec  rdi                ; n - 1
    call fact               ; rax = fact(n - 1)

    pop  rdi                ; RESTORE n (stack is LIFO: same order reversed)
    imul rax, rdi           ; rax = fact(n - 1) * n
    ret

.base:
    mov  rax, 1             ; 0! = 1! = 1
    ret

The push rdi / pop rdi pair is the whole trick. Without it, the recursive call would overwrite rdi with its argument, and the multiplication would use a wrong value. Every recursive assembly routine has this shape: establish the base case, save what the recursive call will destroy, call, then combine.

Compilers often convert recursion to iteration to avoid the stack cost. If you write recursion by hand, remember each level consumes a frame — deep recursion overflows the stack long before it runs out of memory elsewhere.

Calling Assembly from C

The most practical use of assembly in real projects is a single hand-optimized function inside a C program. The contract is the same System V ABI you have been following all along.

; sum_array.asm — long sum_array(const long *p, long n)
; Build into an object file and link with the C driver:
;   nasm -f elf64 sum_array.asm -o sum_array.o
;   gcc -no-pie main.c sum_array.o -o demo

section .text
    global sum_array            ; export it so the linker can find it

sum_array:
    xor  rax, rax               ; rax = running total = 0
    xor  rcx, rcx               ; rcx = index i = 0

.loop:
    cmp  rcx, rsi               ; i < n ?   (rsi holds the second argument)
    jge  .done
    add  rax, [rdi + rcx*8]     ; total += p[i]   (rdi holds the pointer)
    inc  rcx
    jmp  .loop

.done:
    ret                         ; result already in rax — nothing else to do

Three things make this link cleanly: the symbol is global, the arguments come from rdi/rsi as the ABI specifies, and the result is left in rax. On the C side it is an ordinary function declaration:

/* main.c */
#include <stdio.h>

long sum_array(const long *p, long n);   /* implemented in assembly */

int main(void) {
    long values[] = {1, 2, 3, 4, 5};
    printf("sum = %ld\n", sum_array(values, 5));
    return 0;
}

Summary

• call pushes the return address and jumps; ret pops it back into rip.
• A stack frame is a prologue (push rbp; mov rbp, rsp), locals below rbp, and an epilogue (leave/pop rbp + ret).
• Arguments go in rdi, rsi, rdx, rcx, r8, r9; the seventh and later go on the stack.
• The return value is in rax; rdx carries the high half of a 128-bit result.
• Recursion needs the argument saved (push/pop) around each recursive call — registers are not preserved for you.

Next: Macros & Directives — the assembler's own programming system.