ABI Programming with Assembly

What an ABI Is

An API is a source-level contract (function names and signatures). An ABI is the binary-level contract: byte-for-byte, register-for-register. It fixes the sizes and layout of types, the endianness of multi-byte integers (seen on the advanced page), struct padding rules, and — most visibly — the calling convention: which registers carry arguments, which are call-clobbered, how the stack is aligned, and how symbols are named.

C has a unique property in this story: it does no name mangling. The symbol exported for int add(int, int) is literally add. C++ adds type information to its symbol names (mangling); C's plain names are why the C ABI became the universal glue — every language, from Python to Rust, exposes its libraries through a C-compatible ABI.

Calling Conventions

The convention your compiler follows depends on the operating system and architecture. The three you will meet are listed below — note that the same C source compiles against all of them; the convention is decided by the platform, not by you (unless you write assembly).

Floating-point arguments travel separately in vector registers (XMM0–XMM7 in System V, XMM0–XMM3 on Windows), and a returned value lands in RAX (or the XMM0 return register for floats). Two subtle rules matter in practice: which registers the callee must preserve, and stack alignment at the call boundary — violating alignment crashes vectorized library code that assumes it.

Register Roles on x86-64

The System V convention partitions the general registers into caller-saved (a function may overwrite them freely) and callee-saved (a function must preserve them for its caller — practically, save/restore them in its own stack frame). RAX carries the return value; RIP is the instruction pointer; RSP the stack pointer. Learning these six argument slots turns any disassembly of a C function into a readable story.

Figure 1 — System V AMD64: args 1–6 arrive in RDI/RSI/RDX/RCX/R8/R9, extras go on the stack, the result returns in RAX.

Inline Assembly in C

GCC and Clang let you drop raw instructions into a C function — the extended asm form. The syntax is a GNU extension (not ISO C), so it is inherently non-portable; keep it tiny and explain why. The template lists the instructions; the three sections after the colon bind C operands to registers: outputs, inputs, and clobbers (registers or memory the instructions destroy, so the compiler can save them).

#include <stdint.h>
#include <stdio.h>

// read the CPU timestamp counter — the highest-resolution clock on x86-64
static inline uint64_t rdtsc(void) {
    uint32_t lo, hi;
    // "rdtsc" loads EDX:EAX with the counter; bind them to the outputs
    __asm__ __volatile__("rdtsc" : "=a"(lo), "=d"(hi));
    return ((uint64_t)hi << 32) | lo;   // pack the two 32-bit halves
}

int main(void) {
    uint64_t t0 = rdtsc();
    uint64_t t1 = rdtsc();
    printf("elapsed ticks: %llu\n", (unsigned long long)(t1 - t0));
    return 0;
}

Read the constraints left to right: "=a"(lo) means "put the output in register RAX and store it into lo"; "=d"(hi) does the same for EDX; __volatile__ tells the compiler not to move, deduplicate, or delete the block.

AT&T vs Intel Syntax

GNU/GAS assembly writes operands in AT&T order: source first, destination second (mov %rax, %rbx copies into RBX), registers with a % prefix, immediates with a $. Intel syntax — NASM, MASM, and gcc -masm=intel — writes destination first (mov rbx, rax) with no prefixes. Both describe the same machine code. Most inline asm you meet in C libraries is AT&T, because GCC's default assembler is GAS.

Calling Between C and Assembly

Because C names are unmangled, linking a hand-written assembly function into a C program is just a label, an extern declaration, and the same gcc link step you already use. The assembly function must honor the ABI of its platform — below, System V: first argument arrives in RDI, the result must land in EAX.

# add.s — int add(int a, int b)  (System V AMD64, GAS/AT&T syntax)
        .globl add          # export "add" so the linker can see it
add:                        # a = edi, b = esi ; result goes into eax
        leal (%rdi, %rsi), %eax   # eax = a + b in one instruction
        ret                       # return to the C caller

#include <stdio.h>

extern int add(int a, int b);   // declared, defined in add.s

int main(void) {
    printf("2 + 3 = %d\n", add(2, 3));   // 5 — the asm function does the work
    return 0;
}

Build both in one command and the linker resolves add:

gcc -std=c11 -Wall -Wextra -Werror main.c add.s -o app
./app

Two portability traps appear immediately: macOS prefixes every C symbol with an underscore in its object files (your assembler label must be _add), and 32-bit x86 expects all arguments on the stack — memory layout and register choices are always per-platform. When in doubt, disassemble the C compiler's own output (gcc -S or Compiler Explorer) and imitate its prologue and epilogue exactly.

Which Assembly Can Be Used

The phrase "assembly language" hides several dialects, assemblers, and ISAs. Pick the pair that matches your compiler and target:

Target ISAs matter just as much: x86-64 (desktop/server) and ARM64 (phones, Apple Silicon, most Linux boards) dominate today, with RISC-V rising. Assembly for one ISA is not portable to another — ever. That is the entire reason the assembly roadmap is a separate journey, and why you treat assembly as a carefully isolated layer inside a C program.

When to Reach for Assembly

Assembly is the last tool, not the first. Use it — and the ABI knowledge behind it — when nothing else reaches:

• Hardware access — privileged instructions, model-specific registers, fences and cache control that C cannot express.
• Precise timing — rdtsc-style counters for benchmarking (seen above).
• Bootloaders & kernels — the first code on a CPU exists before a C runtime.
• Hot-loop maximums — only after profiling proves the compiler left performance on the table.
• Learning — reading disassembly is the deepest debugging and performance tool a C engineer has.

Do not write assembly for portability, readability, or features — C and its ABI give you those for free. When you do drop down, wrap every asm block in one function, comment the ABI assumptions, and keep the C calling side boring.

That closes the lessons. Move to the lab examples to compile and break these patterns, then the study projects, and finish with references.