Everything at Once

Every lesson in this course has isolated one idea at a time: one lesson for argument registers, one for flags and jumps, one for addressing modes, one for inlining. Real listings are not so polite. When you open the assembly of production code, everything arrives at once: a guard here, a loop there, an inlined helper hiding in the middle, and no headings telling you which is which.

This capstone is the test of everything you have learned. We will take one realistic function, compile it at -O2, and read the entire listing. Along the way you will meet argument registers, a guard clause, a loop with an early exit, pointer arithmetic, an inlined call, lea doing maths, and the return value, all in the same fifteen instructions. Just as importantly, you will learn the order in which an experienced reader attacks a listing, because it is not top to bottom.

The C++

Here is the function, along with the small helper it uses. It computes a weighted sum of readings, stopping at the first reading that reaches a limit:

int weightOf(int x)
{
    return x * 2 + 1;
}

int sumUntilLimit(const int* values, int count, int limit)
{
    int total{0};
    for (int i{0}; i < count; ++i)
    {
        if (values[i] >= limit)
        {
            break;
        }
        total += weightOf(values[i]);
    }
    return total;
}

Nothing exotic: a pointer and a count, a loop, a conditional break, a helper call, a running total. This is the shape of thousands of real functions.

The Full Listing

At -O2, GCC produces this for sumUntilLimit (the demangler writes the parameter as int const*, which is the same type as const int*):

sumUntilLimit(int const*, int, int):
	test	esi, esi
	jle	.L4
	movsx	rsi, esi
	lea	rcx, [rdi+rsi*4]
	xor	eax, eax
.L3:
	mov	esi, DWORD PTR [rdi]
	cmp	esi, edx
	jge	.L1
	lea	eax, 1[rax+rsi*2]
	add	rdi, 4
	cmp	rcx, rdi
	jne	.L3
.L1:
	ret
.L4:
	xor	eax, eax
	ret

Fifteen instructions, three labels, no stack frame, and, notably, no call. Resist the urge to decode it line by line just yet. First, make a map.

First Pass: Sketch the Map

The single most useful habit for reading an unknown listing is to find the labels and jumps before reading anything else. They are the skeleton; everything else hangs off them.

This listing has three labels (.L3, .L1, .L4) and three conditional jumps:

  • jle .L4, near the top, jumps forward past everything to a tiny block that zeroes eax and returns. A forward jump taken before any real work is a guard: an early out for a degenerate case.
  • jge .L1, in the middle, jumps forward to a bare ret. A forward jump out of the middle of a block is an exit, and since it sits between .L3 and the jump back to .L3, it is an exit from inside a loop.
  • jne .L3, at the bottom, jumps backward. A backward jump is the signature of a loop: it is the back-edge, and its target .L3 marks the top of the loop body.

Without decoding a single operand, the shape is already clear: guard, some setup, a loop with an early exit, two ways to return. That matches the C++ exactly. Now we can fill in the details, stage by stage.

Stage 1: The Signature and the Guard

The signature is int(const int*, int, int), so the System V convention assigns rdi = values, esi = count, edx = limit, and the int return travels back in eax. Pin those down before anything else; every operand in the listing now has a chance of meaning something.

	test	esi, esi
	jle	.L4

test esi, esi ANDs count with itself purely to set flags: the idiomatic "compare with zero" you met in Chapter 3. jle is a signed jump, taken when count <= 0. In that case .L4 runs xor eax, eax (return value zero) and returns. This is the for loop's condition i < count failing on the very first check, hoisted out and turned into a guard so the loop itself never has to consider an empty array.

Stage 2: The Loop Setup

	movsx	rsi, esi
	lea	rcx, [rdi+rsi*4]
	xor	eax, eax

movsx rsi, esi sign-extends the 32-bit count to 64 bits, because it is about to take part in address arithmetic, and addresses are 64-bit. Then lea rcx, [rdi+rsi*4] computes values + count * 4: the address one past the last element. rcx is the end pointer.

Notice what does not exist: the variable i. The optimizer replaced the counter with a moving pointer. Instead of "increment i, compute values + i*4, compare i with count", the loop will simply walk rdi forward 4 bytes at a time and stop when it reaches rcx. Same trips, fewer instructions per trip.

Finally xor eax, eax is total{0}, placed directly in the return register so no move is needed at the end.

Stage 3: The Body

.L3:
	mov	esi, DWORD PTR [rdi]
	cmp	esi, edx
	jge	.L1

The load fetches the current element: DWORD PTR [rdi] is a 4-byte read through the walking pointer, i.e. values[i]. Note the register recycling: esi used to hold count, but count's only remaining job was the end-pointer calculation, so the compiler reuses esi for the element. A register's meaning is only valid between writes; always be ready for it to change jobs.

cmp esi, edx / jge .L1 is the break: if the element is greater than or equal to limit, jump out of the loop. Crucially, eax already holds the running total, so the exit path needs no cleanup; it lands straight on ret.

	lea	eax, 1[rax+rsi*2]

Here is the line to savour, because of what it is and what it is not. There is no call weightOf(int) anywhere in this function. The helper was inlined, and its body, x * 2 + 1, fused with total += into a single instruction: lea computes rax + rsi*2 + 1, that is, total + element * 2 + 1, using the address-calculation hardware to do three-term arithmetic in one step. A function call, a multiply, and two additions, all inside one lea.

	add	rdi, 4
	cmp	rcx, rdi
	jne	.L3

add rdi, 4 steps the pointer to the next int. cmp rcx, rdi / jne .L3 asks "have we reached the end pointer?" and takes the back-edge if not. This trio is the entire loop overhead: one add, one compare, one jump.

Stage 4: The Returns

.L1:
	ret
.L4:
	xor	eax, eax
	ret

Both the early break (jge .L1) and normal loop completion (falling through the untaken jne) arrive at .L1 with the answer already in eax. The .L4 block handles the empty case with an explicit zero. Two exits in the C++, two rets in the assembly.

One more observation ties this to the previous lesson: there is not an xmm register in sight. A plain "sum everything" loop would likely be vectorised at -O2, but the data-dependent break means the compiler cannot know in advance how many iterations will run, so the loop stays scalar. The absence of vector code is itself information.

The helper, by the way, still exists as a standalone function for any other caller, exactly as the inlining lesson predicted:

weightOf(int):
	lea	eax, 1[rdi+rdi]
	ret

How to Approach Any Unknown Listing

The method we just used works on any function. Distilled into a checklist:

  1. Start from the signature. Assign the argument registers (rdi, rsi, rdx, rcx, r8, r9 for integers and pointers; xmm0 onward for floats) and remember eax/rax (or xmm0) is the return.
  2. Map the labels and jumps. Backward jump means loop; forward jumps mean guards, if/else, or early exits. Sketch the flow graph before decoding operands.
  3. Find the loop's moving parts. Look at what the cmp before the back-edge tests and what changes each iteration: that is your counter or walking pointer.
  4. Decode addressing modes. [reg] is a dereference, [base+index*4] is an int array, disp[reg] is a struct field or stack slot. The shape of the operand tells you the shape of the data.
  5. Follow the return register. Trace how eax is built up; that is the function's answer taking shape.
  6. Read what is missing. No call means inlining; no stack frame means everything fit in registers; code you expected but cannot find means folding or dead code elimination; no xmm loop means vectorisation was not possible or not worthwhile.
  7. Only then go line by line, with the map in hand, and every instruction will land somewhere you already understand.

Key Takeaways

  • Read structure first: find labels and jumps and sketch the flow graph before decoding any operands.
  • A backward jump is a loop back-edge; forward jumps are guards, branches, or early exits.
  • Pin down the argument registers from the signature (rdi, esi, edx here) and track the return value in eax.
  • The optimizer replaced the index i with a walking pointer plus end pointer (add rdi, 4 / cmp rcx, rdi), and recycled esi once count was no longer needed.
  • The helper call vanished: inlining fused weightOf and total += into a single lea eax, 1[rax+rsi*2].
  • Absences carry meaning: no call means inlining, no stack frame means register-only, no xmm here because the data-dependent break prevents vectorisation.
  • The checklist, signature, flow graph, loop parts, addressing modes, return register, missing pieces, then line by line, works on any listing you will ever open.