GCC MIPS compiler patterns

When reverse engineering assembly you'll see a lot of appearingly strange instructions that doesn't really make sense. Mostly these are the results of compiler optimizations.

Because the compiler generates these optimizations, I'm looking at the inner works of the gcc mips compiler to understand these optimizations and see the patterns it creates.

The md files contains the machine description. https://github.com/gcc-mirror/gcc/blob/master/gcc/config/mips/

The compiler will call the function output_asm_insn to output assembler instructions.

The function prologue for MIPS16 is created at function mips_output_function_prologue using:

/* This is a fixed-form sequence.  The position of the
first two instructions is important because of the
way _gp_disp is defined.  */
output_asm_insn ("li\t$2,%%hi(_gp_disp)", 0);
output_asm_insn ("addiu\t$3,$pc,%%lo(_gp_disp)", 0);
output_asm_insn ("sll\t$2,16", 0);
output_asm_insn ("addu\t$2,$3", 0);

This is nice, now when we're seeing this pattern, we know that this will be a MIPS16 function.

Other interesting pattern is the mips_output_probe_stack_range function. As per documentation:

/* Probe a range of stack addresses from REG1 to REG2 inclusive.  These are absolute addresses.  */

The resulting code for this function is:

	/* Jump to END_LAB if TEST_ADDR == LAST_ADDR.  */
	beq 		%0, %1
	addiu		%0,%0,%1		(64 bit: daddiu		%0,%0,%1)

	/* Probe at TEST_ADDR and branch.  */
	b			loop_lab
	sw			$0,0(%0)		(64 bit: sd	$0,0(%0))

The divide by zero check looks like this ( mips_output_division):

/* Return the assembly code for DIV or DDIV instruction DIVISION, which has the operands given by OPERANDS.  Add in a divide-by-zero check if needed.*/

for mips16:

div			%1,%2
bnez		%2,1
   break	7				

If %2 is zero then it will break 7, which is the divide by zero break.