Today I decided to give a look at Challenge #3 since it promised nasty tricks. Now that looks like a challenge and I love challenges! If you think this is a spoiler then stop reading and come back in a week or so. There is no solution for the challenge; I’m more interested in the “nasty” trick used and why the tools are failing. And I don’t need the Challenge itself to analyse this behavior since I can reproduce it with own code. I have other things to do, so I need to get this out of my mind and maybe someone else can contribute to this analysis.

Let the games begin!

As usual, I started to analyse this challenge by trying to disassemble it with otx. The result was empty, as if __text section can’t be read. Hummm that’s weird. GDB is the next victim… Launch GDB, select binary, run… BANG! GDB not able to run the program. This is getting interesting! So let’s read the Mach-O headers to understand what’s happening here.

$ otool -l Cactus
Load command 0
      cmd LC_SEGMENT
  cmdsize 56
  segname __TEXT
   vmaddr 0x0000b000
   vmsize 0x0000468f
  fileoff 0
 filesize 15789
  maxprot 0x00000007
 initprot 0x00000007
   nsects 0
    flags 0x0
Load command 1
        cmd LC_UNIXTHREAD
    cmdsize 80
     flavor i386_THREAD_STATE
      count i386_THREAD_STATE_COUNT
	    eax 0x00000000 ebx    0x00000000 ecx 0x00000000 edx 0x00000000
	    edi 0x00000000 esi    0x00000000 ebp 0x00000000 esp 0x00000000
	    ss  0x00000000 eflags 0x00000000 eip 0x0000e79c cs  0x00000000
	    ds  0x00000000 es     0x00000000 fs  0x00000000 gs  0x00000000

The binary has only 2 commands and there is no __text section. This is not an usual binary. From other projects I know this is possible and there are at least two articles about this. After searching my bookmarks I found Amit Singh’s post about this here (have I told you that his book rocks? buy it!). The posted code for tiny.asm doesn’t work in Snow Leopard (I remember it works in Leopard), so here is the working version (we just need to specify nsects and flags – the kernel parser must have been updated in Snow Leopard).

; tiny.asm for Mac OS X (Mach-O Object File Format)
; nasm -f bin -o tiny tiny.asm

        org   0x1000

        db    0xce, 0xfa, 0xed, 0xfe       ; magic
        dd    7                            ; cputype (CPU_TYPE_X86)
        dd    3                            ; cpusubtype (CPU_SUBTYPE_I386_ALL)
        dd    2                            ; filetype (MH_EXECUTE)
        dd    2                            ; ncmds
        dd    _start - _cmds               ; cmdsize
        dd    0                            ; flags
        dd    1                            ; cmd (LC_SEGMENT)
        dd    56                           ; cmdsize
        db    "__TEXT"                     ; segname
        db    0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ; segname
        dd    0x1000                       ; vmaddr
        dd    0x1000                       ; vmsize
        dd    0                            ; fileoff
        dd    filesize                     ; filesize
        dd    7                            ; maxprot
        dd    5                            ; initprot
        dd    0                            ; nsects
        dd    0                            ; flags

        dd    5                            ; cmd (LC_UNIXTHREAD)
        dd    80                           ; cmdsize
        dd    1                            ; flvaor (i386_THREAD_STATE)
        dd    16                           ; count (i386_THREAD_STATE_COUNT)
        dd    0, 0, 0, 0, 0, 0, 0, 0       ; state
        dd    0, 0, _start, 0, 0, 0, 0, 0  ; state
        xor   eax,eax
        inc   eax
        push  byte 42
        sub   esp, 4
        int   0x80                         ; _exit(42)

filesize equ  $ - $$

otool isn’t able to disassemble this binary (otx uses otool) but this time GDB is able to run. The difference is that breakpoints aren’t enforced! IDA is able to disassemble the binary directly (else we could point it to the right place taking the EIP as our starting point).

Since I had an idea why otool was failing, I quickly checked its source code to be sure that was the problem (otool is part of cctools package available at Apple Developer Program, if you are interested in checking it). The following can be found at otool/main.c:

else if(cputype == CPU_TYPE_I386)
  j = i386_disassemble(sect, size - i, cur_addr, addr,
              object_byte_sex, relocs, nrelocs, symbols, NULL,
              nsymbols, sorted_symbols, nsorted_symbols,
              strings, strings_size, indirect_symbols,
              nindirect_symbols, cputype, load_commands,
              ncmds, sizeofcmds, verbose);

The function i386_disassemble is located in otool/i386_disasm.c. Only sections are disassembled and since there is none in the binary then otool can’t disassemble it. Proof is easy since that’s the difference between tiny.asm and nicertiny.asm. I further tested this by manually adding that section to the binary; otool was able to disassemble it without problems (although a 100% correct disassembly is another story). I presume that GDB has the same problem since I think there was/is a common code base for the disassembler and other parts (or similar, cannot recall by memory since last time I messed with GDB source code was one year ago).

I tried some experiments with GDB and before writing this I had the idea that it could breakpoint after I added the __text section. I can’t reproduce it again so I must have confused with other test binaries I had. I don’t know why GDB isn’t able to breakpoint these binaries (with or without __text section). I suspect it might have to do with alignment or something. For example, I have tried to compile GNU GDB latest 7.1 version and it can’t even parse these binaries. It crashes like this:

gdb$ file ~/Projects/macserialjunkiechallenge10/Challenge3/tiny

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000010
0x00000001001e10c8 in bfd_mach_o_read_symtab_symbols (abfd=0x1007c2f40) at mach-o.c:1835
1835	  if (sym->symbols)

I have tried to pack some test binaries with UPX and same thing happens (you don’t need to be Sherlock Holmes to see that Challenge #3 is UPX packed). GDB isn’t able to execute the breakpoints, either software or hardware. A quick Google search hasn’t returned any valid information about this.

While this isn’t a big problem for reversing Challenge #3, this could be an issue with more advanced and complex packers. We can attach GDB to Challenge #3 without any problem after the unpacking and dump whatever we need (hint: there’s an even easier solution for this case) – don’t forget that vmmap is also messed up in Snow Leopard. But we could miss important stuff in other packers and this could be a good anti-debug trick (maybe IDA debugger is able to do it, I don’t use it). otool gets confused while disassembling some instructions from Challenge #3 so there is also something else playing in that binary (that could explain why the binary doesn’t even run under GDB).

Now, who can give some hints about this? I’m very much curious about it but I don’t want to spend more time due to other matters that call my attention – GDB source is a pain to read and navigate thru.

Have fun,

Update 1:
psaghelyi left a link in another post to his tool, MachOView. For now it’s a graphical otool -l, that allows you to explore the Mach-O header of binaries. It has potential to be a nice tool as soon as he adds editor capabilities. Help him and contribute to the code.
We need such a tool! Mac is sexy so we need sexy graphical tools!