Reversing the exit(173) from the Mac App Store

This will be a working in progress so this post might be updated a few times.

As promised, a reader sent me the Mac App Store (MAS) validation guidelines (thank you again!) and I got curious about one detail, the exit(173). This guides states if application fails to validate the receipt because it’s not present, then it should exit with status 173. This status will be interpreted by the system and it will try to obtain a valid receipt – this is the reason why you see that message asking for Sign in when the receipt isn’t valid and you can see the email address of the guy who released that app to the wild.

So I got interested in trying to understand how is this done and who is responsible for interpreting that status. I started by diff’ing the 10.6.6 and 10.6.5 kernels but the changes are minimal and don’t appear to be the source of this. I also messed around (lightly) libSystem.B.dylib and couldn’t find any changes around exit() implementation. Weird…

Today I got back to this after reading a post about store_helper being the responsible daemon for that message. It’s a small program and you can see right away CFMessagePort code. store_helper opens a message port to receive messages and they are process by a callout (at 0x209E). This helper is also using Grand Central Dispatch technology to take advantage of multicore CPUs (you can see the callout submitting the message to the queue and the queue is created before the message port is opened). The main “helper” program is storeagent. This one also opens a message port and also uses GCD.

My theory is that storeagent receives the status from exit(173) and then dispatches a message to store_helper. If you rename storeagent, no sign in message will appear. I couldn’t yet intercept storeagent sending messages to store_helper so I still have to verify this.

I was trying to find who is sending the message to storeagent. My idea is to trace exit(173) from the protected app and have a breakpoint on storeagent where it receives the messages. But I’m facing a weird problem. The MAS apps do not run from the command line if you execute the main binary only. The only way is to issue open If I call the main binary directly, the sign in message box never appears – I can verify this because store_helper never receives a message. Storeagent (process dies after a few seconds if not used) is also never executed/called. I will have to understand why this is happening – any ideas? The exit code is correct, 173, 0255 in octal.

Also, does anyone have/knows a quick Dtrace script or something else to trace the messages? I’m interested in finding the origin of each message. I really need to close that item in my todo list that says DTrace.

That’s it for now. Tomorrow is time to get back to this!