Reversing the exit(173) from the Mac App Store

This will be a working in progress so this post might be updated a few times.

As promised, a reader sent me the Mac App Store (MAS) validation guidelines (thank you again!) and I got curious about one detail, the exit(173). This guides states if application fails to validate the receipt because it’s not present, then it should exit with status 173. This status will be interpreted by the system and it will try to obtain a valid receipt – this is the reason why you see that message asking for Sign in when the receipt isn’t valid and you can see the email address of the guy who released that app to the wild.

So I got interested in trying to understand how is this done and who is responsible for interpreting that status. I started by diff’ing the 10.6.6 and 10.6.5 kernels but the changes are minimal and don’t appear to be the source of this. I also messed around (lightly) libSystem.B.dylib and couldn’t find any changes around exit() implementation. Weird…

Today I got back to this after reading a post about store_helper being the responsible daemon for that message. It’s a small program and you can see right away CFMessagePort code. Store_helper opens a message port to receive messages and they are process by a callout (at 0x209E). This helper is also using Grand Central Dispatch technology to take advantage of multicore cpus (you can see the callout submitting the message to the queue and the queue is created before the message port is opened). The main “helper” program is storeagent. This one also opens a message port and also uses GCD.

My theory is that storeagent receives the status from exit(173) and then dispatches a message to store_helper. If you rename storeagent, no sign in message will appear. I couldn’t yet intercept storeagent sending messages to store_helper so I still have to verify this.

I was trying to find who is sending the message to storeagent. My idea is to trace exit(173) from the protected app and have a breakpoint on storeagent where it receives the messages. But I’m facing a weird problem. The MAS apps do not run from the command line if you execute the main binary only. The only way is to issue “open”. If I call the main binary directly, the sign in message box never appears – I can verify this because store_helper never receives a message. Storeagent (process dies after a few seconds if not used) is also never executed/called. I will have to understand why this is happening – any ideas ? 🙂 The exit code is correct, 173, 0255 in octal.

Also, does anyone have/knows a quick dtrace script or something else to trace the messages? I’m interested in finding the origin of each message. I really need to close that item in my todo list that says “Dtrace”!

That’s it for now. Tomorrow is time to get back to this!


3 thoughts on “Reversing the exit(173) from the Mac App Store

  1. if i recall storeagent has a call to ptrace that i patched out to try and run it, it also looks like they read a key called DebugLevel from its plist

  2. It is launchd that is immediately responsible for capturing the exit status of MAS apps, and this is why store_helper and storeagent are launched when you use ‘open’, but not when calling the binary directly.

    There appears to be a private launchd API (see launch_priv.h in launchd source) ‘spawn_via_launchd’ that allows the calling process to request launchd to spawn a process. This API allows feedback about the created process via an ‘observer port’. Looking at the diff of launchd between 10.6.5 and 10.6.6, you can see Apple fixed a bug in their code to make sure that the right exit status is captured by launchd and then forwarded to the original process via that observer port.

    What appears to happen is that when an application is launched via LaunchServices, LaunchApplicationViaLaunchD calls ‘spawn_via_launchd’, and passes the observer port along via NotifyServerSideAboutLaunchedApplication. I think this then passes the observer port along to coreservicesd. I’m guessing coreservicesd is the one that checks the exit code, but I can’t be sure.

    1. Excellent thoughts and tips! After writing the post I saw that those two helpers were under launchd – the ondemand and a single run in CFRun made all sense now.
      I was about to research how launchd detected the different methods to run the app (I must confess I don’t know these Apple technologies that much so it’s always fun to learn!).

      Thanks for the tips, they will be valuable and cut some of my effort to understand the whole thing! Great 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *