Tag: Objective-C

  • When I was building Dashman, I hit a wall that Apple’s paid developer support told me was unsolvable. Then I solved it by reading the binaries in assembly until I understood them well enough to swap them out.

    A note up front: this happened a long time ago and I’m writing it from memory. The shape of the work is right; small specifics, API names, and exact sequence of events may be slightly off.

    What was Dashman?

    Dashman was a B2B SaaS for displaying business dashboards (which are, technically, just web pages) on the screens around an office. An early version of it was a native macOS app with an embedded browser. That’s the version this post is about.

    The problem

    On macOS, an embedded browser meant Apple’s WKWebView, which shared a single, system-wide cookie jar with Safari.

    That was a non-starter. Dashman needed to be logged into the dashboards it displayed, typically with accounts different from the user’s: Google Analytics, AWS consoles, internal admin tools. Mixing Dashman’s and Safari’s cookies would have caused constant overwrites in both directions. The longer-term goal was stricter still: one cookie jar per dashboard, so the same install could display two accounts of the same web application side by side.

    Long story short, I needed full control of the cookie jar. Other forms of local storage existed by then but were still nascent; the cookies were where the leakage actually happened.

    I paid Apple’s developer support fee and asked. The reply was friendly, brief, and unambiguous:

    After investigating whether different WKWebViews can each have their own cookie jar the answer is no they cannot — they all share a single container. If you feel that this is something that you need for your application, please file an enhancement request here:

    Challenge accepted, Mr Apple.

    Reading the cookie jar in assembly

    The macOS cookie jar is NSHTTPCookieStorage, a Foundation class with a small public API: get cookies, set cookies, delete by URL, the usual. Underneath, it talks to CFNetwork, the C framework Foundation wraps. The implementation I’d need to replace lived in a binary I didn’t have source for.

    So I opened CFNetwork.framework in Hopper Disassembler and started reading.

    Disassembled CFNetwork in Hopper, showing the implementation of -[NSHTTPCookieStorageInternal initInternalWithCFStorage:] and the symbol list of every NSHTTPCookieStorage and NSHTTPCookieStorageInternal method.

    I sometimes wonder what this would look like on Apple Silicon today: ARM instead of x86, Swift sneaking into Apple’s frameworks. Different bytes, presumably the same archaeology.

    What the disassembly revealed was a three-layer onion. The public class, NSHTTPCookieStorage, was a thin façade. Most of its methods forwarded to an internal Objective-C class, NSHTTPCookieStorageInternal, that held the actual state behind an NSRecursiveLock. That class in turn called down into a family of plain C functions named _CFHTTPCookieStorageCreateDefault, _CFHTTPCookieStorageGetClass, _CFHTTPCookieStorageCreateInMemory, and friends. Foundation wrapper on top, private Objective-C in the middle, C core at the bottom.

    Three-layer diagram of Apple's cookie storage stack, with the public Foundation façade above the private internal class and the CFNetwork store shared with Safari.

    The disassembly told me five things at once. Which selectors did real work. Which were forwarded. Which were vestigial. Where cookies actually got persisted. And, as a bonus, the error strings the binary fell back to when something went wrong: literal byte sequences like Cannot get default cookie store - using a memory store for this process were sitting right there in the data segment, telling me which failure modes Apple’s engineers had anticipated and which they hadn’t.

    It was, honestly, fun. Hopper’s cross-references made it possible to climb up from any C function to every Objective-C selector that called it, and from any selector back down to the persistence calls. After a while I had a mental map of which methods I’d need to satisfy and which I could safely ignore.

    There’s no symbols in the source, which made this hard. Method names survive (Objective-C runtime needs them) but everything else is x86 instructions. What looks in the source like if (self.policy == NSHTTPCookieAcceptPolicyAlways) shows up as a cmp against an immediate, with a jne to a label that another four objc_msgSend calls eventually reveal as the rejection branch. You build the abstraction back up by hand.

    The replacement: a private cookie jar

    What I built was a single Objective-C class, CAHTTPCookieStorage, that implemented the subset of NSHTTPCookieStorage‘s interface that real callers actually used. Cookies live in a dictionary keyed by a composite of (domain, path, name), all lowercased, so duplicates collapse on insert:

    - (NSUInteger)hash {
    return self.domain.hash ^ self.path.hash ^ self.name.hash;
    }
    - (BOOL)isEqual:(id)other {
    if (![other isKindOfClass:[CookieKey class]]) {
    return NO;
    }
    CookieKey *otherCookieKey = (CookieKey *)other;
    return [self.domain isEqual: otherCookieKey.domain]
    && [self.path isEqual: otherCookieKey.path]
    && [self.name isEqual: otherCookieKey.name];
    }

    The lowercasing matters more than it looks. The HTTP cookie spec is case-insensitive on domains and paths in some places and case-sensitive in others, and NSHTTPCookie itself doesn’t normalise consistently. Without the lowercase hash, you can end up with two distinct entries for www.example.org and WWW.EXAMPLE.ORG and behaviour that diverges from Safari’s depending on which one you set last. The test suite (next section) caught this kind of thing immediately.

    Persistence is NSKeyedArchiver to a path under ~/Library/Application Support/<bundle>/Cookies, with the directory created lazily on first write. If the file is corrupted on read we surface it via a NSLog and start with an empty jar rather than crashing the host process. That seems obvious in retrospect; it wasn’t obvious the first time the archive deserialiser threw an exception three frames deep into application launch.

    Domain matching is the one piece that always trips people up. Cookies with a leading dot match subdomains; without a leading dot, they only match exactly. The whole rule collapses to a single ternary once you accept that:

    + (BOOL)match:(NSURL *)url toDomain:(NSString *)domain {
    NSString *host = [[url host] lowercaseString];
    if (host != nil) {
    return [domain hasPrefix:@"."]
    ? ([[@"." stringByAppendingString: host] hasSuffix: domain])
    : [host isEqualToString:domain];
    } else {
    return false;
    }
    }

    Expiry I made lazy. Every call to cookies walks the dictionary and evicts anything past its expiry. Cheaper than running a timer, and it always returns a clean view:

    - (NSArray *)cookies {
    NSMutableArray *cookiesToRemove = [NSMutableArray array];
    for (CookieKey *key in _cookies) {
    NSHTTPCookie *cookie = [_cookies objectForKey: key];
    if ([cookie expiresDate] != nil
    && [[cookie expiresDate] isLessThan:[NSDate date]]) {
    [cookiesToRemove addObject: key];
    }
    }
    for (CookieKey *key in cookiesToRemove) {
    [_cookies removeObjectForKey: key];
    }
    return [_cookies allValues];
    }

    About 300 lines of core logic across the storage class, the cookie key, and the swizzle category. Most of it is RFC 6265 plumbing.

    A test suite for someone else’s code

    Once I had a working mental model, I wrote a test suite. Not against my own code: against Apple’s. The suite exercised every observable behaviour I could find. Cookie expiry, including Max-Age versus Expires. Domain matching, including the leading-dot wildcard rules and the prefix trap (prefixexample.com must not match a cookie set on example.com). Path prefix matching. Secure flag handling. Set-Cookie parsing. The order in which cookies came back. Case sensitivity quirks across domain, path, and name. Cookie acceptance policies (Always, Never). Round-tripping through archive and unarchive. I deliberately included behaviours I suspected were abandoned, on the theory that if they still existed in the binary, something somewhere depended on them.

    The trick that paid off: every test was written so it could run against either the real NSHTTPCookieStorage or my CAHTTPCookieStorage. Same assertions, swap the class name at the top of the file, run again. When something passed Apple’s run and failed mine, I had a divergence to chase. When it passed both, I knew I’d matched behaviour.

    There was a charming downside, immortalised in a comment at the top of the test file:

    Be aware though that these tests are cookie destructive. If you run them against NSHTTPCookieStorage it will destroy all your Safari cookies and if you run them against CAHTTPCookieStorage it will destroy all the Ninja cookies.

    The tests cleared the jar before each run so the assertions had a known starting state. Validate them against Apple’s class to confirm the spec, and you’d lose every Safari cookie on the machine. Bank logins, session tokens, the lot.

    Once swizzling was in (more on that later), I went a step further. Every test got a twin with the suffix ThroughSharedStorage: same assertion, but reaching in through NSHTTPCookieStorage.sharedHTTPCookieStorage() rather than my class directly. This proved two things at once. First, the swizzle was alive: a call into the Foundation singleton was actually landing in my code. Second, my class behaved identically whether you called it directly or arrived through the Apple façade. Both halves had to be true for the integration to be safe, and the dual-path testing made any drift between them impossible to miss.

    This was the contract. Whatever I built next had to satisfy it.

    Method swizzling, the final step

    The last piece was making the rest of the system call my implementation instead of Apple’s. Objective-C method swizzling lets you swap the implementation of a selector with another at runtime: the class is still NSHTTPCookieStorage, the API surface is unchanged, but underneath it now calls into mine.

    I used JRSwizzle as the runtime helper, ran the swap from a +load method (which the Objective-C runtime calls before main), and guarded it with dispatch_once so a category accidentally loaded twice wouldn’t try to double-swap and undo itself:

    + (void)load {
    static dispatch_once_t swizzleMethodsToken;
    dispatch_once(&swizzleMethodsToken, ^{
    NSError *error = nil;
    [[self class] jr_swizzleMethod:@selector(deleteCookie:)
    withMethod:@selector(caDeleteCookie:) error:&error];
    [[self class] jr_swizzleMethod:@selector(cookieAcceptPolicy)
    withMethod:@selector(caCookieAcceptPolicy) error:&error];
    [[self class] jr_swizzleMethod:@selector(setCookieAcceptPolicy:)
    withMethod:@selector(caSetCookieAcceptPolicy:) error:&error];
    [[self class] jr_swizzleMethod:@selector(cookies)
    withMethod:@selector(caCookies) error:&error];
    [[self class] jr_swizzleMethod:@selector(cookiesForURL:)
    withMethod:@selector(caCookiesForURL:) error:&error];
    [[self class] jr_swizzleMethod:@selector(sortedCookiesUsingDescriptors:)
    withMethod:@selector(caSortedCookiesUsingDescriptors:) error:&error];
    [[self class] jr_swizzleMethod:@selector(setCookie:)
    withMethod:@selector(caSetCookie:) error:&error];
    [[self class] jr_swizzleMethod:@selector(setCookies:forURL:mainDocumentURL:)
    withMethod:@selector(caSetCookies:forURL:mainDocumentURL:) error:&error];
    });
    }

    Eight selectors. That was all the public API anyone called. The list maps one-to-one to the symbols I’d seen in Hopper’s left sidebar.

    There was one subtlety. My first instinct was to override +sharedHTTPCookieStorage so that anyone asking for the singleton would get my object instead of Apple’s. That didn’t work, and I wrote down the reason at the top of the swizzle header so I wouldn’t forget:

    We are not hijacking sharedHTTPCookieStorage because something from Safari calls it, gets a copy of NSHTTPCookieStorage and then calls private methods in it that CAHTTPCookieStorage doesn’t implement.

    The crash was deep in CFNetwork, with a stack trace pointing at internal selectors I’d seen but deliberately not bothered to ship. Swizzling on the class itself, on the real NSHTTPCookieStorage, sidestepped it. The shared instance stays the shared instance. Its public methods just do something different now. The private ones still work, because they’re still Apple’s.

    Two side-by-side process containers: a cyan Dashman process whose NSHTTPCookieStorage has the eight swizzled public selectors routed to a remotely synchronized cookie jar, and a red Safari process whose NSHTTPCookieStorage delegates unchanged through CFNetwork to the local cookie jar.

    There was a second subtlety, easy to miss. WebKit, by default, processes cookies for you on every request. If you only swizzle storage, you still get Safari-cookie behaviour because WebKit’s request pipeline reaches into its own cookie machinery before your storage runs. The fix is to disable WebKit’s cookie handling per-request and call the storage manually:

    request.HTTPShouldHandleCookies = false
    CAHTTPCookieStorage.sharedHTTPCookieStorage().handleCookiesInRequest(request)

    Same on the response side: receive the response yourself, extract Set-Cookie headers, hand them to the storage, then return. This had to be wired into WKWebView‘s request pipeline in two places: once on outgoing requests (to attach our cookies) and once on incoming responses (to harvest Set-Cookie). Redirects had to be handled twice, on both legs, because the redirect response itself can carry cookies that the next request needs.

    What shipped

    A single, fully isolated cookie jar separate from Safari, for the macOS build. That solved the immediate problem: nothing else on the user’s machine could be authenticated as Dashman just by being on the same machine, and Dashman couldn’t accidentally inherit the user’s personal Safari sessions either. The longer-term goal of one jar per site I designed for but didn’t ship before the architecture pivoted to a Java/JavaFX client where I owned the entire HTTP stack and the swizzling story became unnecessary.

    I’d call that a healthy outcome. The hack solved a real, present problem. The future problem got solved a different way, by changing the architecture rather than by deepening the hack.

    What I took from it

    A few things I’ve carried with me since:

    One more proof: “can’t be done” is, for me, a starting line. Long before Apple’s reply, treating “impossible” as a cue to dig in had become a reflex. When someone tells me a thing can’t be done, that’s the moment I commit to doing it, and I’ll go as deep as the problem needs. Apple’s reply was technically accurate from inside their boundaries. I went outside them. Here that meant x86 assembly. Next time it’ll be whatever the next problem demands.

    Behavioural equivalence is a real specification. When the source is unavailable, the binary is the spec, and a test suite written against the binary is the next best thing. The suite was as valuable as the replacement, because it told me, precisely, when I’d drifted. The dual-target trick (run the same tests against Apple’s class and mine) and the dual-path trick (run them direct and through the swizzled singleton) were what made the substitution trustworthy rather than hopeful.

  • Disclaimer: this article is now obsolete as the rules have changed, please don’t waste your time reading it.

    Disclaimer: I am not a lawyer, this is not legal advice. 


    There’s a lot of conflicting information out there about whether you need an ERN or not to publish an app in the App Store. I spoke to Apple representatives as well as various employees of a couple of US agencies. As painful as it is, if your app is capable of the simplest, most standard, of encryptions such as SSL/HTTPS then you need to answer your export compliance questions like this:

    The conclusion from selecting the above answers:

    To make your app available on the App Store, you must submit a copy of your U.S. Encryption Registration (ERN) approval from the U.S. Bureau of Industry (BIS).

    In some places, you’ll see CCATS instead of ERN. I’m not 100% sure, but it seems CCATS was a previous more bureaucratic version of the ERN. Right now, what you need is an ERN and this is our journey to get it. We are publishing as much detail as possible so that you can replicate it for your own application. There are some other blog posts that explain how to do it, but we found that over the years, some of the steps changed and we had to find a new path. Since this is going to happen again, we are adding as much information as possible so that should your path be slightly different, you won’t have much trouble finding your way through it.

    Starting at the beginning

    After being utterly confused by both Apple’s as well as BIS’ FAQ and how to pages, I decided to go the homepage for the Bureau of Industry and Security and see where it took me:

    At this point I new SNAP-R was relevant to my needs. I was almost under the impression of needing one, even though I didn’t know what it was. Going through that page I found this:

    Yes! I’d like to submit an application (SNAP-R) – fourth item in the list. That takes you to this page: http://www.bis.doc.gov/index.php/licensing/simplified-network-application-process-redesign-snap-r, which defines what a SNAP-R is. It stands for Simplified Network Application Process – Redesign. I think a SNAP-R is sort of an account with the BIS. There’s no mention of ERN in that page, but it says:

    You must have a Company Identification Number (CIN) and an active user account to access SNAP-R. The procedures and requirements for obtaining a CIN and user account are set forth below.

    You need to obtain a CIN before you can proceed. If you scroll all the way to the bottom of the page, you’ll see:

    And that link, ladies and gentlemen, is the most promising I’ve seen so far. It takes you to https://snapr.bis.doc.gov/registration/Register.do which looks like this:

    The SNAP-R Company Registration process

    After completing and submitting that form you’ll get an email to confirm your email address. I recommend limiting yourself to ASCII characters here, as the é and á in my name got mangled. That email took only a few minutes to arrive but the confirmation page claims the next step might take up to five days:

    Some people claim to have been finished in 30 minutes or even less. I suppose it depends where you or your company is located. In my case, the five days elapsed so I sent them an email and two days later I got a reply telling me to call their support number: +1-202-482-2227 (later on I learned that another phone number that might help is +1-202-482-0707). When I talked to a representative, he said that I should have received the activation email already and just re-triggered it. Maybe calling them after a couple of days would have been a good approach to speed things up. Shortly after my call I got this email:

    That link takes you to a page to set up your log in and password:

    After entering those details, voila! you have an account:

    You may now log in:

    After logging in, you are now in your SNAP-R Home page:

    Creating a new work item within your SNAP-R account

    The next step is to create a new work item, which you can do from the sidebar. That takes you to a page that looks like this:

    The type of work item that you want, to be able to distribute apps with encryption, is an Encryption Registration:

    Now, about the Reference Number, the question mark next to it sends you to https://snapr.bis.doc.gov/snapr/docs/fieldHelp.html#NewWrkItem1 where it says:

    Enter a valid reference number for the Work Item. Reference numbers must be in the format “AAA1111”.

    which didn’t really answer what a reference number is. I decided to call them again and when I asked the question they put me on hold for 25 minutes. I hung up, called them again and I was speaking with someone else in less than 3 minutes and she answered. The reference number is just something you make up, for yourself. It’s not something you obtain and it seems as long as you follow their convention, it’s fine:

    After creating the work item, you are invited to edit it. It starts partially populated and it’s straight forward:

    Well, it’s straightforward until the last part: Documents. You need to attach the Encryption Registration Supplement No. 5 to Part 742.

    Creating the Encryption Registration Supplement

    Creating the supplement, thankfully, is easier than it looks; that is, when you know what you have to do. There’s a document number 742 that you can download from https://www.bis.doc.gov/index.php/forms-documents/doc_download/1208-742 and  on page 60 it has the Supplement No. 5: Encryption Registration. These are the contents of that page:

    SUPPLEMENT NO. 5 TO PART 742 - ENCRYPTION REGISTRATION
    
    Certain classification requests and self-classification reports for encryption items must be supported by an encryption registration, i.e., the information as described in this Supplement, submitted as a support documentation attachment to an application in accordance with the procedures described in §§ 740.17(b), 740.17(d), 742.15(b), 748.1, 748.3 and Supplement No. 2 to part 748 of the EAR.
    
    (1) Point of Contact Information
      (a) Contact Person
      (b) Telephone Number
      (c) Fax Number
      (d) E-mail address
      (e) Mailing Address
    (2) Company Overview (approximately 100 words).
    (3) Identify which of the following categories apply to your companys technology/families of products:
      (a) Wireless
        (i) 3G cellular
        (ii) 4G cellular/WiMax/LTE
        (iii) Short-range wireless / WLAN
        (iv) Satellite
        (v) Radios
        (vi) Mobile communications, n.e.s.
      (b) Mobile applications
      (c) Computing platforms
      (d) Multimedia over IP
      (e) Trusted computing
      (f) Network infrastructure
      (g) Link layer encryption
      (h) Smartcards or other identity management
      (i) Computer or network forensics
      (j) Software
       (i) Operating systems
       (ii) Applications
      (k) Toolkits / ASICs / components
      (l) Information security including secure storage
      (m) Gaming
      (n) Cryptanalytic tools
      (o) “Open cryptographic interface” (or other support for user-supplied or non-standard cryptography)
      (p) Other (identify any not listed above)
      (q) Not Applicable (Not a producer of encryption or information technology items)
    (4) Describe whether the products incorporate or use proprietary, unpublished or non-standard cryptographic functionality, including encryption algorithms or protocols that have not been adopted or approved by a duly recognized international standards body. (If unsure, please explain)
    (5) Will your company be exporting “encryption source code”?
    (6) Do the products incorporate encryption components produced or furnished by non-U.S. sources or vendors? (If unsure, please explain)
    (7) With respect to your companys encryption products, are any of them manufactured outside the United States? If yes, provide manufacturing locations. (Insert “not applicable”, if you are not the principal producer of encryption products) 

    All you have to do is create a PDF file answering these questions for your application and upload it. I couldn’t find this information anywhere so I called them once again and that’s how I learned that all matters related to encryption were handled by the department… never mind the name, the phone number is +1-202-482-0707. Next time I’m calling them directly – there was no wait, no menu, just a person picking up the phone.

    I created a document for my case saying:

    Dashman Encryption Registration Supplement No. 5 to Part 742
    (1) Point of Contact Information
    (a) José Pablo Fernández Silva
    (b) +44XXXXXXXX
    (c)
    (d) pupeno@carouselapps.com
    (e) 20-22 Wenlock Road, London, N1 7GU, United Kingdom
    (2) Carousel Apps is a small London based company producing software apps such as Dashman. Our main use of encryption (and so far all of it) is the standard SSL (https), OpenSSH, etc. You can learn more about us at https://CarouselApps.com
    (3) We produce
    (j) Software
    (ii) Applications
    (4) Our products use standard off the shelf encryption libraries and tools, such as https (SSL). We don’t develop or intend to develop any proprietary encryption mechanisms
    (5) We don’t plan on exporting “encryption source code”.
    (6) Dashman uses Apple’s Safari component that allows https encrypted communication. This is provided by Apple. I understand that Apple uses OpenSSL which is an open source project and thus may have contributions from all around the world.
    (7) We produce software, so, no manufacturing process are involved. All our software is produced outside the United States. The reason for this application is to distributed an app through Apple’s App store.

    I cannot vouch for this content, I’m not sure this is the appropriate file to submit, this is only what I did. The next step is to click on “View and Manage Supporting Documents” which will take you to a page that looks like this:

    There, click “Upload Supporting Document” and you’ll be greeted by this form:

    I just came up with a title and keywords, entered the current date and my name as author. I think the only really important field is the document type:

    Submitting the ERN

    With that document in place and attached, we seem to have passed some sort of automatic verification procedure.

    I clicked on “Preview Work Item to Submit” and I was given a last chance to look at the application and verify its correctness:

    The submission process, triggered by the “Submit” button of course, asks you for your name, in a special format, one more time:

    And we you click “Submit Work Item” you are done:

    Uploading Encryption Registration to Apple

    I almost immediately got a message in the SNAP-R website:

    And the message was the acceptance of the application including the ERN code (blacked out):

    That is the document you need to upload to Apple. Take a screenshot of that page and save it for your records. Back at Apple’s iTunes connect, when you answer the questions stating that you use encryption, you get an upload box for the document:

    If the upload button doesn’t appear, this is what an Apple representative suggested: “If you do not see the prompt, there could be a glitch in the website. One possible workaround is to change the answer to question 4 to “Yes”. By doing this the upload field should appear.”

    Once you upload it, the “Submit” button will become enabled and you are ready to rock. Click it and your app will be on its way to fame and fortune. Well… that is… after they review your export compliance. For now, your app will be “Waiting for Export Compliance”:

    From Apple’s version statuses, that means: “Your app is reviewed and ready for sale, but your CCATS file is in review with Export Compliance.” CCATS seems to be an older or bigger version of the ERN and in some places we can still find CCATS instead of ERN. Don’t worry, an ERN is all you need if your situation is similar to mine. When the status reaches to “Waiting for Review”:

    Congratulations! Your ERN was accepted.  You are done with this bit of bureaucracy.

    If this blog post was useful or you find differences in the process, please, let us know in the comment section.

    Picture by Yuri Samoilov