Save-on-pay was verified end to end through real IIS. Charge-by-token was verified against the gateway with a real sandbox AuthTransID. The returning-customer UI rendered exactly as designed. Everything was green. Then I fired one no-card probe against the production gateway and got back E00044: Customer Information Manager is not enabled. The feature was 100% built, 100% verified, and 0% shippable. The thing standing in the way was a vendor toggle behind a permission I didn’t have.

The lesson isn’t “sandbox lied.” Sandbox told the truth. It just answered a different question than the one that decides whether you can ship.

What the feature was

A customer asked for what sounds simple: let people re-use a card next season without us holding the card data. Their mental model was “you can just send us the last 4.” You can’t. The last 4 of a card charges nothing, anywhere. It is a display label for “Visa ending 1234” and that is all.

The real way to do this is a gateway token. We run on Authorize.Net (XML API v1), and their Customer Information Manager (CIM) does exactly this: you create a customer profile and a payment profile, the gateway hands you back two IDs, and from then on you charge those IDs with no card number ever touching your code again. The customer profile doesn’t expire. The card behind it can, but Account Updater handles that. So the data model is genuinely small: no new table, just two nullable token columns on the existing transactions row, scoped per account so any admin can re-use a saved card.

Proving it in the sandbox, before any UI

Before writing a line of UI or schema, I proved the full path against apitest.authorize.net. authenticateTestRequest returned I00001, creds good. createCustomerProfile returned a customerProfileId and customerPaymentProfileId. Reading the profile back for display, the gateway returns XXXX1111 and Visa, never the PAN, that’s the label. Then createCustomerProfileTransaction for $12.34 with only the tokens, no card number. Approved.

Then the multi-card path: add a second card to the same profile, charge it, delete it. All green. An hour of createProfile -> token -> charge-by-token retired the single biggest risk in the whole feature: does this third party actually let me re-charge without storing the PAN. Yes, it does.

This is the part I’d defend to anyone. When a feature’s feasibility hinges on a third-party capability, you prove the happy path against the vendor’s sandbox before you build anything around it. That discipline was correct and it paid off. It just wasn’t sufficient, and the gap between “necessary” and “sufficient” is the whole post.

The two credential traps on the way

The sandbox proof wasn’t a straight line. Two config traps hit first, and each one cost real time.

One was a latent bug in our own code: the API endpoint helper was returning an undefined env variable for the sandbox URL, which broke every XML-API call in sandbox mode. Not the feature’s fault, but it had to be fixed before the feature could even be tested.

The other was the stale-credential-that-failed-two-paths-at-once. My Python proof read creds from the primary repo’s env file, but the local IIS runtime loads a different clone’s env. Same sandbox login, different transaction keys. The primary clone’s key worked and returned I00001. The other clone’s key was stale. Because the legacy AIM charge and the modern CIM XML call both authenticate with that same transaction key env var, one stale value failed both legs: the AIM org checkout rejected it with (TESTMODE) merchant login invalid, and the CIM XML call returned E00007 User authentication failed. Two different error surfaces, one root cause. It looked like a charge bug and a tokenization bug. It was neither. It was one wrong copy of one secret. (Compounding it: a later-loaded env override had an empty value for a username variable that silently clobbered the real value from the base config. “The creds are empty” was wrong. The creds were overridden by a blank.)

Lesson buried in there: when the “same” credential works in one tool and fails in another, you’re almost certainly reading two different copies.

Then production

Everything was built. Everything was verified in sandbox, including a real save and a real charge-by-token against the gateway. I went to confirm it works in prod the same way.

The definitive test for a vendor-capability gate is simple and I should have run it on day one: a no-card createCustomerProfile against the production API. It creates an empty profile, moves no money, and the gateway answers unambiguously. Ok or E00039 (duplicate) means CIM is enabled. E00044 means it isn’t. The whole probe is the artifact below; run it before you build, not after.

Prod returned E00044.

CIM is not enabled on our production merchant account. Sandbox accounts get CIM by default, which is why every test passed. The entire feature, which is correct and verified, cannot run against real customers until someone goes into the merchant portal and clicks “Sign up for CIM,” which carries a fee.

The UI was lying in both directions

Before I ran the probe, I’d tried to figure out CIM’s status by looking, and the portals actively misled me. The legacy merchant portal showed an “About + Sign up for CIM” page, which suggests it’s off. The new portal showed “Manage Customers” with no paywall, which suggests it’s on. Neither was authoritative. A human staring at a vendor’s marketing-flavored settings page cannot tell you whether an API capability is live. The API can. That’s the whole argument for probing instead of reading: E00044 is a fact, “this page has a sign-up link” is a vibe.

The permission wall behind the toggle

Here’s where it stops being a five-minute fix. I tried to at least read the activation terms and the fee. “Review terms and fees” was disabled, because the session was a staff sub-user account, not the Account Owner. The portal’s words: “Activating products can only be done by Account Owners.” I couldn’t enable CIM, and I couldn’t even see what it costs.

So the honest status I wrote up was: feature built and verified in sandbox, blocked in prod on a CIM activation plus a fee decision that only the account owner can make. I did not quote the fee from memory, because I genuinely could not see it. Guessing a number you can’t read is exactly the kind of confident-wrong that wastes someone’s afternoon downstream.