A sandbox checkout died with “merchant login ID or password is invalid.” Minutes earlier, the exact same credentials had authenticated cleanly in a standalone Python proof and returned I00001. Same gateway, same login, same machine. One path said yes and the other said the password was wrong. When the “same” credential works in one tool and fails in another, you are almost certainly reading two different copies of it.

The setup

We were building a save-card feature for the site’s checkout. A customer stores a card at checkout and re-uses it next season as a one-click option, with the card held by the payment provider (Authorize.Net) and never stored on our side. The mechanism is CIM, Authorize.Net’s Customer Information Manager. You create a customer profile, get back gateway token IDs, and re-charge later with only those tokens, no PAN.

Before we wrote a line of UI, we proved the full happy path against the Authorize.Net sandbox from Python: authenticate, createCustomerProfile, read it back for display, then charge by token. It worked. Profile created, charge approved, no card number anywhere in the flow.

Then we wired the same capability into the actual checkout running under IIS on a local clone and ran it through the browser. That is where it failed with the invalid-login error.

Two copies of the truth

The site runs as several near-identical local clones: local clone 1, local clone 2, local clone 3, and so on. Each is its own checkout served by its own IIS site (l2 serves local clone 2’s branch). Each loads its own environment from its own gitignored .env-local. That is the trap, and it stays invisible until it bites.

Our Python proof read credentials from the primary local clone’s .env-local. The checkout we tested in the browser was l2, which loads local clone 2’s env. Both clones pointed at the same Authorize.Net sandbox and shared the same sandbox API login ID, which is why the symptom was so disorienting. The login was identical. What had drifted was the transaction key, AUTHNET_TRAN_KEY_TEST. local clone 1 carried a working key (authenticates, I00001). local clone 2 carried a stale one (rejected, E00007 User authentication failed).

So there were two different secrets that happened to share a login ID, and we were reading whichever one the runtime under test happened to load. The Python proof and the IIS checkout never read the same file.

This is per-environment config drift in its purest form. The moment a secret lives in N copies, one of them goes stale, and the failure surfaces as “but it works over here,” which sends you hunting in the wrong place.

Two bugs that were one

Authorize.Net has a legacy charge API (the old AIM-style direct charge) and the modern XML API that CIM tokenization rides on. They are different code paths with different request shapes and different parsers. They authenticate with the same transaction key.

So one stale key failed both legs at once. The legacy charge rejected the credentials with a plaintext “the merchant login ID or password is invalid” from the old gateway. The tokenized CIM call rejected the same credentials too, this time with the structured E00007 User authentication failed from the XML API. The error shapes differed and the cause was the same. For a while it looked like we had a charge bug and a separate tokenization bug, two problems to chase. There was one problem wearing two costumes.

When a single secret feeds multiple code paths and goes bad, you get a multi-symptom failure that masquerades as several unrelated bugs. If two failing paths share exactly one upstream dependency, suspect the dependency, not the paths.

The blank value that clobbered a real one

While we were untangling this, a second, smaller gotcha compounded it. The env loading merges files in order: .env-dev first, then .env-local on top. .env-local had an empty assignment, AGENT_USERNAME=, with no value after the equals sign. That blank silently overrode the perfectly good value .env-dev had already set.

One of our early hypotheses, “the creds are just empty,” was wrong in a specific way. The creds were not missing. A real value existed in the earlier file, and a later-loaded file declared the same key with nothing after it. The merge treated “declared as empty” the same as “set to empty,” and last-write-wins stomped the good value.

If your config layering does last-write-wins, and most do, an empty value in a higher-priority file is a live landmine. It does not look like a value, so you skip past it when reading the file. It behaves like a value, so it wins the merge.

The fix

The fix was small once the diagnosis was right. We synced the working transaction key into local clone 2’s gitignored .env-local so l2 loaded the key that actually authenticates. The clone reads .env-local per request (EnvGet is per-request, not cached at app start), so the corrected key took effect live without an IIS restart. We confirmed that by re-running the checkout immediately.

The durable part was a note. If local clone 2 had drifted, the other clones almost certainly carry the same stale key, so the real fix is reconciling the secret across all of them rather than patching the one clone in front of us. One stale copy is rarely alone.

One more thing about these clones, because it caught us adjacent to this: the l<N> local IIS clones share the production database via an ODBC DSN. Testing on l2 isolates the rendered HTML and the sandbox gateway, but it does not isolate data. That is a separate trap from the credential drift, and you want it pinned down before you run anything that writes.

What we do with it now

Two checks came out of this.

  • Count the shared inputs before you assume you have N independent bugs. If two failing paths lean on one upstream value, look at that value first.
  • In layered config, grep every file in the load order for the key, not just the one you expect to own it. “It’s empty” can mean a blank clobbered it, and a blank that clobbered a real value looks identical to a value that was never set.

Where one secret feeds several code paths, a single stale value lights up as several apparently unrelated bugs. Reconcile that secret across every environment copy.