What Checkbox checks, and what it can't
Straight answers on coverage (rendered pages vs source code), where the rules come from, and how Checkbox avoids passing something it never saw.
Three questions to ask before trusting any compliance check, Checkbox included.
1. Does it test what subscribers actually see?
Checkbox checks in two places, and they do different jobs.
| Live monitoring (the hosted service) | checkbox ci (every code change) | |
|---|---|---|
| Sign-up | Loads your pricing and checkout pages in a real browser and reads the renewal terms, consent checkbox and button as rendered; reads Stripe Checkout Session and Payment Link settings from Stripe | Reads the exact params your code passes to stripe.checkout.sessions.create, and the disclosure text and page HTML you list |
| Emails | Reads the acknowledgment, reminder and cancellation emails subscribers receive (BCC or forwarding), and checks their timing against each state's window | Checks your email templates' wording, and that every email the law requires for each plan has a template |
| Cancellation | Reads your billing platform's cancellation settings and runs test cancellations (Stripe test mode) | Not checked: cancellation is a flow, not a file |
| What it's for | The record of what subscribers actually saw | A fast gate before deploy: stop a regression in the pull request |
CI is not a substitute for live checks: it reads source, not the rendered page. Live monitoring is not a substitute for CI either: it finds a problem after it ships. Use both.
What Checkbox can't see:
- Pages behind a login. These are checked through the in-page snippet. Its results are marked as reported by the snippet until Checkbox's own browser confirms them.
- Cancellation by phone or chat. Checkbox can't run these paths.
- Emails sent outside the BCC address. Checkbox can't tell whether every subscriber received one, so the result says so.
- Billing platforms other than Stripe, which are covered with fewer checks. The dashboard and API say which checks apply.
2. Where do the rules come from?
- Tied to the statute. Each check cites the specific section it comes from, for example ROSCA 15 U.S.C. § 8403(2) or Cal. Bus. & Prof. Code § 17602(a)(1). The state-by-state pages link the statute text, so you can read it yourself.
- Updated. The rules catalog is versioned (every result carries
rules_versionandcatalog_version), and the law updates feed records each change, such as California's July 2025 amendments and New York's November 2025 rewrite. - How they're written. Rules drafted from the statute text and updated as new statutes come into effect. Automated checks, not legal advice. Where a law leaves room for judgment, for example how prominent "clear and conspicuous" has to be, Checkbox doesn't decide for you. It marks the item Your call, with the recommended answer and the reasoning.
3. Can a check pass when it shouldn't?
This was the main design constraint. Checkbox only says "Checks pass" for something it actually saw, and only after every check on it passed. In practice:
- Nothing seen is not a pass. A page the browser couldn't reach, a setting Stripe didn't return, or an email that never arrived is Not checked yet, never "passes". A template that reads correctly but was never seen sent is Looks compliant — live check pending, never "verified".
- CI can't silently skip code.
- If a file listed in
checkbox.jsonno longer has itscheckout.sessions.create(call, or the params stop being readable, the run fails as a configuration error (exit 3). - Values CI can't read, such as a price from a variable, are reported as unread, not assumed.
- Coverage guard: every Stripe Checkout Session and Payment Link call in the project must be covered by an entry in
checkbox.jsonor deliberately ignored. A new call in an unlisted file fails the pull request with an annotation on that line (details). An agent can't add a second checkout that bypasses the checks.
- If a file listed in
- The language model can't approve on its own. When text is ambiguous, Checkbox may use a language model to read it. The model can only turn a failure into a pass by quoting the exact supporting sentence, and Checkbox checks that the quote appears word for word in your page or email.
- Tested against known failures. A corpus of failing checkouts, emails and cancellation flows runs on every change to the rules. A rule change that lets any of them pass is rejected.
- Tamper-evident records. What Checkbox observed is stored with hashes in an evidence chain, and exports can be verified independently (evidence).
What remains your responsibility:
- Telling Checkbox about every way subscribers sign up. That means every checkout, plan and Payment Link. The coverage guard finds Stripe calls in code, but not a checkout built in a tool Checkbox doesn't know about.
- Your answers to "Your call" items. These are legal judgments, not machine checks.
- What the law means for your business. Checkbox is software, not a law firm, and nothing it reports is legal advice.
Checkbox is compliance software, not a law firm; nothing in these docs is legal advice (details). You remain solely responsible for your subscription program, and Checkbox takes on no legal liability for it (Terms). Checkbox is a System1 company; Trust center · Status. For AI agents: every page is available as Markdown at the same URL plus .md; the index is /llms.txt.