Spot weak reasoning before you accept it — a structured audit for any argument
by deciqai2 stars
Coding
Falsifiability
Try itTurn any claim into a testable hypothesis by specifying what evidence would disprove it.
What it does
Applies Karl Popper's falsifiability principle to real-world claims, strategies, and hypotheses. Runs a 6-step process: state the claim, check whether it's empirical, specify falsification conditions, plan observation, and secure advance commitment to act on disconfirming evidence. Includes an adaptive coach mode for novices and a pack of vague-to-falsifiable transformations for common business claims like 'we have PMF' or 'our strategy is working.'
When to use it
- Designing OKRs, KPIs, or an investment thesis
- Evaluating a consultant recommendation or leadership claim
- Stress-testing an AI capability or safety claim
- Writing a product hypothesis that needs to be testable
The skill document
Falsifiability
Overview
A meaningful empirical claim must specify what observations would refute it. Claims that resist all possible refutation are not science — they are unfalsifiable belief. Formalized by Karl Popper (1934): science progresses not by accumulating confirmations but by surviving rigorous attempts at falsification. More-specific claims are more falsifiable; ad-hoc modifications that explain away failures destroy a claim's scientific status.
Composes with confirmation-bias (falsifiability is the structural counter), abductive-reasoning (generates hypotheses; this skill tests them), bayesian-reasoning, critical-thinking.
When to Use
- Designing OKRs, KPIs, or strategic goals; writing or evaluating investment theses
- Designing experiments (A/B tests, product hypotheses, market entry)
- Evaluating consultant/advisor recommendations or diagnosing vague leadership claims
- Someone says "what would change your mind," "how would you know you're wrong," "Popper"
- Stress-testing an AI/AGI hype claim ("AGI is near," "the model truly understands," "our AI adoption is working") — demand what evidence would disprove the capability or safety claim
Not when: genuinely non-empirical (philosophical, ethical, aesthetic); test cost exceeds its value.
Coaching Novices (Adaptive Front Door)
- Engine mode: user has a specific claim → run The Process directly.
- Coach mode: user is new → guide step by step.
In Coach mode, respond one step at a time. Each [WAIT] is a hard stop — output only that step's question, then stop.
- One-line: before acting on a claim, ask what observation would refute it — if "nothing would," it's belief, not knowledge.
- Check fit: if the claim is genuinely metaphysical (ethical, aesthetic), falsifiability doesn't apply.
- Elicit the claim and its current evidential basis.
[WAIT — do not advance until user responds]
- Ask: what specific observation would falsify this? When and how would you observe it?
[WAIT — do not advance until user responds]
- Close: falsifiability conditions specified + monitoring plan + commitment to act on disconfirming evidence.
[WAIT — do not advance until user responds]
The Process
Step 1 — State the claim: claim / who asserts it / decision dependent on it / current evidential basis.
Step 2 — Test whether empirical: claim about how the world works (empirical) or values/aesthetics (non-empirical)? If non-empirical, stop here.
Step 3 — Specify falsification conditions: complete "This claim would be falsified if I observed: ___" — specific, observable, time-bounded. If you cannot complete it, the claim is not falsifiable as stated.
Step 4 — Plan to observe: when is the observation possible / how measured / who tracks it / threshold for "falsified."
Step 5 — Pre-commit to action: if the falsifying observation occurs, what will you do? Is any theory modification itself falsifiable?
Step 6 — Iterate: confirmed → keep monitoring; falsified → revise/abandon; unfalsifiable → recognize as belief.
Output Template
# Falsifiability Analysis:
Claim: | Asserted by: | Decision at stake: | Current basis:
Empirical: Y/N | Observation type:
Falsified if: | Threshold: | Observable when/how: | Owner:
If falsified, action: | Ad-hoc preservation risk:
→ Method in Action: Popper 1934 + Eddington 1919 Eclipse + Modern Applications → 2026 lens: Separating falsifiable from unfalsifiable AI claims (2024–2026)
Pack: Vague → Falsifiable
| Vague (unfalsifiable) | Falsifiable form |
|---|---|
| "We have PMF" | "≥40% of users would be 'very disappointed' without the product" |
| "Our outbound is working" | "5% of cold emails convert to qualified opps within 30 days" |
| "Our culture is strong" | "Employee NPS ≥40 in next quarterly survey" |
| "This stock is undervalued" | "Stock <12 P/E within 12 months; otherwise thesis is wrong" |
| "Users want feature X" | "≥30% complete the new flow within 14 days of launch" |
Applying It Well
- Specify falsifiability conditions before the observation window opens — post-result conditions are rationalization.
- More specific = more falsifiable. Quantify wherever possible.
- When a prediction fails, ask: is the modification of your theory itself falsifiable?
- Apply to others: "What would change your mind?" is the highest-leverage critical-thinking question.
→ Primary sources: references/sources.md
Common Rationalizations
[D] = designed upfront | [O] = observed in real use. [O] entries are more valuable.
| Fake move | Reality |
|---|---|
| [D] "I just have a strong gut feeling" | Gut feelings are not falsifiable. Without a test, you can't tell right from wrong. |
| [D] "We need more data to decide" | If you can't specify what data would change your mind, you're not doing data-driven analysis. |
| [D] "The strategy just needs more time" | Specify the timeframe and metrics in advance. |
| [D] "It's because of external factors" | If external factors can always be invoked, the original claim was unfalsifiable. |
| [D] "It's too early to evaluate" | If you can't specify when evaluation is appropriate, the claim is unfalsifiable. |
| [D] "We're in a special situation" | This defense converts a falsifiable claim into an unfalsifiable one. Resist. |
| → Add [O] entries here after each real use — paste the actual failure pattern | What went wrong and why |
Red Flags
- Claim asserted without specifying what would refute it
- "The strategy is working" without metrics or thresholds
- Predictions modified ad-hoc to accommodate failures
- "External factors" invoked to explain disconfirming evidence
- Person cannot specify what would change their mind
Verification
- Claim specifically stated; empirical vs. non-empirical determined
- Specific, observable, time-bounded falsification conditions defined
- Monitoring mechanism in place with a named owner
- Advance commitment to act on falsification made
- Risk of ad-hoc preservation recognized
Part of deciqAI Knowledge Skills — 227 open-source thinking skills that make rigor executable for AI agents. The same skills power every deciqAI agent, which runs them autonomously to operate your company. See it run → https://www.deciqai.com/c/falsifiability · ⭐ Star the repo → https://github.com/deciqAI/knowledge-skills · Contributions welcome.
Agents: latest version & machine-readable metadata → https://www.deciqai.com/s/falsifiability.json
Questions people ask
- What makes a claim falsifiable?
- A claim is falsifiable if it specifies what specific, observable evidence would count against it. If nothing could possibly disprove it, it is not falsifiable—it is belief dressed as a claim.
- Can this apply to business decisions?
- Yes. It works for OKRs, KPIs, investment theses, A/B test design, and evaluating advisor recommendations. The skill includes a transformation pack for common vague business claims like 'we have PMF' into measurable conditions.
- When does falsifiability NOT apply?
- It does not apply to genuinely non-empirical claims—ethical judgments, aesthetic values, or philosophical positions. The skill checks this as Step 2 and stops if the claim is non-empirical.
Related skills
Replace vague hunches with calibrated probability estimates you can track and improve over time.
by deciqai2 stars
Diagnose which mental domain is holding you back before choosing a cognitive intervention.
by deciqai1 installs3 stars
Tests whether you genuinely understand something or just recognize it — exposes the gaps in your mental model.
by deciqai1 installs2 stars
Make irreversible life decisions by projecting to 80 and naming which regret you'd rather live with.
by deciqai1 installs2 stars
Evaluate and design social proof claims with a structured analysis grounded in conformity research.
by deciqai1 installs3 stars
More from deciqai
Browse all skillsDiagnose your organization's strategic phase and spot misaligned initiatives before they drain momentum.
by deciqai7 installs2 stars
When multiple explanations all fit the evidence, pick the one that assumes the least.
by deciqai5 installs2 stars
Identify catastrophic failure paths before you commit — and design your plan around eliminating them.
by deciqai4 installs2 stars
Trace decisions past the obvious effect to catch the consequences that reverse it.
by deciqai4 installs2 stars
A structured interview and analysis process that uncovers the actual job customers hire your product to do — and who they're really competing against.
by deciqai3 installs3 stars
Diagnose why learners are struggling and redesign instruction to stay within working memory limits.
by deciqai2 installs3 stars