You have inherited a codebase with no trustworthy tests, and a release is approaching. The obvious move is to start recording current behavior as regression tests. I would first spend a short, fixed window proving that the test harness can reproduce a meaningful result. Otherwise, the suite may grow around a database, network dependency, or build process that nobody can reliably control.
A stack probe should usually come before a regression suite
The choice is between characterization-first and stack-probe-first. Characterization-first runs the application as it exists, records an important output, and fails when that output changes. It wins when a release is imminent, the behavior can already be reproduced, and a known defect needs a guardrail. Its cost is that the test may preserve an accidental behavior or depend on unstable infrastructure.
Stack-probe-first tests whether you can establish a repeatable test environment before recording many behaviors. It wins when the application is difficult to start, test data is shared, or CI results differ from local results. Its cost is a delay before the first product-level regression assertion. For an inherited codebase with no evidence that its harness works, I would pay that cost first: an assertion is less useful if tomorrow’s engineer cannot run it against the same conditions.
Keep the probe narrow. Set a provisional 2-hour limit—a value to tune to the release deadline—and answer one question: can a clean checkout execute the same read-only scenario twice with the same result? Record the command, environment variables, database source, and output from each run. If the runs disagree, identify the dependency that changed before adding more assertions.
Read Add regression tests to legacy code without rewriting it QA playbook critically because capturing behavior is a sound tactic only after the captured behavior is reproducible. I would not begin by raising a coverage target, because coverage.py 7.x can report executed lines while leaving a shared database or intermittent service responsible for the test result.
The first probe should expose the dependency you cannot control
Pick one workflow that has caused an incident or is about to change, then trace only what it needs to run. On a Python service, that might mean Python 3.12, pytest 8.x, PostgreSQL 16, and a Docker Compose v2 service definition. On a browser-facing workflow, Playwright 1.x may expose a different obstacle: authentication state or asynchronous updates that make the same action produce different screenshots. The tools are examples, not a proposed migration; use the versions already supported by the application unless a version mismatch itself prevents a clean run.
Start with the cheapest boundary. Can the workflow use a disposable database instead of a developer’s shared instance? Can a read-only HTTP request reach a stable fixture? Can a third-party response be replaced at the network boundary without changing production code? Testcontainers for Python 4.x can help create a disposable PostgreSQL instance when database behavior matters; a plain fixture is cheaper when it does not. Choose the smallest boundary that preserves the failure you are trying to catch, because a realistic environment that takes too long to reset will be run less often.
Here is a runnable pytest check for a deterministic, read-only endpoint. It requires a reachable URL and a previously reviewed response digest; it does not generate or approve its own expected value.
import hashlib
import os
from urllib.request import urlopen
def test_legacy_read_contract():
url = os.environ["LEGACY_READ_URL"]
expected = os.environ["LEGACY_BODY_SHA256"]
with urlopen(url, timeout=3) as response:
assert response.status == 200
body = response.read()
assert hashlib.sha256(body).hexdigest() == expected
Run it with pytest -q after setting both environment variables. The 3-second timeout is a team-selected setting, not a universal service objective; increase it only if the endpoint’s normal response time justifies doing so. A body hash is useful for a genuinely stable response, but it is the wrong long-term assertion for JSON containing timestamps or generated IDs, because harmless changes would fail the check. In that case, parse the JSON and compare the fields that define the behavior instead.
The probe should produce an artifact even if it fails: the exact command, a sanitized log, the dependency versions, and the point at which repetition broke down. That evidence lets you choose a test boundary without mistaking an environmental failure for a product regression.
Characterization-first wins when a release needs one defensible guardrail
Do not make the stack probe a ritual. If a defect can be reproduced now, a release is near, and the existing environment is repeatable, characterize the defect first. For example, an invoice export that duplicates a line for a particular input deserves a test of that input and output before anyone tidies the export module. The immediate assertion protects a known risk; a broad investigation of the build would defer that protection without answering the release question.
Choose the narrowest observable result that would have caught the defect. For an HTTP API, an OpenAPI 3.1 schema can identify required fields, but a schema check alone will miss a wrong total because a well-formed number can still be incorrect. For an integration with another service, Pact V4 may be appropriate when both sides can review and run the contract; it adds coordination cost if you only control one side. For a browser path, Playwright is justified when the failure crosses rendering and interaction boundaries; a pytest assertion against a service response is cheaper when the browser adds no relevant behavior.
Before approving a captured result, ask who can say it is correct. An incident report, an existing customer-visible example, or a domain owner’s explicit decision can support the expected value. “That is what the application returned today” is weaker evidence, because the inherited implementation may contain the very defect you are trying to prevent. Mark uncertain assertions as observations in the test name or review notes until someone confirms the rule.
In an illustrative dry run, suppose the same export takes 11 minutes on the first attempt and 12 minutes on the second; those are measured times in the example, not performance targets. The close results support characterizing that path now, though they do not prove every dependency is stable. If one run instead requires a manual database repair, stop expanding the suite and return to the stack probe: the repair is part of the test setup whether or not the repository admits it.
The switching point is repeatability, not test count
Make the decision explicit in the issue or pull request. Use characterization-first if you can state the expected result, run the scenario twice without manual intervention, and connect it to a release risk. Use stack-probe-first if any of those conditions fails because of setup or dependency uncertainty. This rule is deliberately stricter than “write a test for every bug,” since a test that fails for unrelated reasons will soon be ignored.
Set a trial rule of 3 consecutive clean runs before promoting a fragile scenario into required CI; treat that number as an adjustable threshold, not a statistical guarantee. A failure on the third run is useful evidence: inspect test-data cleanup, clock use, network calls, and ordering before applying pytest-xdist 3.x to run more tests in parallel. Parallelism can make shared-state failures harder to diagnose because several tests may mutate the same resource at once.
Question Before You Scope It, Test the Stack’s Hidden Costs alongside the test log because a dependency’s cost matters here only when it changes which regression boundary you can run reliably. You do not need a full stack assessment to decide whether tomorrow’s test should use a disposable PostgreSQL database or a controlled HTTP fixture.
Once the boundary works, put the command in GitHub Actions and keep the required job focused on the risks it actually covers. GitHub documents a 6-hour maximum for a GitHub-hosted job, but that vendor-published limit is not a sensible target for regression feedback: a job can be permitted to run and still arrive too late to help a developer investigate a change. Track elapsed time and failure cause separately, because “red CI” does not distinguish a changed product result from a failed setup.
The next useful step is one repeatable run
Choose a recent defect or an imminent change, then run its smallest read-only scenario twice from a clean checkout. Save the commands and both results before writing an assertion. If they match and someone can confirm the expected behavior, add the regression test. If they do not, spend the next short session isolating the dependency that changed; that is the decision your inherited codebase needs first.


