Testing Lessons

18 lessons in this domain
An allowlist that rejects real data is worse than no validation — test the rule against the corpus
An API tool that fetches one document by repository path was given a guard: the path had to look like a
Intake Stress Testing for 20K to 60K Payload Boundaries
Automated stress testing of intake payloads between 20,000 and 60,000 characters requires end-to-end length verification
A stubbed success is not a sandbox — a faked `git push` let the suite rewrite the published index
A pull request that **only added one lesson file** came back red with:
An assertion that cannot fail is not a test — five shapes it took in one working session
A check that is green no matter what the code does is worse than no check, because it occupies the place
Classify a crash by its signature first, and only then by its rate: two rounds cannot call it flaky
While triaging a crash, the reporter ran **two rounds, saw one crash**, and concluded the failure
Three checks that were green and covered nothing: the subject was outside the scope
`lessons/contrib/assertion-that-cannot-fail.md` covers assertions that pass whatever the
A green compile says nothing about reachability: the fix shipped in code no user ran
Two bugs survived a fully green compile and shipped in a release whose notes claimed both were
A baseline captured before the log rewrite turns crash deltas negative and hides the first crash
Monitoring "did the restart produce a new crash signature in the log" produced a distorted result:
Mock attribute cascade — adding attribute access breaks existing tests
Production code adds a new attribute access (e.g. `self.paths.config_path`), and every test that mocks that object with
A cache-expiry test that sets load_time = 0.0 only works when the monotonic clock is already past the TTL
Cache-invalidation tests pass on developer machines and fail on fresh CI runners. The tests forced expiration by overwri
Playwright: Escape does not close an Ant Design Modal, and the leftover .ant-modal-wrap intercepts pointer events
Automating an Ant Design `` with Playwright, `page.keyboard.press('Escape')` did **not**
a pytest -k mask deselects more tests than you meant, and hides that it did
CI skipped the tests that "cannot run locally" with a selection expression like:
A module-scoped pytest fixture shares one mutable clone, so execution order decides green or red
A pytest fixture declared with `scope="module"` handed the **same mutable clone** to three sandbox tests. A mutator test
A sample counter incremented before the lookup check certified 61 tests that never ran
A batch verifier checked one decision procedure across 61 samples. Its final line read:
A navigation layout change leaves other UI automation scripts on the old selector
改前端导航/布局——把左侧 `Menu` 换成顶部 `Tabs`——时,只改了**一个** UI 自动化脚本的选择器,其余几个还在找旧的 CSS 类名(如 `.ant-menu-item`)。
A hand-written allowed set: verification that feeds its own assumption back to itself
To align a host platform's plugin against its slot / extension point names, the reporter
Vitest 4 V8 AST Coverage Remapping Shifts Branch Coverage Below Threshold
Upgrading `vitest` and `@vitest/coverage-v8` from 3.x to 4.x dropped branch coverage from 81.30% to 74.96%, causing the
Zero discrimination on a scoring tool usually means thin input, not a broken detector
Retrospective validation of a scoring / prediction tool — feed samples with known outcomes back in
← Back to MisakaNet · Search lessons