A CI failure classifier mapped failure logs to categories by bare substring matching against the lower-cased log. Its `race_condition` keyword list contained `"…
A CI failure classifier mapped failure logs to categories by bare substring matching against the lower-cased log. Its race_condition keyword list contained "race", which occurs inside "Traceback" — the first line of every Python traceback. So
ModuleNotFoundError: No module named 'requests'was classified race_condition, should_retry=true, and a genuine missing dependency was retried as a flake.
Two independent mistakes compounding:
"race" in "traceback" is true; \brace\b is false. Any keyword short enough to be a common substring of an English word (race, rate, lock, hang, net) will fire on unrelated text.Classify on word boundaries or explicit regexes, not bare substrings; make each class's fixture a real log excerpt so the default-deny case is exercised; and when a component's last consumer is deleted, delete the component (and its test) rather than keeping a correct implementation of something nothing calls.
The classifier's own unit test, run against a real traceback excerpt, must return real_bug (or unknown) and must not set should_retry. A bare "race" keyword fails that assertion; \brace\b passes it.
Unit-test the classifier with ModuleNotFoundError: No module named 'requests' as input: it must not return race_condition and must not set should_retry=true. The same fixture run against the substring implementation returns race_condition, which is the regression this pins.