Skip to content

Further Improve robustness of model classifier tests#4450

Open
chrishalcrow wants to merge 1 commit intoSpikeInterface:mainfrom
chrishalcrow:further-improve-robustness-unitrefine-tests
Open

Further Improve robustness of model classifier tests#4450
chrishalcrow wants to merge 1 commit intoSpikeInterface:mainfrom
chrishalcrow:further-improve-robustness-unitrefine-tests

Conversation

@chrishalcrow
Copy link
Member

After merging (#4447), there were still flakey failures in the tests.

This PRs makes the training data more robust in two ways:

  1. The number of units in the generated recording has been reduces. This is better because the units are spaced out with some minimum distance between each one. If the number is too high, a unit can be produced quite far from the probe and hence have a small amplitude.
  2. We give the model repeated data, ensuring it sees all the data in its training. Hence it really should not make a mistake. You don't want to do this when actually training a model, but good for testing.

When testing before, I used pytests count flag. This was stupid: the test was using a cache so wasn't retrained each time. This time I ran the test repeatedly from scratch using something like

for i in {1..1000}; do echo "--- Run #$i ---"; pytest test_file.py::test_name -q || break; rm -r path_to_cache; done

Thanks Gemini!

No failures on 1000 runs :)

@chrishalcrow chrishalcrow added the testing Related to test routines label Mar 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Related to test routines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant