Conversation
517d2b2 to
ab571a6
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #159 +/- ##
==========================================
- Coverage 94.79% 94.78% -0.01%
==========================================
Files 79 79
Lines 2498 2494 -4
==========================================
- Hits 2368 2364 -4
Misses 130 130 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
To test this PR, post_benchmark_comment.yml needs to be on master already... So I'll play around a bit in some playground repositories, then I'll report if it works. |
ab571a6 to
5f53814
Compare
|
I opened a PR on AirspeedVelocity (MilesCranmer/AirspeedVelocity.jl#142) that adds this two-stage benchmarking workflow. @lostella For now, there are three options, I think:
Regarding the automatic detection of performance regressions, I considered it and concluded that making the job fail might not be the best solution. I opened another PR on AirspeedVelocity (MilesCranmer/AirspeedVelocity.jl#140) that adds emojis to its result table to mark significant speedups and slowdowns (actually, I just re-heated the original author's outdated PR). If it gets merged, I'm considering opening another one that adds a notice before the collapsible table in the posted comment, warning about the negative performance regressions. |
In this PR, I try to fix this problem by dividing the benchmarking into two phases: