Repository navigation

Unreliable coverage testing #326

Closed

Labels

opened

There seems to be some variability in the coverage results reported. See discussion in PR #324.

A potential solution is changing the coverage testing to disallow lowering the coverage, not only to compare against a specified threshold.

mihaic
added
bug
on Feb 1, 2018
mihaic commented on Feb 2, 2018
mihaic
on Feb 2, 2018
MemberAuthor
The increase in coverage from PR #327 should get us over the threshold again, but it does not help with reliability.

Also, see this code comment in Coverage itself which indicates expected randomness:
https://bitbucket.org/ned/coveragepy/src/7b080c304b9219cdc33dd0b303db14d411c419fe/tests/test_concurrency.py?at=default&fileviewer=file-view-default#test_concurrency.py-362
danielsuo
closed this as completedin 03dcd50 on Feb 3, 2018
mihaic
added a commit that references this issue on Feb 14, 2018
dev: Add coverage variation to deal with flakiness
82d30b8
mihaic
added a commit that references this issue on Mar 19, 2018
dev: Add coverage variation to deal with flakiness (#335)
b51f831

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests