mirror of
https://github.com/beetbox/beets.git
synced 2026-02-18 13:25:06 +01:00
3b670cdf18
I got a little bit nerdsniped by the problems observed in #5027. In short, my high-level diagnosis in https://github.com/beetbox/beets/pull/5027#issuecomment-1857953929 seems to have been correct: other tests were suppressing the legitimate failure of a flaky test. I found the problem by running other tests before the problem test, like this: ``` $ pytest -k 'test_nonexistant_db or test_delete_removes_item' test/test_ui.py ``` When running `test_nonexistant_db` alone, it fails. When running it like this with another test that goes first, it passes. That's the problem. However, `test_delete_removes_item` is just one example that works to make this problem happen. It appeared that _any_ test in a class that used our `_common.TestCase` base class had this power. I tracked down the issue to our `DummyIO` utility, which was having an unintentional effect even when it was never actually used. Here's the solution. Instead of restoring `sys.stdin` to `sys.__stdin__`, we now restore it to whatever it was before we installed out dummy I/O hooks. This is relevant in pytest, for example, which installs its *own* `sys.stdin`, which we were then clobbering. This was leading to the suppression of test failures observed in #5021 and addressed in #5027. The CI will fail for this PR because it now (correctly) exposes a failing test. Hopefully by combining this with the fixes in the works in #5027, we'll be back to a passing test suite. 😃 @Phil305, could you perhaps help validate that hypothesis? Edit: @snejus: I've now consolidated test I/O handling by removing the legacy `control_stdin`/`capture_stdout` context managers and the custom `DummyOut` stream, replacing them with a pytest-driven `io` fixture that: - provides controllable `stdin` via a lightweight `DummyIn` - captures `stdout` via `capteesys` - attaches a `DummyIO` helper to test classes as `self.io` |
||
|---|---|---|
| .. | ||
| utils | ||
| __init__.py | ||
| conftest.py | ||
| lyrics_pages.py | ||
| test_acousticbrainz.py | ||
| test_advancedrewrite.py | ||
| test_albumtypes.py | ||
| test_art.py | ||
| test_aura.py | ||
| test_autobpm.py | ||
| test_bareasc.py | ||
| test_beatport.py | ||
| test_bpd.py | ||
| test_bucket.py | ||
| test_convert.py | ||
| test_discogs.py | ||
| test_edit.py | ||
| test_embedart.py | ||
| test_embyupdate.py | ||
| test_export.py | ||
| test_fetchart.py | ||
| test_filefilter.py | ||
| test_fromfilename.py | ||
| test_ftintitle.py | ||
| test_hook.py | ||
| test_ihate.py | ||
| test_importadded.py | ||
| test_importfeeds.py | ||
| test_importsource.py | ||
| test_info.py | ||
| test_inline.py | ||
| test_ipfs.py | ||
| test_keyfinder.py | ||
| test_lastgenre.py | ||
| test_limit.py | ||
| test_listenbrainz.py | ||
| test_lyrics.py | ||
| test_mbcollection.py | ||
| test_mbpseudo.py | ||
| test_mbsubmit.py | ||
| test_mbsync.py | ||
| test_missing.py | ||
| test_mpdstats.py | ||
| test_musicbrainz.py | ||
| test_parentwork.py | ||
| test_permissions.py | ||
| test_play.py | ||
| test_playlist.py | ||
| test_plexupdate.py | ||
| test_plugin_mediafield.py | ||
| test_random.py | ||
| test_replace.py | ||
| test_replaygain.py | ||
| test_scrub.py | ||
| test_smartplaylist.py | ||
| test_spotify.py | ||
| test_subsonicupdate.py | ||
| test_substitute.py | ||
| test_the.py | ||
| test_thumbnails.py | ||
| test_titlecase.py | ||
| test_types_plugin.py | ||
| test_web.py | ||
| test_zero.py | ||