Fixes#5229, is part of #5361 and relates to #5285.
I have to admit thsi was a fairly tough task - I initially assumed that
the problem lies
with how the tests are setup, and that we're probably missing some
`teardown_beets` calls
here and there.
Unfortunately, it was not so simple. I came across several issues that
gave rise to
leftover temporary files:
1. `fetchart`, `artresizer` and `play` handling of temporary files.
These plugins created
isolated temporary files outside of the directories that tests clean up.
You will find
I added a couple of functions (namely `get_module_tempdir`) that force
these plugins to
create files in directories determined by their module names. This way
we can clean up
after them using the new `CleanupModulesMixin`.
2. Tests that ran temporary directories setup twice, running
`_common.TestCase.setUp` and
`test.helper.TestHelper.setup_beets`. Both of these ran `self.temp_dir =
mkdtemp()`,
therefore the directories created by the initial setup persisted since
those have been
overridden and thus unreachable in the teardown. Here, I removed the
`setUp` calls, see
- `test/plugins/test_embedart.py`
- `test/test_importer.py`
- and `test/test_plugins.py` where `setup_beets` was called twice
3. `test/test_config_command.py` attempted to manage the temporary
directory by itself,
where I found that `tearDown` failed to remove the directory for four
tests. Could not
figure out the cause, and found that delegating this task to
`TestHelper` fixed the
issue.
4. Mediafile fixture removal depended on calling
`remove_mediafile_fixtures` method, which
`test/plugins/test_zero.py` failed to do. I made the fixtures to be
created within the
same `temp_dir` directory that gets removed in the teardown, so now they
are taken care
of automatically.
In summary, see the test modules that left files behind:
```
Temp files created by test/__init__.py
Temp files created by test/plugins/__init__.py
Temp files created by test/plugins/lyrics_download_samples.py
Temp files created by test/plugins/test_acousticbrainz.py
Temp files created by test/plugins/test_advancedrewrite.py
Temp files created by test/plugins/test_albumtypes.py
Temp files created by test/plugins/test_art.py
/tmp/tmp11nicahe.jpg
/tmp/tmp1bjmodum.png
/tmp/tmped7nhls4.jpg
/tmp/tmpflnzr9wz.jpg
/tmp/tmpjngkauqs.png
/tmp/tmpkzy9mn6t.jpg
/tmp/tmpph_wmuea.jpg
/tmp/tmps6gk58i_.jpg
/tmp/tmpz2eji_o4.jpg
Temp files created by test/plugins/test_aura.py
Temp files created by test/plugins/test_bareasc.py
/tmp/tmphl3kzhug
/tmp/tmpnh2q6v02
/tmp/tmpppw5qrhz
Temp files created by test/plugins/test_beatport.py
Temp files created by test/plugins/test_bucket.py
Temp files created by test/plugins/test_convert.py
Temp files created by test/plugins/test_discogs.py
Temp files created by test/plugins/test_edit.py
Temp files created by test/plugins/test_embedart.py
/tmp/tmp1ayvqzhx
/tmp/tmp58k6mdfx.jpg
/tmp/tmp64c2lqiv
/tmp/tmp6nar4kr5
/tmp/tmp6u0d5dex
/tmp/tmpacoq7w_f
/tmp/tmpajnr_sxr
/tmp/tmpasj16beh
/tmp/tmpboyaixb5
/tmp/tmpcrmcyt5r
/tmp/tmpdomje5g3
/tmp/tmplu3o6t6g
/tmp/tmpns_xvkns
/tmp/tmpo87o1h6o.jpg
/tmp/tmpqem39h_j
/tmp/tmprlzm18pb
/tmp/tmpt22v4u6x
/tmp/tmptp3rxdgv
Temp files created by test/plugins/test_embyupdate.py
Temp files created by test/plugins/test_export.py
Temp files created by test/plugins/test_fetchart.py
Temp files created by test/plugins/test_filefilter.py
Temp files created by test/plugins/test_ftintitle.py
Temp files created by test/plugins/test_hook.py
Temp files created by test/plugins/test_ihate.py
Temp files created by test/plugins/test_importadded.py
Temp files created by test/plugins/test_importfeeds.py
Temp files created by test/plugins/test_info.py
Temp files created by test/plugins/test_ipfs.py
Temp files created by test/plugins/test_keyfinder.py
Temp files created by test/plugins/test_lastgenre.py
Temp files created by test/plugins/test_limit.py
Temp files created by test/plugins/test_lyrics.py
Temp files created by test/plugins/test_mbsubmit.py
Temp files created by test/plugins/test_mbsync.py
Temp files created by test/plugins/test_mpdstats.py
Temp files created by test/plugins/test_parentwork.py
Temp files created by test/plugins/test_permissions.py
Temp files created by test/plugins/test_player.py
Temp files created by test/plugins/test_playlist.py
Temp files created by test/plugins/test_play.py
/tmp/tmp6ohknmve.m3u
/tmp/tmp8rw2z_j4.m3u
/tmp/tmp9vi27ypx.m3u
/tmp/tmpa_s66jh8.m3u
/tmp/tmpb7h3cn3n.m3u
/tmp/tmpexbmqvry.m3u
/tmp/tmpinbqrt80.m3u
/tmp/tmpql02hax5.m3u
/tmp/tmpvbdzprsf.m3u
/tmp/tmpzipim36x.m3u
Temp files created by test/plugins/test_plexupdate.py
Temp files created by test/plugins/test_plugin_mediafield.py
Temp files created by test/plugins/test_random.py
Temp files created by test/plugins/test_replaygain.py
Temp files created by test/plugins/test_smartplaylist.py
Temp files created by test/plugins/test_spotify.py
Temp files created by test/plugins/test_subsonicupdate.py
Temp files created by test/plugins/test_the.py
Temp files created by test/plugins/test_thumbnails.py
Temp files created by test/plugins/test_types_plugin.py
Temp files created by test/plugins/test_web.py
Temp files created by test/plugins/test_zero.py
/tmp/tmp3ub9xmzy
Temp files created by test/rsrc/beetsplug/test.py
Temp files created by test/rsrc/convert_stub.py
Temp files created by test/testall.py
Temp files created by test/test_art_resize.py
/tmp/tmp3p7p60ih.jpg
/tmp/tmp8exclgit.jpg
/tmp/tmpkrrjsitl.jpg
/tmp/tmpw6n8ee8e.jpg
/tmp/tmpygws_0aw.jpg
Temp files created by test/test_autotag.py
Temp files created by test/test_config_command.py
/tmp/tmp333f0r2j
/tmp/tmphr356z5r
/tmp/tmporp4rag2
/tmp/tmpy7sjqdsw
Temp files created by test/test_datequery.py
Temp files created by test/test_dbcore.py
Temp files created by test/test_files.py
Temp files created by test/test_hidden.py
Temp files created by test/test_importer.py
/tmp/tmp0m363gfb
/tmp/tmp2n3i13mc
/tmp/tmpxk3v304s
Temp files created by test/test_library.py
Temp files created by test/test_logging.py
Temp files created by test/test_m3ufile.py
Temp files created by test/test_mb.py
Temp files created by test/test_metasync.py
Temp files created by test/test_pipeline.py
Temp files created by test/test_plugins.py
/tmp/tmp6pxhx67u
/tmp/tmpb8pqi9ui
/tmp/tmpcx_658g7
/tmp/tmp_giqb9jz
/tmp/tmpgm9xk94_
/tmp/tmpk60l6bt3
/tmp/tmpqoj4la68
/tmp/tmptcdu20rp
/tmp/tmpvr7k5shn
/tmp/tmpwnfnzs91
Temp files created by test/test_query.py
Temp files created by test/test_sort.py
Temp files created by test/test_template.py
Temp files created by test/test_ui_commands.py
/tmp/tmpns2u94w6
Temp files created by test/test_ui_importer.py
Temp files created by test/test_ui_init.py
Temp files created by test/test_ui.py
Temp files created by test/test_util.py
Temp files created by test/test_vfs.py
```
And that's what we have right now:
```
Temp files created by test/__init__.py
Temp files created by test/plugins/__init__.py
Temp files created by test/plugins/lyrics_download_samples.py
Temp files created by test/plugins/test_acousticbrainz.py
Temp files created by test/plugins/test_advancedrewrite.py
Temp files created by test/plugins/test_albumtypes.py
Temp files created by test/plugins/test_art.py
Temp files created by test/plugins/test_aura.py
Temp files created by test/plugins/test_bareasc.py
Temp files created by test/plugins/test_beatport.py
Temp files created by test/plugins/test_bucket.py
Temp files created by test/plugins/test_convert.py
Temp files created by test/plugins/test_discogs.py
Temp files created by test/plugins/test_edit.py
Temp files created by test/plugins/test_embedart.py
Temp files created by test/plugins/test_embyupdate.py
Temp files created by test/plugins/test_export.py
Temp files created by test/plugins/test_fetchart.py
Temp files created by test/plugins/test_filefilter.py
Temp files created by test/plugins/test_ftintitle.py
Temp files created by test/plugins/test_hook.py
Temp files created by test/plugins/test_ihate.py
Temp files created by test/plugins/test_importadded.py
Temp files created by test/plugins/test_importfeeds.py
Temp files created by test/plugins/test_info.py
Temp files created by test/plugins/test_ipfs.py
Temp files created by test/plugins/test_keyfinder.py
Temp files created by test/plugins/test_lastgenre.py
Temp files created by test/plugins/test_limit.py
Temp files created by test/plugins/test_lyrics.py
Temp files created by test/plugins/test_mbsubmit.py
Temp files created by test/plugins/test_mbsync.py
Temp files created by test/plugins/test_mpdstats.py
Temp files created by test/plugins/test_parentwork.py
Temp files created by test/plugins/test_permissions.py
Temp files created by test/plugins/test_player.py
Temp files created by test/plugins/test_playlist.py
Temp files created by test/plugins/test_play.py
Temp files created by test/plugins/test_plexupdate.py
Temp files created by test/plugins/test_plugin_mediafield.py
Temp files created by test/plugins/test_random.py
Temp files created by test/plugins/test_replaygain.py
Temp files created by test/plugins/test_smartplaylist.py
Temp files created by test/plugins/test_spotify.py
Temp files created by test/plugins/test_subsonicupdate.py
Temp files created by test/plugins/test_the.py
Temp files created by test/plugins/test_thumbnails.py
Temp files created by test/plugins/test_types_plugin.py
Temp files created by test/plugins/test_web.py
Temp files created by test/plugins/test_zero.py
Temp files created by test/rsrc/beetsplug/test.py
Temp files created by test/rsrc/convert_stub.py
Temp files created by test/testall.py
Temp files created by test/test_art_resize.py
Temp files created by test/test_autotag.py
Temp files created by test/test_config_command.py
Temp files created by test/test_datequery.py
Temp files created by test/test_dbcore.py
Temp files created by test/test_files.py
Temp files created by test/test_hidden.py
Temp files created by test/test_importer.py
Temp files created by test/test_library.py
Temp files created by test/test_logging.py
Temp files created by test/test_m3ufile.py
Temp files created by test/test_mb.py
Temp files created by test/test_metasync.py
Temp files created by test/test_pipeline.py
Temp files created by test/test_plugins.py
Temp files created by test/test_query.py
Temp files created by test/test_sort.py
Temp files created by test/test_template.py
Temp files created by test/test_ui_commands.py
Temp files created by test/test_ui_importer.py
Temp files created by test/test_ui_init.py
Temp files created by test/test_ui.py
Temp files created by test/test_util.py
Temp files created by test/test_vfs.py
```
Note that the command which provides the output is now available through
`poe`.
Due to some weird race conditions the temporary directories were not
getting torn down for four of the tests. I failed to figure out why, and
I found that using TestHelper to manage the temporary directory somehow
fixes this.
This way they get automatically removed with removal of temp_dir.
After this change `test/plugins/test_zero.py` does not any more leave
any mediafield fixtures hanging around.
And move the definition to a shared module.
The problem was that EmbedartCliTest ran `_common.TestCase.setUp`
method which initialised temporary directory for the tests AND ran
`helper.TestHelper.setup_beets` method which initialised another set of
temporary directories. This meant that the first set of directories
could not be tracked down for the cleanup.
In order to include the table name for fields in this query, use the
`field_query` method.
Since `AnyFieldQuery` is just an `OrQuery` under the hood, remove it and
construct `OrQuery` explicitly instead.
Unify query creation logic from
- queryparse.py:construct_query_part,
- Model.field_query,
- DefaultTemplateFunctions._tmpl_unique
to a single implementation under `LibModel.field_query` class method.
This method should be used for query resolution for model (flex)fields.
Allow filtering item attributes in album queries and vice versa by
merging `flex_attrs` from Album and Item together as `all_flex_attrs`.
This field is only used for filtering and is discarded after.
For a flexible attribute query, replace the `col_name` property with
a function call that extracts that attribute from the `field_attrs`
field introduced in the earlier commit.
Additionally, for boolean, numeric and date queries CAST the value to
NUMERIC SQLite affinity to ensure that our queries like 'flex:1..5' and
'flex:true' continue working fine.
This removes the concept of 'slow query', since every query for any
field now has an SQL clause.
In #4746 I was making a small adjustment in beetsplug/aura.py and found
that the module wasn't tested. So this PR adds some high-level tests to
act a safeguard for any future adjustments.
The docs say:
> The `auto` option uses reflinks when possible and falls back to plain
> copying when necessary.
I've been using this option for a while, and recently discovered that
despite the option, copying fails between two BTRFS filesystems with:
Error: OS/filesystem does not support reflinks. during link of paths /mnt/fs1/file, /mnt/fs2/file
I tracked this down to how the configuration is handled in the importer.
Remove 'NamedQuery' since it is not being used by any queries any more.
This simplifies query parsing logic in 'queryparse.py'. We know that
this logic can only receive 'FieldQuery' thus I adjusted types and
removed the logic that handles other cases.
Effectively, this means that the query parsing logic does not any more
care whether the query is named by the corresponding DB field. Instead,
queries like 'SingletonQuery' and 'PlaylistQuery' are responsible for
translating 'singleton' and 'playlist' to the underlying DB filters.
When unix tools make use of an external editor, they typically check the
environment variable VISUAL and fall back to EDITOR. This commit adds the
additional check for VISUAL to the existing EDITOR check (where VISUAL is
preferred over EDITOR).
Allow generating extm3u playlists so that they contain additional item fields such as the `id`.
The feature is required by the mgoltzsche/beets-webm3u plugin (M3U server) to transform playlists using a request based item URI template which may require additional fields such as the `id`, e.g. `beets:library:track;$id`.
