Another and (hopefully) final attempt to improve querying speed.
Fixes#4360Fixes#3515
and possibly more issues to do with slow queries.
This PR supersedes #4746.
## What's been done
The `album` and `item` tables are joined, and corresponding data from
`item_attributes` and `album_attributes` is merged and made available
for filtering. This enables to achieve the following:
- [x] Faster album path queries, `beet list -a path::some/path`
- [x] Faster flexible attributes queries, both albums and tracks, `beet
list play_count:10`
- [x] (New) Ability to filter albums with track-level (and vice-versa)
**db** field queries, `beet list -a title:something`
- [x] (New) Ability to filter tracks with album-level **flexible** field
queries, `beet list artpath:cover`
- [x] (New) Ability to filter albums with track-level **flexible** field
queries, `beet list -a art_source:something`
## Benchmarks
You can see that now querying speed is more or less constant regardless
of the query, and the speed is mostly influenced by how many results
need to be printed out
Compare this with what we had previously
## Later
https://github.com/beetbox/beets/issues/5318https://github.com/beetbox/beets/issues/5319
While the commit merged yesterday fixed the coverage upload to
coveralls, it broke the source files preview on the platform.
This was due to the missing actions/checkout step in the coverage upload
job.
In order to include the table name for fields in this query, use the
`field_query` method.
Since `AnyFieldQuery` is just an `OrQuery` under the hood, remove it and
construct `OrQuery` explicitly instead.
It seems like previously filtering by flexible attributes did not work
- I'd receive '{"data": []}' trying to GET `/aura/tracks?filter[play_count]=11`
Now this works, not only for tracks, but for `/aura/artists` and
`/aura/albums` too.
Additionally, this improves `/aura/tracks` response time significantly.
I tried loading the default of 500 tracks from my library:
On `master`, it took ~20s
After this commit, it takes under 1s.
Unify query creation logic from
- queryparse.py:construct_query_part,
- Model.field_query,
- DefaultTemplateFunctions._tmpl_unique
to a single implementation under `LibModel.field_query` class method.
This method should be used for query resolution for model (flex)fields.
Allow filtering item attributes in album queries and vice versa by
merging `flex_attrs` from Album and Item together as `all_flex_attrs`.
This field is only used for filtering and is discarded after.
For a flexible attribute query, replace the `col_name` property with
a function call that extracts that attribute from the `field_attrs`
field introduced in the earlier commit.
Additionally, for boolean, numeric and date queries CAST the value to
NUMERIC SQLite affinity to ensure that our queries like 'flex:1..5' and
'flex:true' continue working fine.
This removes the concept of 'slow query', since every query for any
field now has an SQL clause.
Use `json_group_object` SQLite function to aggregate flexible attributes
into `flex_attrs` field.
Register SQLite converter `json.loads` to automatically convert the JSON
string to a Python dictionary. Remove the code that had this task
previously.
Replace py3_path by standard os.fsdecode and use native os.cpu_count.
These functions had been written before builtins achieving the same have
been made available (in Python 3.2 and 3.4).
Now that they are available though, this PR updates the codebase to use
builtin implementations rather than the custom ones.
## Description
This is a very simple PR, just removing the (apparently unused) `beet`
executable from the repository and substituting `bin/env python` for
`bin/python3` where possible.
Fixes#4604.
This adds a step to the GitHub release so that it sends a toot on
Fosstodon, beets' Mastodon account, to publicise that and get some
engagement!
Not tested, but this is lifted straight from the action repo so it
*should* be okay. @snejus for a sanity check if you wouldn't mind
