This works around a bug that does not exist in Python 3.x, and the
workaround (by calling the underlying shlex.split function with bytes)
was causing crashes on some versions of Python 3. Seemed to work fine on
3.10-dev, though, oddly.
The previous code had the potential to crash if (when?) Tekstowo changes
their website structure sufficiently.
The new code is rather ugly due to the explicit checks after each and
every function call. Unfortunately, the alternative would be to catch a
bunch of very generic Exceptions (AttributeError, ...), since there's no
such thing as a `BeautifulSoupNotFoundError`.
I experienced a failure to parse Tekstowo for song lyrics.
This patch allowed the lyrics plugin to fetch the lyrics from another provider as opposed to failing.
This changes greatly improves the speed of `beet export` and `beet info`
when the `--include-keys` option is used. It also removes the globbing
feature of `--include-keys` that was added in #1295. (See #3762 for
discussion).
Listing all fields for an item requires querying the database to find
any flex attributes. This is slow when done for every item being
exported. We already have a way for the user to specify a fixed set
of keys, but we previously queried everything and filtered it afterwards.
The new approach is more efficient.
Code that iterates through all fields now have to handle invalid field
names. The export and info plugins output invalid fields as None.
Timings before:
> /usr/bin/time beet export -i title,path,artist -l Bob Dylan
13.26user 20.22system 0:34.01elapsed 98%CPU (0avgtext+0avgdata 52544maxresident)k
> /usr/bin/time beet export -l Bob Dylan
12.93user 20.15system 0:33.58elapsed 98%CPU (0avgtext+0avgdata 53632maxresident)k
Timings after:
> /usr/bin/time beet export -l Bob Dylan
13.33user 20.17system 0:34.02elapsed 98%CPU (0avgtext+0avgdata 53500maxresident)k
> /usr/bin/time beet export -i title,path,artist -l Bob Dylan
0.49user 0.07system 0:00.56elapsed 98%CPU (0avgtext+0avgdata 50496maxresident)k
Notice the dramatic speedup in the last example!
Squashed from the PR, relevant commit messages follow below:
Added file size option to artresizer
- In line with comments on PR, adjusted the ArtResizer API to add
functionality to "resize to X bytes" through `max_filesize` arg
- Adjustment to changelog.rst to include max_filesize change to ArtResizer
and addition of new plugin.
Added explicit tests for PIL & Imagemagick Methods
- Checks new resizing functions do reduce the filesize of images
Expose max_filesize logic to fetchart plugin
- Add syspath escaping for OS cross compatibility
- Return smaller PIL image even if max filesize not reached.
- Test resize logic against known smaller filesize (//2)
- Pass integer (not float) quality argument to PIL
- Remove Pillow from dependencies
- Implement "max_filesize" fetchart option, including
logic to resize and rescale if maxwidth is also set.
Added tests & documentation for fetchart additions.
Tests now check that a target filesize is reached with a
higher initial quality (a difficult check to pass).
With a starting quality of 95% PIL takes 4 iterations to succeed
in lowering the example cover image to 90% its original size.
To cover all bases, the PIL loop has been changed to 5 iterations
in the worst case, and the documentation altered to reflect the
50% loss in quality this implies. This seems reasonable as users
concerned about performance would most likely be persuaded to
install ImageMagick, or remove the maximum filesize constraint.
The previous 30% figure was arbitrary.
Also simplified the setup of the `readonly` value in the tests which
fixes a test ordering issue found using --random-order.
Signed-off-by: Graham R. Cobb <g+beets@cobb.uk.net>
As discussed in bug #3867, backslash replacement in query strings is a bit of a
hack but it is useful (see #3566 and #3567 for more discussion). However,
it breaks many regular expressions so this patch stops the replacement if the
query term contains '::', indicating it is a regex match.
This commit fixes#3867.
Signed-off-by: Graham R. Cobb <g+beets@cobb.uk.net>
displayable_path may remove 'bad' characters, yielding a wrong path.
Also use track.path rather than track.destination() as that is where
the file is actually located rather than where it should be located
according to the beets path system.
Track item paths and album artpaths should be removed from results unless
INCLUDE_PATHS is set. This works for items but for albums the artpath is always
removed.
This patch makes the artpath removal conditional on INCLUDE_PATHS not being set
and includes a regression test. Note: the default value for INCLUDE_PATHS is
False so no changes will be seen by users unless they already have
INCLUDE_PATHS set.
Signed-off-by: Graham R. Cobb <g+beets@cobb.uk.net>
discogs-client has been deprecated since June 2020, the replacement
is actively developed by the community and does not have any breaking
API changes.
Signed-off-by: George Rawlinson <george@rawlinson.net.nz>
References in the documentation to this plugin were removed in
beetbox/beets#3127 (beetbox/beets#3130) but no actual code
changes were made.
This PR removes support for this dependency entirely.
This PR fixes a bug (#3834) where tracks which have already been fingerprinted do not return to be used by `beet submit` (part of the Chroma plugin). This results in submission errors, as the fingerprint is omitted from the resultant payload sent to acoustID.
This adds support for the JSON Lines format as documented at
https://jsonlines.org/.
In this mode the data is output incrementally, whereas the other
modes load every item into memory and don't produce output until
the end.
* fetchart: Improve Cover Art Archive source.
Instead of blindly selecting the first image, we now treat all "front"
images as candidates.
This is useful where some digital releases have both an animated cover
and a still image and the animated image is the first image returned
from the API.
MPD keeps the current track in the queue when stopping, so it's not
really like a skip, and I use it so that I can stop the music, and later
start at the beginning of a track.
I do this by keeping track of the current song id, and then comparing
them when we receive a stop signal.
* If import move is true, files will be deleted after converting.
Fixes#2947
* Removed trailing whitespace to comply with W293, fixing build
* Add period to the end of the comment
Co-Authored-By: Adrian Sampson <adrian@radbox.org>
* Added changelog entry for this fix.
* Added delete_originals option to remove source files after transcode
* Added unit test, removed redundant syspath call
Co-authored-by: Logan Arens <logan-arens@users.noreply.github.com>
Co-authored-by: Logan Arens <heresmygithub@protonmail.com>
Co-authored-by: Adrian Sampson <adrian@radbox.org>
Co-authored-by: Logan Arens <logan.arens@protonmail.com>
* clean-up code & add tests for genius lyrics backend
* add genius fetch tests
* organize imports: standard lib -> pip -> local
* check in sample genius lyrics page
* fix mock import
* force utf-8 encoding for opened files
* use io.open to force utf-8 encoding w/ python2.7
The Cover Art Archive API offers pre-resized thumbnails of cover
art. If the `maxwidth` option of `fetchart` matches one of the
supported Cover Art Archive thumbnail sizes, and a thumbnail of
that size exists in the Cover Art Archive, fetch it directly
instead of fetching the full size image then resizing it.
The Cover Art Archive API offers pre-resized thumbnails of cover
art. If the `maxwidth` option of `fetchart` matches one of the
supported Cover Art Archive thumbnail sizes fetch it directly
instead of fetching the full size image then resizing it.
* Skip ReST writing & sphinx info messages if query doesn't yield anything
* `writerest` into `appendrest` and `writerest`, don't call `writerest(item=None)` to flush state at the end.
Searching only for the title and just verifying the artist afterwards leads to songs with very common titles not being found, since Genius limits the amount of returned hits.
An example would be 'Saviour' by 'Circa Waves'.
Without this change the web plugin does not support path queries as
slashes are currently used for joining keywords in the QueryConverter.
Moreover, flask cannot distinguish between an URL encoded and a plain
'/' character during routing [0]. To work around this issue without
introducing a breaking change (i.e. removing the QueryConverter) use the
backslash character for path queries and convert it later on.
Fixes#3566
[0]: https://github.com/pallets/flask/issues/900
This will handle cases where the artist name includes some
characters which can cause a search failure for some
websites - mainly unicode.
Fixes https://github.com/beetbox/beets/issues/3340.
Fish shell previously interpreted question marks as glob characters, but
that behavior has been deprecated and will soon be removed. Plus, the
completion for `help` and its alias `?` does not currently seem to behave
as expected anyway and is thus, at present, of limited utility.
From the ImageMagick docs: "The default is to use the estimated quality
of your input image if it can be determined, otherwise 92."
In order to get the original behaviour we need to conditional add the
quality parameter to the `magick` call. The quality range can be
anything from 1 to 100, which gives us the convenience of using 0 to
specify no specific quality level.
* ExceptionWatcher instance running in parallel to the pool, monitoring a queue for exceptions
* Pooled threads push exceptions to this queue, log non-fatal exceptions
* Application exits on fatal exception in pooled thread
* More front end info logs in the CLI
* safer version comparison
* regex bytes directly
* handle b'\x08 ...' case
* test_replaygain.py: injected command output should match the type of the actual output
* Fix unspecified `gain_adjustment` when method defined in config
* Fix difference between dB and LUFS values in case of mismatched `target_level`/`method`:
```
db_to_lufs( target_level <dB> ) - lufs_to_dB( -23 <LUFS> )
```
* Ignore single assertion in case of bs1770gain
(cherry picked from commit 2395bf224032c44f1ea5d28e0c63af96a92b96df)
* Add `--jobs` or `-j` to `replaygain`-> set the pool size
* Single-threaded execution by default, if `--jobs` is unset
* If multithreaded, calls `Backend.compute_album_gain` or `Backend.compute_track_gain` asynchronously with metadata storing/writing in the callback
Apply improvements suggested in GitHub PullRequest #3065:
- be idiomatic
- 0 is falsy
- check enum equality, not identity
- mutate list by constructing a new one
- improve documentation
- fix a typo
- do not mention deprecation of a config option
Configure the replaygain analysis by passing arguments to the Backends. This
avoids the difference between ReplayGain and EBU r128 backends; every Backend
can now fulfil both tasks. Additionally it eases Backend development as the
difference between the two tag formats is now completely handled in the main
Plugin, not in the Backends.
Use the POSIX character class instead of `\s` to match all whitespace in a
regular expression describing the language of valid inputs, in order to avoid a
test failure for the invalid escape sequence `\s` in Python strings.
This commit mostly addresses feedback:
- remove some unused parenthesis
- fix a typo
- expand some docstrings
- document that ffmpeg is usually easy to install
Add replaygain backend using ffmpeg's ebur128 filter.
The album gain is calculated as the mean of all BS.1770 gating block powers.
Besides differences in gating block offset, this should be equivalent to a
BS.1770 analysis of a proper concatenation of all tracks.
Just calculating the mean of all track gains (as implemented by the bs1770gain
backend) yields incorrect results as that would:
- completely ignore track lengths
- just using length in seconds won't work either (e.g. BS.1770 ignores
passages below a threshold)
- take the mean of track loudness, not power
When using the ffmpeg replaygain backend to create R128_*_GAIN tags, the
targetlevel will be set to -23 LUFS. GitHub PullRequest #3065 will make this
configurable.
It will also skip peak calculation, as there is no R128_*_PEAK tag.
It is checked if the libavfilter library supports replaygain calculation. Before
version 6.67.100 that did require the `--enable-libebur128` compile-time-option,
after that the ebur128 library is included in libavfilter itself. Thus we
require either a recent enough libavfilter version or the `--enable-libebur128`
option.
Previously using EBU R128 forced the use of the bs1770gain backend.
This change adds a whitelist of backends supporting R128. When the configured
backend is in that list it will also be used for R128 calculations. Otherwise
bs1770gain is still used as a default.
This should not change the overall behaviour of the program at all, but allow
for further R128-supporting backends to be added.
Return a namedtuple CommandOutput(stdout, stderr) instead of just stdout from
util.command_ouput, allowing separate access to stdout and stderr.
This change is required by the ffmpeg replaygain backend (GitHub
PullRequest #3056) as ffmpeg's ebur128 filter outputs only to stderr.
Since commit 95e569a, mediafile takes care of the float -> Q7.8 conversion in
R128 GAIN tags by itself.
From `store_album_r128_gain` this conversion was already missing, remove it from
`store_track_r128_gain`, too.
fixes#3311
I'm not sure where these are used, but the website supports https and
the API url already uses https, so this should be a safe call and not
require a util.SNI_SUPPORTED check.
*All* URLs were checked manually, but only once per domain!
I mostly concerned myself with URLs in documentation rather than source
code because the latter may or may not have impactful changes, while the
former should be straight forward.
Changes in addition to simply adding an s:
- changed pip and pypi references as their location has changed
- MPoD (iOS app) url redirects to Regelian, so I replaced those
- updated homebrew references
Notable observations:
- beets.io does have HTTPS set up properly (via gh-pages)
- beatport.py uses the old HTTP url for beatport
- as does lyrics.py for lyrics.wikia.com
- https://tomahawk-player.org/ expired long ago, but the http page
redirects to https regardless
- none of the sourceforge subdomains have https (in 2019!)
Use Album.load() whenever the album is requested, which causes it to be
reloaded from the database.
Drawback: This adds a slowdown of 100% (6.2s to 12.6s) to `beet list`
on my setup.
The real MPD ignores `noidle` when the client is not idle. It doesn't
even send a successful response, just ignores the command. Although
I don't understand why a client would fail to keep track of its own
state, it seems that this is necessary to get ncmpcpp working.
The playlistid command is supposed to list the whole playlist if no
argument is provided, but we were accidentally trying to look up an
impossible negative id in that case causing an error to always be
returned.
As detailed here:
https://github.com/beetbox/beets/issues/2406#issuecomment-274423601
In a *function-style* definition, we didn't properly *un-define* the
values for a given item after each function invocation. So when a field
wasn't defined, it would get the value for the previously-formatted
object instead. It now properly throws a NameError.
This uses GStreamer APIs to extract a list of audio decoders and the
relevant MIME types and file extensions. Some clients like ncmpcpp use
this command to fetch a list of supported file extensions.
Some clients list the albums belonging to an artist by issuing the
command `list album <ARTIST NAME>`. This change inserts the tag `artist`
before the artist name so that this succeeds. Fixes#3007
A new `ControlConnection` is created each time a client connects over
a new control socket. This is used to forward events from the player,
and also for debugging utilities that are not part of the real MPD
protocol.
This new feature reuses as much infrastructure from the normal protocol
handling as possible (e.g. `Command` for parsing messages). While the
normal connection delegates to server `cmd_*` methods which are string
generators, the control connections delegate to `ctrl_*` methods defined
on the connection itself that are full coroutines.
Getting this command puts the connection into a special mode where it
awaits MPD events (like the player changing state or the playlist
changing due to other clients interacting with the server.
The MPD specification states that events should queue while a client is
connected, and when it issues the `idle` command any matching events
should be sent immediately if there are any, or as soon as they happen
otherwise.
Keep track of a list of currently-connected clients.
Use `socket.getpeername()` to get an identifier for each connection and
include this in each log message. This function is documented as not
being available on all systems, but it's unclear which systems this
involves.
Also log a message on client connect and disconnect events. If the
disconnection reason is because the client sent a blank line, match MPD
by returning a protocol error then hanging up. Escape curly braces.
Improved the method to get the path of the current song. Before, the complete playlist was fetched from the server. Now, the command "currentsong" is used for this purpose. This improves performance when a huge playlist is active.
Check function signature instead of using TypeError to crudely guess
that the wrong number of arguments were provided.
Prevent bpd from crashing when trying to log a traceback. The
`traceback.format_exc` function takes an optional argument which is
supposed to be an integer restricting the length of the backtrace to
show. Instead we were passing the exception object to this function and
causing a new exception to be raised.
Previously issuing the 'previous' command when at position 0 on the
playlist would cause bpd to stop playing. MPD instead just restarts the
currently playing song instead, so we now match this behaviour.
The songs are indexed starting from zero for the play command, however
the bound check was off by one. An index matching the length of the
playlist would crash the server instead of responding with an error
message over the protocol.
The repeat flag indicates that the entire playlist should be repeated.
If both the repeat and single flags are set then this triggers the old
behaviour of looping over a single track.
This command instructs bpd to stop playing when the current song
finishes. In the MPD 0.20 protocol this flag gains a value 'oneshot' but
for now we just support its older version with a boolean value.
The real MPD offers persistent playlist manipulation, storing the
playlists in a directory set in the config file. If that directory is
not available then the feature is disabled and the relevant commands all
respond with errors. Based on this, the initial support in bpd just
returns errors matching the MPD server in the disabled mode.
For playlistadd, extend the _bpd_add helper to work with playlists other
than the queue in order to support testing the real implementations of
these commands in the future.
There's a special status command for checking the replay gain mode,
which can be set to one of a short list of possible values. For now at
least we can ignore this feature, but track the setting anyway.
