Merge branch 'master' into replaygain

.github/ISSUE_TEMPLATE/bug-report.md

@@ -1,6 +1,17 @@
-### Problem
+---
+name: "\U0001F41B Bug report"
+about: Report a problem with beets
 
-(Describe your problem, feature request, or discussion topic here. If you're reporting a bug, please fill out this and the "Setup" section below. Otherwise, you can delete them.)
+---
+
+<!--
+Describe your problem, feature request, or discussion topic here.
+
+Please fill out this and the "Setup" section below and remember to include
+enough detail so that other people can reproduce the problem.
+-->
+
+### Problem
 
 Running this command in verbose (`-vv`) mode:
 

.github/ISSUE_TEMPLATE/feature-request.md

@@ -0,0 +1,26 @@
+---
+name: "\U0001F680 Feature request"
+about: Suggest a new idea for beets
+
+---
+
+### Use case
+
+I'm trying to use beets to...
+
+
+### Solution
+<!--
+Do you have a proposal for how beets should work?
+
+Try to be as specific as possible—for example, you could propose the name for
+a new command-line option or refer to the particular ID3 frame you wish
+were supported.
+-->
+
+
+### Alternatives
+<!--
+Have you tried using an existing plugin to do something similar?
+Is there any current feature that _almost_ does what you need?
+-->

.gitignore

@@ -89,3 +89,4 @@ ENV/
 /.project
 /.pydevproject
 /.settings
+.vscode

.travis.yml

@@ -24,18 +24,21 @@ matrix:
         - python: 3.7
           env: {TOX_ENV: py37-test}
           dist: xenial
+        # - python: 3.8-dev
+        #   env: {TOX_ENV: py38-test}
+        #   dist: xenial
         # - python: pypy
         #   - env: {TOX_ENV: pypy-test}
-        - python: 3.4
+        - python: 3.6
-          env: {TOX_ENV: py34-flake8}
+          env: {TOX_ENV: py36-flake8}
         - python: 2.7.13
           env: {TOX_ENV: docs}
 # Non-Python dependencies.
 addons:
     apt:
         sources:
-            - sourceline: "deb http://archive.ubuntu.com/ubuntu/ trusty multiverse"
+            - sourceline: "deb https://archive.ubuntu.com/ubuntu/ trusty multiverse"
-            - sourceline: "deb http://archive.ubuntu.com/ubuntu/ trusty-updates multiverse"
+            - sourceline: "deb https://archive.ubuntu.com/ubuntu/ trusty-updates multiverse"
         packages:
             - bash-completion
             - gir1.2-gst-plugins-base-1.0

README.rst

@@ -1,12 +1,15 @@
-.. image:: http://img.shields.io/pypi/v/beets.svg
+.. image:: https://img.shields.io/pypi/v/beets.svg
     :target: https://pypi.python.org/pypi/beets
 
-.. image:: http://img.shields.io/codecov/c/github/beetbox/beets.svg
+.. image:: https://img.shields.io/codecov/c/github/beetbox/beets.svg
     :target: https://codecov.io/github/beetbox/beets
 
 .. image:: https://travis-ci.org/beetbox/beets.svg?branch=master
     :target: https://travis-ci.org/beetbox/beets
 
+.. image:: https://repology.org/badge/tiny-repos/beets.svg
+    :target: https://repology.org/project/beets/versions
+
 
 beets
 =====
@@ -51,37 +54,39 @@ imagine for your music collection. Via `plugins`_, beets becomes a panacea:
 If beets doesn't do what you want yet, `writing your own plugin`_ is
 shockingly simple if you know a little Python.
 
-.. _plugins: http://beets.readthedocs.org/page/plugins/
+.. _plugins: https://beets.readthedocs.org/page/plugins/
-.. _MPD: http://www.musicpd.org/
+.. _MPD: https://www.musicpd.org/
-.. _MusicBrainz music collection: http://musicbrainz.org/doc/Collections/
+.. _MusicBrainz music collection: https://musicbrainz.org/doc/Collections/
 .. _writing your own plugin:
-    http://beets.readthedocs.org/page/dev/plugins.html
+    https://beets.readthedocs.org/page/dev/plugins.html
 .. _HTML5 Audio:
     http://www.w3.org/TR/html-markup/audio.html
 .. _albums that are missing tracks:
-    http://beets.readthedocs.org/page/plugins/missing.html
+    https://beets.readthedocs.org/page/plugins/missing.html
 .. _duplicate tracks and albums:
-    http://beets.readthedocs.org/page/plugins/duplicates.html
+    https://beets.readthedocs.org/page/plugins/duplicates.html
 .. _Transcode audio:
-    http://beets.readthedocs.org/page/plugins/convert.html
+    https://beets.readthedocs.org/page/plugins/convert.html
-.. _Discogs: http://www.discogs.com/
+.. _Discogs: https://www.discogs.com/
 .. _acoustic fingerprints:
-    http://beets.readthedocs.org/page/plugins/chroma.html
+    https://beets.readthedocs.org/page/plugins/chroma.html
-.. _ReplayGain: http://beets.readthedocs.org/page/plugins/replaygain.html
+.. _ReplayGain: https://beets.readthedocs.org/page/plugins/replaygain.html
-.. _tempos: http://beets.readthedocs.org/page/plugins/acousticbrainz.html
+.. _tempos: https://beets.readthedocs.org/page/plugins/acousticbrainz.html
-.. _genres: http://beets.readthedocs.org/page/plugins/lastgenre.html
+.. _genres: https://beets.readthedocs.org/page/plugins/lastgenre.html
-.. _album art: http://beets.readthedocs.org/page/plugins/fetchart.html
+.. _album art: https://beets.readthedocs.org/page/plugins/fetchart.html
-.. _lyrics: http://beets.readthedocs.org/page/plugins/lyrics.html
+.. _lyrics: https://beets.readthedocs.org/page/plugins/lyrics.html
-.. _MusicBrainz: http://musicbrainz.org/
+.. _MusicBrainz: https://musicbrainz.org/
 .. _Beatport: https://www.beatport.com
 
 Install
 -------
 
-You can install beets by typing ``pip install beets``. Then check out the
+You can install beets by typing ``pip install beets``.
-`Getting Started`_ guide.
+Beets has also been packaged in the `software repositories`_ of several distributions.
+Check out the `Getting Started`_ guide for more information.
 
-.. _Getting Started: http://beets.readthedocs.org/page/guides/main.html
+.. _Getting Started: https://beets.readthedocs.org/page/guides/main.html
+.. _software repositories: https://repology.org/project/beets/versions
 
 Contribute
 ----------
@@ -90,7 +95,7 @@ Check out the `Hacking`_ page on the wiki for tips on how to help out.
 You might also be interested in the `For Developers`_ section in the docs.
 
 .. _Hacking: https://github.com/beetbox/beets/wiki/Hacking
-.. _For Developers: http://docs.beets.io/page/dev/
+.. _For Developers: https://beets.readthedocs.io/en/stable/dev/
 
 Read More
 ---------
@@ -98,8 +103,8 @@ Read More
 Learn more about beets at `its Web site`_. Follow `@b33ts`_ on Twitter for
 news and updates.
 
-.. _its Web site: http://beets.io/
+.. _its Web site: https://beets.io/
-.. _@b33ts: http://twitter.com/b33ts/
+.. _@b33ts: https://twitter.com/b33ts/
 
 Authors
 -------
@@ -108,4 +113,4 @@ Beets is by `Adrian Sampson`_ with a supporting cast of thousands. For help,
 please visit our `forum`_.
 
 .. _forum: https://discourse.beets.io
-.. _Adrian Sampson: http://www.cs.cornell.edu/~asampson/
+.. _Adrian Sampson: https://www.cs.cornell.edu/~asampson/

README_kr.rst

@@ -1,7 +1,7 @@
-.. image:: http://img.shields.io/pypi/v/beets.svg
+.. image:: https://img.shields.io/pypi/v/beets.svg
     :target: https://pypi.python.org/pypi/beets
 
-.. image:: http://img.shields.io/codecov/c/github/beetbox/beets.svg
+.. image:: https://img.shields.io/codecov/c/github/beetbox/beets.svg
     :target: https://codecov.io/github/beetbox/beets
 
 .. image:: https://travis-ci.org/beetbox/beets.svg?branch=master
@@ -48,28 +48,28 @@ Beets는 라이브러리로 디자인 되었기 때문에, 당신이 음악들
 만약 Beets에 당신이 원하는게 아직 없다면,
 당신이 python을 안다면 `writing your own plugin`_ _은 놀라울정도로 간단하다.
 
-.. _plugins: http://beets.readthedocs.org/page/plugins/
+.. _plugins: https://beets.readthedocs.org/page/plugins/
-.. _MPD: http://www.musicpd.org/
+.. _MPD: https://www.musicpd.org/
-.. _MusicBrainz music collection: http://musicbrainz.org/doc/Collections/
+.. _MusicBrainz music collection: https://musicbrainz.org/doc/Collections/
 .. _writing your own plugin:
-    http://beets.readthedocs.org/page/dev/plugins.html
+    https://beets.readthedocs.org/page/dev/plugins.html
 .. _HTML5 Audio:
     http://www.w3.org/TR/html-markup/audio.html
 .. _albums that are missing tracks:
-    http://beets.readthedocs.org/page/plugins/missing.html
+    https://beets.readthedocs.org/page/plugins/missing.html
 .. _duplicate tracks and albums:
-    http://beets.readthedocs.org/page/plugins/duplicates.html
+    https://beets.readthedocs.org/page/plugins/duplicates.html
 .. _Transcode audio:
-    http://beets.readthedocs.org/page/plugins/convert.html
+    https://beets.readthedocs.org/page/plugins/convert.html
-.. _Discogs: http://www.discogs.com/
+.. _Discogs: https://www.discogs.com/
 .. _acoustic fingerprints:
-    http://beets.readthedocs.org/page/plugins/chroma.html
+    https://beets.readthedocs.org/page/plugins/chroma.html
-.. _ReplayGain: http://beets.readthedocs.org/page/plugins/replaygain.html
+.. _ReplayGain: https://beets.readthedocs.org/page/plugins/replaygain.html
-.. _tempos: http://beets.readthedocs.org/page/plugins/acousticbrainz.html
+.. _tempos: https://beets.readthedocs.org/page/plugins/acousticbrainz.html
-.. _genres: http://beets.readthedocs.org/page/plugins/lastgenre.html
+.. _genres: https://beets.readthedocs.org/page/plugins/lastgenre.html
-.. _album art: http://beets.readthedocs.org/page/plugins/fetchart.html
+.. _album art: https://beets.readthedocs.org/page/plugins/fetchart.html
-.. _lyrics: http://beets.readthedocs.org/page/plugins/lyrics.html
+.. _lyrics: https://beets.readthedocs.org/page/plugins/lyrics.html
-.. _MusicBrainz: http://musicbrainz.org/
+.. _MusicBrainz: https://musicbrainz.org/
 .. _Beatport: https://www.beatport.com
 
 설치
@@ -78,7 +78,7 @@ Beets는 라이브러리로 디자인 되었기 때문에, 당신이 음악들
 당신은 ``pip install beets`` 을 사용해서 Beets를 설치할 수 있다.
 그리고 `Getting Started`_ 가이드를 확인할 수 있다.
 
-.. _Getting Started: http://beets.readthedocs.org/page/guides/main.html
+.. _Getting Started: https://beets.readthedocs.org/page/guides/main.html
 
 컨트리뷰션
 ----------
@@ -87,7 +87,7 @@ Beets는 라이브러리로 디자인 되었기 때문에, 당신이 음악들
 당신은 docs 안에 `For Developers`_ 에도 관심이 있을수 있다.
 
 .. _Hacking: https://github.com/beetbox/beets/wiki/Hacking
-.. _For Developers: http://docs.beets.io/page/dev/
+.. _For Developers: https://beets.readthedocs.io/en/stable/dev/
 
 Read More
 ---------
@@ -95,8 +95,8 @@ Read More
 `its Web site`_ 에서 Beets에 대해 조금 더 알아볼 수 있다.
 트위터에서 `@b33ts`_ 를 팔로우하면 새 소식을 볼 수 있다.
 
-.. _its Web site: http://beets.io/
+.. _its Web site: https://beets.io/
-.. _@b33ts: http://twitter.com/b33ts/
+.. _@b33ts: https://twitter.com/b33ts/
 
 저자들
 -------
@@ -105,4 +105,4 @@ Read More
 돕고 싶다면 `forum`_.를 방문하면 된다.
 
 .. _forum: https://discourse.beets.io
-.. _Adrian Sampson: http://www.cs.cornell.edu/~asampson/
+.. _Adrian Sampson: https://www.cs.cornell.edu/~asampson/

appveyor.yml

@@ -16,14 +16,16 @@ environment:
           TOX_ENV: py35-test
         - PYTHON: C:\Python36
           TOX_ENV: py36-test
+        - PYTHON: C:\Python37
+          TOX_ENV: py37-test
 
 # Install Tox for running tests.
 install:
-    - cinst imagemagick -y
+    - appveyor-retry cinst imagemagick -y
     # TODO: remove --allow-empty-checksums when unrar offers a proper checksum
-    - cinst unrar -y --allow-empty-checksums
+    - appveyor-retry cinst unrar -y --allow-empty-checksums
-    - "%PYTHON%/Scripts/pip.exe install tox"
+    - 'appveyor-retry %PYTHON%/Scripts/pip.exe install "tox<=3.8.1"'
-    - "%PYTHON%/Scripts/tox.exe -e %TOX_ENV% --notest"
+    - "appveyor-retry %PYTHON%/Scripts/tox.exe -e %TOX_ENV% --notest"
 
 test_script:
     - "%PYTHON%/Scripts/tox.exe -e %TOX_ENV%"

beets/__init__.py

@@ -17,14 +17,14 @@ from __future__ import division, absolute_import, print_function
 
 import os
 
-from beets.util import confit
+import confuse
 
-__version__ = u'1.4.8'
+__version__ = u'1.5.0'
 __author__ = u'Adrian Sampson <adrian@radbox.org>'
 
 
-class IncludeLazyConfig(confit.LazyConfig):
+class IncludeLazyConfig(confuse.LazyConfig):
-    """A version of Confit's LazyConfig that also merges in data from
+    """A version of Confuse's LazyConfig that also merges in data from
     YAML files specified in an `include` setting.
     """
     def read(self, user=True, defaults=True):
@@ -35,7 +35,7 @@ class IncludeLazyConfig(confit.LazyConfig):
                 filename = view.as_filename()
                 if os.path.isfile(filename):
                     self.set_file(filename)
-        except confit.NotFoundError:
+        except confuse.NotFoundError:
             pass
 
 

beets/art.py

@@ -26,7 +26,7 @@ import os
 
 from beets.util import displayable_path, syspath, bytestring_path
 from beets.util.artresizer import ArtResizer
-from beets import mediafile
+import mediafile
 
 
 def mediafile_image(image_path, maxwidth=None):
@@ -51,7 +51,8 @@ def get_art(log, item):
 
 
 def embed_item(log, item, imagepath, maxwidth=None, itempath=None,
-               compare_threshold=0, ifempty=False, as_album=False):
+               compare_threshold=0, ifempty=False, as_album=False,
+               id3v23=None):
     """Embed an image into the item's media file.
     """
     # Conditions and filters.
@@ -80,7 +81,7 @@ def embed_item(log, item, imagepath, maxwidth=None, itempath=None,
                  image.mime_type)
         return
 
-    item.try_write(path=itempath, tags={'images': [image]})
+    item.try_write(path=itempath, tags={'images': [image]}, id3v23=id3v23)
 
 
 def embed_album(log, album, maxwidth=None, quiet=False,

beets/autotag/__init__.py

@@ -54,6 +54,12 @@ def apply_item_metadata(item, track_info):
         item.composer_sort = track_info.composer_sort
     if track_info.arranger is not None:
         item.arranger = track_info.arranger
+    if track_info.work is not None:
+        item.work = track_info.work
+    if track_info.mb_workid is not None:
+        item.mb_workid = track_info.mb_workid
+    if track_info.work_disambig is not None:
+        item.work_disambig = track_info.work_disambig
 
     # At the moment, the other metadata is left intact (including album
     # and track number). Perhaps these should be emptied?
@@ -167,6 +173,9 @@ def apply_metadata(album_info, mapping):
                 'composer',
                 'composer_sort',
                 'arranger',
+                'work',
+                'mb_workid',
+                'work_disambig',
             )
         }
 

beets/autotag/hooks.py

@@ -159,6 +159,9 @@ class TrackInfo(object):
     - ``composer_sort``: individual track composer sort name
     - ``arranger`: individual track arranger name
     - ``track_alt``: alternative track number (tape, vinyl, etc.)
+    - ``work`: individual track work title
+    - ``mb_workid`: individual track work id
+    - ``work_disambig`: individual track work diambiguation
 
     Only ``title`` and ``track_id`` are required. The rest of the fields
     may be None. The indices ``index``, ``medium``, and ``medium_index``
@@ -169,7 +172,8 @@ class TrackInfo(object):
                  medium_index=None, medium_total=None, artist_sort=None,
                  disctitle=None, artist_credit=None, data_source=None,
                  data_url=None, media=None, lyricist=None, composer=None,
-                 composer_sort=None, arranger=None, track_alt=None):
+                 composer_sort=None, arranger=None, track_alt=None,
+                 work=None, mb_workid=None, work_disambig=None):
         self.title = title
         self.track_id = track_id
         self.release_track_id = release_track_id
@@ -191,6 +195,9 @@ class TrackInfo(object):
         self.composer_sort = composer_sort
         self.arranger = arranger
         self.track_alt = track_alt
+        self.work = work
+        self.mb_workid = mb_workid
+        self.work_disambig = work_disambig
 
     # As above, work around a bug in python-musicbrainz-ngs.
     def decode(self, codec='utf-8'):

beets/autotag/mb.py

@@ -39,7 +39,7 @@ else:
 SKIPPED_TRACKS = ['[data track]']
 
 musicbrainzngs.set_useragent('beets', beets.__version__,
-                             'http://beets.io/')
+                             'https://beets.io/')
 
 
 class MusicBrainzAPIError(util.HumanReadableException):
@@ -213,6 +213,11 @@ def track_info(recording, index=None, medium=None, medium_index=None,
     for work_relation in recording.get('work-relation-list', ()):
         if work_relation['type'] != 'performance':
             continue
+        info.work = work_relation['work']['title']
+        info.mb_workid = work_relation['work']['id']
+        if 'disambiguation' in work_relation['work']:
+            info.work_disambig = work_relation['work']['disambiguation']
+
         for artist_relation in work_relation['work'].get(
                 'artist-relation-list', ()):
             if 'type' in artist_relation:

beets/dbcore/db.py

@@ -23,14 +23,17 @@ from collections import defaultdict
 import threading
 import sqlite3
 import contextlib
-import collections
 
 import beets
-from beets.util.functemplate import Template
+from beets.util import functemplate
 from beets.util import py3_path
 from beets.dbcore import types
 from .query import MatchQuery, NullSort, TrueQuery
 import six
+if six.PY2:
+    from collections import Mapping
+else:
+    from collections.abc import Mapping
 
 
 class DBAccessError(Exception):
@@ -42,7 +45,7 @@ class DBAccessError(Exception):
     """
 
 
-class FormattedMapping(collections.Mapping):
+class FormattedMapping(Mapping):
     """A `dict`-like formatted view of a model.
 
     The accessor `mapping[key]` returns the formatted version of
@@ -88,6 +91,100 @@ class FormattedMapping(collections.Mapping):
         return value
 
 
+class LazyConvertDict(object):
+    """Lazily convert types for attributes fetched from the database
+    """
+
+    def __init__(self, model_cls):
+        """Initialize the object empty
+        """
+        self.data = {}
+        self.model_cls = model_cls
+        self._converted = {}
+
+    def init(self, data):
+        """Set the base data that should be lazily converted
+        """
+        self.data = data
+
+    def _convert(self, key, value):
+        """Convert the attribute type according the the SQL type
+        """
+        return self.model_cls._type(key).from_sql(value)
+
+    def __setitem__(self, key, value):
+        """Set an attribute value, assume it's already converted
+        """
+        self._converted[key] = value
+
+    def __getitem__(self, key):
+        """Get an attribute value, converting the type on demand
+        if needed
+        """
+        if key in self._converted:
+            return self._converted[key]
+        elif key in self.data:
+            value = self._convert(key, self.data[key])
+            self._converted[key] = value
+            return value
+
+    def __delitem__(self, key):
+        """Delete both converted and base data
+        """
+        if key in self._converted:
+            del self._converted[key]
+        if key in self.data:
+            del self.data[key]
+
+    def keys(self):
+        """Get a list of available field names for this object.
+        """
+        return list(self._converted.keys()) + list(self.data.keys())
+
+    def copy(self):
+        """Create a copy of the object.
+        """
+        new = self.__class__(self.model_cls)
+        new.data = self.data.copy()
+        new._converted = self._converted.copy()
+        return new
+
+    # Act like a dictionary.
+
+    def update(self, values):
+        """Assign all values in the given dict.
+        """
+        for key, value in values.items():
+            self[key] = value
+
+    def items(self):
+        """Iterate over (key, value) pairs that this object contains.
+        Computed fields are not included.
+        """
+        for key in self:
+            yield key, self[key]
+
+    def get(self, key, default=None):
+        """Get the value for a given key or `default` if it does not
+        exist.
+        """
+        if key in self:
+            return self[key]
+        else:
+            return default
+
+    def __contains__(self, key):
+        """Determine whether `key` is an attribute on this object.
+        """
+        return key in self.keys()
+
+    def __iter__(self):
+        """Iterate over the available field names (excluding computed
+        fields).
+        """
+        return iter(self.keys())
+
+
 # Abstract base for model classes.
 
 class Model(object):
@@ -143,6 +240,11 @@ class Model(object):
     are subclasses of `Sort`.
     """
 
+    _queries = {}
+    """Named queries that use a field-like `name:value` syntax but which
+    do not relate to any specific field.
+    """
+
     _always_dirty = False
     """By default, fields only become "dirty" when their value actually
     changes. Enabling this flag marks fields as dirty even when the new
@@ -172,8 +274,8 @@ class Model(object):
         """
         self._db = db
         self._dirty = set()
-        self._values_fixed = {}
+        self._values_fixed = LazyConvertDict(self)
-        self._values_flex = {}
+        self._values_flex = LazyConvertDict(self)
 
         # Initial contents.
         self.update(values)
@@ -187,10 +289,10 @@ class Model(object):
         ordinary construction are bypassed.
         """
         obj = cls(db)
-        for key, value in fixed_values.items():
+
-            obj._values_fixed[key] = cls._type(key).from_sql(value)
+        obj._values_fixed.init(fixed_values)
-        for key, value in flex_values.items():
+        obj._values_flex.init(flex_values)
-            obj._values_flex[key] = cls._type(key).from_sql(value)
+
         return obj
 
     def __repr__(self):
@@ -251,7 +353,10 @@ class Model(object):
         if key in getters:  # Computed.
             return getters[key](self)
         elif key in self._fields:  # Fixed.
-            return self._values_fixed.get(key, self._type(key).null)
+            if key in self._values_fixed:
+                return self._values_fixed[key]
+            else:
+                return self._type(key).null
         elif key in self._values_flex:  # Flexible.
             return self._values_flex[key]
         else:
@@ -431,8 +536,8 @@ class Model(object):
         self._check_db()
         stored_obj = self._db._get(type(self), self.id)
         assert stored_obj is not None, u"object {0} not in DB".format(self.id)
-        self._values_fixed = {}
+        self._values_fixed = LazyConvertDict(self)
-        self._values_flex = {}
+        self._values_flex = LazyConvertDict(self)
         self.update(dict(stored_obj))
         self.clear_dirty()
 
@@ -492,7 +597,7 @@ class Model(object):
         """
         # Perform substitution.
         if isinstance(template, six.string_types):
-            template = Template(template)
+            template = functemplate.template(template)
         return template.substitute(self.formatted(for_path),
                                    self._template_funcs())
 
@@ -519,7 +624,8 @@ class Results(object):
     """An item query result set. Iterating over the collection lazily
     constructs LibModel objects that reflect database rows.
     """
-    def __init__(self, model_class, rows, db, query=None, sort=None):
+    def __init__(self, model_class, rows, db, flex_rows,
+                 query=None, sort=None):
         """Create a result set that will construct objects of type
         `model_class`.
 
@@ -539,6 +645,7 @@ class Results(object):
         self.db = db
         self.query = query
         self.sort = sort
+        self.flex_rows = flex_rows
 
         # We keep a queue of rows we haven't yet consumed for
         # materialization. We preserve the original total number of
@@ -560,6 +667,10 @@ class Results(object):
         a `Results` object a second time should be much faster than the
         first.
         """
+
+        # Index flexible attributes by the item ID, so we have easier access
+        flex_attrs = self._get_indexed_flex_attrs()
+
         index = 0  # Position in the materialized objects.
         while index < len(self._objects) or self._rows:
             # Are there previously-materialized objects to produce?
@@ -572,7 +683,7 @@ class Results(object):
             else:
                 while self._rows:
                     row = self._rows.pop(0)
-                    obj = self._make_model(row)
+                    obj = self._make_model(row, flex_attrs.get(row['id'], {}))
                     # If there is a slow-query predicate, ensurer that the
                     # object passes it.
                     if not self.query or self.query.match(obj):
@@ -594,20 +705,24 @@ class Results(object):
             # Objects are pre-sorted (i.e., by the database).
             return self._get_objects()
 
-    def _make_model(self, row):
+    def _get_indexed_flex_attrs(self):
-        # Get the flexible attributes for the object.
+        """ Index flexible attributes by the entity id they belong to
-        with self.db.transaction() as tx:
+        """
-            flex_rows = tx.query(
+        flex_values = dict()
-                'SELECT * FROM {0} WHERE entity_id=?'.format(
+        for row in self.flex_rows:
-                    self.model_class._flex_table
+            if row['entity_id'] not in flex_values:
-                ),
+                flex_values[row['entity_id']] = dict()
-                (row['id'],)
-            )
 
+            flex_values[row['entity_id']][row['key']] = row['value']
+
+        return flex_values
+
+    def _make_model(self, row, flex_values={}):
+        """ Create a Model object for the given row
+        """
         cols = dict(row)
         values = dict((k, v) for (k, v) in cols.items()
                       if not k[:4] == 'flex')
-        flex_values = dict((row['key'], row['value']) for row in flex_rows)
 
         # Construct the Python object
         obj = self.model_class._awaken(self.db, values, flex_values)
@@ -735,16 +850,21 @@ class Database(object):
     """A container for Model objects that wraps an SQLite database as
     the backend.
     """
+
     _models = ()
     """The Model subclasses representing tables in this database.
     """
 
+    supports_extensions = hasattr(sqlite3.Connection, 'enable_load_extension')
+    """Whether or not the current version of SQLite supports extensions"""
+
     def __init__(self, path, timeout=5.0):
         self.path = path
         self.timeout = timeout
 
         self._connections = {}
         self._tx_stacks = defaultdict(list)
+        self._extensions = []
 
         # A lock to protect the _connections and _tx_stacks maps, which
         # both map thread IDs to private resources.
@@ -794,6 +914,13 @@ class Database(object):
             py3_path(self.path), timeout=self.timeout
         )
 
+        if self.supports_extensions:
+            conn.enable_load_extension(True)
+
+            # Load any extension that are already loaded for other connections.
+            for path in self._extensions:
+                conn.load_extension(path)
+
         # Access SELECT results like dictionaries.
         conn.row_factory = sqlite3.Row
         return conn
@@ -822,6 +949,18 @@ class Database(object):
         """
         return Transaction(self)
 
+    def load_extension(self, path):
+        """Load an SQLite extension into all open connections."""
+        if not self.supports_extensions:
+            raise ValueError(
+                    'this sqlite3 installation does not support extensions')
+
+        self._extensions.append(path)
+
+        # Load the extension into every open connection.
+        for conn in self._connections.values():
+            conn.load_extension(path)
+
     # Schema setup and migration.
 
     def _make_table(self, table, fields):
@@ -894,11 +1033,25 @@ class Database(object):
             "ORDER BY {0}".format(order_by) if order_by else '',
         )
 
+        # Fetch flexible attributes for items matching the main query.
+        # Doing the per-item filtering in python is faster than issuing
+        # one query per item to sqlite.
+        flex_sql = ("""
+            SELECT * FROM {0} WHERE entity_id IN
+                (SELECT id FROM {1} WHERE {2});
+            """.format(
+                model_cls._flex_table,
+                model_cls._table,
+                where or '1',
+            )
+        )
+
         with self.transaction() as tx:
             rows = tx.query(sql, subvals)
+            flex_rows = tx.query(flex_sql, subvals)
 
         return Results(
-            model_cls, rows, self,
+            model_cls, rows, self, flex_rows,
             None if where else query,  # Slow query component.
             sort if sort.is_slow() else None,  # Slow sort component.
         )

beets/dbcore/queryparse.py

@@ -20,7 +20,6 @@ from __future__ import division, absolute_import, print_function
 import re
 import itertools
 from . import query
-import beets
 
 PARSE_QUERY_PART_REGEX = re.compile(
     # Non-capturing optional segment for the keyword.
@@ -119,12 +118,13 @@ def construct_query_part(model_cls, prefixes, query_part):
     if not query_part:
         return query.TrueQuery()
 
-    # Use `model_cls` to build up a map from field names to `Query`
+    # Use `model_cls` to build up a map from field (or query) names to
-    # classes.
+    # `Query` classes.
     query_classes = {}
     for k, t in itertools.chain(model_cls._fields.items(),
                                 model_cls._types.items()):
         query_classes[k] = t.query
+    query_classes.update(model_cls._queries)  # Non-field queries.
 
     # Parse the string.
     key, pattern, query_class, negate = \
@@ -137,26 +137,27 @@ def construct_query_part(model_cls, prefixes, query_part):
             # The query type matches a specific field, but none was
             # specified. So we use a version of the query that matches
             # any field.
-            q = query.AnyFieldQuery(pattern, model_cls._search_fields,
+            out_query = query.AnyFieldQuery(pattern, model_cls._search_fields,
                                             query_class)
-            if negate:
-                return query.NotQuery(q)
-            else:
-                return q
         else:
             # Non-field query type.
-            if negate:
+            out_query = query_class(pattern)
-                return query.NotQuery(query_class(pattern))
-            else:
-                return query_class(pattern)
 
-    # Otherwise, this must be a `FieldQuery`. Use the field name to
+    # Field queries get constructed according to the name of the field
-    # construct the query object.
+    # they are querying.
+    elif issubclass(query_class, query.FieldQuery):
         key = key.lower()
-    q = query_class(key.lower(), pattern, key in model_cls._fields)
+        out_query = query_class(key.lower(), pattern, key in model_cls._fields)
+
+    # Non-field (named) query.
+    else:
+        out_query = query_class(pattern)
+
+    # Apply negation.
     if negate:
-        return query.NotQuery(q)
+        return query.NotQuery(out_query)
-    return q
+    else:
+        return out_query
 
 
 def query_from_strings(query_cls, model_cls, prefixes, query_parts):
@@ -172,11 +173,13 @@ def query_from_strings(query_cls, model_cls, prefixes, query_parts):
     return query_cls(subqueries)
 
 
-def construct_sort_part(model_cls, part):
+def construct_sort_part(model_cls, part, case_insensitive=True):
     """Create a `Sort` from a single string criterion.
 
     `model_cls` is the `Model` being queried. `part` is a single string
-    ending in ``+`` or ``-`` indicating the sort.
+    ending in ``+`` or ``-`` indicating the sort. `case_insensitive`
+    indicates whether or not the sort should be performed in a case
+    sensitive manner.
     """
     assert part, "part must be a field name and + or -"
     field = part[:-1]
@@ -185,7 +188,6 @@ def construct_sort_part(model_cls, part):
     assert direction in ('+', '-'), "part must end with + or -"
     is_ascending = direction == '+'
 
-    case_insensitive = beets.config['sort_case_insensitive'].get(bool)
     if field in model_cls._sorts:
         sort = model_cls._sorts[field](model_cls, is_ascending,
                                        case_insensitive)
@@ -197,21 +199,23 @@ def construct_sort_part(model_cls, part):
     return sort
 
 
-def sort_from_strings(model_cls, sort_parts):
+def sort_from_strings(model_cls, sort_parts, case_insensitive=True):
     """Create a `Sort` from a list of sort criteria (strings).
     """
     if not sort_parts:
         sort = query.NullSort()
     elif len(sort_parts) == 1:
-        sort = construct_sort_part(model_cls, sort_parts[0])
+        sort = construct_sort_part(model_cls, sort_parts[0], case_insensitive)
     else:
         sort = query.MultipleSort()
         for part in sort_parts:
-            sort.add_sort(construct_sort_part(model_cls, part))
+            sort.add_sort(construct_sort_part(model_cls, part,
+                                              case_insensitive))
     return sort
 
 
-def parse_sorted_query(model_cls, parts, prefixes={}):
+def parse_sorted_query(model_cls, parts, prefixes={},
+                       case_insensitive=True):
     """Given a list of strings, create the `Query` and `Sort` that they
     represent.
     """
@@ -246,5 +250,5 @@ def parse_sorted_query(model_cls, parts, prefixes={}):
 
     # Avoid needlessly wrapping single statements in an OR
     q = query.OrQuery(query_parts) if len(query_parts) > 1 else query_parts[0]
-    s = sort_from_strings(model_cls, sort_parts)
+    s = sort_from_strings(model_cls, sort_parts, case_insensitive)
     return q, s

beets/dbcore/types.py

@@ -97,7 +97,7 @@ class Type(object):
         For fixed fields the type of `value` is determined by the column
         type affinity given in the `sql` property and the SQL to Python
         mapping of the database adapter. For more information see:
-        http://www.sqlite.org/datatype3.html
+        https://www.sqlite.org/datatype3.html
         https://docs.python.org/2/library/sqlite3.html#sqlite-and-python-types
 
         Flexible fields have the type affinity `TEXT`. This means the
@@ -173,14 +173,18 @@ class Id(Integer):
 
 
 class Float(Type):
-    """A basic floating-point type.
+    """A basic floating-point type. The `digits` parameter specifies how
+    many decimal places to use in the human-readable representation.
     """
     sql = u'REAL'
     query = query.NumericQuery
     model_type = float
 
+    def __init__(self, digits=1):
+        self.digits = digits
+
     def format(self, value):
-        return u'{0:.1f}'.format(value or 0.0)
+        return u'{0:.{1}f}'.format(value or 0, self.digits)
 
 
 class NullFloat(Float):

beets/importer.py

@@ -40,7 +40,7 @@ from beets import config
 from beets.util import pipeline, sorted_walk, ancestry, MoveOperation
 from beets.util import syspath, normpath, displayable_path
 from enum import Enum
-from beets import mediafile
+import mediafile
 
 action = Enum('action',
               ['SKIP', 'ASIS', 'TRACKS', 'APPLY', 'ALBUMS', 'RETAG'])

beets/library.py

@@ -26,12 +26,12 @@ import six
 import string
 
 from beets import logging
-from beets.mediafile import MediaFile, UnreadableFileError
+from mediafile import MediaFile, UnreadableFileError
 from beets import plugins
 from beets import util
 from beets.util import bytestring_path, syspath, normpath, samefile, \
-    MoveOperation
+    MoveOperation, lazy_property
-from beets.util.functemplate import Template
+from beets.util.functemplate import template, Template
 from beets import dbcore
 from beets.dbcore import types
 import beets
@@ -376,13 +376,25 @@ class FormattedItemMapping(dbcore.db.FormattedMapping):
 
     def __init__(self, item, for_path=False):
         super(FormattedItemMapping, self).__init__(item, for_path)
-        self.album = item.get_album()
+        self.item = item
-        self.album_keys = []
+
+    @lazy_property
+    def all_keys(self):
+        return set(self.model_keys).union(self.album_keys)
+
+    @lazy_property
+    def album_keys(self):
+        album_keys = []
         if self.album:
             for key in self.album.keys(True):
-                if key in Album.item_keys or key not in item._fields.keys():
+                if key in Album.item_keys \
-                    self.album_keys.append(key)
+                        or key not in self.item._fields.keys():
-        self.all_keys = set(self.model_keys).union(self.album_keys)
+                    album_keys.append(key)
+        return album_keys
+
+    @lazy_property
+    def album(self):
+        return self.item.get_album()
 
     def _get(self, key):
         """Get the value for a key, either from the album or the item.
@@ -439,6 +451,9 @@ class Item(LibModel):
         'lyricist':             types.STRING,
         'composer':             types.STRING,
         'composer_sort':        types.STRING,
+        'work':                 types.STRING,
+        'mb_workid':            types.STRING,
+        'work_disambig':        types.STRING,
         'arranger':             types.STRING,
         'grouping':             types.STRING,
         'year':                 types.PaddedInt(4),
@@ -611,7 +626,7 @@ class Item(LibModel):
 
         self.path = read_path
 
-    def write(self, path=None, tags=None):
+    def write(self, path=None, tags=None, id3v23=None):
         """Write the item's metadata to a media file.
 
         All fields in `_media_fields` are written to disk according to
@@ -623,6 +638,9 @@ class Item(LibModel):
         `tags` is a dictionary of additional metadata the should be
         written to the file. (These tags need not be in `_media_fields`.)
 
+        `id3v23` will override the global `id3v23` config option if it is
+        set to something other than `None`.
+
         Can raise either a `ReadError` or a `WriteError`.
         """
         if path is None:
@@ -630,6 +648,9 @@ class Item(LibModel):
         else:
             path = normpath(path)
 
+        if id3v23 is None:
+            id3v23 = beets.config['id3v23'].get(bool)
+
         # Get the data to write to the file.
         item_tags = dict(self)
         item_tags = {k: v for k, v in item_tags.items()
@@ -640,8 +661,7 @@ class Item(LibModel):
 
         # Open the file.
         try:
-            mediafile = MediaFile(syspath(path),
+            mediafile = MediaFile(syspath(path), id3v23=id3v23)
-                                  id3v23=beets.config['id3v23'].get(bool))
         except UnreadableFileError as exc:
             raise ReadError(path, exc)
 
@@ -657,14 +677,14 @@ class Item(LibModel):
             self.mtime = self.current_mtime()
         plugins.send('after_write', item=self, path=path)
 
-    def try_write(self, path=None, tags=None):
+    def try_write(self, *args, **kwargs):
         """Calls `write()` but catches and logs `FileOperationError`
         exceptions.
 
         Returns `False` an exception was caught and `True` otherwise.
         """
         try:
-            self.write(path, tags)
+            self.write(*args, **kwargs)
             return True
         except FileOperationError as exc:
             log.error(u"{0}", exc)
@@ -850,7 +870,7 @@ class Item(LibModel):
         if isinstance(path_format, Template):
             subpath_tmpl = path_format
         else:
-            subpath_tmpl = Template(path_format)
+            subpath_tmpl = template(path_format)
 
         # Evaluate the selected template.
         subpath = self.evaluate_template(subpath_tmpl, True)
@@ -930,7 +950,7 @@ class Album(LibModel):
         'releasegroupdisambig': types.STRING,
         'rg_album_gain':        types.NULL_FLOAT,
         'rg_album_peak':        types.NULL_FLOAT,
-        'r128_album_gain':      types.PaddedInt(6),
+        'r128_album_gain':      types.NullPaddedInt(6),
         'original_year':        types.PaddedInt(4),
         'original_month':       types.PaddedInt(2),
         'original_day':         types.PaddedInt(2),
@@ -1129,7 +1149,7 @@ class Album(LibModel):
         image = bytestring_path(image)
         item_dir = item_dir or self.item_dir()
 
-        filename_tmpl = Template(
+        filename_tmpl = template(
             beets.config['art_filename'].as_str())
         subpath = self.evaluate_template(filename_tmpl, True)
         if beets.config['asciify_paths']:
@@ -1234,8 +1254,10 @@ def parse_query_parts(parts, model_cls):
         else:
             non_path_parts.append(s)
 
+    case_insensitive = beets.config['sort_case_insensitive'].get(bool)
+
     query, sort = dbcore.parse_sorted_query(
-        model_cls, non_path_parts, prefixes
+        model_cls, non_path_parts, prefixes, case_insensitive
     )
 
     # Add path queries to aggregate query.

beets/plugins.py

@@ -17,16 +17,16 @@
 
 from __future__ import division, absolute_import, print_function
 
-import inspect
 import traceback
 import re
+import inspect
 from collections import defaultdict
 from functools import wraps
 
 
 import beets
 from beets import logging
-from beets import mediafile
+import mediafile
 import six
 
 PLUGIN_NAMESPACE = 'beetsplug'
@@ -127,7 +127,10 @@ class BeetsPlugin(object):
         value after the function returns). Also determines which params may not
         be sent for backwards-compatibility.
         """
-        argspec = inspect.getargspec(func)
+        if six.PY2:
+            func_args = inspect.getargspec(func).args
+        else:
+            func_args = inspect.getfullargspec(func).args
 
         @wraps(func)
         def wrapper(*args, **kwargs):
@@ -142,7 +145,7 @@ class BeetsPlugin(object):
                     if exc.args[0].startswith(func.__name__):
                         # caused by 'func' and not stuff internal to 'func'
                         kwargs = dict((arg, val) for arg, val in kwargs.items()
-                                      if arg in argspec.args)
+                                      if arg in func_args)
                         return func(*args, **kwargs)
                     else:
                         raise
@@ -344,6 +347,16 @@ def types(model_cls):
     return types
 
 
+def named_queries(model_cls):
+    # Gather `item_queries` and `album_queries` from the plugins.
+    attr_name = '{0}_queries'.format(model_cls.__name__.lower())
+    queries = {}
+    for plugin in find_plugins():
+        plugin_queries = getattr(plugin, attr_name, {})
+        queries.update(plugin_queries)
+    return queries
+
+
 def track_distance(item, info):
     """Gets the track distance calculated by all loaded plugins.
     Returns a Distance object.
@@ -513,7 +526,7 @@ def sanitize_choices(choices, choices_all):
 
 def sanitize_pairs(pairs, pairs_all):
     """Clean up a single-element mapping configuration attribute as returned
-    by `confit`'s `Pairs` template: keep only two-element tuples present in
+    by Confuse's `Pairs` template: keep only two-element tuples present in
     pairs_all, remove duplicate elements, expand ('str', '*') and ('*', '*')
     wildcards while keeping the original order. Note that ('*', '*') and
     ('*', 'whatever') have the same effect.

beets/random.py

@@ -0,0 +1,115 @@
+# -*- coding: utf-8 -*-
+# This file is part of beets.
+# Copyright 2016, Philippe Mongeau.
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+
+"""Get a random song or album from the library.
+"""
+from __future__ import division, absolute_import, print_function
+
+import random
+from operator import attrgetter
+from itertools import groupby
+
+
+def _length(obj, album):
+    """Get the duration of an item or album.
+    """
+    if album:
+        return sum(i.length for i in obj.items())
+    else:
+        return obj.length
+
+
+def _equal_chance_permutation(objs, field='albumartist', random_gen=None):
+    """Generate (lazily) a permutation of the objects where every group
+    with equal values for `field` have an equal chance of appearing in
+    any given position.
+    """
+    rand = random_gen or random
+
+    # Group the objects by artist so we can sample from them.
+    key = attrgetter(field)
+    objs.sort(key=key)
+    objs_by_artists = {}
+    for artist, v in groupby(objs, key):
+        objs_by_artists[artist] = list(v)
+
+    # While we still have artists with music to choose from, pick one
+    # randomly and pick a track from that artist.
+    while objs_by_artists:
+        # Choose an artist and an object for that artist, removing
+        # this choice from the pool.
+        artist = rand.choice(list(objs_by_artists.keys()))
+        objs_from_artist = objs_by_artists[artist]
+        i = rand.randint(0, len(objs_from_artist) - 1)
+        yield objs_from_artist.pop(i)
+
+        # Remove the artist if we've used up all of its objects.
+        if not objs_from_artist:
+            del objs_by_artists[artist]
+
+
+def _take(iter, num):
+    """Return a list containing the first `num` values in `iter` (or
+    fewer, if the iterable ends early).
+    """
+    out = []
+    for val in iter:
+        out.append(val)
+        num -= 1
+        if num <= 0:
+            break
+    return out
+
+
+def _take_time(iter, secs, album):
+    """Return a list containing the first values in `iter`, which should
+    be Item or Album objects, that add up to the given amount of time in
+    seconds.
+    """
+    out = []
+    total_time = 0.0
+    for obj in iter:
+        length = _length(obj, album)
+        if total_time + length <= secs:
+            out.append(obj)
+            total_time += length
+    return out
+
+
+def random_objs(objs, album, number=1, time=None, equal_chance=False,
+                random_gen=None):
+    """Get a random subset of the provided `objs`.
+
+    If `number` is provided, produce that many matches. Otherwise, if
+    `time` is provided, instead select a list whose total time is close
+    to that number of minutes. If `equal_chance` is true, give each
+    artist an equal chance of being included so that artists with more
+    songs are not represented disproportionately.
+    """
+    rand = random_gen or random
+
+    # Permute the objects either in a straightforward way or an
+    # artist-balanced way.
+    if equal_chance:
+        perm = _equal_chance_permutation(objs)
+    else:
+        perm = objs
+        rand.shuffle(perm)  # N.B. This shuffles the original list.
+
+    # Select objects by time our count.
+    if time:
+        return _take_time(perm, time * 60, album)
+    else:
+        return _take(perm, number)

beets/ui/__init__.py

@@ -36,12 +36,13 @@ from beets import logging
 from beets import library
 from beets import plugins
 from beets import util
-from beets.util.functemplate import Template
+from beets.util.functemplate import template
 from beets import config
-from beets.util import confit, as_string
+from beets.util import as_string
 from beets.autotag import mb
 from beets.dbcore import query as db_query
 from beets.dbcore import db
+import confuse
 import six
 
 # On Windows platforms, use colorama to support "ANSI" terminal colors.
@@ -203,7 +204,7 @@ def input_(prompt=None):
     """
     # raw_input incorrectly sends prompts to stderr, not stdout, so we
     # use print_() explicitly to display prompts.
-    # http://bugs.python.org/issue1927
+    # https://bugs.python.org/issue1927
     if prompt:
         print_(prompt, end=u' ')
 
@@ -474,7 +475,7 @@ def human_seconds_short(interval):
 # Colorization.
 
 # ANSI terminal colorization code heavily inspired by pygments:
-# http://dev.pocoo.org/hg/pygments-main/file/b2deea5b5030/pygments/console.py
+# https://bitbucket.org/birkenfeld/pygments-main/src/default/pygments/console.py
 # (pygments is by Tim Hatch, Armin Ronacher, et al.)
 COLOR_ESCAPE = "\x1b["
 DARK_COLORS = {
@@ -529,7 +530,9 @@ def colorize(color_name, text):
     """Colorize text if colored output is enabled. (Like _colorize but
     conditional.)
     """
-    if config['ui']['color']:
+    if not config['ui']['color'] or 'NO_COLOR' in os.environ.keys():
+        return text
+
     global COLORS
     if not COLORS:
         COLORS = dict((name,
@@ -542,8 +545,6 @@ def colorize(color_name, text):
         log.debug(u'Invalid color_name: {0}', color_name)
         color = color_name
     return _colorize(color, text)
-    else:
-        return text
 
 
 def _colordiff(a, b, highlight='text_highlight',
@@ -616,12 +617,12 @@ def get_path_formats(subview=None):
     subview = subview or config['paths']
     for query, view in subview.items():
         query = PF_KEY_QUERIES.get(query, query)  # Expand common queries.
-        path_formats.append((query, Template(view.as_str())))
+        path_formats.append((query, template(view.as_str())))
     return path_formats
 
 
 def get_replacements():
-    """Confit validation function that reads regex/string pairs.
+    """Confuse validation function that reads regex/string pairs.
     """
     replacements = []
     for pattern, repl in config['replace'].get(dict).items():
@@ -928,7 +929,7 @@ class CommonOptionsParser(optparse.OptionParser, object):
 #
 # This is a fairly generic subcommand parser for optparse. It is
 # maintained externally here:
-# http://gist.github.com/462717
+# https://gist.github.com/462717
 # There you will also find a better description of the code and a more
 # succinct example program.
 
@@ -1143,8 +1144,12 @@ def _setup(options, lib=None):
     if lib is None:
         lib = _open_library(config)
         plugins.send("library_opened", lib=lib)
+
+    # Add types and queries defined by plugins.
     library.Item._types.update(plugins.types(library.Item))
     library.Album._types.update(plugins.types(library.Album))
+    library.Item._queries.update(plugins.named_queries(library.Item))
+    library.Album._queries.update(plugins.named_queries(library.Album))
 
     return subcommands, plugins, lib
 
@@ -1273,7 +1278,7 @@ def main(args=None):
         log.debug('{}', traceback.format_exc())
         log.error('{}', exc)
         sys.exit(1)
-    except confit.ConfigError as exc:
+    except confuse.ConfigError as exc:
         log.error(u'configuration error: {0}', exc)
         sys.exit(1)
     except db_query.InvalidQueryError as exc:

beets/ui/commands.py

@@ -39,7 +39,6 @@ from beets.util import syspath, normpath, ancestry, displayable_path, \
 from beets import library
 from beets import config
 from beets import logging
-from beets.util.confit import _package_path
 import six
 from . import _store_dict
 
@@ -543,7 +542,7 @@ def choose_candidate(candidates, singleton, rec, cur_artist=None,
             print_(u"No matching release found for {0} tracks."
                    .format(itemcount))
             print_(u'For help, see: '
-                   u'http://beets.readthedocs.org/en/latest/faq.html#nomatch')
+                   u'https://beets.readthedocs.org/en/latest/faq.html#nomatch')
         sel = ui.input_options(choice_opts)
         if sel in choice_actions:
             return choice_actions[sel]
@@ -1177,7 +1176,7 @@ def update_items(lib, query, album, move, pretend, fields):
                 # Manually moving and storing the album.
                 items = list(album.items())
                 for item in items:
-                    item.move(store=False)
+                    item.move(store=False, with_album=False)
                     item.store(fields=fields)
                 album.move(store=False)
                 album.store(fields=fields)
@@ -1726,7 +1725,7 @@ def completion_script(commands):
     ``commands`` is alist of ``ui.Subcommand`` instances to generate
     completion data for.
     """
-    base_script = os.path.join(_package_path('beets.ui'), 'completion_base.sh')
+    base_script = os.path.join(os.path.dirname(__file__), 'completion_base.sh')
     with open(base_script, 'r') as base_script:
         yield util.text_string(base_script.read())
 

beets/util/__init__.py

@@ -23,7 +23,9 @@ import locale
 import re
 import shutil
 import fnmatch
+import functools
 from collections import Counter
+from multiprocessing.pool import ThreadPool
 import traceback
 import subprocess
 import platform
@@ -282,14 +284,14 @@ def prune_dirs(path, root=None, clutter=('.DS_Store', 'Thumbs.db')):
             continue
         clutter = [bytestring_path(c) for c in clutter]
         match_paths = [bytestring_path(d) for d in os.listdir(directory)]
+        try:
             if fnmatch_all(match_paths, clutter):
                 # Directory contains only clutter (or nothing).
-            try:
                 shutil.rmtree(directory)
-            except OSError:
-                break
             else:
                 break
+        except OSError:
+            break
 
 
 def components(path):
@@ -410,7 +412,7 @@ def syspath(path, prefix=True):
             path = path.decode(encoding, 'replace')
 
     # Add the magic prefix if it isn't already there.
-    # http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247.aspx
+    # https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247.aspx
     if prefix and not path.startswith(WINDOWS_MAGIC_PREFIX):
         if path.startswith(u'\\\\'):
             # UNC path. Final path should look like \\?\UNC\...
@@ -561,7 +563,7 @@ def unique_path(path):
 # Note: The Windows "reserved characters" are, of course, allowed on
 # Unix. They are forbidden here because they cause problems on Samba
 # shares, which are sufficiently common as to cause frequent problems.
-# http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247.aspx
+# https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247.aspx
 CHAR_REPLACE = [
     (re.compile(r'[\\/]'), u'_'),  # / and \ -- forbidden everywhere.
     (re.compile(r'^\.'), u'_'),  # Leading dot (hidden files on Unix).
@@ -1009,3 +1011,47 @@ def asciify_path(path, sep_replace):
                 sep_replace
             )
     return os.sep.join(path_components)
+
+
+def par_map(transform, items):
+    """Apply the function `transform` to all the elements in the
+    iterable `items`, like `map(transform, items)` but with no return
+    value. The map *might* happen in parallel: it's parallel on Python 3
+    and sequential on Python 2.
+
+    The parallelism uses threads (not processes), so this is only useful
+    for IO-bound `transform`s.
+    """
+    if sys.version_info[0] < 3:
+        # multiprocessing.pool.ThreadPool does not seem to work on
+        # Python 2. We could consider switching to futures instead.
+        for item in items:
+            transform(item)
+    else:
+        pool = ThreadPool()
+        pool.map(transform, items)
+        pool.close()
+        pool.join()
+
+
+def lazy_property(func):
+    """A decorator that creates a lazily evaluated property. On first access,
+    the property is assigned the return value of `func`. This first value is
+    stored, so that future accesses do not have to evaluate `func` again.
+
+    This behaviour is useful when `func` is expensive to evaluate, and it is
+    not certain that the result will be needed.
+    """
+    field_name = '_' + func.__name__
+
+    @property
+    @functools.wraps(func)
+    def wrapper(self):
+        if hasattr(self, field_name):
+            return getattr(self, field_name)
+
+        value = func(self)
+        setattr(self, field_name, value)
+        return value
+
+    return wrapper

beets/util/artresizer.py

@@ -81,8 +81,10 @@ def pil_resize(maxwidth, path_in, path_out=None):
 
 
 def im_resize(maxwidth, path_in, path_out=None):
-    """Resize using ImageMagick's ``convert`` tool.
+    """Resize using ImageMagick.
-    Return the output path of resized image.
+
+    Use the ``magick`` program or ``convert`` on older versions. Return
+    the output path of resized image.
     """
     path_out = path_out or temp_file_for(path_in)
     log.debug(u'artresizer: ImageMagick resizing {0} to {1}',
@@ -91,16 +93,18 @@ def im_resize(maxwidth, path_in, path_out=None):
     # "-resize WIDTHx>" shrinks images with the width larger
     # than the given width while maintaining the aspect ratio
     # with regards to the height.
-    try:
+    cmd = ArtResizer.shared.im_convert_cmd + \
-        util.command_output([
+        [util.syspath(path_in, prefix=False),
-            'convert', util.syspath(path_in, prefix=False),
             '-resize', '{0}x>'.format(maxwidth),
-            util.syspath(path_out, prefix=False),
+            util.syspath(path_out, prefix=False)]
-        ])
+
+    try:
+        util.command_output(cmd)
     except subprocess.CalledProcessError:
         log.warning(u'artresizer: IM convert failed for {0}',
                     util.displayable_path(path_in))
         return path_in
+
     return path_out
 
 
@@ -121,8 +125,9 @@ def pil_getsize(path_in):
 
 
 def im_getsize(path_in):
-    cmd = ['identify', '-format', '%w %h',
+    cmd = ArtResizer.shared.im_identify_cmd + \
-           util.syspath(path_in, prefix=False)]
+        ['-format', '%w %h', util.syspath(path_in, prefix=False)]
+
     try:
         out = util.command_output(cmd)
     except subprocess.CalledProcessError as exc:
@@ -173,6 +178,18 @@ class ArtResizer(six.with_metaclass(Shareable, object)):
         log.debug(u"artresizer: method is {0}", self.method)
         self.can_compare = self._can_compare()
 
+        # Use ImageMagick's magick binary when it's available. If it's
+        # not, fall back to the older, separate convert and identify
+        # commands.
+        if self.method[0] == IMAGEMAGICK:
+            self.im_legacy = self.method[2]
+            if self.im_legacy:
+                self.im_convert_cmd = ['convert']
+                self.im_identify_cmd = ['identify']
+            else:
+                self.im_convert_cmd = ['magick']
+                self.im_identify_cmd = ['magick', 'identify']
+
     def resize(self, maxwidth, path_in, path_out=None):
         """Manipulate an image file according to the method, returning a
         new path. For PIL or IMAGEMAGIC methods, resizes the image to a
@@ -218,10 +235,20 @@ class ArtResizer(six.with_metaclass(Shareable, object)):
 
     @staticmethod
     def _check_method():
-        """Return a tuple indicating an available method and its version."""
+        """Return a tuple indicating an available method and its version.
+
+        The result has at least two elements:
+        - The method, eitehr WEBPROXY, PIL, or IMAGEMAGICK.
+        - The version.
+
+        If the method is IMAGEMAGICK, there is also a third element: a
+        bool flag indicating whether to use the `magick` binary or
+        legacy single-purpose executables (`convert`, `identify`, etc.)
+        """
         version = get_im_version()
         if version:
-            return IMAGEMAGICK, version
+            version, legacy = version
+            return IMAGEMAGICK, version, legacy
 
         version = get_pil_version()
         if version:
@@ -231,29 +258,32 @@ class ArtResizer(six.with_metaclass(Shareable, object)):
 
 
 def get_im_version():
-    """Return Image Magick version or None if it is unavailable
+    """Get the ImageMagick version and legacy flag as a pair. Or return
-    Try invoking ImageMagick's "convert".
+    None if ImageMagick is not available.
     """
-    try:
+    for cmd_name, legacy in ((['magick'], False), (['convert'], True)):
-        out = util.command_output(['convert', '--version'])
+        cmd = cmd_name + ['--version']
 
+        try:
+            out = util.command_output(cmd)
+        except (subprocess.CalledProcessError, OSError) as exc:
+            log.debug(u'ImageMagick version check failed: {}', exc)
+        else:
             if b'imagemagick' in out.lower():
                 pattern = br".+ (\d+)\.(\d+)\.(\d+).*"
                 match = re.search(pattern, out)
                 if match:
-                return (int(match.group(1)),
+                    version = (int(match.group(1)),
                                int(match.group(2)),
                                int(match.group(3)))
-            return (0,)
+                    return version, legacy
 
-    except (subprocess.CalledProcessError, OSError) as exc:
-        log.debug(u'ImageMagick check `convert --version` failed: {}', exc)
     return None
 
 
 def get_pil_version():
-    """Return Image Magick version or None if it is unavailable
+    """Get the PIL/Pillow version, or None if it is unavailable.
-    Try importing PIL."""
+    """
     try:
         __import__('PIL', fromlist=[str('Image')])
         return (0,)

beets/util/bluelet.py

@@ -346,6 +346,10 @@ def run(root_coro):
                             exc.args[0] == errno.EPIPE:
                         # Broken pipe. Remote host disconnected.
                         pass
+                    elif isinstance(exc.args, tuple) and \
+                            exc.args[0] == errno.ECONNRESET:
+                        # Connection was reset by peer.
+                        pass
                     else:
                         traceback.print_exc()
                     # Abort the coroutine.

beets/util/functemplate.py

@@ -35,6 +35,7 @@ import dis
 import types
 import sys
 import six
+import functools
 
 SYMBOL_DELIM = u'$'
 FUNC_DELIM = u'%'
@@ -117,31 +118,38 @@ def compile_func(arg_names, statements, name='_the_func', debug=False):
     bytecode of the compiled function.
     """
     if six.PY2:
-        func_def = ast.FunctionDef(
+        name = name.encode('utf-8')
-            name=name.encode('utf-8'),
+        args = ast.arguments(
-            args=ast.arguments(
             args=[ast.Name(n, ast.Param()) for n in arg_names],
             vararg=None,
             kwarg=None,
             defaults=[ex_literal(None) for _ in arg_names],
-            ),
-            body=statements,
-            decorator_list=[],
         )
     else:
+        args_fields = {
+            'args': [ast.arg(arg=n, annotation=None) for n in arg_names],
+            'kwonlyargs': [],
+            'kw_defaults': [],
+            'defaults': [ex_literal(None) for _ in arg_names],
+        }
+        if 'posonlyargs' in ast.arguments._fields:  # Added in Python 3.8.
+            args_fields['posonlyargs'] = []
+        args = ast.arguments(**args_fields)
+
     func_def = ast.FunctionDef(
         name=name,
-            args=ast.arguments(
+        args=args,
-                args=[ast.arg(arg=n, annotation=None) for n in arg_names],
-                kwonlyargs=[],
-                kw_defaults=[],
-                defaults=[ex_literal(None) for _ in arg_names],
-            ),
         body=statements,
         decorator_list=[],
     )
 
+    # The ast.Module signature changed in 3.8 to accept a list of types to
+    # ignore.
+    if sys.version_info >= (3, 8):
+        mod = ast.Module([func_def], [])
+    else:
         mod = ast.Module([func_def])
+
     ast.fix_missing_locations(mod)
 
     prog = compile(mod, '<generated>', 'exec')
@@ -547,8 +555,23 @@ def _parse(template):
     return Expression(parts)
 
 
-# External interface.
+def cached(func):
+    """Like the `functools.lru_cache` decorator, but works (as a no-op)
+    on Python < 3.2.
+    """
+    if hasattr(functools, 'lru_cache'):
+        return functools.lru_cache(maxsize=128)(func)
+    else:
+        # Do nothing when lru_cache is not available.
+        return func
 
+
+@cached
+def template(fmt):
+    return Template(fmt)
+
+
+# External interface.
 class Template(object):
     """A string template, including text, Symbols, and Calls.
     """

beetsplug/absubmit.py

@@ -24,9 +24,7 @@ import json
 import os
 import subprocess
 import tempfile
-import sys
 
-from multiprocessing.pool import ThreadPool
 from distutils.spawn import find_executable
 import requests
 
@@ -75,8 +73,8 @@ class AcousticBrainzSubmitPlugin(plugins.BeetsPlugin):
                 call([self.extractor])
             except OSError:
                 raise ui.UserError(
-                    u'No extractor command found: please install the '
+                    u'No extractor command found: please install the extractor'
-                    u'extractor binary from http://acousticbrainz.org/download'
+                    u' binary from https://acousticbrainz.org/download'
                 )
             except ABSubmitError:
                 # Extractor found, will exit with an error if not called with
@@ -106,15 +104,7 @@ class AcousticBrainzSubmitPlugin(plugins.BeetsPlugin):
     def command(self, lib, opts, args):
         # Get items from arguments
         items = lib.items(ui.decargs(args))
-        if sys.version_info[0] < 3:
+        util.par_map(self.analyze_submit, items)
-            for item in items:
-                self.analyze_submit(item)
-        else:
-            # Analyze in parallel using a thread pool.
-            pool = ThreadPool()
-            pool.map(self.analyze_submit, items)
-            pool.close()
-            pool.join()
 
     def analyze_submit(self, item):
         analysis = self._get_analysis(item)

beetsplug/acousticbrainz.py

@@ -17,10 +17,12 @@
 """
 from __future__ import division, absolute_import, print_function
 
+from collections import defaultdict
+
 import requests
 
-from collections import defaultdict
 from beets import plugins, ui
+from beets.dbcore import types
 
 ACOUSTIC_BASE = "https://acousticbrainz.org/"
 LEVELS = ["/low-level", "/high-level"]
@@ -104,6 +106,29 @@ ABSCHEME = {
 
 
 class AcousticPlugin(plugins.BeetsPlugin):
+    item_types = {
+        'average_loudness': types.Float(6),
+        'chords_changes_rate': types.Float(6),
+        'chords_key': types.STRING,
+        'chords_number_rate': types.Float(6),
+        'chords_scale': types.STRING,
+        'danceable': types.Float(6),
+        'gender': types.STRING,
+        'genre_rosamerica': types.STRING,
+        'initial_key': types.STRING,
+        'key_strength': types.Float(6),
+        'mood_acoustic': types.Float(6),
+        'mood_aggressive': types.Float(6),
+        'mood_electronic': types.Float(6),
+        'mood_happy': types.Float(6),
+        'mood_party': types.Float(6),
+        'mood_relaxed': types.Float(6),
+        'mood_sad': types.Float(6),
+        'rhythm': types.Float(6),
+        'tonal': types.Float(6),
+        'voice_instrumental': types.STRING,
+    }
+
     def __init__(self):
         super(AcousticPlugin, self).__init__()
 

beetsplug/badfiles.py

@@ -18,16 +18,18 @@
 
 from __future__ import division, absolute_import, print_function
 
-from beets.plugins import BeetsPlugin
-from beets.ui import Subcommand
-from beets.util import displayable_path, confit
-from beets import ui
 from subprocess import check_output, CalledProcessError, list2cmdline, STDOUT
+
 import shlex
 import os
 import errno
 import sys
 import six
+import confuse
+from beets.plugins import BeetsPlugin
+from beets.ui import Subcommand
+from beets.util import displayable_path, par_map
+from beets import ui
 
 
 class CheckerCommandException(Exception):
@@ -48,6 +50,10 @@ class CheckerCommandException(Exception):
 
 
 class BadFiles(BeetsPlugin):
+    def __init__(self):
+        super(BadFiles, self).__init__()
+        self.verbose = False
+
     def run_command(self, cmd):
         self._log.debug(u"running command: {}",
                         displayable_path(list2cmdline(cmd)))
@@ -61,7 +67,7 @@ class BadFiles(BeetsPlugin):
             status = e.returncode
         except OSError as e:
             raise CheckerCommandException(cmd, e)
-        output = output.decode(sys.getfilesystemencoding())
+        output = output.decode(sys.getdefaultencoding(), 'replace')
         return status, errors, [line for line in output.split("\n") if line]
 
     def check_mp3val(self, path):
@@ -85,18 +91,16 @@ class BadFiles(BeetsPlugin):
         ext = ext.lower()
         try:
             command = self.config['commands'].get(dict).get(ext)
-        except confit.NotFoundError:
+        except confuse.NotFoundError:
             command = None
         if command:
             return self.check_custom(command)
-        elif ext == "mp3":
+        if ext == "mp3":
             return self.check_mp3val
-        elif ext == "flac":
+        if ext == "flac":
             return self.check_flac
 
-    def check_bad(self, lib, opts, args):
+    def check_item(self, item):
-        for item in lib.items(ui.decargs(args)):
-
         # First, check whether the path exists. If not, the user
         # should probably run `beet update` to cleanup your library.
         dpath = displayable_path(item.path)
@@ -111,7 +115,7 @@ class BadFiles(BeetsPlugin):
         if not checker:
             self._log.error(u"no checker specified in the config for {}",
                             ext)
-                continue
+            return
         path = item.path
         if not isinstance(path, six.text_type):
             path = item.path.decode(sys.getfilesystemencoding())
@@ -126,20 +130,26 @@ class BadFiles(BeetsPlugin):
                 )
             else:
                 self._log.error(u"error invoking {}: {}", e.checker, e.msg)
-                continue
+            return
         if status > 0:
             ui.print_(u"{}: checker exited with status {}"
                       .format(ui.colorize('text_error', dpath), status))
             for line in output:
-                    ui.print_(u"  {}".format(displayable_path(line)))
+                ui.print_(u"  {}".format(line))
         elif errors > 0:
             ui.print_(u"{}: checker found {} errors or warnings"
                       .format(ui.colorize('text_warning', dpath), errors))
             for line in output:
-                    ui.print_(u"  {}".format(displayable_path(line)))
+                ui.print_(u"  {}".format(line))
-            elif opts.verbose:
+        elif self.verbose:
             ui.print_(u"{}: ok".format(ui.colorize('text_success', dpath)))
 
+    def command(self, lib, opts, args):
+        # Get items from arguments
+        items = lib.items(ui.decargs(args))
+        self.verbose = opts.verbose
+        par_map(self.check_item, items)
+
     def commands(self):
         bad_command = Subcommand('bad',
                                  help=u'check for corrupt or missing files')
@@ -148,5 +158,5 @@ class BadFiles(BeetsPlugin):
             action='store_true', default=False, dest='verbose',
             help=u'view results for both the bad and uncorrupted files'
         )
-        bad_command.func = self.check_bad
+        bad_command.func = self.command
         return [bad_command]

beetsplug/beatport.py

@@ -30,11 +30,11 @@ import beets
 import beets.ui
 from beets.autotag.hooks import AlbumInfo, TrackInfo, Distance
 from beets.plugins import BeetsPlugin
-from beets.util import confit
+import confuse
 
 
 AUTH_ERRORS = (TokenRequestDenied, TokenMissing, VerifierMissing)
-USER_AGENT = u'beets/{0} +http://beets.io/'.format(beets.__version__)
+USER_AGENT = u'beets/{0} +https://beets.io/'.format(beets.__version__)
 
 
 class BeatportAPIError(Exception):
@@ -109,7 +109,7 @@ class BeatportClient(object):
         :rtype:             (unicode, unicode) tuple
         """
         self.api.parse_authorization_response(
-            "http://beets.io/auth?" + auth_data)
+            "https://beets.io/auth?" + auth_data)
         access_data = self.api.fetch_access_token(
             self._make_url('/identity/1/oauth/access-token'))
         return access_data['oauth_token'], access_data['oauth_token_secret']
@@ -191,7 +191,7 @@ class BeatportClient(object):
             response = self.api.get(self._make_url(endpoint), params=kwargs)
         except Exception as e:
             raise BeatportAPIError("Error connecting to Beatport API: {}"
-                                   .format(e.message))
+                                   .format(e))
         if not response:
             raise BeatportAPIError(
                 "Error {0.status_code} for '{0.request.path_url}"
@@ -224,7 +224,7 @@ class BeatportRelease(BeatportObject):
         if 'category' in data:
             self.category = data['category']
         if 'slug' in data:
-            self.url = "http://beatport.com/release/{0}/{1}".format(
+            self.url = "https://beatport.com/release/{0}/{1}".format(
                 data['slug'], data['id'])
 
 
@@ -252,8 +252,8 @@ class BeatportTrack(BeatportObject):
             except ValueError:
                 pass
         if 'slug' in data:
-            self.url = "http://beatport.com/track/{0}/{1}".format(data['slug'],
+            self.url = "https://beatport.com/track/{0}/{1}" \
-                                                                  data['id'])
+                .format(data['slug'], data['id'])
         self.track_number = data.get('trackNumber')
 
 
@@ -318,7 +318,7 @@ class BeatportPlugin(BeetsPlugin):
     def _tokenfile(self):
         """Get the path to the JSON file for storing the OAuth token.
         """
-        return self.config['tokenfile'].get(confit.Filename(in_app_dir=True))
+        return self.config['tokenfile'].get(confuse.Filename(in_app_dir=True))
 
     def album_distance(self, items, album_info, mapping):
         """Returns the beatport source weight and the maximum source weight

beetsplug/bpd/gstplayer.py

@@ -64,7 +64,8 @@ class GstPlayer(object):
         """
 
         # Set up the Gstreamer player. From the pygst tutorial:
-        # http://pygstdocs.berlios.de/pygst-tutorial/playbin.html
+        # https://pygstdocs.berlios.de/pygst-tutorial/playbin.html (gone)
+        # https://brettviren.github.io/pygst-tutorial-org/pygst-tutorial.html
         ####
         # Updated to GStreamer 1.0 with:
         # https://wiki.ubuntu.com/Novacut/GStreamer1.0
@@ -177,12 +178,12 @@ class GstPlayer(object):
             posq = self.player.query_position(fmt)
             if not posq[0]:
                 raise QueryError("query_position failed")
-            pos = posq[1] // (10 ** 9)
+            pos = posq[1] / (10 ** 9)
 
             lengthq = self.player.query_duration(fmt)
             if not lengthq[0]:
                 raise QueryError("query_duration failed")
-            length = lengthq[1] // (10 ** 9)
+            length = lengthq[1] / (10 ** 9)
 
             self.cached_time = (pos, length)
             return (pos, length)
@@ -215,6 +216,59 @@ class GstPlayer(object):
         while self.playing:
             time.sleep(1)
 
+    def get_decoders(self):
+        return get_decoders()
+
+
+def get_decoders():
+    """Get supported audio decoders from GStreamer.
+    Returns a dict mapping decoder element names to the associated media types
+    and file extensions.
+    """
+    # We only care about audio decoder elements.
+    filt = (Gst.ELEMENT_FACTORY_TYPE_DEPAYLOADER |
+            Gst.ELEMENT_FACTORY_TYPE_DEMUXER |
+            Gst.ELEMENT_FACTORY_TYPE_PARSER |
+            Gst.ELEMENT_FACTORY_TYPE_DECODER |
+            Gst.ELEMENT_FACTORY_TYPE_MEDIA_AUDIO)
+
+    decoders = {}
+    mime_types = set()
+    for f in Gst.ElementFactory.list_get_elements(filt, Gst.Rank.NONE):
+        for pad in f.get_static_pad_templates():
+            if pad.direction == Gst.PadDirection.SINK:
+                caps = pad.static_caps.get()
+                mimes = set()
+                for i in range(caps.get_size()):
+                    struct = caps.get_structure(i)
+                    mime = struct.get_name()
+                    if mime == 'unknown/unknown':
+                        continue
+                    mimes.add(mime)
+                    mime_types.add(mime)
+                if mimes:
+                    decoders[f.get_name()] = (mimes, set())
+
+    # Check all the TypeFindFactory plugin features form the registry. If they
+    # are associated with an audio media type that we found above, get the list
+    # of corresponding file extensions.
+    mime_extensions = {mime: set() for mime in mime_types}
+    for feat in Gst.Registry.get().get_feature_list(Gst.TypeFindFactory):
+        caps = feat.get_caps()
+        if caps:
+            for i in range(caps.get_size()):
+                struct = caps.get_structure(i)
+                mime = struct.get_name()
+                if mime in mime_types:
+                    mime_extensions[mime].update(feat.get_extensions())
+
+    # Fill in the slot we left for file extensions.
+    for name, (mimes, exts) in decoders.items():
+        for mime in mimes:
+            exts.update(mime_extensions[mime])
+
+    return decoders
+
 
 def play_simple(paths):
     """Play the files in paths in a straightforward way, without

beetsplug/chroma.py

@@ -22,8 +22,8 @@ from beets import plugins
 from beets import ui
 from beets import util
 from beets import config
-from beets.util import confit
 from beets.autotag import hooks
+import confuse
 import acoustid
 from collections import defaultdict
 from functools import partial
@@ -221,7 +221,7 @@ class AcoustidPlugin(plugins.BeetsPlugin):
         def submit_cmd_func(lib, opts, args):
             try:
                 apikey = config['acoustid']['apikey'].as_str()
-            except confit.NotFoundError:
+            except confuse.NotFoundError:
                 raise ui.UserError(u'no Acoustid user API key provided')
             submit_items(self._log, apikey, lib.items(ui.decargs(args)))
         submit_cmd.func = submit_cmd_func

beetsplug/convert.py

@@ -28,7 +28,7 @@ import platform
 
 from beets import ui, util, plugins, config
 from beets.plugins import BeetsPlugin
-from beets.util.confit import ConfigTypeError
+from confuse import ConfigTypeError
 from beets import art
 from beets.util.artresizer import ArtResizer
 from beets.library import parse_query_string
@@ -116,6 +116,7 @@ class ConvertPlugin(BeetsPlugin):
             u'pretend': False,
             u'threads': util.cpu_count(),
             u'format': u'mp3',
+            u'id3v23': u'inherit',
             u'formats': {
                 u'aac': {
                     u'command': u'ffmpeg -i $source -y -vn -acodec aac '
@@ -316,8 +317,12 @@ class ConvertPlugin(BeetsPlugin):
             if pretend:
                 continue
 
+            id3v23 = self.config['id3v23'].as_choice([True, False, 'inherit'])
+            if id3v23 == 'inherit':
+                id3v23 = None
+
             # Write tags from the database to the converted file.
-            item.try_write(path=converted)
+            item.try_write(path=converted, id3v23=id3v23)
 
             if keep_new:
                 # If we're keeping the transcoded file, read it again (after
@@ -332,7 +337,7 @@ class ConvertPlugin(BeetsPlugin):
                     self._log.debug(u'embedding album art from {}',
                                     util.displayable_path(album.artpath))
                     art.embed_item(self._log, item, album.artpath,
-                                   itempath=converted)
+                                   itempath=converted, id3v23=id3v23)
 
             if keep_new:
                 plugins.send('after_convert', item=item,

beetsplug/discogs.py

@@ -22,7 +22,7 @@ import beets.ui
 from beets import config
 from beets.autotag.hooks import AlbumInfo, TrackInfo, Distance
 from beets.plugins import BeetsPlugin
-from beets.util import confit
+import confuse
 from discogs_client import Release, Master, Client
 from discogs_client.exceptions import DiscogsAPIError
 from requests.exceptions import ConnectionError
@@ -37,7 +37,7 @@ import traceback
 from string import ascii_lowercase
 
 
-USER_AGENT = u'beets/{0} +http://beets.io/'.format(beets.__version__)
+USER_AGENT = u'beets/{0} +https://beets.io/'.format(beets.__version__)
 
 # Exceptions that discogs_client should really handle but does not.
 CONNECTION_ERRORS = (ConnectionError, socket.error, http_client.HTTPException,
@@ -61,6 +61,8 @@ class DiscogsPlugin(BeetsPlugin):
         self.config['user_token'].redact = True
         self.discogs_client = None
         self.register_listener('import_begin', self.setup)
+        self.rate_limit_per_minute = 25
+        self.last_request_timestamp = 0
 
     def setup(self, session=None):
         """Create the `discogs_client` field. Authenticate if necessary.
@@ -71,6 +73,9 @@ class DiscogsPlugin(BeetsPlugin):
         # Try using a configured user token (bypassing OAuth login).
         user_token = self.config['user_token'].as_str()
         if user_token:
+            # The rate limit for authenticated users goes up to 60
+            # requests per minute.
+            self.rate_limit_per_minute = 60
             self.discogs_client = Client(USER_AGENT, user_token=user_token)
             return
 
@@ -88,6 +93,26 @@ class DiscogsPlugin(BeetsPlugin):
         self.discogs_client = Client(USER_AGENT, c_key, c_secret,
                                      token, secret)
 
+    def _time_to_next_request(self):
+        seconds_between_requests = 60 / self.rate_limit_per_minute
+        seconds_since_last_request = time.time() - self.last_request_timestamp
+        seconds_to_wait = seconds_between_requests - seconds_since_last_request
+        return seconds_to_wait
+
+    def request_start(self):
+        """wait for rate limit if needed
+        """
+        time_to_next_request = self._time_to_next_request()
+        if time_to_next_request > 0:
+            self._log.debug('hit rate limit, waiting for {0} seconds',
+                            time_to_next_request)
+            time.sleep(time_to_next_request)
+
+    def request_finished(self):
+        """update timestamp for rate limiting
+        """
+        self.last_request_timestamp = time.time()
+
     def reset_auth(self):
         """Delete token file & redo the auth steps.
         """
@@ -97,7 +122,7 @@ class DiscogsPlugin(BeetsPlugin):
     def _tokenfile(self):
         """Get the path to the JSON file for storing the OAuth token.
         """
-        return self.config['tokenfile'].get(confit.Filename(in_app_dir=True))
+        return self.config['tokenfile'].get(confuse.Filename(in_app_dir=True))
 
     def authenticate(self, c_key, c_secret):
         # Get the link for the OAuth page.
@@ -206,9 +231,13 @@ class DiscogsPlugin(BeetsPlugin):
         # Strip medium information from query, Things like "CD1" and "disk 1"
         # can also negate an otherwise positive result.
         query = re.sub(br'(?i)\b(CD|disc)\s*\d+', b'', query)
+
+        self.request_start()
         try:
             releases = self.discogs_client.search(query,
                                                   type='release').page(1)
+            self.request_finished()
+
         except CONNECTION_ERRORS:
             self._log.debug(u"Communication error while searching for {0!r}",
                             query, exc_info=True)
@@ -222,8 +251,11 @@ class DiscogsPlugin(BeetsPlugin):
         """
         self._log.debug(u'Searching for master release {0}', master_id)
         result = Master(self.discogs_client, {'id': master_id})
+
+        self.request_start()
         try:
             year = result.fetch('year')
+            self.request_finished()
             return year
         except DiscogsAPIError as e:
             if e.status_code != 404:
@@ -252,7 +284,7 @@ class DiscogsPlugin(BeetsPlugin):
         # https://www.discogs.com/help/doc/submission-guidelines-general-rules
         if not all([result.data.get(k) for k in ['artists', 'title', 'id',
                                                  'tracklist']]):
-            self._log.warn(u"Release does not contain the required fields")
+            self._log.warning(u"Release does not contain the required fields")
             return None
 
         artist, artist_id = self.get_artist([a.data for a in result.artists])

beetsplug/edit.py

@@ -74,7 +74,7 @@ def load(s):
     """
     try:
         out = []
-        for d in yaml.load_all(s):
+        for d in yaml.safe_load_all(s):
             if not isinstance(d, dict):
                 raise ParseError(
                     u'each entry must be a dictionary; found {}'.format(

beetsplug/export.py

@@ -24,7 +24,7 @@ import codecs
 from datetime import datetime, date
 from beets.plugins import BeetsPlugin
 from beets import ui
-from beets import mediafile
+import mediafile
 from beetsplug.info import make_key_filter, library_data, tag_data
 
 

beetsplug/fetchart.py

@@ -29,10 +29,11 @@ from beets import importer
 from beets import ui
 from beets import util
 from beets import config
-from beets.mediafile import image_mime_type
+from mediafile import image_mime_type
 from beets.util.artresizer import ArtResizer
-from beets.util import confit, sorted_walk
+from beets.util import sorted_walk
 from beets.util import syspath, bytestring_path, py3_path
+import confuse
 import six
 
 CONTENT_TYPES = {
@@ -310,6 +311,9 @@ class CoverArtArchive(RemoteArtSource):
 
 class Amazon(RemoteArtSource):
     NAME = u"Amazon"
+    if util.SNI_SUPPORTED:
+        URL = 'https://images.amazon.com/images/P/%s.%02i.LZZZZZZZ.jpg'
+    else:
         URL = 'http://images.amazon.com/images/P/%s.%02i.LZZZZZZZ.jpg'
     INDICES = (1, 2)
 
@@ -324,6 +328,9 @@ class Amazon(RemoteArtSource):
 
 class AlbumArtOrg(RemoteArtSource):
     NAME = u"AlbumArt.org scraper"
+    if util.SNI_SUPPORTED:
+        URL = 'https://www.albumart.org/index_detail.php'
+    else:
         URL = 'http://www.albumart.org/index_detail.php'
     PAT = r'href\s*=\s*"([^>"]*)"[^>]*title\s*=\s*"View larger image"'
 
@@ -365,12 +372,17 @@ class GoogleImages(RemoteArtSource):
         if not (album.albumartist and album.album):
             return
         search_string = (album.albumartist + ',' + album.album).encode('utf-8')
+
+        try:
             response = self.request(self.URL, params={
                 'key': self.key,
                 'cx': self.cx,
                 'q': search_string,
                 'searchType': 'image'
             })
+        except requests.RequestException:
+            self._log.debug(u'google: error receiving response')
+            return
 
         # Get results using JSON.
         try:
@@ -406,10 +418,14 @@ class FanartTV(RemoteArtSource):
         if not album.mb_releasegroupid:
             return
 
+        try:
             response = self.request(
                 self.API_ALBUMS + album.mb_releasegroupid,
                 headers={'api-key': self.PROJECT_KEY,
                          'client-key': self.client_key})
+        except requests.RequestException:
+            self._log.debug(u'fanart.tv: error receiving response')
+            return
 
         try:
             data = response.json()
@@ -545,6 +561,8 @@ class Wikipedia(RemoteArtSource):
 
         # Find the name of the cover art filename on DBpedia
         cover_filename, page_id = None, None
+
+        try:
             dbpedia_response = self.request(
                 self.DBPEDIA_URL,
                 params={
@@ -555,6 +573,10 @@ class Wikipedia(RemoteArtSource):
                 },
                 headers={'content-type': 'application/json'},
             )
+        except requests.RequestException:
+            self._log.debug(u'dbpedia: error receiving response')
+            return
+
         try:
             data = dbpedia_response.json()
             results = data['results']['bindings']
@@ -584,6 +606,7 @@ class Wikipedia(RemoteArtSource):
             lpart, rpart = cover_filename.rsplit(' .', 1)
 
             # Query all the images in the page
+            try:
                 wikipedia_response = self.request(
                     self.WIKIPEDIA_URL,
                     params={
@@ -595,6 +618,9 @@ class Wikipedia(RemoteArtSource):
                     },
                     headers={'content-type': 'application/json'},
                 )
+            except requests.RequestException:
+                self._log.debug(u'wikipedia: error receiving response')
+                return
 
             # Try to see if one of the images on the pages matches our
             # incomplete cover_filename
@@ -613,6 +639,7 @@ class Wikipedia(RemoteArtSource):
                 return
 
         # Find the absolute url of the cover art on Wikipedia
+        try:
             wikipedia_response = self.request(
                 self.WIKIPEDIA_URL,
                 params={
@@ -625,6 +652,9 @@ class Wikipedia(RemoteArtSource):
                 },
                 headers={'content-type': 'application/json'},
             )
+        except requests.RequestException:
+            self._log.debug(u'wikipedia: error receiving response')
+            return
 
         try:
             data = wikipedia_response.json()
@@ -753,9 +783,9 @@ class FetchArtPlugin(plugins.BeetsPlugin, RequestMixin):
 
         # allow both pixel and percentage-based margin specifications
         self.enforce_ratio = self.config['enforce_ratio'].get(
-            confit.OneOf([bool,
+            confuse.OneOf([bool,
-                          confit.String(pattern=self.PAT_PX),
+                           confuse.String(pattern=self.PAT_PX),
-                          confit.String(pattern=self.PAT_PERCENT)]))
+                           confuse.String(pattern=self.PAT_PERCENT)]))
         self.margin_px = None
         self.margin_percent = None
         if type(self.enforce_ratio) is six.text_type:
@@ -765,7 +795,7 @@ class FetchArtPlugin(plugins.BeetsPlugin, RequestMixin):
                 self.margin_px = int(self.enforce_ratio[:-2])
             else:
                 # shouldn't happen
-                raise confit.ConfigValueError()
+                raise confuse.ConfigValueError()
             self.enforce_ratio = True
 
         cover_names = self.config['cover_names'].as_str_seq()

beetsplug/gmusic.py

@@ -31,12 +31,19 @@ class Gmusic(BeetsPlugin):
     def __init__(self):
         super(Gmusic, self).__init__()
         self.m = Musicmanager()
+
+        # OAUTH_FILEPATH was moved in gmusicapi 12.0.0.
+        if hasattr(Musicmanager, 'OAUTH_FILEPATH'):
+            oauth_file = Musicmanager.OAUTH_FILEPATH
+        else:
+            oauth_file = gmusicapi.clients.OAUTH_FILEPATH
+
         self.config.add({
             u'auto': False,
             u'uploader_id': '',
             u'uploader_name': '',
             u'device_id': '',
-            u'oauth_file': gmusicapi.clients.OAUTH_FILEPATH,
+            u'oauth_file': oauth_file,
         })
         if self.config['auto']:
             self.import_stages = [self.autoupload]
@@ -62,7 +69,7 @@ class Gmusic(BeetsPlugin):
             return
         # Checks for OAuth2 credentials,
         # if they don't exist - performs authorization
-        oauth_file = self.config['oauth_file'].as_str()
+        oauth_file = self.config['oauth_file'].as_filename()
         if os.path.isfile(oauth_file):
             uploader_id = self.config['uploader_id']
             uploader_name = self.config['uploader_name']

beetsplug/hook.py

@@ -18,7 +18,6 @@ from __future__ import division, absolute_import, print_function
 
 import string
 import subprocess
-import six
 
 from beets.plugins import BeetsPlugin
 from beets.util import shlex_split, arg_encoding
@@ -46,10 +45,8 @@ class CodingFormatter(string.Formatter):
 
         See str.format and string.Formatter.format.
         """
-        try:
+        if isinstance(format_string, bytes):
             format_string = format_string.decode(self._coding)
-        except UnicodeEncodeError:
-            pass
 
         return super(CodingFormatter, self).format(format_string, *args,
                                                    **kwargs)
@@ -96,10 +93,7 @@ class HookPlugin(BeetsPlugin):
                 return
 
             # Use a string formatter that works on Unicode strings.
-            if six.PY2:
             formatter = CodingFormatter(arg_encoding())
-            else:
-                formatter = string.Formatter()
 
             command_pieces = shlex_split(command)
 

beetsplug/info.py

@@ -23,7 +23,7 @@ import re
 
 from beets.plugins import BeetsPlugin
 from beets import ui
-from beets import mediafile
+import mediafile
 from beets.library import Item
 from beets.util import displayable_path, normpath, syspath
 

beetsplug/inline.py

@@ -117,9 +117,13 @@ class InlinePlugin(BeetsPlugin):
             # For function bodies, invoke the function with values as global
             # variables.
             def _func_func(obj):
+                old_globals = dict(func.__globals__)
                 func.__globals__.update(_dict_for(obj))
                 try:
                     return func()
                 except Exception as exc:
                     raise InlineError(python_code, exc)
+                finally:
+                    func.__globals__.clear()
+                    func.__globals__.update(old_globals)
             return _func_func

beetsplug/keyfinder.py

@@ -66,7 +66,7 @@ class KeyFinderPlugin(BeetsPlugin):
                 continue
             except UnicodeEncodeError:
                 # Workaround for Python 2 Windows bug.
-                # http://bugs.python.org/issue1759845
+                # https://bugs.python.org/issue1759845
                 self._log.error(u'execution failed for Unicode path: {0!r}',
                                 item.path)
                 continue

beetsplug/lastgenre/__init__.py

@@ -14,6 +14,7 @@
 # included in all copies or substantial portions of the Software.
 
 from __future__ import division, absolute_import, print_function
+
 import six
 
 """Gets genres for imported music based on Last.fm tags.
@@ -152,7 +153,7 @@ class LastGenrePlugin(plugins.BeetsPlugin):
             self._log.debug('Loading canonicalization tree {0}', c14n_filename)
             c14n_filename = normpath(c14n_filename)
             with codecs.open(c14n_filename, 'r', encoding='utf-8') as f:
-                genres_tree = yaml.load(f)
+                genres_tree = yaml.safe_load(f)
             flatten_tree(genres_tree, [], self.c14n_branches)
 
     @property
@@ -373,18 +374,27 @@ class LastGenrePlugin(plugins.BeetsPlugin):
         lastgenre_cmd = ui.Subcommand('lastgenre', help=u'fetch genres')
         lastgenre_cmd.parser.add_option(
             u'-f', u'--force', dest='force',
-            action='store_true', default=False,
+            action='store_true',
             help=u're-download genre when already present'
         )
         lastgenre_cmd.parser.add_option(
             u'-s', u'--source', dest='source', type='string',
             help=u'genre source: artist, album, or track'
         )
+        lastgenre_cmd.parser.add_option(
+            u'-A', u'--items', action='store_false', dest='album',
+            help=u'match items instead of albums')
+        lastgenre_cmd.parser.add_option(
+            u'-a', u'--albums', action='store_true', dest='album',
+            help=u'match albums instead of items')
+        lastgenre_cmd.parser.set_defaults(album=True)
 
         def lastgenre_func(lib, opts, args):
             write = ui.should_write()
             self.config.set_args(opts)
 
+            if opts.album:
+                # Fetch genres for whole albums
                 for album in lib.albums(ui.decargs(args)):
                     album.genre, src = self._get_genre(album)
                     self._log.info(u'genre for album {0} ({1}): {0.genre}',
@@ -397,11 +407,20 @@ class LastGenrePlugin(plugins.BeetsPlugin):
                         if 'track' in self.sources:
                             item.genre, src = self._get_genre(item)
                             item.store()
-                        self._log.info(u'genre for track {0} ({1}): {0.genre}',
+                            self._log.info(
+                                u'genre for track {0} ({1}): {0.genre}',
                                 item, src)
 
                         if write:
                             item.try_write()
+            else:
+                # Just query singletons, i.e. items that are not part of
+                # an album
+                for item in lib.items(ui.decargs(args)):
+                    item.genre, src = self._get_genre(item)
+                    self._log.debug(u'added last.fm item genre ({0}): {1}',
+                                    src, item.genre)
+                    item.store()
 
         lastgenre_cmd.func = lastgenre_func
         return [lastgenre_cmd]

beetsplug/lastimport.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 # This file is part of beets.
-# Copyright 2016, Rafael Bodill http://github.com/rafi
+# Copyright 2016, Rafael Bodill https://github.com/rafi
 #
 # Permission is hereby granted, free of charge, to any person obtaining
 # a copy of this software and associated documentation files (the

beetsplug/loadext.py

@@ -0,0 +1,46 @@
+# -*- coding: utf-8 -*-
+# This file is part of beets.
+# Copyright 2019, Jack Wilsdon <jack.wilsdon@gmail.com>
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+
+"""Load SQLite extensions.
+"""
+
+from __future__ import division, absolute_import, print_function
+
+from beets.dbcore import Database
+from beets.plugins import BeetsPlugin
+import sqlite3
+
+
+class LoadExtPlugin(BeetsPlugin):
+    def __init__(self):
+        super(LoadExtPlugin, self).__init__()
+
+        if not Database.supports_extensions:
+            self._log.warn('loadext is enabled but the current SQLite '
+                           'installation does not support extensions')
+            return
+
+        self.register_listener('library_opened', self.library_opened)
+
+    def library_opened(self, lib):
+        for v in self.config:
+            ext = v.as_filename()
+
+            self._log.debug(u'loading extension {}', ext)
+
+            try:
+                lib.load_extension(ext)
+            except sqlite3.OperationalError as e:
+                self._log.error(u'failed to load extension {}: {}', ext, e)

beetsplug/lyrics.py

@@ -55,6 +55,7 @@ except ImportError:
 
 from beets import plugins
 from beets import ui
+from beets import util
 import beets
 
 DIV_RE = re.compile(r'<(/?)div>?', re.I)
@@ -406,6 +407,9 @@ class Genius(Backend):
 class LyricsWiki(SymbolsReplaced):
     """Fetch lyrics from LyricsWiki."""
 
+    if util.SNI_SUPPORTED:
+        URL_PATTERN = 'https://lyrics.wikia.com/%s:%s'
+    else:
         URL_PATTERN = 'http://lyrics.wikia.com/%s:%s'
 
     def fetch(self, artist, title):
@@ -446,7 +450,7 @@ def _scrape_strip_cruft(html, plain_text_out=False):
     html = html.replace('\r', '\n')  # Normalize EOL.
     html = re.sub(r' +', ' ', html)  # Whitespaces collapse.
     html = BREAK_RE.sub('\n', html)  # <br> eats up surrounding '\n'.
-    html = re.sub(r'<(script).*?</\1>(?s)', '', html)  # Strip script tags.
+    html = re.sub(r'(?s)<(script).*?</\1>', '', html)  # Strip script tags.
 
     if plain_text_out:  # Strip remaining HTML tags
         html = COMMENT_RE.sub('', html)

beetsplug/mbsubmit.py

@@ -19,7 +19,7 @@ This plugin allows the user to print track information in a format that is
 parseable by the MusicBrainz track parser [1]. Programmatic submitting is not
 implemented by MusicBrainz yet.
 
-[1] http://wiki.musicbrainz.org/History:How_To_Parse_Track_Listings
+[1] https://wiki.musicbrainz.org/History:How_To_Parse_Track_Listings
 """
 
 from __future__ import division, absolute_import, print_function

beetsplug/metasync/__init__.py

@@ -21,7 +21,7 @@ from __future__ import division, absolute_import, print_function
 from abc import abstractmethod, ABCMeta
 from importlib import import_module
 
-from beets.util.confit import ConfigValueError
+from confuse import ConfigValueError
 from beets import ui
 from beets.plugins import BeetsPlugin
 import six

beetsplug/metasync/itunes.py

@@ -24,13 +24,14 @@ import shutil
 import tempfile
 import plistlib
 
+import six
 from six.moves.urllib.parse import urlparse, unquote
 from time import mktime
 
 from beets import util
 from beets.dbcore import types
 from beets.library import DateType
-from beets.util.confit import ConfigValueError
+from confuse import ConfigValueError
 from beetsplug.metasync import MetaSource
 
 
@@ -84,7 +85,11 @@ class Itunes(MetaSource):
             self._log.debug(
                 u'loading iTunes library from {0}'.format(library_path))
             with create_temporary_copy(library_path) as library_copy:
+                if six.PY2:
                     raw_library = plistlib.readPlist(library_copy)
+                else:
+                    with open(library_copy, 'rb') as library_copy_f:
+                        raw_library = plistlib.load(library_copy_f)
         except IOError as e:
             raise ConfigValueError(u'invalid iTunes library: ' + e.strerror)
         except Exception:

beetsplug/mpdstats.py

@@ -107,17 +107,17 @@ class MPDClientWrapper(object):
         self.connect()
         return self.get(command, retries=retries - 1)
 
-    def playlist(self):
+    def currentsong(self):
-        """Return the currently active playlist.  Prefixes paths with the
+        """Return the path to the currently playing song.  Prefixes paths with the
         music_directory, to get the absolute path.
         """
-        result = {}
+        result = None
-        for entry in self.get('playlistinfo'):
+        entry = self.get('currentsong')
+        if 'file' in entry:
             if not is_url(entry['file']):
-                result[entry['id']] = os.path.join(
+                result = os.path.join(self.music_directory, entry['file'])
-                    self.music_directory, entry['file'])
             else:
-                result[entry['id']] = entry['file']
+                result = entry['file']
         return result
 
     def status(self):
@@ -250,8 +250,8 @@ class MPDStats(object):
         self.now_playing = None
 
     def on_play(self, status):
-        playlist = self.mpd.playlist()
+
-        path = playlist.get(status['songid'])
+        path = self.mpd.currentsong()
 
         if not path:
             return
@@ -326,7 +326,7 @@ class MPDStatsPlugin(plugins.BeetsPlugin):
             'rating':          True,
             'rating_mix':      0.75,
             'host':            os.environ.get('MPD_HOST', u'localhost'),
-            'port':            6600,
+            'port':            int(os.environ.get('MPD_PORT', 6600)),
             'password':        u'',
         })
         mpd_config['password'].redact = True

beetsplug/mpdupdate.py

@@ -69,7 +69,7 @@ class MPDUpdatePlugin(BeetsPlugin):
         super(MPDUpdatePlugin, self).__init__()
         config['mpd'].add({
             'host':     os.environ.get('MPD_HOST', u'localhost'),
-            'port':     6600,
+            'port':     int(os.environ.get('MPD_PORT', 6600)),
             'password': u'',
         })
         config['mpd']['password'].redact = True

beetsplug/playlist.py

@@ -0,0 +1,181 @@
+# -*- coding: utf-8 -*-
+# This file is part of beets.
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+
+from __future__ import division, absolute_import, print_function
+
+import os
+import fnmatch
+import tempfile
+import beets
+
+
+class PlaylistQuery(beets.dbcore.Query):
+    """Matches files listed by a playlist file.
+    """
+    def __init__(self, pattern):
+        self.pattern = pattern
+        config = beets.config['playlist']
+
+        # Get the full path to the playlist
+        playlist_paths = (
+            pattern,
+            os.path.abspath(os.path.join(
+                config['playlist_dir'].as_filename(),
+                '{0}.m3u'.format(pattern),
+            )),
+        )
+
+        self.paths = []
+        for playlist_path in playlist_paths:
+            if not fnmatch.fnmatch(playlist_path, '*.[mM]3[uU]'):
+                # This is not am M3U playlist, skip this candidate
+                continue
+
+            try:
+                f = open(beets.util.syspath(playlist_path), mode='rb')
+            except (OSError, IOError):
+                continue
+
+            if config['relative_to'].get() == 'library':
+                relative_to = beets.config['directory'].as_filename()
+            elif config['relative_to'].get() == 'playlist':
+                relative_to = os.path.dirname(playlist_path)
+            else:
+                relative_to = config['relative_to'].as_filename()
+            relative_to = beets.util.bytestring_path(relative_to)
+
+            for line in f:
+                if line[0] == '#':
+                    # ignore comments, and extm3u extension
+                    continue
+
+                self.paths.append(beets.util.normpath(
+                    os.path.join(relative_to, line.rstrip())
+                ))
+            f.close()
+            break
+
+    def col_clause(self):
+        if not self.paths:
+            # Playlist is empty
+            return '0', ()
+        clause  = 'path IN ({0})'.format(', '.join('?' for path in self.paths))
+        return clause, (beets.library.BLOB_TYPE(p) for p in self.paths)
+
+    def match(self, item):
+        return item.path in self.paths
+
+
+class PlaylistPlugin(beets.plugins.BeetsPlugin):
+    item_queries = {'playlist': PlaylistQuery}
+
+    def __init__(self):
+        super(PlaylistPlugin, self).__init__()
+        self.config.add({
+            'auto': False,
+            'playlist_dir': '.',
+            'relative_to': 'library',
+        })
+
+        self.playlist_dir = self.config['playlist_dir'].as_filename()
+        self.changes = {}
+
+        if self.config['relative_to'].get() == 'library':
+            self.relative_to = beets.util.bytestring_path(
+                beets.config['directory'].as_filename())
+        elif self.config['relative_to'].get() != 'playlist':
+            self.relative_to = beets.util.bytestring_path(
+                self.config['relative_to'].as_filename())
+        else:
+            self.relative_to = None
+
+        if self.config['auto']:
+            self.register_listener('item_moved', self.item_moved)
+            self.register_listener('item_removed', self.item_removed)
+            self.register_listener('cli_exit', self.cli_exit)
+
+    def item_moved(self, item, source, destination):
+        self.changes[source] = destination
+
+    def item_removed(self, item):
+        if not os.path.exists(beets.util.syspath(item.path)):
+            self.changes[item.path] = None
+
+    def cli_exit(self, lib):
+        for playlist in self.find_playlists():
+            self._log.info('Updating playlist: {0}'.format(playlist))
+            base_dir = beets.util.bytestring_path(
+                self.relative_to if self.relative_to
+                else os.path.dirname(playlist)
+            )
+
+            try:
+                self.update_playlist(playlist, base_dir)
+            except beets.util.FilesystemError:
+                self._log.error('Failed to update playlist: {0}'.format(
+                    beets.util.displayable_path(playlist)))
+
+    def find_playlists(self):
+        """Find M3U playlists in the playlist directory."""
+        try:
+            dir_contents = os.listdir(beets.util.syspath(self.playlist_dir))
+        except OSError:
+            self._log.warning('Unable to open playlist directory {0}'.format(
+                beets.util.displayable_path(self.playlist_dir)))
+            return
+
+        for filename in dir_contents:
+            if fnmatch.fnmatch(filename, '*.[mM]3[uU]'):
+                yield os.path.join(self.playlist_dir, filename)
+
+    def update_playlist(self, filename, base_dir):
+        """Find M3U playlists in the specified directory."""
+        changes = 0
+        deletions = 0
+
+        with tempfile.NamedTemporaryFile(mode='w+b', delete=False) as tempfp:
+            new_playlist = tempfp.name
+            with open(filename, mode='rb') as fp:
+                for line in fp:
+                    original_path = line.rstrip(b'\r\n')
+
+                    # Ensure that path from playlist is absolute
+                    is_relative = not os.path.isabs(line)
+                    if is_relative:
+                        lookup = os.path.join(base_dir, original_path)
+                    else:
+                        lookup = original_path
+
+                    try:
+                        new_path = self.changes[beets.util.normpath(lookup)]
+                    except KeyError:
+                        tempfp.write(line)
+                    else:
+                        if new_path is None:
+                            # Item has been deleted
+                            deletions += 1
+                            continue
+
+                        changes += 1
+                        if is_relative:
+                            new_path = os.path.relpath(new_path, base_dir)
+
+                        tempfp.write(line.replace(original_path, new_path))
+
+        if changes or deletions:
+            self._log.info(
+                'Updated playlist {0} ({1} changes, {2} deletions)'.format(
+                    filename, changes, deletions))
+            beets.util.copy(new_playlist, filename, replace=True)
+        beets.util.remove(new_playlist)

beetsplug/random.py

@@ -19,97 +19,7 @@ from __future__ import division, absolute_import, print_function
 
 from beets.plugins import BeetsPlugin
 from beets.ui import Subcommand, decargs, print_
-import random
+from beets.random import random_objs
-from operator import attrgetter
-from itertools import groupby
-
-
-def _length(obj, album):
-    """Get the duration of an item or album.
-    """
-    if album:
-        return sum(i.length for i in obj.items())
-    else:
-        return obj.length
-
-
-def _equal_chance_permutation(objs, field='albumartist'):
-    """Generate (lazily) a permutation of the objects where every group
-    with equal values for `field` have an equal chance of appearing in
-    any given position.
-    """
-    # Group the objects by artist so we can sample from them.
-    key = attrgetter(field)
-    objs.sort(key=key)
-    objs_by_artists = {}
-    for artist, v in groupby(objs, key):
-        objs_by_artists[artist] = list(v)
-
-    # While we still have artists with music to choose from, pick one
-    # randomly and pick a track from that artist.
-    while objs_by_artists:
-        # Choose an artist and an object for that artist, removing
-        # this choice from the pool.
-        artist = random.choice(list(objs_by_artists.keys()))
-        objs_from_artist = objs_by_artists[artist]
-        i = random.randint(0, len(objs_from_artist) - 1)
-        yield objs_from_artist.pop(i)
-
-        # Remove the artist if we've used up all of its objects.
-        if not objs_from_artist:
-            del objs_by_artists[artist]
-
-
-def _take(iter, num):
-    """Return a list containing the first `num` values in `iter` (or
-    fewer, if the iterable ends early).
-    """
-    out = []
-    for val in iter:
-        out.append(val)
-        num -= 1
-        if num <= 0:
-            break
-    return out
-
-
-def _take_time(iter, secs, album):
-    """Return a list containing the first values in `iter`, which should
-    be Item or Album objects, that add up to the given amount of time in
-    seconds.
-    """
-    out = []
-    total_time = 0.0
-    for obj in iter:
-        length = _length(obj, album)
-        if total_time + length <= secs:
-            out.append(obj)
-            total_time += length
-    return out
-
-
-def random_objs(objs, album, number=1, time=None, equal_chance=False):
-    """Get a random subset of the provided `objs`.
-
-    If `number` is provided, produce that many matches. Otherwise, if
-    `time` is provided, instead select a list whose total time is close
-    to that number of minutes. If `equal_chance` is true, give each
-    artist an equal chance of being included so that artists with more
-    songs are not represented disproportionately.
-    """
-    # Permute the objects either in a straightforward way or an
-    # artist-balanced way.
-    if equal_chance:
-        perm = _equal_chance_permutation(objs)
-    else:
-        perm = objs
-        random.shuffle(perm)  # N.B. This shuffles the original list.
-
-    # Select objects by time our count.
-    if time:
-        return _take_time(perm, time * 60, album)
-    else:
-        return _take(perm, number)
 
 
 def random_func(lib, opts, args):

beetsplug/replaygain.py

@@ -250,7 +250,14 @@ class Bs1770gainBackend(Backend):
                 state['gain'] = state['peak'] = None
         parser.StartElementHandler = start_element_handler
         parser.EndElementHandler = end_element_handler
+
+        try:
             parser.Parse(text, True)
+        except xml.parsers.expat.ExpatError:
+            raise ReplayGainError(
+                u'The bs1770gain tool produced malformed XML. '
+                'Using version >=0.4.10 may solve this problem.'
+            )
 
         if len(per_file_gain) != len(path_list):
             raise ReplayGainError(

beetsplug/scrub.py

@@ -23,7 +23,7 @@ from beets.plugins import BeetsPlugin
 from beets import ui
 from beets import util
 from beets import config
-from beets import mediafile
+import mediafile
 import mutagen
 
 _MUTAGEN_FORMATS = {

beetsplug/spotify.py

@@ -14,7 +14,7 @@ import requests
 
 from beets import ui
 from beets.plugins import BeetsPlugin
-from beets.util import confit
+import confuse
 from beets.autotag.hooks import AlbumInfo, TrackInfo, Distance
 
 
@@ -22,7 +22,7 @@ class SpotifyPlugin(BeetsPlugin):
     # Base URLs for the Spotify API
     # Documentation: https://developer.spotify.com/web-api
     oauth_token_url = 'https://accounts.spotify.com/api/token'
-    open_track_url = 'http://open.spotify.com/track/'
+    open_track_url = 'https://open.spotify.com/track/'
     search_url = 'https://api.spotify.com/v1/search'
     album_url = 'https://api.spotify.com/v1/albums/'
     track_url = 'https://api.spotify.com/v1/tracks/'
@@ -49,7 +49,7 @@ class SpotifyPlugin(BeetsPlugin):
         self.config['client_secret'].redact = True
 
         self.tokenfile = self.config['tokenfile'].get(
-            confit.Filename(in_app_dir=True)
+            confuse.Filename(in_app_dir=True)
         )  # Path to the JSON file for storing the OAuth access token.
         self.setup()
 

beetsplug/the.py

@@ -93,8 +93,8 @@ class ThePlugin(BeetsPlugin):
             for p in self.patterns:
                 r = self.unthe(text, p)
                 if r != text:
-                    break
                     self._log.debug(u'\"{0}\" -> \"{1}\"', text, r)
+                    break
             return r
         else:
             return u''

beetsplug/thumbnails.py

@@ -160,7 +160,7 @@ class ThumbnailsPlugin(BeetsPlugin):
 
     def thumbnail_file_name(self, path):
         """Compute the thumbnail file name
-        See http://standards.freedesktop.org/thumbnail-spec/latest/x227.html
+        See https://standards.freedesktop.org/thumbnail-spec/latest/x227.html
         """
         uri = self.get_uri(path)
         hash = md5(uri.encode('utf-8')).hexdigest()
@@ -168,7 +168,7 @@ class ThumbnailsPlugin(BeetsPlugin):
 
     def add_tags(self, album, image_path):
         """Write required metadata to the thumbnail
-        See http://standards.freedesktop.org/thumbnail-spec/latest/x142.html
+        See https://standards.freedesktop.org/thumbnail-spec/latest/x142.html
         """
         mtime = os.stat(album.artpath).st_mtime
         metadata = {"Thumb::URI": self.get_uri(album.artpath),

beetsplug/types.py

@@ -17,7 +17,7 @@ from __future__ import division, absolute_import, print_function
 
 from beets.plugins import BeetsPlugin
 from beets.dbcore import types
-from beets.util.confit import ConfigValueError
+from confuse import ConfigValueError
 from beets import library
 
 

beetsplug/web/static/beets.js

@@ -129,7 +129,7 @@ $.fn.player = function(debug) {
 
 // Simple selection disable for jQuery.
 // Cut-and-paste from:
-// http://stackoverflow.com/questions/2700000
+// https://stackoverflow.com/questions/2700000
 $.fn.disableSelection = function() {
     $(this).attr('unselectable', 'on')
            .css('-moz-user-select', 'none')

beetsplug/zero.py

@@ -21,10 +21,10 @@ import six
 import re
 
 from beets.plugins import BeetsPlugin
-from beets.mediafile import MediaFile
+from mediafile import MediaFile
 from beets.importer import action
 from beets.ui import Subcommand, decargs, input_yn
-from beets.util import confit
+import confuse
 
 __author__ = 'baobab@heresiarch.info'
 
@@ -98,7 +98,7 @@ class ZeroPlugin(BeetsPlugin):
                 for pattern in self.config[field].as_str_seq():
                     prog = re.compile(pattern, re.IGNORECASE)
                     self.fields_to_progs.setdefault(field, []).append(prog)
-            except confit.NotFoundError:
+            except confuse.NotFoundError:
                 # Matches everything
                 self.fields_to_progs[field] = []
 

docs/changelog.rst

@@ -1,44 +1,119 @@
 Changelog
 =========
 
-1.4.8 (in development)
+1.5.0 (in development)
 ----------------------
 
 New features:
 
-* The disambiguation string for identifying albums in the importer now shows
+* We now fetch information about `works`_ from MusicBrainz.
-  the catalog number.
+  MusicBrainz matches provide the fields ``work`` (the title), ``mb_workid``
-  Thanks to :user:`8h2a`.
+  (the MBID), and ``work_disambig`` (the disambiguation string).
-  :bug:`2951`
+  Thanks to :user:`dosoe`.
-* :doc:`/plugins/play`: The plugin can now emit a UTF-8 BOM, fixing some
+  :bug:`2580` :bug:`3272`
-  issues with foobar2000 and Winamp.
+* :doc:`/plugins/bpd`: BPD now supports most of the features of version 0.16
-  Thanks to :user:`mz2212`.
+  of the MPD protocol. This is enough to get it talking to more complicated
-  :bug:`2944`
+  clients like ncmpcpp, but there are still some incompatibilities, largely due
-* Added whitespace padding to missing tracks dialog to improve readability.
+  to MPD commands we don't support yet. Let us know if you find an MPD client
-  Thanks to :user:`jams2`.
+  that doesn't get along with BPD!
-  :bug:`2962`
+  :bug:`3214` :bug:`800`
-* :bug:`/plugins/gmusic`: Add a new option to automatically upload to Google
+
-  Play Music library on track import.
+Fixes:
-  Thanks to :user:`shuaiscott`.
+
-* :doc:`/plugins/gmusic`: Add new options for Google Play Music
+* :doc:`/plugins/inline`: In function-style field definitions that refer to
-  authentication.
+  flexible attributes, values could stick around from one function invocation
-  Thanks to :user:`thetarkus`.
+  to the next. This meant that, when displaying a list of objects, later
-  :bug:`3002`
+  objects could seem to reuse values from earlier objects when they were
-* :doc:`/plugins/absubmit`: Analysis now works in parallel (on Python 3 only).
+  missing a value for a given field. These values are now properly undefined.
-  Thanks to :user:`bemeurer`.
+  :bug:`2406`
-  :bug:`2442` :bug:`3003`
+* :doc:`/plugins/bpd`: Seeking by fractions of a second now works as intended,
-* :doc:`/plugins/replaygain`: albumpeak on large collections is calculated as
+  fixing crashes in MPD clients like mpDris2 on seek.
-  the average, not the maximum.
+  The ``playlistid`` command now works properly in its zero-argument form.
-  :bug:`3008` :bug:`3009`
+  :bug:`3214`
-* A new :doc:`/plugins/subsonicupdate` can automatically update your Subsonic library.
+
-  Thanks to :user:`maffo999`.
+For plugin developers:
-  :bug:`3001`
+
-* :doc:`/plugins/chroma`: Now optionally has a bias toward looking up more
+* `MediaFile`_ has been split into a standalone project. Where you used to do
-  relevant releases according to the :ref:`preferred` configuration options.
+  ``from beets import mediafile``, now just do ``import mediafile``. Beets
-  Thanks to :user:`archer4499`.
+  re-exports MediaFile at the old location for backwards-compatibility, but a
-  :bug:`3017`
+  deprecation warning is raised if you do this since we might drop this wrapper
-* A new ``aunique`` configuration option allows setting default options
+  in a future release.
-  for the :ref:`aunique` template function.
+* We've replaced beets' configuration library confit with a standalone
+  version called `Confuse`_. Where you used to do
+  ``from beets.util import confit``, now just do ``import confuse``. The code
+  is almost identical apart from the name change. Again, we'll re-export at the
+  old location (with a deprecation warning) for backwards compatibility, but
+  might stop doing this in a future release.
+
+For packagers:
+
+* Beets' library for manipulating media file metadata has now been split to a
+  standalone project called `MediaFile`_, released as :pypi:`mediafile`. Beets
+  now depends on this new package. Beets now depends on Mutagen transitively
+  through MediaFile rather than directly, except in the case of one of beets'
+  plugins (scrub).
+* Beets' library for configuration has been split into a standalone project
+  called `Confuse`_, released as :pypi:`confuse`. Beets now depends on this
+  package. Confuse has existed separately for some time and is used by
+  unrelated projects, but until now we've been bundling a copy within beets.
+* We attempted to fix an unreliable test, so a patch to `skip <https://sources.debian.org/src/beets/1.4.7-2/debian/patches/skip-broken-test/>`_
+  or `repair <https://build.opensuse.org/package/view_file/openSUSE:Factory/beets/fix_test_command_line_option_relative_to_working_dir.diff?expand=1>`_
+  the test may no longer be necessary.
+
+.. _MediaFile: https://github.com/beetbox/mediafile
+.. _Confuse: https://github.com/beetbox/confuse
+.. _works: https://musicbrainz.org/doc/Work
+
+
+1.4.9 (May 30, 2019)
+--------------------
+
+This small update is part of our attempt to release new versions more often!
+There are a few important fixes, and we're clearing the deck for a change to
+beets' dependencies in the next version.
+
+The new feature is:
+
+* You can use the `NO_COLOR`_ environment variable to disable terminal colors.
+  :bug:`3273`
+
+There are some fixes in this release:
+
+* Fix a regression in the last release that made the image resizer fail to
+  detect older versions of ImageMagick.
+  :bug:`3269`
+* :doc:`/plugins/gmusic`: The ``oauth_file`` config option now supports more
+  flexible path values, including ``~`` for the home directory.
+  :bug:`3270`
+* :doc:`/plugins/gmusic`: Fix a crash when using version 12.0.0 or later of
+  the ``gmusicapi`` module.
+  :bug:`3270`
+* Fix an incompatibility with Python 3.8's AST changes.
+  :bug:`3278`
+
+Here's a note for packagers:
+
+* ``pathlib`` is now an optional test dependency on Python 3.4+, removing the
+  need for `a Debian patch <https://sources.debian.org/src/beets/1.4.7-2/debian/patches/pathlib-is-stdlib/>`_.
+  :bug:`3275`
+
+.. _NO_COLOR: https://no-color.org
+
+
+1.4.8 (May 16, 2019)
+--------------------
+
+This release is far too long in coming, but it's a good one. There is the
+usual torrent of new features and a ridiculously long line of fixes, but there
+are also some crucial maintenance changes.
+We officially support Python 3.7 and 3.8, and some performance optimizations
+can (anecdotally) make listing your library more than three times faster than
+in the previous version.
+
+The new core features are:
+
+* A new :ref:`config-aunique` configuration option allows setting default
+  options for the :ref:`aunique` template function.
 * The ``albumdisambig`` field no longer includes the MusicBrainz release group
   disambiguation comment. A new ``releasegroupdisambig`` field has been added.
   :bug:`3024`
@@ -70,30 +145,216 @@ New features:
   level.
   Thanks to :user:`samuelnilsson`
 
-
+=======
-Changes:
-
-* :doc:`/plugins/mbsync` no longer queries MusicBrainz when either the
-  ``mb_albumid`` or ``mb_trackid`` field is invalid
-  See also the discussion on Google Groups_
-  Thanks to :user:`arogl`.
-* :doc:`/plugins/export` now also exports ``path`` field if user explicitly
-  specifies it with ``-i`` parameter. Only works when exporting library fields.
-  :bug:`3084`
-
-.. _Groups: https://groups.google.com/forum/#!searchin/beets-users/mbsync|sort:date/beets-users/iwCF6bNdh9A/i1xl4Gx8BQAJ
-
-Fixes:
-
 * A new importer option, :ref:`ignore_data_tracks`, lets you skip audio tracks
-  contained in data files :bug:`3021`
+  contained in data files. :bug:`3021`
+
+There are some new plugins:
+
+* The :doc:`/plugins/playlist` can query the beets library using M3U playlists.
+  Thanks to :user:`Holzhaus` and :user:`Xenopathic`.
+  :bug:`123` :bug:`3145`
+* The :doc:`/plugins/loadext` allows loading of SQLite extensions, primarily
+  for use with the ICU SQLite extension for internationalization.
+  :bug:`3160` :bug:`3226`
+* The :doc:`/plugins/subsonicupdate` can automatically update your Subsonic
+  library.
+  Thanks to :user:`maffo999`.
+  :bug:`3001`
+
+And many improvements to existing plugins:
+
+* :doc:`/plugins/lastgenre`: Added option ``-A`` to match individual tracks
+  and singletons.
+  :bug:`3220` :bug:`3219`
+* :doc:`/plugins/play`: The plugin can now emit a UTF-8 BOM, fixing some
+  issues with foobar2000 and Winamp.
+  Thanks to :user:`mz2212`.
+  :bug:`2944`
+* :doc:`/plugins/gmusic`:
+   * Add a new option to automatically upload to Google Play Music library on
+     track import.
+     Thanks to :user:`shuaiscott`.
+   * Add new options for Google Play Music authentication.
+     Thanks to :user:`thetarkus`.
+     :bug:`3002`
+* :doc:`/plugins/replaygain`: ``albumpeak`` on large collections is calculated
+  as the average, not the maximum.
+  :bug:`3008` :bug:`3009`
+* :doc:`/plugins/chroma`:
+   * Now optionally has a bias toward looking up more relevant releases
+     according to the :ref:`preferred` configuration options.
+     Thanks to :user:`archer4499`.
+     :bug:`3017`
+   * Fingerprint values are now properly stored as strings, which prevents
+     strange repeated output when running ``beet write``.
+     Thanks to :user:`Holzhaus`.
+     :bug:`3097` :bug:`2942`
+* :doc:`/plugins/convert`: The plugin now has an ``id3v23`` option that allows
+  you to override the global ``id3v23`` option.
+  Thanks to :user:`Holzhaus`.
+  :bug:`3104`
+* :doc:`/plugins/spotify`:
+   * The plugin now uses OAuth for authentication to the Spotify API.
+     Thanks to :user:`rhlahuja`.
+     :bug:`2694` :bug:`3123`
+   * The plugin now works as an import metadata
+     provider: you can match tracks and albums using the Spotify database.
+     Thanks to :user:`rhlahuja`.
+     :bug:`3123`
+* :doc:`/plugins/ipfs`: The plugin now supports a ``nocopy`` option which
+  passes that flag to ipfs.
+  Thanks to :user:`wildthyme`.
+* :doc:`/plugins/discogs`: The plugin now has rate limiting for the Discogs API.
+  :bug:`3081`
+* :doc:`/plugins/mpdstats`, :doc:`/plugins/mpdupdate`: These plugins now use
+  the ``MPD_PORT`` environment variable if no port is specified in the
+  configuration file.
+  :bug:`3223`
+* :doc:`/plugins/bpd`:
+   * MPD protocol commands ``consume`` and ``single`` are now supported along
+     with updated semantics for ``repeat`` and ``previous`` and new fields for
+     ``status``. The bpd server now understands and ignores some additional
+     commands.
+     :bug:`3200` :bug:`800`
+   * MPD protocol command ``idle`` is now supported, allowing the MPD version
+     to be bumped to 0.14.
+     :bug:`3205` :bug:`800`
+   * MPD protocol command ``decoders`` is now supported.
+     :bug:`3222`
+   * The plugin now uses the main beets logging system.
+     The special-purpose ``--debug`` flag has been removed.
+     Thanks to :user:`arcresu`.
+     :bug:`3196`
+* :doc:`/plugins/mbsync`: The plugin no longer queries MusicBrainz when either
+  the ``mb_albumid`` or ``mb_trackid`` field is invalid.
+  See also the discussion on `Google Groups`_
+  Thanks to :user:`arogl`.
+* :doc:`/plugins/export`: The plugin now also exports ``path`` field if the user
+  explicitly specifies it with ``-i`` parameter. This only works when exporting
+  library fields.
+  :bug:`3084`
+* :doc:`/plugins/acousticbrainz`: The plugin now declares types for all its
+  fields, which enables easier querying and avoids a problem where very small
+  numbers would be stored as strings.
+  Thanks to :user:`rain0r`.
+  :bug:`2790` :bug:`3238`
+
+.. _Google Groups: https://groups.google.com/forum/#!searchin/beets-users/mbsync|sort:date/beets-users/iwCF6bNdh9A/i1xl4Gx8BQAJ
+
+Some improvements have been focused on improving beets' performance:
+
+* Querying the library is now faster:
+    * We only convert fields that need to be displayed.
+      Thanks to :user:`pprkut`.
+      :bug:`3089`
+    * We now compile templates once and reuse them instead of recompiling them
+      to print out each matching object.
+      Thanks to :user:`SimonPersson`.
+      :bug:`3258`
+    * Querying the library for items is now faster, for all queries that do not
+      need to access album level properties. This was implemented by lazily
+      fetching the album only when needed.
+      Thanks to :user:`SimonPersson`.
+      :bug:`3260`
+* :doc:`/plugins/absubmit`, :doc:`/plugins/badfiles`: Analysis now works in
+  parallel (on Python 3 only).
+  Thanks to :user:`bemeurer`.
+  :bug:`2442` :bug:`3003`
+* :doc:`/plugins/mpdstats`: Use the ``currentsong`` MPD command instead of
+  ``playlist`` to get the current song, improving performance when the playlist
+  is long.
+  Thanks to :user:`ray66`.
+  :bug:`3207` :bug:`2752`
+
+Several improvements are related to usability:
+
+* The disambiguation string for identifying albums in the importer now shows
+  the catalog number.
+  Thanks to :user:`8h2a`.
+  :bug:`2951`
+* Added whitespace padding to missing tracks dialog to improve readability.
+  Thanks to :user:`jams2`.
+  :bug:`2962`
+* The :ref:`move-cmd` command now lists the number of items already in-place.
+  Thanks to :user:`RollingStar`.
+  :bug:`3117`
+* Modify selection can now be applied early without selecting every item.
+  :bug:`3083`
+* Beets now emits more useful messages during startup if SQLite returns an error. The
+  SQLite error message is now attached to the beets message.
+  :bug:`3005`
+* Fixed a confusing typo when the :doc:`/plugins/convert` plugin copies the art
+  covers.
+  :bug:`3063`
+
+Many fixes have been focused on issues where beets would previously crash:
+
+* Avoid a crash when archive extraction fails during import.
+  :bug:`3041`
+* Missing album art file during an update no longer causes a fatal exception
+  (instead, an error is logged and the missing file path is removed from the
+  library).
+  :bug:`3030`
+* When updating the database, beets no longer tries to move album art twice.
+  :bug:`3189`
+* Fix an unhandled exception when pruning empty directories.
+  :bug:`1996` :bug:`3209`
+* :doc:`/plugins/fetchart`: Added network connection error handling to backends
+  so that beets won't crash if a request fails.
+  Thanks to :user:`Holzhaus`.
+  :bug:`1579`
+* :doc:`/plugins/badfiles`: Avoid a crash when the underlying tool emits
+  undecodable output.
+  :bug:`3165`
+* :doc:`/plugins/beatport`: Avoid a crash when the server produces an error.
+  :bug:`3184`
+* :doc:`/plugins/bpd`: Fix crashes in the bpd server during exception handling.
+  :bug:`3200`
+* :doc:`/plugins/bpd`: Fix a crash triggered when certain clients tried to list
+  the albums belonging to a particular artist.
+  :bug:`3007` :bug:`3215`
+* :doc:`/plugins/replaygain`: Avoid a crash when the ``bs1770gain`` tool emits
+  malformed XML.
+  :bug:`2983` :bug:`3247`
+
+There are many fixes related to compatibility with our dependencies including
+addressing changes interfaces:
+
+* On Python 2, pin the :pypi:`jellyfish` requirement to version 0.6.0 for
+  compatibility.
+* Fix compatibility with Python 3.7 and its change to a name in the
+  :stdlib:`re` module.
+  :bug:`2978`
+* Fix several uses of deprecated standard-library features on Python 3.7.
+  Thanks to :user:`arcresu`.
+  :bug:`3197`
+* Fix compatibility with pre-release versions of Python 3.8.
+  :bug:`3201` :bug:`3202`
+* :doc:`/plugins/web`: Fix an error when using more recent versions of Flask
+  with CORS enabled.
+  Thanks to :user:`rveachkc`.
+  :bug:`2979`: :bug:`2980`
+* Avoid some deprecation warnings with certain versions of the MusicBrainz
+  library.
+  Thanks to :user:`zhelezov`.
+  :bug:`2826` :bug:`3092`
 * Restore iTunes Store album art source, and remove the dependency on
-  python-itunes_, which had gone unmaintained and was not py3 compatible.
+  :pypi:`python-itunes`, which had gone unmaintained and was not
-  Thanks to :user:`ocelma` for creating python-itunes_ in the first place.
+  Python-3-compatible.
+  Thanks to :user:`ocelma` for creating :pypi:`python-itunes` in the first place.
   Thanks to :user:`nathdwek`.
   :bug:`2371` :bug:`2551` :bug:`2718`
-* Fix compatibility Python 3.7 and its change to a name in the ``re`` module.
+* :doc:`/plugins/lastgenre`, :doc:`/plugins/edit`: Avoid a deprecation warnings
-  :bug:`2978`
+  from the :pypi:`PyYAML` library by switching to the safe loader.
+  Thanks to :user:`translit` and :user:`sbraz`.
+  :bug:`3192` :bug:`3225`
+* Fix a problem when resizing images with :pypi:`PIL`/:pypi:`pillow` on Python 3.
+  Thanks to :user:`architek`.
+  :bug:`2504` :bug:`3029`
+
+And there are many other fixes:
+
 * R128 normalization tags are now properly deleted from files when the values
   are missing.
   Thanks to :user:`autrimpo`.
@@ -104,38 +365,49 @@ Fixes:
 * With the :ref:`from_scratch` configuration option set, only writable fields
   are cleared. Beets now no longer ignores the format your music is saved in.
   :bug:`2972`
-* LastGenre: Allow to set the configuration option ``prefer_specific``
-  without setting ``canonical``.
-  :bug:`2973`
-* :doc:`/plugins/web`: Fix an error when using more recent versions of Flask
-  with CORS enabled.
-  Thanks to :user:`rveachkc`.
-  :bug:`2979`: :bug:`2980`
-* Improve error reporting: during startup if sqlite returns an error the
-  sqlite error message is attached to the beets message.
-  :bug:`3005`
-* Fix a problem when resizing images with PIL/Pillow on Python 3.
-  Thanks to :user:`architek`.
-  :bug:`2504` :bug:`3029`
-* Avoid a crash when archive extraction fails during import.
-  :bug:`3041`
 * The ``%aunique`` template function now works correctly with the
   ``-f/--format`` option.
   :bug:`3043`
-* Missing album art file during an update no longer causes a fatal exception
-  (instead, an error is logged and the missing file path is removed from the
-  library). :bug:`3030`
 * Fixed the ordering of items when manually selecting changes while updating
   tags
   Thanks to :user:`TaizoSimpson`.
   :bug:`3501`
-* Confusing typo when the convert plugin copies the art covers. :bug:`3063`
 * The ``%title`` template function now works correctly with apostrophes.
   Thanks to :user:`GuilhermeHideki`.
   :bug:`3033`
-* Fetchart now respects the ``ignore`` and ``ignore_hidden`` settings. :bug:`1632`
+* :doc:`/plugins/lastgenre`: It's now possible to set the ``prefer_specific``
+  option without also setting ``canonical``.
+  :bug:`2973`
+* :doc:`/plugins/fetchart`: The plugin now respects the ``ignore`` and
+  ``ignore_hidden`` settings.
+  :bug:`1632`
+* :doc:`/plugins/hook`: Fix byte string interpolation in hook commands.
+  :bug:`2967` :bug:`3167`
+* :doc:`/plugins/the`: Log a message when something has changed, not when it
+  hasn't.
+  Thanks to :user:`arcresu`.
+  :bug:`3195`
+* :doc:`/plugins/lastgenre`: The ``force`` config option now actually works.
+  :bug:`2704` :bug:`3054`
+* Resizing image files with ImageMagick now avoids problems on systems where
+  there is a ``convert`` command that is *not* ImageMagick's by using the
+  ``magick`` executable when it is available.
+  Thanks to :user:`ababyduck`.
+  :bug:`2093` :bug:`3236`
 
-.. _python-itunes: https://github.com/ocelma/python-itunes
+There is one new thing for plugin developers to know about:
+
+* In addition to prefix-based field queries, plugins can now define *named
+  queries* that are not associated with any specific field.
+  For example, the new :doc:`/plugins/playlist` supports queries like
+  ``playlist:name`` although there is no field named ``playlist``.
+  See :ref:`extend-query` for details.
+
+And some messages for packagers:
+
+* Note the changes to the dependencies on :pypi:`jellyfish` and :pypi:`munkres`.
+* The optional :pypi:`python-itunes` dependency has been removed.
+* Python versions 3.7 and 3.8 are now supported.
 
 
 1.4.7 (May 29, 2018)
@@ -969,7 +1241,7 @@ There are even more new features:
   don't actually need to be moved. :bug:`1583`
 
 .. _Google Code-In: https://codein.withgoogle.com/
-.. _AcousticBrainz: http://acousticbrainz.org/
+.. _AcousticBrainz: https://acousticbrainz.org/
 
 Fixes:
 
@@ -994,7 +1266,7 @@ Fixes:
 * :doc:`/plugins/replaygain`: Fix a crash using the Python Audio Tools
   backend. :bug:`1873`
 
-.. _beets.io: http://beets.io/
+.. _beets.io: https://beets.io/
 .. _Beetbox: https://github.com/beetbox
 
 
@@ -1111,7 +1383,7 @@ Fixes:
   communication errors. The backend has also been disabled by default, since
   the API it depends on is currently down. :bug:`1770`
 
-.. _Emby: http://emby.media
+.. _Emby: https://emby.media
 
 
 1.3.15 (October 17, 2015)
@@ -1273,8 +1545,8 @@ Fixes:
 * :doc:`/plugins/convert`: Fix a problem with filename encoding on Windows
   under Python 3. :bug:`2515` :bug:`2516`
 
-.. _Python bug: http://bugs.python.org/issue16512
+.. _Python bug: https://bugs.python.org/issue16512
-.. _ipfs: http://ipfs.io
+.. _ipfs: https://ipfs.io
 
 
 1.3.13 (April 24, 2015)
@@ -1625,7 +1897,7 @@ As usual, there are loads of little fixes and improvements:
 * The :ref:`config-cmd` command can now use ``$EDITOR`` variables with
   arguments.
 
-.. _API changes: http://developer.echonest.com/forums/thread/3650
+.. _API changes: https://developer.echonest.com/forums/thread/3650
 .. _Plex: https://plex.tv/
 .. _musixmatch: https://www.musixmatch.com/
 
@@ -2105,7 +2377,7 @@ Fixes:
 * :doc:`/plugins/convert`: Display a useful error message when the FFmpeg
   executable can't be found.
 
-.. _requests: http://www.python-requests.org/
+.. _requests: https://www.python-requests.org/
 
 
 1.3.3 (February 26, 2014)
@@ -2287,7 +2559,7 @@ As usual, there are also innumerable little fixes and improvements:
 
 
 .. _Acoustic Attributes: http://developer.echonest.com/acoustic-attributes.html
-.. _MPD: http://www.musicpd.org/
+.. _MPD: https://www.musicpd.org/
 
 
 1.3.1 (October 12, 2013)
@@ -2354,7 +2626,7 @@ And some fixes:
 * :doc:`/plugins/scrub`: Avoid preserving certain non-standard ID3 tags such
   as NCON.
 
-.. _Opus: http://www.opus-codec.org/
+.. _Opus: https://www.opus-codec.org/
 .. _@Verrus: https://github.com/Verrus
 
 
@@ -2392,7 +2664,7 @@ previous versions would spit out a warning and then list your entire library.
 
 There's more detail than you could ever need `on the beets blog`_.
 
-.. _on the beets blog: http://beets.io/blog/flexattr.html
+.. _on the beets blog: https://beets.io/blog/flexattr.html
 
 
 1.2.2 (August 27, 2013)
@@ -2586,8 +2858,8 @@ And a batch of fixes:
 * :doc:`/plugins/lyrics`: Lyrics searches should now turn up more results due
   to some fixes in dealing with special characters.
 
-.. _Discogs: http://discogs.com/
+.. _Discogs: https://discogs.com/
-.. _Beatport: http://www.beatport.com/
+.. _Beatport: https://www.beatport.com/
 
 
 1.1.0 (April 29, 2013)
@@ -2636,7 +2908,7 @@ will automatically migrate your configuration to the new system.
   header. Thanks to Uwe L. Korn.
 * :doc:`/plugins/lastgenre`: Fix an error when using genre canonicalization.
 
-.. _Tomahawk: http://www.tomahawk-player.org/
+.. _Tomahawk: https://tomahawk-player.org/
 
 1.1b3 (March 16, 2013)
 ----------------------
@@ -2811,7 +3083,7 @@ Other new stuff:
   (YAML doesn't like tabs.)
 * Fix the ``-l`` (log path) command-line option for the ``import`` command.
 
-.. _iTunes Sound Check: http://support.apple.com/kb/HT2425
+.. _iTunes Sound Check: https://support.apple.com/kb/HT2425
 
 1.1b1 (January 29, 2013)
 ------------------------
@@ -2820,7 +3092,7 @@ This release entirely revamps beets' configuration system. The configuration
 file is now a `YAML`_ document and is located, along with other support files,
 in a common directory (e.g., ``~/.config/beets`` on Unix-like systems).
 
-.. _YAML: http://en.wikipedia.org/wiki/YAML
+.. _YAML: https://en.wikipedia.org/wiki/YAML
 
 * Renamed plugins: The ``rdm`` plugin has been renamed to ``random`` and
   ``fuzzy_search`` has been renamed to ``fuzzy``.
@@ -2980,9 +3252,9 @@ begins today on features for version 1.1.
   unintentionally loading the plugins they contain.
 
 .. _The Echo Nest: http://the.echonest.com/
-.. _Tomahawk resolver: http://beets.io/blog/tomahawk-resolver.html
+.. _Tomahawk resolver: https://beets.io/blog/tomahawk-resolver.html
 .. _mp3gain: http://mp3gain.sourceforge.net/download.php
-.. _aacgain: http://aacgain.altosdesign.com
+.. _aacgain: https://aacgain.altosdesign.com
 
 1.0b15 (July 26, 2012)
 ----------------------
@@ -3091,7 +3363,7 @@ fetching cover art for your music, enable this plugin after upgrading to beets
   database with ``beet import -AWC /path/to/music``.
 * Fix ``import`` with relative path arguments on Windows.
 
-.. _artist credits: http://wiki.musicbrainz.org/Artist_Credit
+.. _artist credits: https://wiki.musicbrainz.org/Artist_Credit
 
 1.0b14 (May 12, 2012)
 ---------------------
@@ -3249,7 +3521,7 @@ to come in the next couple of releases.
   data.
 * Fix the ``list`` command in BPD (thanks to Simon Chopin).
 
-.. _Colorama: http://pypi.python.org/pypi/colorama
+.. _Colorama: https://pypi.python.org/pypi/colorama
 
 1.0b12 (January 16, 2012)
 -------------------------
@@ -3362,12 +3634,12 @@ release: one for assigning genres and another for ReplayGain analysis.
   corrupted.
 
 .. _KraYmer: https://github.com/KraYmer
-.. _Next Generation Schema: http://musicbrainz.org/doc/XML_Web_Service/Version_2
+.. _Next Generation Schema: https://musicbrainz.org/doc/XML_Web_Service/Version_2
 .. _python-musicbrainzngs: https://github.com/alastair/python-musicbrainzngs
-.. _acoustid: http://acoustid.org/
+.. _acoustid: https://acoustid.org/
 .. _Peter Brunner: https://github.com/Lugoues
 .. _Simon Chopin: https://github.com/laarmen
-.. _albumart.org: http://www.albumart.org/
+.. _albumart.org: https://www.albumart.org/
 
 1.0b10 (September 22, 2011)
 ---------------------------
@@ -3536,8 +3808,8 @@ below, for a plethora of new features.
 
 * Fix a crash on album queries with item-only field names.
 
-.. _xargs: http://en.wikipedia.org/wiki/xargs
+.. _xargs: https://en.wikipedia.org/wiki/xargs
-.. _unidecode: http://pypi.python.org/pypi/Unidecode/0.04.1
+.. _unidecode: https://pypi.python.org/pypi/Unidecode/0.04.1
 
 1.0b8 (April 28, 2011)
 ----------------------
@@ -3680,7 +3952,7 @@ new configuration options and the ability to clean up empty directory subtrees.
 
 * The old "albumify" plugin for upgrading databases was removed.
 
-.. _as specified by MusicBrainz: http://wiki.musicbrainz.org/ReleaseType
+.. _as specified by MusicBrainz: https://wiki.musicbrainz.org/ReleaseType
 
 1.0b6 (January 20, 2011)
 ------------------------
@@ -3796,7 +4068,7 @@ are also rolled into this release.
 
 * Fixed escaping of ``/`` characters in paths on Windows.
 
-.. _!!!: http://musicbrainz.org/artist/f26c72d3-e52c-467b-b651-679c73d8e1a7.html
+.. _!!!: https://musicbrainz.org/artist/f26c72d3-e52c-467b-b651-679c73d8e1a7.html
 
 1.0b4 (August 9, 2010)
 ----------------------
@@ -3985,7 +4257,7 @@ Vorbis) and an option to log untaggable albums during import.
   removed dependency on the aging ``cmdln`` module in favor of `a hand-rolled
   solution`_.
 
-.. _a hand-rolled solution: http://gist.github.com/462717
+.. _a hand-rolled solution: https://gist.github.com/462717
 
 1.0b1 (June 17, 2010)
 ---------------------

docs/conf.py

@@ -15,15 +15,17 @@ master_doc = 'index'
 project = u'beets'
 copyright = u'2016, Adrian Sampson'
 
-version = '1.4'
+version = '1.5'
-release = '1.4.8'
+release = '1.5.0'
 
 pygments_style = 'sphinx'
 
-# External links to the bug tracker.
+# External links to the bug tracker and other sites.
 extlinks = {
     'bug': ('https://github.com/beetbox/beets/issues/%s', '#'),
     'user': ('https://github.com/%s', ''),
+    'pypi': ('https://pypi.org/project/%s/', ''),
+    'stdlib': ('https://docs.python.org/3/library/%s.html', ''),
 }
 
 # Options for HTML output

docs/dev/api.rst

@@ -1,84 +0,0 @@
-API Documentation
-=================
-
-.. currentmodule:: beets.library
-
-This page describes the internal API of beets' core. It's a work in
-progress---since beets is an application first and a library second, its API
-has been mainly undocumented until recently. Please file bugs if you run
-across incomplete or incorrect docs here.
-
-The :class:`Library` object is the central repository for data in beets. It
-represents a database containing songs, which are :class:`Item` instances, and
-groups of items, which are :class:`Album` instances.
-
-The Library Class
------------------
-
-.. autoclass:: Library(path, directory[, path_formats[, replacements]])
-
-    .. automethod:: items
-
-    .. automethod:: albums
-
-    .. automethod:: get_item
-
-    .. automethod:: get_album
-
-    .. automethod:: add
-
-    .. automethod:: add_album
-
-    .. automethod:: transaction
-
-Transactions
-''''''''''''
-
-The :class:`Library` class provides the basic methods necessary to access and
-manipulate its contents. To perform more complicated operations atomically, or
-to interact directly with the underlying SQLite database, you must use a
-*transaction*. For example::
-
-    lib = Library()
-    with lib.transaction() as tx:
-        items = lib.items(query)
-        lib.add_album(list(items))
-
-.. currentmodule:: beets.dbcore.db
-
-.. autoclass:: Transaction
-    :members:
-
-Model Classes
--------------
-
-The two model entities in beets libraries, :class:`Item` and :class:`Album`,
-share a base class, :class:`Model`, that provides common functionality and
-ORM-like abstraction.
-
-The fields model classes can be accessed using attributes (dots, as in
-``item.artist``) or items (brackets, as in ``item['artist']``). The
-:class:`Model` base class provides some methods that resemble `dict`
-objects.
-
-Model base
-''''''''''
-
-.. currentmodule:: beets.dbcore
-
-.. autoclass:: Model
-    :members:
-
-Item
-''''
-
-.. currentmodule:: beets.library
-
-.. autoclass:: Item
-    :members:
-
-Album
-'''''
-
-.. autoclass:: Album
-    :members:

docs/dev/cli.rst

@@ -0,0 +1,9 @@
+Providing a CLI
+===============
+
+The ``beets.ui`` module houses interactions with the user via a terminal, the
+:doc:`/reference/cli`.
+The main function is called when the user types beet on the command line.
+The CLI functionality is organized into commands, some of which are built-in
+and some of which are provided by plugins. The built-in commands are all
+implemented in the ``beets.ui.commands`` submodule.

docs/dev/importer.rst

@@ -0,0 +1,19 @@
+Music Importer
+==============
+
+The importer component is responsible for the user-centric workflow that adds
+music to a library. This is one of the first aspects that a user experiences
+when using beets: it finds music in the filesystem, groups it into albums,
+finds corresponding metadata in MusicBrainz, asks the user for intervention,
+applies changes, and moves/copies files. A description of its user interface is
+given in :doc:`/guides/tagger`.
+
+The workflow is implemented in the ``beets.importer`` module and is
+distinct from the core logic for matching MusicBrainz metadata (in the
+``beets.autotag`` module). The workflow is also decoupled from the command-line
+interface with the hope that, eventually, other (graphical) interfaces can be
+bolted onto the same importer implementation.
+
+The importer is multithreaded and follows the pipeline pattern. Each pipeline
+stage is a Python coroutine. The ``beets.util.pipeline`` module houses
+a generic, reusable implementation of a multithreaded pipeline.

docs/dev/index.rst

@@ -4,8 +4,14 @@ For Developers
 This section contains information for developers. Read on if you're interested
 in hacking beets itself or creating plugins for it.
 
+See also the documentation for `MediaFile`_, the library used by beets to read
+and write metadata tags in media files.
+
+.. _MediaFile: https://mediafile.readthedocs.io/
+
 .. toctree::
 
     plugins
-    api
+    library
-    media_file
+    importer
+    cli

docs/dev/library.rst

@@ -0,0 +1,279 @@
+Library Database API
+====================
+
+.. currentmodule:: beets.library
+
+This page describes the internal API of beets' core database features. It
+doesn't exhaustively document the API, but is aimed at giving an overview of
+the architecture to orient anyone who wants to dive into the code.
+
+The :class:`Library` object is the central repository for data in beets. It
+represents a database containing songs, which are :class:`Item` instances, and
+groups of items, which are :class:`Album` instances.
+
+The Library Class
+-----------------
+
+The :class:`Library` is typically instantiated as a singleton. A single
+invocation of beets usually has only one :class:`Library`. It's powered by
+:class:`dbcore.Database` under the hood, which handles the `SQLite`_
+abstraction, something like a very minimal `ORM`_. The library is also
+responsible for handling queries to retrieve stored objects.
+
+.. autoclass:: Library(path, directory[, path_formats[, replacements]])
+
+    .. automethod:: __init__
+
+    You can add new items or albums to the library:
+
+    .. automethod:: add
+
+    .. automethod:: add_album
+
+    And there are methods for querying the database:
+
+    .. automethod:: items
+
+    .. automethod:: albums
+
+    .. automethod:: get_item
+
+    .. automethod:: get_album
+
+    Any modifications must go through a :class:`Transaction` which you get can
+    using this method:
+
+    .. automethod:: transaction
+
+.. _SQLite: https://sqlite.org/
+.. _ORM: https://en.wikipedia.org/wiki/Object-relational_mapping
+
+
+Model Classes
+-------------
+
+The two model entities in beets libraries, :class:`Item` and :class:`Album`,
+share a base class, :class:`LibModel`, that provides common functionality. That
+class itself specialises :class:`dbcore.Model` which provides an ORM-like
+abstraction.
+
+To get or change the metadata of a model (an item or album), either access its
+attributes (e.g., ``print(album.year)`` or ``album.year = 2012``) or use the
+``dict``-like interface (e.g. ``item['artist']``).
+
+
+Model base
+''''''''''
+
+Models use dirty-flags to track when the object's metadata goes out of
+sync with the database. The dirty dictionary maps field names to booleans
+indicating whether the field has been written since the object was last
+synchronized (via load or store) with the database.
+
+.. autoclass:: LibModel
+
+    .. automethod:: all_keys
+
+    .. automethod:: __init__
+
+    .. autoattribute:: _types
+
+    .. autoattribute:: _fields
+
+    There are CRUD-like methods for interacting with the database:
+
+    .. automethod:: store
+
+    .. automethod:: load
+
+    .. automethod:: remove
+
+    .. automethod:: add
+
+    The base class :class:`dbcore.Model` has a ``dict``-like interface, so
+    normal the normal mapping API is supported:
+
+    .. automethod:: keys
+
+    .. automethod:: update
+
+    .. automethod:: items
+
+    .. automethod:: get
+
+Item
+''''
+
+Each :class:`Item` object represents a song or track. (We use the more generic
+term item because, one day, beets might support non-music media.) An item can
+either be purely abstract, in which case it's just a bag of metadata fields,
+or it can have an associated file (indicated by ``item.path``).
+
+In terms of the underlying SQLite database, items are backed by a single table
+called items with one column per metadata fields. The metadata fields currently
+in use are listed in ``library.py`` in ``Item._fields``.
+
+To read and write a file's tags, we use the `MediaFile`_ library.
+To make changes to either the database or the tags on a file, you
+update an item's fields (e.g., ``item.title = "Let It Be"``) and then call
+``item.write()``.
+
+.. _MediaFile: https://mediafile.readthedocs.io/
+
+Items also track their modification times (mtimes) to help detect when they
+become out of sync with on-disk metadata, mainly to speed up the
+:ref:`update-cmd` (which needs to check whether the database is in sync with
+the filesystem). This feature turns out to be sort of complicated.
+
+For any :class:`Item`, there are two mtimes: the on-disk mtime (maintained by
+the OS) and the database mtime (maintained by beets). Correspondingly, there is
+on-disk metadata (ID3 tags, for example) and DB metadata. The goal with the
+mtime is to ensure that the on-disk and DB mtimes match when the on-disk and DB
+metadata are in sync; this lets beets do a quick mtime check and avoid
+rereading files in some circumstances.
+
+Specifically, beets attempts to maintain the following invariant:
+
+    If the on-disk metadata differs from the DB metadata, then the on-disk
+    mtime must be greater than the DB mtime.
+
+As a result, it is always valid for the DB mtime to be zero (assuming that real
+disk mtimes are always positive). However, whenever possible, beets tries to
+set ``db_mtime = disk_mtime`` at points where it knows the metadata is
+synchronized. When it is possible that the metadata is out of sync, beets can
+then just set ``db_mtime = 0`` to return to a consistent state.
+
+This leads to the following implementation policy:
+
+  * On every write of disk metadata (``Item.write()``), the DB mtime is updated
+    to match the post-write disk mtime.
+  * Same for metadata reads (``Item.read()``).
+  * On every modification to DB metadata (``item.field = ...``), the DB mtime
+    is reset to zero.
+
+
+.. autoclass:: Item
+
+    .. automethod:: __init__
+
+    .. automethod:: from_path
+
+    .. automethod:: get_album
+
+    .. automethod:: destination
+
+    .. automethod:: current_mtime
+
+    The methods ``read()`` and ``write()`` are complementary: one reads a
+    file's tags and updates the item's metadata fields accordingly while the
+    other takes the item's fields and writes them to the file's tags.
+
+    .. automethod:: read
+
+    .. automethod:: write
+
+    .. automethod:: try_write
+
+    .. automethod:: try_sync
+
+    The :class:`Item` class supplements the normal model interface so that they
+    interacting with the filesystem as well:
+
+    .. automethod:: move
+
+    .. automethod:: remove
+
+Album
+'''''
+
+An :class:`Album` is a collection of Items in the database. Every item in the
+database has either zero or one associated albums (accessible via
+``item.album_id``).  An item that has no associated album is called a
+singleton.
+Changing fields on an album (e.g. ``album.year = 2012``) updates the album
+itself and also changes the same field in all associated items.
+
+An :class:`Album` object keeps track of album-level metadata, which is (mostly)
+a subset of the track-level metadata. The album-level metadata fields are
+listed in ``Album._fields``.
+For those fields that are both item-level and album-level (e.g., ``year`` or
+``albumartist``), every item in an album should share the same value. Albums
+use an SQLite table called ``albums``, in which each column is an album
+metadata field.
+
+.. autoclass:: Album
+
+    .. automethod:: __init__
+
+    .. automethod:: item_dir
+
+    Albums extend the normal model interface to also forward changes to their
+    items:
+
+    .. autoattribute:: item_keys
+
+    .. automethod:: store
+
+    .. automethod:: try_sync
+
+    .. automethod:: move
+
+    .. automethod:: remove
+
+    Albums also manage album art, image files that are associated with each
+    album:
+
+    .. automethod:: set_art
+
+    .. automethod:: move_art
+
+    .. automethod:: art_destination
+
+Transactions
+''''''''''''
+
+The :class:`Library` class provides the basic methods necessary to access and
+manipulate its contents. To perform more complicated operations atomically, or
+to interact directly with the underlying SQLite database, you must use a
+*transaction* (see this `blog post`_ for motivation). For example::
+
+    lib = Library()
+    with lib.transaction() as tx:
+        items = lib.items(query)
+        lib.add_album(list(items))
+
+.. _blog post: https://beets.io/blog/sqlite-nightmare.html
+
+.. currentmodule:: beets.dbcore.db
+
+.. autoclass:: Transaction
+    :members:
+
+
+Queries
+-------
+
+To access albums and items in a library, we use :doc:`/reference/query`.
+In beets, the :class:`Query` abstract base class represents a criterion that
+matches items or albums in the database.
+Every subclass of :class:`Query` must implement two methods, which implement
+two different ways of identifying matching items/albums.
+
+The ``clause()`` method should return an SQLite ``WHERE`` clause that matches
+appropriate albums/items. This allows for efficient batch queries.
+Correspondingly, the ``match(item)`` method should take an :class:`Item` object
+and return a boolean, indicating whether or not a specific item matches the
+criterion. This alternate implementation allows clients to determine whether
+items that have already been fetched from the database match the query.
+
+There are many different types of queries. Just as an example,
+:class:`FieldQuery` determines whether a certain field matches a certain value
+(an equality query).
+:class:`AndQuery` (like its abstract superclass, :class:`CollectionQuery`)
+takes a set of other query objects and bundles them together, matching only
+albums/items that match all constituent queries.
+
+Beets has a human-writable plain-text query syntax that can be parsed into
+:class:`Query` objects. Calling ``AndQuery.from_strings`` parses a list of
+query parts into a query object that can then be used with :class:`Library`
+objects.

docs/dev/media_file.rst

@@ -1,21 +0,0 @@
-.. _mediafile:
-
-MediaFile
----------
-
-.. currentmodule:: beets.mediafile
-
-.. autoclass:: MediaFile
-
-    .. automethod:: __init__
-    .. automethod:: fields
-    .. automethod:: readable_fields
-    .. automethod:: save
-    .. automethod:: update
-
-.. autoclass:: MediaField
-
-    .. automethod:: __init__
-
-.. autoclass:: StorageStyle
-    :members:

docs/dev/plugins.rst

@@ -15,7 +15,7 @@ structure should look like this::
         myawesomeplugin.py
 
 .. _Stack Overflow question about namespace packages:
-    http://stackoverflow.com/questions/1675734/how-do-i-create-a-namespace-package-in-python/1676069#1676069
+    https://stackoverflow.com/questions/1675734/how-do-i-create-a-namespace-package-in-python/1676069#1676069
 
 Then, you'll need to put this stuff in ``__init__.py`` to make ``beetsplug`` a
 namespace package::
@@ -42,7 +42,7 @@ Then, as described above, edit your ``config.yaml`` to include
 ``plugins: myawesomeplugin`` (substituting the name of the Python module
 containing your plugin).
 
-.. _virtualenv: http://pypi.python.org/pypi/virtualenv
+.. _virtualenv: https://pypi.org/project/virtualenv
 
 .. _add_subcommands:
 
@@ -73,7 +73,7 @@ but it defaults to an empty parser (you can extend it later). ``help`` is a
 description of your command, and ``aliases`` is a list of shorthand versions of
 your command name.
 
-.. _OptionParser instance: http://docs.python.org/library/optparse.html
+.. _OptionParser instance: https://docs.python.org/library/optparse.html
 
 You'll need to add a function to your command by saying ``mycommand.func =
 myfunction``. This function should take the following parameters: ``lib`` (a
@@ -81,7 +81,7 @@ beets ``Library`` object) and ``opts`` and ``args`` (command-line options and
 arguments as returned by `OptionParser.parse_args`_).
 
 .. _OptionParser.parse_args:
-    http://docs.python.org/library/optparse.html#parsing-arguments
+    https://docs.python.org/library/optparse.html#parsing-arguments
 
 The function should use any of the utility functions defined in ``beets.ui``.
 Try running ``pydoc beets.ui`` to see what's available.
@@ -103,19 +103,18 @@ operation. For instance, a plugin could write a log message every time an album
 is successfully autotagged or update MPD's index whenever the database is
 changed.
 
-You can "listen" for events using the ``BeetsPlugin.listen`` decorator. Here's
+You can "listen" for events using ``BeetsPlugin.register_listener``. Here's
 an example::
 
     from beets.plugins import BeetsPlugin
 
-    class SomePlugin(BeetsPlugin):
-        pass
-
-    @SomePlugin.listen('pluginload')
     def loaded():
         print 'Plugin loaded!'
 
-Pass the name of the event in question to the ``listen`` decorator.
+    class SomePlugin(BeetsPlugin):
+      def __init__(self):
+        super(SomePlugin, self).__init__()
+        self.register_listener('pluginload', loaded)
 
 Note that if you want to access an attribute of your plugin (e.g. ``config`` or
 ``log``) you'll have to define a method and not a function. Here is the usual
@@ -299,10 +298,10 @@ this in their ``config.yaml``::
         foo: bar
 
 To access this value, say ``self.config['foo'].get()`` at any point in your
-plugin's code. The `self.config` object is a *view* as defined by the `Confit`_
+plugin's code. The `self.config` object is a *view* as defined by the `Confuse`_
 library.
 
-.. _Confit: http://confit.readthedocs.org/
+.. _Confuse: https://confuse.readthedocs.org/
 
 If you want to access configuration values *outside* of your plugin's section,
 import the `config` object from the `beets` module. That is, just put ``from
@@ -371,17 +370,16 @@ template fields by adding a function accepting an ``Album`` argument to the
 Extend MediaFile
 ^^^^^^^^^^^^^^^^
 
-:ref:`MediaFile` is the file tag abstraction layer that beets uses to make
+`MediaFile`_ is the file tag abstraction layer that beets uses to make
 cross-format metadata manipulation simple. Plugins can add fields to MediaFile
 to extend the kinds of metadata that they can easily manage.
 
 The ``MediaFile`` class uses ``MediaField`` descriptors to provide
-access to file tags. Have a look at the ``beets.mediafile`` source code
+access to file tags. If you have created a descriptor you can add it through
-to learn how to use this descriptor class. If you have created a
+your plugins ``add_media_field()`` method.
-descriptor you can add it through your plugins ``add_media_field()``
-method.
 
 .. automethod:: beets.plugins.BeetsPlugin.add_media_field
+.. _MediaFile: https://mediafile.readthedocs.io/
 
 
 Here's an example plugin that provides a meaningless new field "foo"::
@@ -443,15 +441,24 @@ Extend the Query Syntax
 ^^^^^^^^^^^^^^^^^^^^^^^
 
 You can add new kinds of queries to beets' :doc:`query syntax
-</reference/query>` indicated by a prefix. As an example, beets already
+</reference/query>`. There are two ways to add custom queries: using a prefix
+and using a name. Prefix-based query extension can apply to *any* field, while
+named queries are not associated with any field. For example, beets already
 supports regular expression queries, which are indicated by a colon
 prefix---plugins can do the same.
 
-To do so, define a subclass of the ``Query`` type from the
+For either kind of query extension, define a subclass of the ``Query`` type
-``beets.dbcore.query`` module. Then, in the ``queries`` method of your plugin
+from the ``beets.dbcore.query`` module. Then:
-class, return a dictionary mapping prefix strings to query classes.
 
-One simple kind of query you can extend is the ``FieldQuery``, which
+- To define a prefix-based query, define a ``queries`` method in your plugin
+  class. Return from this method a dictionary mapping prefix strings to query
+  classes.
+- To define a named query, defined dictionaries named either ``item_queries``
+  or ``album_queries``. These should map names to query types. So if you
+  use ``{ "foo": FooQuery }``, then the query ``foo:bar`` will construct a
+  query like ``FooQuery("bar")``.
+
+For prefix-based queries, you will want to extend ``FieldQuery``, which
 implements string comparisons on fields. To use it, create a subclass
 inheriting from that class and override the ``value_match`` class method.
 (Remember the ``@classmethod`` decorator!) The following example plugin

docs/faq.rst

@@ -6,8 +6,8 @@ Got a question that isn't answered here? Try `IRC`_, the `discussion board`_, or
 :ref:`filing an issue <bugs>` in the bug tracker.
 
 .. _IRC: irc://irc.freenode.net/beets
-.. _mailing list: http://groups.google.com/group/beets-users
+.. _mailing list: https://groups.google.com/group/beets-users
-.. _discussion board: http://discourse.beets.io
+.. _discussion board: https://discourse.beets.io
 
 .. contents::
     :local:
@@ -94,14 +94,14 @@ the tracks into a single directory to force them to be tagged together.
 
 An MBID looks like one of these:
 
--  ``http://musicbrainz.org/release/ded77dcf-7279-457e-955d-625bd3801b87``
+-  ``https://musicbrainz.org/release/ded77dcf-7279-457e-955d-625bd3801b87``
 -  ``d569deba-8c6b-4d08-8c43-d0e5a1b8c7f3``
 
 Beets can recognize either the hex-with-dashes UUID-style string or the
 full URL that contains it (as of 1.0b11).
 
 You can get these IDs by `searching on the MusicBrainz web
-site <http://musicbrainz.org/>`__ and going to a *release* page (when
+site <https://musicbrainz.org/>`__ and going to a *release* page (when
 tagging full albums) or a *recording* page (when tagging singletons).
 Then, copy the URL of the page and paste it into beets.
 
@@ -119,7 +119,7 @@ Run a command like this::
 
     pip install -U beets
 
-The ``-U`` flag tells `pip <http://www.pip-installer.org>`__ to upgrade
+The ``-U`` flag tells `pip <https://pip.pypa.io/>`__ to upgrade
 beets to the latest version. If you want a specific version, you can
 specify with using ``==`` like so::
 
@@ -163,10 +163,10 @@ on GitHub. `Enter a new issue <https://github.com/beetbox/beets/issues/new>`__
 there to report a bug. Please follow these guidelines when reporting an issue:
 
 -  Most importantly: if beets is crashing, please `include the
-   traceback <http://imgur.com/jacoj>`__. Tracebacks can be more
+   traceback <https://imgur.com/jacoj>`__. Tracebacks can be more
    readable if you put them in a pastebin (e.g.,
    `Gist <https://gist.github.com/>`__ or
-   `Hastebin <http://hastebin.com/>`__), especially when communicating
+   `Hastebin <https://hastebin.com/>`__), especially when communicating
    over IRC or email.
 -  Turn on beets' debug output (using the -v option: for example,
    ``beet -v import ...``) and include that with your bug report. Look
@@ -188,7 +188,7 @@ there to report a bug. Please follow these guidelines when reporting an issue:
 
 If you've never reported a bug before, Mozilla has some well-written
 `general guidelines for good bug
-reports <http://www.mozilla.org/bugs/>`__.
+reports <https://www.mozilla.org/bugs/>`__.
 
 
 .. _find-config:
@@ -237,7 +237,7 @@ Why does beets…
 There are a number of possibilities:
 
 -  First, make sure the album is in `the MusicBrainz
-   database <http://musicbrainz.org/>`__. You
+   database <https://musicbrainz.org/>`__. You
    can search on their site to make sure it's cataloged there. (If not,
    anyone can edit MusicBrainz---so consider adding the data yourself.)
 -  If the album in question is a multi-disc release, see the relevant
@@ -320,7 +320,7 @@ it encounters files that *look* like music files (according to their
 extension) but seem to be broken. Most of the time, this is because the
 file is corrupted. To check whether the file is intact, try opening it
 in another media player (e.g.,
-`VLC <http://www.videolan.org/vlc/index.html>`__) to see whether it can
+`VLC <https://www.videolan.org/vlc/index.html>`__) to see whether it can
 read the file. You can also use specialized programs for checking file
 integrity---for example, type ``metaflac --list music.flac`` to check
 FLAC files.
@@ -378,4 +378,4 @@ installed using pip, the command ``pip show -f beets`` can show you where
 ``beet`` was placed on your system. If you need help extending your ``$PATH``,
 try `this Super User answer`_.
 
-.. _this Super User answer: http://superuser.com/a/284361/4569
+.. _this Super User answer: https://superuser.com/a/284361/4569

docs/guides/advanced.rst

@@ -93,7 +93,7 @@ everything by the Long Winters for listening on the go.
 The plugin has many more dials you can fiddle with to get your conversions how
 you like them. Check out :doc:`its documentation </plugins/convert>`.
 
-.. _ffmpeg: http://www.ffmpeg.org
+.. _ffmpeg: https://www.ffmpeg.org
 
 
 Store any data you like
@@ -127,7 +127,7 @@ And, unlike :ref:`built-in fields <itemfields>`, such fields can be removed::
 Read more than you ever wanted to know about the *flexible attributes*
 feature `on the beets blog`_.
 
-.. _on the beets blog: http://beets.io/blog/flexattr.html
+.. _on the beets blog: https://beets.io/blog/flexattr.html
 
 
 Choose a path style manually for some music
@@ -151,3 +151,55 @@ differently. Put something like this in your configuration file::
 
 Used together, flexible attributes and path format conditions let you sort
 your music by any criteria you can imagine.
+
+
+Automatically add new music to your library
+-------------------------------------------
+
+As a command-line tool, beets is perfect for automated operation via a cron job
+or the like. To use it this way, you might want to use these options in your
+:doc:`config file </reference/config>`:
+
+.. code-block:: yaml
+
+    import:
+        incremental: yes
+        quiet: yes
+        log: /path/to/log.txt
+
+The :ref:`incremental` option will skip importing any directories that have
+been imported in the past.
+:ref:`quiet` avoids asking you any questions (since this will be run
+automatically, no input is possible).
+You might also want to use the :ref:`quiet_fallback` options to configure
+what should happen when no near-perfect match is found -- this option depends
+on your level of paranoia.
+Finally, :ref:`import_log` will make beets record its decisions so you can come
+back later and see what you need to handle manually.
+
+The last step is to set up cron or some other automation system to run
+``beet import /path/to/incoming/music``.
+
+
+Useful reports
+--------------
+
+Since beets has a quite powerful query tool, this list contains some useful and
+powerful queries to run on your library.
+
+* See a list of all albums which have files which are 128 bit rate::
+
+      beet list bitrate:128000
+
+* See a list of all albums with the tracks listed in order of bit rate::
+
+      beet ls -f '$bitrate $artist - $title' bitrate+
+
+* See a list of albums and their formats::
+
+      beet ls -f '$albumartist $album $format' | sort | uniq
+
+  Note that ``beet ls --album -f '... $format'`` doesn't do what you want,
+  because ``format`` is an item-level field, not an album-level one.
+  If an album's tracks exist in multiple formats, the album will appear in the
+  list once for each format.

docs/guides/main.rst

@@ -4,7 +4,7 @@ Getting Started
 Welcome to `beets`_! This guide will help you begin using it to make your music
 collection better.
 
-.. _beets: http://beets.io/
+.. _beets: https://beets.io/
 
 Installing
 ----------
@@ -12,7 +12,7 @@ Installing
 You will need Python.
 Beets works on `Python 2.7`_ and Python 3.4 or later.
 
-.. _Python 2.7: http://www.python.org/download/
+.. _Python 2.7: https://www.python.org/download/
 
 * **macOS** v10.7 (Lion) and later include Python 2.7 out of the box.
   You can opt for Python 3 by installing it via `Homebrew`_:
@@ -26,37 +26,37 @@ Beets works on `Python 2.7`_ and Python 3.4 or later.
   as described below by running:
   ``apt-get install python-dev python-pip``
 
-* On **Arch Linux**, `beets is in [community]`_, so just run ``pacman -S
+* On **Arch Linux**, `beets is in [community] <Arch community_>`_, so just run ``pacman -S
-  beets``. (There's also a bleeding-edge `dev package`_ in the AUR, which will
+  beets``. (There's also a bleeding-edge `dev package <AUR_>`_ in the AUR, which will
   probably set your computer on fire.)
 
 * For **Gentoo Linux**, beets is in Portage as ``media-sound/beets``. Just run
   ``emerge beets`` to install. There are several USE flags available for
   optional plugin dependencies.
 
-* On **FreeBSD**, there's a `beets port`_ at ``audio/beets``.
+* On **FreeBSD**, there's a `beets port <FreeBSD_>`_ at ``audio/beets``.
 
-* On **OpenBSD**, beets can be installed with ``pkg_add beets``.
+* On **OpenBSD**, there's a `beets port <OpenBSD_>`_ can be installed with ``pkg_add beets``.
 
 * For **Slackware**, there's a `SlackBuild`_ available.
 
-* On **Fedora** 22 or later, there is a `DNF package`_ (or three)::
+* On **Fedora** 22 or later, there is a `DNF package`_::
 
       $ sudo dnf install beets beets-plugins beets-doc
 
 * On **Solus**, run ``eopkg install beets``.
 
-* On **NixOS**, run ``nix-env -i beets``.
+* On **NixOS**, there's a `package <NixOS_>`_ you can install with ``nix-env -i beets``.
 
-.. _copr: https://copr.fedoraproject.org/coprs/afreof/beets/
+.. _DNF package: https://apps.fedoraproject.org/packages/beets
-.. _dnf package: https://apps.fedoraproject.org/packages/beets
+.. _SlackBuild: https://slackbuilds.org/repository/14.2/multimedia/beets/
-.. _SlackBuild: http://slackbuilds.org/repository/14.1/multimedia/beets/
+.. _FreeBSD: http://portsmon.freebsd.org/portoverview.py?category=audio&portname=beets
-.. _beets port: http://portsmon.freebsd.org/portoverview.py?category=audio&portname=beets
+.. _AUR: https://aur.archlinux.org/packages/beets-git/
-.. _beets from AUR: https://aur.archlinux.org/packages/beets-git/
+.. _Debian details: https://tracker.debian.org/pkg/beets
-.. _dev package: https://aur.archlinux.org/packages/beets-git/
-.. _Debian details: http://packages.qa.debian.org/b/beets.html
 .. _Ubuntu details: https://launchpad.net/ubuntu/+source/beets
-.. _beets is in [community]: https://www.archlinux.org/packages/community/any/beets/
+.. _OpenBSD: http://openports.se/audio/beets
+.. _Arch community: https://www.archlinux.org/packages/community/any/beets/
+.. _NixOS: https://github.com/NixOS/nixpkgs/tree/master/pkgs/tools/audio/beets
 
 If you have `pip`_, just say ``pip install beets`` (or ``pip install --user
 beets`` if you run into permissions problems).
@@ -64,14 +64,14 @@ beets`` if you run into permissions problems).
 To install without pip, download beets from `its PyPI page`_ and run ``python
 setup.py install`` in the directory therein.
 
-.. _its PyPI page: http://pypi.python.org/pypi/beets#downloads
+.. _its PyPI page: https://pypi.org/project/beets#downloads
-.. _pip: http://www.pip-installer.org/
+.. _pip: https://pip.pypa.io
 
 The best way to upgrade beets to a new version is by running ``pip install -U
 beets``. You may want to follow `@b33ts`_ on Twitter to hear about progress on
 new versions.
 
-.. _@b33ts: http://twitter.com/b33ts
+.. _@b33ts: https://twitter.com/b33ts
 
 Installing on macOS 10.11 and Higher
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -87,7 +87,7 @@ If this happens, you can install beets for the current user only by typing
 ``~/Library/Python/3.6/bin`` to your ``$PATH``.
 
 .. _System Integrity Protection: https://support.apple.com/en-us/HT204899
-.. _Homebrew: http://brew.sh
+.. _Homebrew: https://brew.sh
 
 Installing on Windows
 ^^^^^^^^^^^^^^^^^^^^^
@@ -122,10 +122,10 @@ Because I don't use Windows myself, I may have missed something. If you have
 trouble or you have more detail to contribute here, please direct it to
 `the mailing list`_.
 
-.. _install Python: http://python.org/download/
+.. _install Python: https://python.org/download/
 .. _beets.reg: https://github.com/beetbox/beets/blob/master/extra/beets.reg
-.. _install pip: http://www.pip-installer.org/en/latest/installing.html#install-pip
+.. _install pip: https://pip.pypa.io/en/stable/installing/
-.. _get-pip.py: https://raw.github.com/pypa/pip/master/contrib/get-pip.py
+.. _get-pip.py: https://bootstrap.pypa.io/get-pip.py
 
 
 Configuring
@@ -179,7 +179,7 @@ There are approximately six million other configuration options you can set
 here, including the directory and file naming scheme. See
 :doc:`/reference/config` for a full reference.
 
-.. _YAML: http://yaml.org/
+.. _YAML: https://yaml.org/
 
 Importing Your Library
 ----------------------
@@ -300,6 +300,6 @@ import`` gives more specific help about the ``import`` command.
 Please let me know what you think of beets via `the discussion board`_ or
 `Twitter`_.
 
-.. _the mailing list: http://groups.google.com/group/beets-users
+.. _the mailing list: https://groups.google.com/group/beets-users
-.. _the discussion board: http://discourse.beets.io
+.. _the discussion board: https://discourse.beets.io
-.. _twitter: http://twitter.com/b33ts
+.. _twitter: https://twitter.com/b33ts

docs/guides/tagger.rst

@@ -272,7 +272,7 @@ Before you jump into acoustic fingerprinting with both feet, though, give beets
 a try without it. You may be surprised at how well metadata-based matching
 works.
 
-.. _Chromaprint: http://acoustid.org/chromaprint
+.. _Chromaprint: https://acoustid.org/chromaprint
 
 Album Art, Lyrics, Genres and Such
 ----------------------------------
@@ -292,7 +292,7 @@ sure the album is present in `the MusicBrainz database`_.  You can search on
 their site to make sure it's cataloged there. If not, anyone can edit
 MusicBrainz---so consider adding the data yourself.
 
-.. _the MusicBrainz database: http://musicbrainz.org/
+.. _the MusicBrainz database: https://musicbrainz.org/
 
 If you think beets is ignoring an album that's listed in MusicBrainz, please
 `file a bug report`_.
@@ -305,5 +305,5 @@ I Hope That Makes Sense
 If we haven't made the process clear, please post on `the discussion
 board`_ and we'll try to improve this guide.
 
-.. _the mailing list: http://groups.google.com/group/beets-users
+.. _the mailing list: https://groups.google.com/group/beets-users
-.. _the discussion board: http://discourse.beets.io
+.. _the discussion board: https://discourse.beets.io

docs/index.rst

@@ -17,10 +17,10 @@ Freenode, drop by `the discussion board`_, send email to `the mailing list`_,
 or `file a bug`_ in the issue tracker. Please let us know where you think this
 documentation can be improved.
 
-.. _beets: http://beets.io/
+.. _beets: https://beets.io/
-.. _the mailing list: http://groups.google.com/group/beets-users
+.. _the mailing list: https://groups.google.com/group/beets-users
 .. _file a bug: https://github.com/beetbox/beets/issues
-.. _the discussion board: http://discourse.beets.io
+.. _the discussion board: https://discourse.beets.io
 
 Contents
 --------

docs/plugins/absubmit.rst

@@ -1,15 +1,15 @@
 AcousticBrainz Submit Plugin
 ============================
 
-The `absubmit` plugin lets you submit acoustic analysis results to the
+The ``absubmit`` plugin lets you submit acoustic analysis results to the
 `AcousticBrainz`_ server.
 
 Installation
 ------------
 
-The `absubmit` plugin requires the `streaming_extractor_music`_ program to run. Its source can be found on `GitHub`_, and while it is possible to compile the extractor from source, AcousticBrainz would prefer if you used their binary (see the AcousticBrainz `FAQ`_).
+The ``absubmit`` plugin requires the `streaming_extractor_music`_ program to run. Its source can be found on `GitHub`_, and while it is possible to compile the extractor from source, AcousticBrainz would prefer if you used their binary (see the AcousticBrainz `FAQ`_).
 
-The `absubmit` also plugin requires `requests`_, which you can install using `pip`_ by typing::
+The ``absubmit`` plugin also requires `requests`_, which you can install using `pip`_ by typing::
 
     pip install requests
 
@@ -41,9 +41,9 @@ To configure the plugin, make a ``absubmit:`` section in your configuration file
 - **extractor**: The absolute path to the `streaming_extractor_music`_ binary.
   Default: search for the program in your ``$PATH``
 
-.. _streaming_extractor_music: http://acousticbrainz.org/download
+.. _streaming_extractor_music: https://acousticbrainz.org/download
-.. _FAQ: http://acousticbrainz.org/faq
+.. _FAQ: https://acousticbrainz.org/faq
-.. _pip: http://www.pip-installer.org/
+.. _pip: https://pip.pypa.io
-.. _requests: http://docs.python-requests.org/en/master/
+.. _requests: https://docs.python-requests.org/en/master/
 .. _github: https://github.com/MTG/essentia
 .. _AcousticBrainz: https://acousticbrainz.org

docs/plugins/acousticbrainz.rst

@@ -4,7 +4,7 @@ AcousticBrainz Plugin
 The ``acousticbrainz`` plugin gets acoustic-analysis information from the
 `AcousticBrainz`_ project.
 
-.. _AcousticBrainz: http://acousticbrainz.org/
+.. _AcousticBrainz: https://acousticbrainz.org/
 
 Enable the ``acousticbrainz`` plugin in your configuration (see :ref:`using-plugins`) and run it by typing::
 

docs/plugins/badfiles.rst

@@ -48,7 +48,7 @@ Here is an example where the FLAC decoder signals a corrupt file::
       00.flac: ERROR while decoding data
                  state = FLAC__STREAM_DECODER_READ_FRAME
 
-Note that the default `mp3val` checker is a bit verbose and can output a lot
+Note that the default ``mp3val`` checker is a bit verbose and can output a lot
 of "stream error" messages, even for files that play perfectly well.
 Generally, if more than one stream error happens, or if a stream error happens
 in the middle of a file, this is a bad sign.

docs/plugins/beatport.rst

@@ -31,6 +31,6 @@ from MusicBrainz and other sources.
 If you have a Beatport ID or a URL for a release or track you want to tag, you
 can just enter one of the two at the "enter Id" prompt in the importer.
 
-.. _requests: http://docs.python-requests.org/en/latest/
+.. _requests: https://docs.python-requests.org/en/latest/
 .. _requests_oauthlib: https://github.com/requests/requests-oauthlib
-.. _Beatport: http://beatport.com
+.. _Beatport: https://beetport.com

docs/plugins/bpd.rst

@@ -6,7 +6,7 @@ implements the MPD protocol, so it's compatible with all the great MPD clients
 out there. I'm using `Theremin`_, `gmpc`_, `Sonata`_, and `Ario`_ successfully.
 
 .. _Theremin: https://theremin.sigterm.eu/
-.. _gmpc: http://gmpc.wikia.com/wiki/Gnome_Music_Player_Client
+.. _gmpc: https://gmpc.wikia.com/wiki/Gnome_Music_Player_Client
 .. _Sonata: http://sonata.berlios.de/
 .. _Ario: http://ario-player.sourceforge.net/
 
@@ -20,7 +20,7 @@ with its Python bindings) on your system.
   gst-plugins-base pygobject3``.
 
 * On Linux, you need to install GStreamer 1.0 and the GObject bindings for
-  python. Under Ubuntu, they are called `python-gi` and `gstreamer1.0`.
+  python. Under Ubuntu, they are called ``python-gi`` and ``gstreamer1.0``.
 
 * On Windows, you may want to try `GStreamer WinBuilds`_ (caveat emptor: I
   haven't tried this).
@@ -29,8 +29,8 @@ You will also need the various GStreamer plugin packages to make everything
 work. See the :doc:`/plugins/chroma` documentation for more information on
 installing GStreamer plugins.
 
-.. _GStreamer WinBuilds: http://www.gstreamer-winbuild.ylatuya.es/
+.. _GStreamer WinBuilds: https://www.gstreamer-winbuild.ylatuya.es/
-.. _Homebrew: http://mxcl.github.com/homebrew/
+.. _Homebrew: https://brew.sh
 
 Usage
 -----
@@ -44,7 +44,7 @@ Then, you can run BPD by invoking::
 Fire up your favorite MPD client to start playing music. The MPD site has `a
 long list of available clients`_. Here are my favorites:
 
-.. _a long list of available clients: http://mpd.wikia.com/wiki/Clients
+.. _a long list of available clients: https://mpd.wikia.com/wiki/Clients
 
 * Linux: `gmpc`_, `Sonata`_
 
@@ -52,9 +52,9 @@ long list of available clients`_. Here are my favorites:
 
 * Windows: I don't know. Get in touch if you have a recommendation.
 
-* iPhone/iPod touch: `MPoD`_
+* iPhone/iPod touch: `Rigelian`_
 
-.. _MPoD: http://www.katoemba.net/makesnosenseatall/mpod/
+.. _Rigelian: https://www.rigelian.net/
 
 One nice thing about MPD's (and thus BPD's) client-server architecture is that
 the client can just as easily on a different computer from the server as it can
@@ -75,6 +75,8 @@ The available options are:
   Default: No password.
 - **volume**: Initial volume, as a percentage.
   Default: 100
+- **control_port**: Port for the internal control socket.
+  Default: 6601
 
 Here's an example::
 
@@ -95,40 +97,41 @@ on-disk directory structure can. (Note that an obvious solution to this is just
 string matching on items' destination, but this requires examining the entire
 library Python-side for every query.)
 
-We don't currently support versioned playlists. Many clients, however, use
+BPD plays music using GStreamer's ``playbin`` player, which has a simple API
-plchanges instead of playlistinfo to get the current playlist, so plchanges
+but doesn't support many advanced playback features.
-contains a dummy implementation that just calls playlistinfo.
 
-The ``stats`` command always send zero for ``playtime``, which is supposed to
+Differences from the real MPD
-indicate the amount of time the server has spent playing music. BPD doesn't
+-----------------------------
-currently keep track of this.
 
-The ``update`` command regenerates the directory tree from the beets database.
+BPD currently supports version 0.16 of `the MPD protocol`_, but several of the
+commands and features are "pretend" implementations or have slightly different
+behaviour to their MPD equivalents. BPD aims to look enough like MPD that it
+can interact with the ecosystem of clients, but doesn't try to be
+a fully-fledged MPD replacement in terms of its playback capabilities.
 
-Unimplemented Commands
+.. _the MPD protocol: https://www.musicpd.org/doc/protocol/
-----------------------
 
-These are the commands from `the MPD protocol`_ that have not yet been
+These are some of the known differences between BPD and MPD:
-implemented in BPD.
 
-.. _the MPD protocol: http://www.musicpd.org/doc/protocol/
+* BPD doesn't currently support versioned playlists. Many clients, however, use
-
+  plchanges instead of playlistinfo to get the current playlist, so plchanges
-Saved playlists:
+  contains a dummy implementation that just calls playlistinfo.
-
+* Stored playlists aren't supported (BPD understands the commands though).
-* playlistclear
+* The ``stats`` command always send zero for ``playtime``, which is supposed to
-* playlistdelete
+  indicate the amount of time the server has spent playing music. BPD doesn't
-* playlistmove
+  currently keep track of this.
-* playlistadd
+* The ``update`` command regenerates the directory tree from the beets database
-* playlistsearch
+  synchronously, whereas MPD does this in the background.
-* listplaylist
+* Advanced playback features like cross-fade, ReplayGain and MixRamp are not
-* listplaylistinfo
+  supported due to BPD's simple audio player backend.
-* playlistfind
+* Advanced query syntax is not currently supported.
-* rm
+* Clients can't use the ``tagtypes`` mask to hide fields.
-* save
+* BPD's ``random`` mode is not deterministic and doesn't support priorities.
-* load
+* Mounts and streams are not supported. BPD can only play files from disk.
-* rename
+* Stickers are not supported (although this is basically a flexattr in beets
-
+  nomenclature so this is feasible to add).
-Deprecated:
+* There is only a single password, and is enabled it grants access to all
-
+  features rather than having permissions-based granularity.
-* playlist
+* Partitions and alternative outputs are not supported; BPD can only play one
-* volume
+  song at a time.
+* Client channels are not implemented.

docs/plugins/bucket.rst

@@ -27,19 +27,19 @@ The ``bucket_year`` parameter is used for all substitutions occurring on the
 The definition of a range is somewhat loose, and multiple formats are allowed:
 
 - For alpha ranges: the range is defined by the lowest and highest (ASCII-wise)
-  alphanumeric characters in the string you provide. For example, *ABCD*,
+  alphanumeric characters in the string you provide. For example, ``ABCD``,
-  *A-D*, *A->D*, and *[AD]* are all equivalent.
+  ``A-D``, ``A->D``, and ``[AD]`` are all equivalent.
 - For year ranges: digits characters are extracted and the two extreme years
-  define the range. For example, *1975-77*, *1975,76,77* and *1975-1977* are
+  define the range. For example, ``1975-77``, ``1975,76,77`` and ``1975-1977`` are
   equivalent. If no upper bound is given, the range is extended to current year
-  (unless a later range is defined). For example, *1975* encompasses all years
+  (unless a later range is defined). For example, ``1975`` encompasses all years
   from 1975 until now.
 
-The `%bucket` template function guesses whether to use alpha- or year-style
+The ``%bucket`` template function guesses whether to use alpha- or year-style
 buckets depending on the text it receives. It can guess wrong if, for example,
-an artist or album happens to begin with four digits. Provide `alpha` as the
+an artist or album happens to begin with four digits. Provide ``alpha`` as the
 second argument to the template to avoid this automatic detection: for
-example, use `%bucket{$artist,alpha}`.
+example, use ``%bucket{$artist,alpha}``.
 
 
 Configuration
@@ -56,7 +56,7 @@ The available options are:
   overrides original range definition.
   Default: none.
 - **bucket_year**: Ranges to use for all substitutions occurring on the
-  `$year` field.
+  ``$year`` field.
   Default: none.
 - **extrapolate**: Enable this if you want to group your files into multiple
   year ranges without enumerating them all. This option will generate year
@@ -73,5 +73,5 @@ Here's an example::
            'A - D': ^[0-9a-dA-D…äÄ]
 
 This configuration creates five-year ranges for any input year.
-The *A - D* bucket now matches also all artists starting with ä or Ä and 0 to 9
+The `A - D` bucket now matches also all artists starting with ä or Ä and 0 to 9
 and … (ellipsis). The other alpha buckets work as ranges.

docs/plugins/chroma.rst

@@ -8,8 +8,8 @@ information at all (or have completely incorrect data).  This plugin uses an
 open-source fingerprinting technology called `Chromaprint`_ and its associated
 Web service, called `Acoustid`_.
 
-.. _Chromaprint: http://acoustid.org/chromaprint
+.. _Chromaprint: https://acoustid.org/chromaprint
-.. _acoustid: http://acoustid.org/
+.. _acoustid: https://acoustid.org/
 
 Turning on fingerprinting can increase the accuracy of the
 autotagger---especially on files with very poor metadata---but it comes at a
@@ -31,7 +31,7 @@ First, install pyacoustid itself. You can do this using `pip`_, like so::
 
     $ pip install pyacoustid
 
-.. _pip: http://www.pip-installer.org/
+.. _pip: https://pip.pypa.io
 
 Then, you will need to install `Chromaprint`_, either as a dynamic library or
 in the form of a command-line tool (``fpcalc``).
@@ -45,7 +45,7 @@ The simplest way to get up and running, especially on Windows, is to
 means something like ``C:\\Program Files``. On OS X or Linux, put the
 executable somewhere like ``/usr/local/bin``.
 
-.. _download: http://acoustid.org/chromaprint
+.. _download: https://acoustid.org/chromaprint
 
 Installing the Library
 ''''''''''''''''''''''
@@ -56,7 +56,7 @@ site has links to packages for major Linux distributions. If you use
 `Homebrew`_ on Mac OS X, you can install the library with ``brew install
 chromaprint``.
 
-.. _Homebrew: http://mxcl.github.com/homebrew/
+.. _Homebrew: https://brew.sh/
 
 You will also need a mechanism for decoding audio files supported by the
 `audioread`_ library:
@@ -78,12 +78,12 @@ You will also need a mechanism for decoding audio files supported by the
 * On Windows, builds are provided by `GStreamer`_
 
 .. _audioread: https://github.com/beetbox/audioread
-.. _pyacoustid: http://github.com/beetbox/pyacoustid
+.. _pyacoustid: https://github.com/beetbox/pyacoustid
-.. _FFmpeg: http://ffmpeg.org/
+.. _FFmpeg: https://ffmpeg.org/
-.. _MAD: http://spacepants.org/src/pymad/
+.. _MAD: https://spacepants.org/src/pymad/
-.. _pymad: http://www.underbit.com/products/mad/
+.. _pymad: https://www.underbit.com/products/mad/
-.. _Core Audio: http://developer.apple.com/technologies/mac/audio-and-video.html
+.. _Core Audio: https://developer.apple.com/technologies/mac/audio-and-video.html
-.. _Gstreamer: http://gstreamer.freedesktop.org/
+.. _Gstreamer: https://gstreamer.freedesktop.org/
 .. _PyGObject: https://wiki.gnome.org/Projects/PyGObject
 
 To decode audio formats (MP3, FLAC, etc.) with GStreamer, you'll need the
@@ -132,4 +132,4 @@ Then, run ``beet submit``. (You can also provide a query to submit a subset of
 your library.) The command will use stored fingerprints if they're available;
 otherwise it will fingerprint each file before submitting it.
 
-.. _get an API key: http://acoustid.org/api-key
+.. _get an API key: https://acoustid.org/api-key

docs/plugins/convert.rst

@@ -14,7 +14,7 @@ To use the ``convert`` plugin, first enable it in your configuration (see
 :ref:`using-plugins`). By default, the plugin depends on `FFmpeg`_ to
 transcode the audio, so you might want to install it.
 
-.. _FFmpeg: http://ffmpeg.org
+.. _FFmpeg: https://ffmpeg.org
 
 
 Usage
@@ -68,6 +68,8 @@ file. The available options are:
 - **dest**: The directory where the files will be converted (or copied) to.
   Default: none.
 - **embed**: Embed album art in converted items. Default: ``yes``.
+- **id3v23**: Can be used to override the global ``id3v23`` option. Default:
+  ``inherit``.
 - **max_bitrate**: All lossy files with a higher bitrate will be
   transcoded and those with a lower bitrate will simply be copied. Note that
   this does not guarantee that all converted files will have a lower
@@ -120,7 +122,7 @@ and select a command with the ``--format`` command-line option or the
 
 In this example ``beet convert`` will use the *speex* command by
 default. To convert the audio to `wav`, run ``beet convert -f wav``.
-This will also use the format key (`wav`) as the file extension.
+This will also use the format key (``wav``) as the file extension.
 
 Each entry in the ``formats`` map consists of a key (the name of the
 format) as well as the command and optionally the file extension.
@@ -168,6 +170,6 @@ can use the :doc:`/plugins/replaygain` to do this analysis. See the LAME
 options and a thorough discussion of MP3 encoding.
 
 .. _documentation: http://lame.sourceforge.net/using.php
-.. _HydrogenAudio wiki: http://wiki.hydrogenaud.io/index.php?title=LAME
+.. _HydrogenAudio wiki: https://wiki.hydrogenaud.io/index.php?title=LAME
-.. _gapless: http://wiki.hydrogenaud.io/index.php?title=Gapless_playback
+.. _gapless: https://wiki.hydrogenaud.io/index.php?title=Gapless_playback
-.. _LAME: http://lame.sourceforge.net/
+.. _LAME: https://lame.sourceforge.net/

docs/plugins/discogs.rst

@@ -4,7 +4,7 @@ Discogs Plugin
 The ``discogs`` plugin extends the autotagger's search capabilities to
 include matches from the `Discogs`_ database.
 
-.. _Discogs: http://discogs.com
+.. _Discogs: https://discogs.com
 
 Installation
 ------------

docs/plugins/embedart.rst

@@ -69,7 +69,7 @@ Note: ``compare_threshold`` option requires `ImageMagick`_, and ``maxwidth``
 requires either `ImageMagick`_ or `Pillow`_.
 
 .. _Pillow: https://github.com/python-pillow/Pillow
-.. _ImageMagick: http://www.imagemagick.org/
+.. _ImageMagick: https://www.imagemagick.org/
 .. _PHASH: http://www.fmwconcepts.com/misc_tests/perceptual_hash_test_results_510/
 
 Manually Embedding and Extracting Art

docs/plugins/embyupdate.rst

@@ -17,8 +17,8 @@ To use the ``embyupdate`` plugin you need to install the `requests`_ library wit
 
 With that all in place, you'll see beets send the "update" command to your Emby server every time you change your beets library.
 
-.. _Emby: http://emby.media/
+.. _Emby: https://emby.media/
-.. _requests: http://docs.python-requests.org/en/latest/
+.. _requests: https://docs.python-requests.org/en/latest/
 
 Configuration
 -------------
@@ -34,5 +34,5 @@ The available options under the ``emby:`` section are:
 - **password**: The password for the user. (This is only necessary if no API
   key is provided.)
 
-You can choose to authenticate either with `apikey` or `password`, but only
+You can choose to authenticate either with ``apikey`` or ``password``, but only
 one of those two is required.

docs/plugins/export.rst

@@ -4,7 +4,7 @@ Export Plugin
 The ``export`` plugin lets you get data from the items and export the content
 as `JSON`_.
 
-.. _JSON: http://www.json.org
+.. _JSON: https://www.json.org
 
 Enable the ``export`` plugin (see :ref:`using-plugins` for help). Then, type ``beet export`` followed by a :doc:`query </reference/query>` to get the data from
 your library. For example, run this::
@@ -42,7 +42,7 @@ Configuration
 To configure the plugin, make a ``export:`` section in your configuration
 file. Under the ``json`` key, these options are available:
 
-- **ensure_ascii**: Escape non-ASCII characters with `\uXXXX` entities.
+- **ensure_ascii**: Escape non-ASCII characters with ``\uXXXX`` entities.
 
 - **indent**: The number of spaces for indentation.
 

docs/plugins/fetchart.rst

@@ -11,7 +11,7 @@ To use the ``fetchart`` plugin, first enable it in your configuration (see
 
 The plugin uses `requests`_ to fetch album art from the Web.
 
-.. _requests: http://docs.python-requests.org/en/latest/
+.. _requests: https://docs.python-requests.org/en/latest/
 
 Fetching Album Art During Import
 --------------------------------
@@ -73,18 +73,18 @@ or `Pillow`_.
 
 .. note::
 
-    Previously, there was a `remote_priority` option to specify when to
+    Previously, there was a ``remote_priority`` option to specify when to
     look for art on the filesystem. This is
     still respected, but a deprecation message will be shown until you
-    replace this configuration with the new `filesystem` value in the
+    replace this configuration with the new ``filesystem`` value in the
-    `sources` array.
+    ``sources`` array.
 
 .. _beets custom search engine: https://cse.google.com.au:443/cse/publicurl?cx=001442825323518660753:hrh5ch1gjzm
 .. _Pillow: https://github.com/python-pillow/Pillow
-.. _ImageMagick: http://www.imagemagick.org/
+.. _ImageMagick: https://www.imagemagick.org/
 
-Here's an example that makes plugin select only images that contain *front* or
+Here's an example that makes plugin select only images that contain ``front`` or
-*back* keywords in their filenames and prioritizes the iTunes source over
+``back`` keywords in their filenames and prioritizes the iTunes source over
 others::
 
     fetchart:
@@ -135,7 +135,7 @@ On some versions of Windows, the program can be shadowed by a system-provided
 environment variable so that ImageMagick comes first or use Pillow instead.
 
 .. _Pillow: https://github.com/python-pillow/Pillow
-.. _ImageMagick: http://www.imagemagick.org/
+.. _ImageMagick: https://www.imagemagick.org/
 
 .. _album-art-sources:
 
@@ -191,7 +191,7 @@ Optionally, you can `define a custom search engine`_. Get your search engine's
 token and use it for your ``google_engine`` configuration option. The
 default engine searches the entire web for cover art.
 
-.. _define a custom search engine: http://www.google.com/cse/all
+.. _define a custom search engine: https://www.google.com/cse/all
 
 Note that the Google custom search API is limited to 100 queries per day.
 After that, the fetchart plugin will fall back on other declared data sources.

docs/plugins/freedesktop.rst

@@ -3,4 +3,4 @@ Freedesktop Plugin
 
 The ``freedesktop`` plugin created .directory files in your album folders.
 This plugin is now deprecated and replaced by the :doc:`/plugins/thumbnails`
-with the `dolphin` option enabled.
+with the ``dolphin`` option enabled.

docs/plugins/ftintitle.rst

@@ -41,4 +41,4 @@ your entire collection.
 Use the ``-d`` flag to remove featured artists (equivalent of the ``drop``
 config option).
 
-.. _MusicBrainz style: http://musicbrainz.org/doc/Style
+.. _MusicBrainz style: https://musicbrainz.org/doc/Style

docs/plugins/gmusic.rst

@@ -8,7 +8,7 @@ songs in your library.
 Installation
 ------------
 
-The plugin requires `gmusicapi`_. You can install it using `pip`::
+The plugin requires :pypi:`gmusicapi`. You can install it using ``pip``::
 
     pip install gmusicapi
 

docs/plugins/index.rst

@@ -13,11 +13,11 @@ Using Plugins
 -------------
 
 To use one of the plugins included with beets (see the rest of this page for a
-list), just use the `plugins` option in your :doc:`config.yaml </reference/config>` file, like so::
+list), just use the ``plugins`` option in your :doc:`config.yaml </reference/config>` file, like so::
 
     plugins: inline convert web
 
-The value for `plugins` can be a space-separated list of plugin names or a
+The value for ``plugins`` can be a space-separated list of plugin names or a
 YAML list like ``[foo, bar]``. You can see which plugins are currently enabled
 by typing ``beet version``.
 
@@ -30,7 +30,7 @@ Each plugin has its own set of options that can be defined in a section bearing
 
 Some plugins have special dependencies that you'll need to install. The
 documentation page for each plugin will list them in the setup instructions.
-For some, you can use `pip`'s "extras" feature to install the dependencies,
+For some, you can use ``pip``'s "extras" feature to install the dependencies,
 like this::
 
     pip install beets[fetchart,lyrics,lastgenre]
@@ -71,6 +71,7 @@ like this::
    kodiupdate
    lastgenre
    lastimport
+   loadext
    lyrics
    mbcollection
    mbsubmit
@@ -81,6 +82,7 @@ like this::
    mpdupdate
    permissions
    play
+   playlist
    plexupdate
    random
    replaygain
@@ -105,7 +107,7 @@ Autotagger Extensions
 * :doc:`fromfilename`: Guess metadata for untagged tracks from their
   filenames.
 
-.. _Discogs: http://www.discogs.com/
+.. _Discogs: https://www.discogs.com/
 
 Metadata
 --------
@@ -134,7 +136,7 @@ Metadata
 * :doc:`zero`: Nullify fields by pattern or unconditionally.
 
 .. _KeyFinder: http://www.ibrahimshaath.co.uk/keyfinder/
-.. _streaming_extractor_music: http://acousticbrainz.org/download
+.. _streaming_extractor_music: https://acousticbrainz.org/download
 
 Path Formats
 ------------
@@ -158,6 +160,7 @@ Interoperability
 * :doc:`mpdupdate`: Automatically notifies `MPD`_ whenever the beets library
   changes.
 * :doc:`play`: Play beets queries in your music player.
+* :doc:`playlist`: Use M3U playlists to query the beets library.
 * :doc:`plexupdate`: Automatically notifies `Plex`_ whenever the beets library
   changes.
 * :doc:`smartplaylist`: Generate smart playlists based on beets queries.
@@ -166,10 +169,10 @@ Interoperability
 * :doc:`thumbnails`: Get thumbnails with the cover art on your album folders.
 
 
-.. _Emby: http://emby.media
+.. _Emby: https://emby.media
-.. _Plex: http://plex.tv
+.. _Plex: https://plex.tv
-.. _Kodi: http://kodi.tv
+.. _Kodi: https://kodi.tv
-.. _Sonos: http://sonos.com
+.. _Sonos: https://sonos.com
 
 Miscellaneous
 -------------
@@ -187,6 +190,7 @@ Miscellaneous
 * :doc:`hook`: Run a command when an event is emitted by beets.
 * :doc:`ihate`: Automatically skip albums and tracks during the import process.
 * :doc:`info`: Print music files' tags to the console.
+* :doc:`loadext`: Load SQLite extensions.
 * :doc:`mbcollection`: Maintain your MusicBrainz collection list.
 * :doc:`mbsubmit`: Print an album's tracks in a MusicBrainz-friendly format.
 * :doc:`missing`: List missing tracks.
@@ -196,8 +200,8 @@ Miscellaneous
 * :doc:`types`: Declare types for flexible attributes.
 * :doc:`web`: An experimental Web-based GUI for beets.
 
-.. _MPD: http://www.musicpd.org/
+.. _MPD: https://www.musicpd.org/
-.. _MPD clients: http://mpd.wikia.com/wiki/Clients
+.. _MPD clients: https://mpd.wikia.com/wiki/Clients
 .. _mstream: https://github.com/IrosTheBeggar/mStream
 
 .. _other-plugins:
@@ -209,14 +213,14 @@ In addition to the plugins that come with beets, there are several plugins
 that are maintained by the beets community. To use an external plugin, there
 are two options for installation:
 
-* Make sure it's in the Python path (known as `sys.path` to developers). This
+* Make sure it's in the Python path (known as ``sys.path`` to developers). This
   just means the plugin has to be installed on your system (e.g., with a
-  `setup.py` script or a command like `pip` or `easy_install`).
+  ``setup.py`` script or a command like ``pip`` or ``easy_install``).
 
-* Set the `pluginpath` config variable to point to the directory containing the
+* Set the ``pluginpath`` config variable to point to the directory containing the
   plugin. (See :doc:`/reference/config`.)
 
-Once the plugin is installed, enable it by placing its name on the `plugins`
+Once the plugin is installed, enable it by placing its name on the ``plugins``
 line in your config file.
 
 Here are a few of the plugins written by the beets community:
@@ -254,6 +258,11 @@ Here are a few of the plugins written by the beets community:
 * `beets-barcode`_ lets you scan or enter barcodes for physical media to
   search for their metadata.
 
+* `beets-ydl`_ downloads audio from youtube-dl sources and import into beets.
+
+* `beet-summarize`_ can compute lots of counts and statistics about your music
+  library.
+
 .. _beets-barcode: https://github.com/8h2a/beets-barcode
 .. _beets-check: https://github.com/geigerzaehler/beets-check
 .. _copyartifacts: https://github.com/sbarakat/beets-copyartifacts
@@ -273,3 +282,5 @@ Here are a few of the plugins written by the beets community:
 .. _whatlastgenre: https://github.com/YetAnotherNerd/whatlastgenre/tree/master/plugin/beets
 .. _beets-usertag: https://github.com/igordertigor/beets-usertag
 .. _beets-popularity: https://github.com/abba23/beets-popularity
+.. _beets-ydl: https://github.com/vmassuchetto/beets-ydl
+.. _beet-summarize: https://github.com/steven-murray/beet-summarize

docs/plugins/info.rst

@@ -42,4 +42,4 @@ Additional command-line options include:
 * ``--keys-only`` or ``-k``: Show the name of the tags without the values.
 
 .. _id3v2: http://id3v2.sourceforge.net
-.. _mp3info: http://www.ibiblio.org/mp3info/
+.. _mp3info: https://www.ibiblio.org/mp3info/

docs/plugins/ipfs.rst

@@ -4,7 +4,7 @@ IPFS Plugin
 The ``ipfs`` plugin makes it easy to share your library and music with friends.
 The plugin uses `ipfs`_ for storing the library and file content.
 
-.. _ipfs: http://ipfs.io/
+.. _ipfs: https://ipfs.io/
 
 Installation
 ------------

docs/plugins/keyfinder.rst

@@ -29,4 +29,4 @@ configuration file. The available options are:
   `initial_key` value.
   Default: ``no``.
 
-.. _KeyFinder: http://www.ibrahimshaath.co.uk/keyfinder/
+.. _KeyFinder: https://www.ibrahimshaath.co.uk/keyfinder/

docs/plugins/kodiupdate.rst

@@ -26,8 +26,8 @@ In Kodi's interface, navigate to System/Settings/Network/Services and choose "Al
 With that all in place, you'll see beets send the "update" command to your Kodi
 host every time you change your beets library.
 
-.. _Kodi: http://kodi.tv/
+.. _Kodi: https://kodi.tv/
-.. _requests: http://docs.python-requests.org/en/latest/
+.. _requests: https://docs.python-requests.org/en/latest/
 
 Configuration
 -------------

docs/plugins/lastgenre.rst

@@ -7,8 +7,8 @@ importing and autotagging music, beets does not assign a genre.  The
 to your albums and items.
 
 .. _does not contain genre information:
-    http://musicbrainz.org/doc/General_FAQ#Why_does_MusicBrainz_not_support_genre_information.3F
+    https://musicbrainz.org/doc/General_FAQ#Why_does_MusicBrainz_not_support_genre_information.3F
-.. _Last.fm: http://last.fm/
+.. _Last.fm: https://last.fm/
 
 Installation
 ------------
@@ -34,7 +34,7 @@ The genre list file should contain one genre per line. Blank lines are ignored.
 For the curious, the default genre list is generated by a `script that scrapes
 Wikipedia`_.
 
-.. _pip: http://www.pip-installer.org/
+.. _pip: https://pip.pypa.io
 .. _pylast: https://github.com/pylast/pylast
 .. _script that scrapes Wikipedia: https://gist.github.com/1241307
 .. _internal whitelist: https://raw.githubusercontent.com/beetbox/beets/master/beetsplug/lastgenre/genres.txt
@@ -72,7 +72,7 @@ nothing would ever be matched to a more generic node since all the specific
 subgenres are in the whitelist to begin with.
 
 
-.. _YAML: http://www.yaml.org/
+.. _YAML: https://www.yaml.org/
 .. _tree of nested genre names: https://raw.githubusercontent.com/beetbox/beets/master/beetsplug/lastgenre/genres-tree.yaml
 
 
@@ -155,7 +155,11 @@ Running Manually
 
 In addition to running automatically on import, the plugin can also be run manually
 from the command line. Use the command ``beet lastgenre [QUERY]`` to fetch
-genres for albums matching a certain query.
+genres for albums or items matching a certain query.
+
+By default, ``beet lastgenre`` matches albums. To match
+individual tracks or singletons, use the ``-A`` switch:
+``beet lastgenre -A [QUERY]``.
 
 To disable automatic genre fetching on import, set the ``auto`` config option
 to false.

docs/plugins/lastimport.rst

@@ -6,7 +6,7 @@ library into beets' database. You can later create :doc:`smart playlists
 </plugins/smartplaylist>` by querying ``play_count`` and do other fun stuff
 with this field.
 
-.. _Last.fm: http://last.fm
+.. _Last.fm: https://last.fm
 
 Installation
 ------------
@@ -23,7 +23,7 @@ Next, add your Last.fm username to your beets configuration file::
     lastfm:
         user: beetsfanatic
 
-.. _pip: http://www.pip-installer.org/
+.. _pip: https://pip.pypa.io
 .. _pylast: https://github.com/pylast/pylast
 
 Importing Play Counts