Add broken link checker to integration test (#3703)

* fix broken links
* add link check command to tox
* add link check to the weekly integration test

.github/workflows/integration_test.yaml

@@ -27,6 +27,10 @@ jobs:
         run: |
           tox -e int
 
+      - name: Check external links in docs
+        run: |
+          tox -e links
+
       - name: Notify on failure
         if: ${{ failure() }}
         env:

CONTRIBUTING.rst

@@ -28,7 +28,7 @@ Non-Programming
 
 -  Promote beets! Help get the word out by telling your friends, writing
    a blog post, or discussing it on a forum you frequent.
--  Improve the `documentation <http://beets.readthedocs.org/>`__. It’s
+-  Improve the `documentation`_. It’s
    incredibly easy to contribute here: just find a page you want to
    modify and hit the “Edit on GitHub” button in the upper-right. You
    can automatically send us a pull request for your changes.
@@ -62,7 +62,7 @@ Getting the Source
 ^^^^^^^^^^^^^^^^^^
 
 The easiest way to get started with the latest beets source is to use
-`pip <https://pip.pypa.io/>`__ to install an “editable” package. This
+`pip`_ to install an “editable” package. This
 can be done with one command:
 
 .. code-block:: bash
@@ -147,8 +147,7 @@ request and your code will ship in no time.
 5. Add a changelog entry to ``docs/changelog.rst`` near the top of the
    document.
 6. Run the tests and style checker. The easiest way to run the tests is
-   to use `tox <https://tox.readthedocs.org/en/latest/>`__. For more
-   information on running tests, see :ref:`testing`.
+   to use `tox`_. For more information on running tests, see :ref:`testing`.
 7. Push to your fork and open a pull request! We’ll be in touch shortly.
 8. If you add commits to a pull request, please add a comment or
    re-request a review after you push them since GitHub doesn’t
@@ -253,7 +252,7 @@ guidelines to follow:
 Editor Settings
 ---------------
 
-Personally, I work on beets with `vim <http://www.vim.org/>`__. Here are
+Personally, I work on beets with `vim`_. Here are
 some ``.vimrc`` lines that might help with PEP 8-compliant Python
 coding::
 
@@ -318,7 +317,7 @@ To install the test dependencies, run ``python -m pip install .[test]``.
 Or, just run a test suite with ``tox`` which will install them
 automatically.
 
-.. _setup.py: https://github.com/beetbox/beets/blob/master/setup.py#L99`
+.. _setup.py: https://github.com/beetbox/beets/blob/master/setup.py
 
 Writing Tests
 -------------
@@ -352,9 +351,9 @@ others. See `unittest.mock`_ for more info.
 .. _Python unittest: https://docs.python.org/2/library/unittest.html
 .. _Codecov: https://codecov.io/github/beetbox/beets
 .. _pytest-random: https://github.com/klrmn/pytest-random
-.. _tox: http://tox.readthedocs.org
-.. _detox: https://pypi.python.org/pypi/detox/
-.. _pytest: http://pytest.org
+.. _tox: https://tox.readthedocs.io/en/latest/
+.. _detox: https://pypi.org/project/detox/
+.. _pytest: https://docs.pytest.org/en/stable/
 .. _Linux: https://github.com/beetbox/beets/actions
 .. _Windows: https://ci.appveyor.com/project/beetbox/beets/
 .. _`https://github.com/beetbox/beets/blob/master/setup.py#L99`: https://github.com/beetbox/beets/blob/master/setup.py#L99
@@ -364,3 +363,6 @@ others. See `unittest.mock`_ for more info.
 .. _integration test: https://github.com/beetbox/beets/actions?query=workflow%3A%22integration+tests%22
 .. _unittest.mock: https://docs.python.org/3/library/unittest.mock.html
 .. _Python unittest: https://docs.python.org/2/library/unittest.html
+.. _documentation: https://beets.readthedocs.io/en/stable/
+.. _pip: https://pip.pypa.io/en/stable/
+.. _vim: https://www.vim.org/

docs/changelog.rst

@@ -196,7 +196,7 @@ Fixes:
 * ``beet update`` will now confirm that the user still wants to update if
   their library folder cannot be found, preventing the user from accidentally
   wiping out their beets database.
-  Thanks to :user:`logan-arens`.
+  Thanks to user: `logan-arens`.
   :bug:`1934`
 * :doc:`/plugins/bpd`: Fix the transition to next track when in consume mode.
   Thanks to :user:`aereaux`.
@@ -1273,7 +1273,7 @@ And there are a few bug fixes too:
 The last release, 1.3.19, also erroneously reported its version as "1.3.18"
 when you typed ``beet version``. This has been corrected.
 
-.. _six: https://pythonhosted.org/six/
+.. _six: https://pypi.org/project/six/
 
 
 1.3.19 (June 25, 2016)
@@ -2119,7 +2119,7 @@ As usual, there are loads of little fixes and improvements:
 * The :ref:`config-cmd` command can now use ``$EDITOR`` variables with
   arguments.
 
-.. _API changes: https://developer.echonest.com/forums/thread/3650
+.. _API changes: https://web.archive.org/web/20160814092627/https://developer.echonest.com/forums/thread/3650
 .. _Plex: https://plex.tv/
 .. _musixmatch: https://www.musixmatch.com/
 
@@ -2344,7 +2344,7 @@ The big new features are:
 * A new :ref:`asciify-paths` configuration option replaces all non-ASCII
   characters in paths.
 
-.. _Mutagen: https://bitbucket.org/lazka/mutagen
+.. _Mutagen: https://github.com/quodlibet/mutagen
 .. _Spotify: https://www.spotify.com/
 
 And the multitude of little improvements and fixes:
@@ -2599,7 +2599,7 @@ Fixes:
 * :doc:`/plugins/convert`: Display a useful error message when the FFmpeg
   executable can't be found.
 
-.. _requests: https://www.python-requests.org/
+.. _requests: https://requests.readthedocs.io/en/master/
 
 
 1.3.3 (February 26, 2014)
@@ -2780,7 +2780,7 @@ As usual, there are also innumerable little fixes and improvements:
   Bezman.
 
 
-.. _Acoustic Attributes: http://developer.echonest.com/acoustic-attributes.html
+.. _Acoustic Attributes: https://web.archive.org/web/20160701063109/http://developer.echonest.com/acoustic-attributes.html
 .. _MPD: https://www.musicpd.org/
 
 
@@ -3130,7 +3130,7 @@ will automatically migrate your configuration to the new system.
   header. Thanks to Uwe L. Korn.
 * :doc:`/plugins/lastgenre`: Fix an error when using genre canonicalization.
 
-.. _Tomahawk: https://tomahawk-player.org/
+.. _Tomahawk: https://github.com/tomahawk-player/tomahawk
 
 1.1b3 (March 16, 2013)
 ----------------------
@@ -3473,7 +3473,7 @@ begins today on features for version 1.1.
 * Changed plugin loading so that modules can be imported without
   unintentionally loading the plugins they contain.
 
-.. _The Echo Nest: http://the.echonest.com/
+.. _The Echo Nest: https://web.archive.org/web/20180329103558/http://the.echonest.com/
 .. _Tomahawk resolver: https://beets.io/blog/tomahawk-resolver.html
 .. _mp3gain: http://mp3gain.sourceforge.net/download.php
 .. _aacgain: https://aacgain.altosdesign.com
@@ -3911,7 +3911,7 @@ plugin.
 
 * The :doc:`/plugins/web` encapsulates a simple **Web-based GUI for beets**. The
   current iteration can browse the library and play music in browsers that
-  support `HTML5 Audio`_.
+  support HTML5 Audio.
 
 * When moving items that are part of an album, the album art implicitly moves
   too.
@@ -3928,8 +3928,6 @@ plugin.
 
 * Fix crash when "copying" an art file that's already in place.
 
-.. _HTML5 Audio: http://www.w3.org/TR/html-markup/audio.html
-
 1.0b9 (July 9, 2011)
 --------------------
 

docs/conf.py

@@ -28,6 +28,13 @@ extlinks = {
     'stdlib': ('https://docs.python.org/3/library/%s.html', ''),
 }
 
+linkcheck_ignore = [
+    r'https://github.com/beetbox/beets/issues/',
+    r'https://github.com/\w+$',  # ignore user pages
+    r'.*localhost.*',
+    r'https://www.musixmatch.com/',  # blocks requests
+]
+
 # Options for HTML output
 htmlhelp_basename = 'beetsdoc'
 

docs/dev/index.rst

@@ -7,7 +7,7 @@ in hacking beets itself or creating plugins for it.
 See also the documentation for `MediaFile`_, the library used by beets to read
 and write metadata tags in media files.
 
-.. _MediaFile: https://mediafile.readthedocs.io/
+.. _MediaFile: https://mediafile.readthedocs.io/en/latest/
 
 .. toctree::
 

docs/dev/library.rst

@@ -45,7 +45,7 @@ responsible for handling queries to retrieve stored objects.
 
     .. automethod:: transaction
 
-.. _SQLite: https://sqlite.org/
+.. _SQLite: https://sqlite.org/index.html
 .. _ORM: https://en.wikipedia.org/wiki/Object-relational_mapping
 
 
@@ -118,7 +118,7 @@ To make changes to either the database or the tags on a file, you
 update an item's fields (e.g., ``item.title = "Let It Be"``) and then call
 ``item.write()``.
 
-.. _MediaFile: https://mediafile.readthedocs.io/
+.. _MediaFile: https://mediafile.readthedocs.io/en/latest/
 
 Items also track their modification times (mtimes) to help detect when they
 become out of sync with on-disk metadata, mainly to speed up the

docs/dev/plugins.rst

@@ -301,7 +301,7 @@ To access this value, say ``self.config['foo'].get()`` at any point in your
 plugin's code. The `self.config` object is a *view* as defined by the `Confuse`_
 library.
 
-.. _Confuse: https://confuse.readthedocs.org/
+.. _Confuse: https://confuse.readthedocs.io/en/latest/
 
 If you want to access configuration values *outside* of your plugin's section,
 import the `config` object from the `beets` module. That is, just put ``from
@@ -379,7 +379,7 @@ access to file tags. If you have created a descriptor you can add it through
 your plugins ``add_media_field()`` method.
 
 .. automethod:: beets.plugins.BeetsPlugin.add_media_field
-.. _MediaFile: https://mediafile.readthedocs.io/
+.. _MediaFile: https://mediafile.readthedocs.io/en/latest/
 
 
 Here's an example plugin that provides a meaningless new field "foo"::

docs/faq.rst

@@ -2,10 +2,9 @@ FAQ
 ###
 
 Here are some answers to frequently-asked questions from IRC and elsewhere.
-Got a question that isn't answered here? Try `IRC`_, the `discussion board`_, or
+Got a question that isn't answered here? Try the `discussion board`_, or
 :ref:`filing an issue <bugs>` in the bug tracker.
 
-.. _IRC: irc://irc.freenode.net/beets
 .. _mailing list: https://groups.google.com/group/beets-users
 .. _discussion board: https://discourse.beets.io
 
@@ -119,7 +118,7 @@ Run a command like this::
 
     pip install -U beets
 
-The ``-U`` flag tells `pip <https://pip.pypa.io/>`__ to upgrade
+The ``-U`` flag tells `pip`_ to upgrade
 beets to the latest version. If you want a specific version, you can
 specify with using ``==`` like so::
 
@@ -188,7 +187,9 @@ there to report a bug. Please follow these guidelines when reporting an issue:
 
 If you've never reported a bug before, Mozilla has some well-written
 `general guidelines for good bug
-reports <https://www.mozilla.org/bugs/>`__.
+reports`_.
+
+.. _general guidelines for good bug reports: https://developer.mozilla.org/en-US/docs/Mozilla/QA/Bug_writing_guidelines
 
 
 .. _find-config:
@@ -300,8 +301,7 @@ a flag. There is no simple way to remedy this.)
 …not change my ID3 tags?
 ------------------------
 
-Beets writes `ID3v2.4 <http://www.id3.org/id3v2.4.0-structure>`__ tags by
-default.
+Beets writes `ID3v2.4`_ tags by default.
 Some software, including Windows (i.e., Windows Explorer and Windows
 Media Player) and `id3lib/id3v2 <http://id3v2.sourceforge.net/>`__,
 don't support v2.4 tags. When using 2.4-unaware software, it might look
@@ -311,6 +311,7 @@ To enable ID3v2.3 tags, enable the :ref:`id3v23` config option.
 
 
 .. _invalid:
+.. _ID3v2.4: https://id3.org/id3v2.4.0-structure
 
 …complain that a file is "unreadable"?
 --------------------------------------
@@ -379,3 +380,4 @@ installed using pip, the command ``pip show -f beets`` can show you where
 try `this Super User answer`_.
 
 .. _this Super User answer: https://superuser.com/a/284361/4569
+.. _pip: https://pip.pypa.io/en/stable/

docs/guides/main.rst

@@ -64,7 +64,7 @@ beets`` if you run into permissions problems).
 To install without pip, download beets from `its PyPI page`_ and run ``python
 setup.py install`` in the directory therein.
 
-.. _its PyPI page: https://pypi.org/project/beets#downloads
+.. _its PyPI page: https://pypi.org/project/beets/#files
 .. _pip: https://pip.pypa.io
 
 The best way to upgrade beets to a new version is by running ``pip install -U

docs/plugins/absubmit.rst

@@ -62,6 +62,6 @@ file. The available options are:
 .. _streaming_extractor_music: https://acousticbrainz.org/download
 .. _FAQ: https://acousticbrainz.org/faq
 .. _pip: https://pip.pypa.io
-.. _requests: https://docs.python-requests.org/en/master/
+.. _requests: https://requests.readthedocs.io/en/master/
 .. _github: https://github.com/MTG/essentia
 .. _AcousticBrainz: https://acousticbrainz.org

docs/plugins/beatport.rst

@@ -41,6 +41,6 @@ Configuration
 
 This plugin can be configured like other metadata source plugins as described in :ref:`metadata-source-plugin-configuration`.
 
-.. _requests: https://docs.python-requests.org/en/latest/
+.. _requests: https://requests.readthedocs.io/en/master/
 .. _requests_oauthlib: https://github.com/requests/requests-oauthlib
-.. _Beatport: https://beetport.com
+.. _Beatport: https://www.beatport.com/

docs/plugins/bpd.rst

@@ -5,7 +5,7 @@ BPD is a music player using music from a beets library. It runs as a daemon and
 implements the MPD protocol, so it's compatible with all the great MPD clients
 out there. I'm using `Theremin`_, `gmpc`_, `Sonata`_, and `Ario`_ successfully.
 
-.. _Theremin: https://theremin.sigterm.eu/
+.. _Theremin: https://github.com/TheStalwart/Theremin
 .. _gmpc: https://gmpc.wikia.com/wiki/Gnome_Music_Player_Client
 .. _Sonata: http://sonata.berlios.de/
 .. _Ario: http://ario-player.sourceforge.net/
@@ -13,7 +13,7 @@ out there. I'm using `Theremin`_, `gmpc`_, `Sonata`_, and `Ario`_ successfully.
 Dependencies
 ------------
 
-Before you can use BPD, you'll need the media library called GStreamer (along
+Before you can use BPD, you'll need the media library called `GStreamer`_ (along
 with its Python bindings) on your system.
 
 * On Mac OS X, you can use `Homebrew`_. Run ``brew install gstreamer
@@ -22,14 +22,11 @@ with its Python bindings) on your system.
 * On Linux, you need to install GStreamer 1.0 and the GObject bindings for
   python. Under Ubuntu, they are called ``python-gi`` and ``gstreamer1.0``.
 
-* On Windows, you may want to try `GStreamer WinBuilds`_ (caveat emptor: I
-  haven't tried this).
-
 You will also need the various GStreamer plugin packages to make everything
 work. See the :doc:`/plugins/chroma` documentation for more information on
 installing GStreamer plugins.
 
-.. _GStreamer WinBuilds: https://www.gstreamer-winbuild.ylatuya.es/
+.. _GStreamer: https://gstreamer.freedesktop.org/download
 .. _Homebrew: https://brew.sh
 
 Usage

docs/plugins/convert.rst

@@ -191,7 +191,7 @@ can use the :doc:`/plugins/replaygain` to do this analysis. See the LAME
 `documentation`_ and the `HydrogenAudio wiki`_ for other LAME configuration
 options and a thorough discussion of MP3 encoding.
 
-.. _documentation: http://lame.sourceforge.net/using.php
+.. _documentation: https://lame.sourceforge.io/index.php
 .. _HydrogenAudio wiki: https://wiki.hydrogenaud.io/index.php?title=LAME
 .. _gapless: https://wiki.hydrogenaud.io/index.php?title=Gapless_playback
-.. _LAME: https://lame.sourceforge.net/
+.. _LAME: https://lame.sourceforge.io/index.php

docs/plugins/embyupdate.rst

@@ -18,7 +18,7 @@ To use the ``embyupdate`` plugin you need to install the `requests`_ library wit
 With that all in place, you'll see beets send the "update" command to your Emby server every time you change your beets library.
 
 .. _Emby: https://emby.media/
-.. _requests: https://docs.python-requests.org/en/latest/
+.. _requests: https://requests.readthedocs.io/en/master/
 
 Configuration
 -------------

docs/plugins/keyfinder.rst

@@ -31,5 +31,5 @@ configuration file. The available options are:
   `initial_key` value.
   Default: ``no``.
 
-.. _KeyFinder: https://www.ibrahimshaath.co.uk/keyfinder/
+.. _KeyFinder: http://www.ibrahimshaath.co.uk/keyfinder/
 .. _keyfinder-cli: https://github.com/EvanPurkhiser/keyfinder-cli/

docs/plugins/kodiupdate.rst

@@ -27,7 +27,7 @@ With that all in place, you'll see beets send the "update" command to your Kodi
 host every time you change your beets library.
 
 .. _Kodi: https://kodi.tv/
-.. _requests: https://docs.python-requests.org/en/latest/
+.. _requests: https://requests.readthedocs.io/en/master/
 
 Configuration
 -------------

docs/plugins/lastgenre.rst

@@ -1,13 +1,10 @@
 LastGenre Plugin
 ================
 
-The MusicBrainz database `does not contain genre information`_. Therefore, when
-importing and autotagging music, beets does not assign a genre.  The
-``lastgenre`` plugin fetches *tags* from `Last.fm`_ and assigns them as genres
+
+The ``lastgenre`` plugin fetches *tags* from `Last.fm`_ and assigns them as genres
 to your albums and items.
 
-.. _does not contain genre information:
-    https://musicbrainz.org/doc/General_FAQ#Why_does_MusicBrainz_not_support_genre_information.3F
 .. _Last.fm: https://last.fm/
 
 Installation
@@ -72,7 +69,7 @@ nothing would ever be matched to a more generic node since all the specific
 subgenres are in the whitelist to begin with.
 
 
-.. _YAML: https://www.yaml.org/
+.. _YAML: https://yaml.org/
 .. _tree of nested genre names: https://raw.githubusercontent.com/beetbox/beets/master/beetsplug/lastgenre/genres-tree.yaml
 
 

docs/plugins/lyrics.rst

@@ -26,7 +26,7 @@ already have them. The lyrics will be stored in the beets database. If the
 ``import.write`` config option is on, then the lyrics will also be written to
 the files' tags.
 
-.. _requests: https://docs.python-requests.org/en/latest/
+.. _requests: https://requests.readthedocs.io/en/master/
 
 
 Configuration
@@ -180,8 +180,7 @@ You also need to register for a Microsoft Azure Marketplace free account and
 to the `Microsoft Translator API`_. Follow the four steps process, specifically
 at step 3 enter ``beets`` as *Client ID* and copy/paste the generated
 *Client secret* into your ``bing_client_secret`` configuration, alongside
-``bing_lang_to`` target `language code`_.
+``bing_lang_to`` target `language code`.
 
 .. _langdetect: https://pypi.python.org/pypi/langdetect
-.. _Microsoft Translator API: https://www.microsoft.com/en-us/translator/getstarted.aspx
-.. _language code: https://msdn.microsoft.com/en-us/library/hh456380.aspx
+.. _Microsoft Translator API: https://docs.microsoft.com/en-us/azure/cognitive-services/translator/translator-how-to-signup

docs/plugins/plexupdate.rst

@@ -25,7 +25,7 @@ With that all in place, you'll see beets send the "update" command to your Plex
 server every time you change your beets library.
 
 .. _Plex: https://plex.tv/
-.. _requests: https://docs.python-requests.org/en/latest/
+.. _requests: https://requests.readthedocs.io/en/master/
 .. _documentation about tokens: https://support.plex.tv/hc/en-us/articles/204059436-Finding-your-account-token-X-Plex-Token
 
 Configuration

docs/plugins/subsonicupdate.rst

@@ -4,7 +4,7 @@ SubsonicUpdate Plugin
 ``subsonicupdate`` is a very simple plugin for beets that lets you automatically
 update `Subsonic`_'s index whenever you change your beets library.
 
-.. _Subsonic: https://www.subsonic.org
+.. _Subsonic: http://www.subsonic.org/pages/index.jsp
 
 To use ``subsonicupdate`` plugin, enable it in your configuration
 (see :ref:`using-plugins`).

docs/plugins/web.rst

@@ -19,8 +19,6 @@ The Web interface depends on `Flask`_. To get it, just run ``pip install
 flask``. Then enable the ``web`` plugin in your configuration (see
 :ref:`using-plugins`).
 
-.. _Flask: https://flask.pocoo.org/
-
 If you need CORS (it's disabled by default---see :ref:`web-cors`, below), then
 you also need `flask-cors`_. Just type ``pip install flask-cors``.
 
@@ -47,9 +45,7 @@ Usage
 -----
 
 Type queries into the little search box. Double-click a track to play it with
-`HTML5 Audio`_.
-
-.. _HTML5 Audio: http://www.w3.org/TR/html-markup/audio.html
+HTML5 Audio.
 
 Configuration
 -------------
@@ -78,7 +74,7 @@ The Web backend is built using a simple REST+JSON API with the excellent
 `Flask`_ library. The frontend is a single-page application written with
 `Backbone.js`_. This allows future non-Web clients to use the same backend API.
 
-.. _Flask: https://flask.pocoo.org/
+
 .. _Backbone.js: https://backbonejs.org
 
 Eventually, to make the Web player really viable, we should use a Flash fallback
@@ -90,7 +86,7 @@ for unsupported formats/browsers. There are a number of options for this:
 
 .. _audio.js: https://kolber.github.io/audiojs/
 .. _html5media: https://html5media.info/
-.. _MediaElement.js: https://mediaelementjs.com/
+.. _MediaElement.js: https://www.mediaelementjs.com/
 
 .. _web-cors:
 
@@ -262,3 +258,5 @@ Responds with the number of tracks and albums in the database. ::
       "items": 5,
       "albums": 3
     }
+
+.. _Flask: https://flask.palletsprojects.com/en/1.1.x/

docs/reference/config.rst

@@ -689,7 +689,7 @@ to one request per second.
 .. _your own MusicBrainz database: https://musicbrainz.org/doc/MusicBrainz_Server/Setup
 .. _main server: https://musicbrainz.org/
 .. _limited: https://musicbrainz.org/doc/XML_Web_Service/Rate_Limiting
-.. _Building search indexes: https://musicbrainz.org/doc/MusicBrainz_Server/Setup#Building_search_indexes
+.. _Building search indexes: https://musicbrainz.org/doc/Development/Search_server_setup
 
 .. _searchlimit:
 

tox.ini

@@ -27,6 +27,12 @@ basepython = python2.7
 deps = sphinx
 commands = sphinx-build -W -q -b html docs {envtmpdir}/html {posargs}
 
+# checks all links in the docs
+[testenv:links]
+deps = sphinx
+allowlist_externals = /bin/bash
+commands = /bin/bash -c '! sphinx-build -b linkcheck docs {envtmpdir}/linkcheck | grep "broken\s"'
+
 [testenv:int]
 deps = {[_test]deps}
 setenv = INTEGRATION_TEST = 1