Include tests and manual in sdist and update dependencies, and include changelog to the release (#5520)

* Fixes #5513
* Fixes #5518

This PR adds the `test` folder and manuals to the package source tarball
and upgrades the dependencies to fix a few vulnerabilities.

I also added a fix for the changelog not making it to the release notes.

.github/workflows/make_release.yaml

@@ -40,6 +40,8 @@ jobs:
     name: Get changelog and build the distribution package
     runs-on: ubuntu-latest
     needs: increment-version
+    outputs:
+      changelog: ${{ steps.generate_changelog.outputs.changelog }}
     steps:
       - uses: actions/checkout@v4
         with:
@@ -53,12 +55,13 @@ jobs:
           cache: poetry
 
       - name: Install dependencies
-        run: poetry install --only=release
+        run: poetry install --with=release --extras=docs
 
       - name: Install pandoc
         run: sudo apt update && sudo apt install pandoc -y
 
       - name: Obtain the changelog
+        id: generate_changelog
         run: |
           {
             echo 'changelog<<EOF'
@@ -67,7 +70,7 @@ jobs:
           } >> "$GITHUB_OUTPUT"
 
       - name: Build a binary wheel and a source tarball
-        run: poetry build
+        run: poe build
 
       - name: Store the distribution packages
         uses: actions/upload-artifact@v4

.pre-commit-config.yaml

@@ -3,6 +3,6 @@
 
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.6.6
+    rev: v0.8.0
     hooks:
       - id: ruff-format

docs/changelog.rst

@@ -20,15 +20,18 @@ Bug fixes:
 * Fix bug where matcher doesn't consider medium number when importing. This makes
   it difficult to import hybrid SACDs and other releases with duplicate tracks.
   :bug:`5148`
+* Bring back test files and the manual to the source distribution tarball.
+  :bug:`5513`
 
 For packagers:
 
 Other changes:
 
-* Changed `bitesize` label to `good first issue`. Our
-  [contribute](https://github.com/beetbox/beets/contribute) page is now
+* Changed `bitesize` label to `good first issue`. Our `contribute`_ page is now
   automatically populated with these issues. :bug:`4855`
 
+.. _contribute: https://github.com/beetbox/beets/contribute
+
 2.1.0 (November 22, 2024)
 -------------------------
 
@@ -37,10 +40,10 @@ New features:
 * New template function added: ``%capitalize``. Converts the first letter of
   the text to uppercase and the rest to lowercase.
 * Ability to query albums with track db fields and vice-versa, for example
-  `beet list -a title:something` or `beet list artpath:cover`. Consequently
-  album queries involving `path` field have been sped up, like `beet list -a
-  path:/path/`.
-* :doc:`plugins/ftintitle`: New `keep_in_artist` option for the plugin, which
+  ``beet list -a title:something`` or ``beet list artpath:cover``. Consequently
+  album queries involving ``path`` field have been sped up, like ``beet list -a
+  path:/path/``.
+* :doc:`plugins/ftintitle`: New ``keep_in_artist`` option for the plugin, which
   allows keeping the "feat." part in the artist metadata while still changing
   the title.
 * :doc:`plugins/autobpm`: Add new configuration option ``beat_track_kwargs``
@@ -68,7 +71,7 @@ Bug fixes:
   issues in the future.
   :bug:`5289`
 * :doc:`plugins/discogs`: Fix the ``TypeError`` when there is no description.
-* Remove single quotes from all SQL queries
+* Use single quotes in all SQL queries
   :bug:`4709`
 * :doc:`plugins/lyrics`: Update ``tekstowo`` backend to fetch lyrics directly
   since recent updates to their website made it unsearchable.
@@ -82,22 +85,22 @@ Bug fixes:
 For packagers:
 
 * The minimum supported Python version is now 3.8.
-* The `beet` script has been removed from the repository.
-* The `typing_extensions` is required for Python 3.10 and below.
+* The ``beet`` script has been removed from the repository.
+* The ``typing_extensions`` is required for Python 3.10 and below.
 
 Other changes:
 
-* :doc:`contributing`: The project now uses `poetry` for packaging and
+* :doc:`contributing`: The project now uses ``poetry`` for packaging and
   dependency management. This change affects project management and mostly
   affects beets developers. Please see updates in :ref:`getting-the-source` and
   :ref:`testing` for more information.
-* :doc:`contributing`: Since `poetry` now manages local virtual environments,
-  `tox` has been replaced by a task runner `poethepoet`. This change affects
+* :doc:`contributing`: Since ``poetry`` now manages local virtual environments,
+  `tox` has been replaced by a task runner ``poethepoet``. This change affects
   beets developers and contributors. Please see updates in the
   :ref:`development-tools` section for more details. Type ``poe`` while in
   the project directory to see the available commands.
 * Installation instructions have been made consistent across plugins
-  documentation. Users should simply install `beets` with an `extra` of the
+  documentation. Users should simply install ``beets`` with an ``extra`` of the
   corresponding plugin name in order to install extra dependencies for that
   plugin.
 * GitHub workflows have been reorganised for clarity: style, linting, type and
@@ -108,10 +111,10 @@ Other changes:
   documentation is changed, and they only check the changed files. When
   dependencies are updated (``poetry.lock``), then the entire code base is
   checked.
-* The long-deprecated `beets.util.confit` module has been removed.  This may
+* The long-deprecated ``beets.util.confit`` module has been removed.  This may
   cause extremely outdated external plugins to fail to load.
-* :doc:`plugins/autobpm`: Add plugin dependencies to `pyproject.toml` under
-  the `autobpm` extra and update the plugin installation instructions in the
+* :doc:`plugins/autobpm`: Add plugin dependencies to ``pyproject.toml`` under
+  the ``autobpm`` extra and update the plugin installation instructions in the
   docs.
   Since importing the bpm calculation functionality from ``librosa`` takes
   around 4 seconds, update the plugin to only do so when it actually needs to

pyproject.toml

@@ -26,6 +26,7 @@ packages = [
     { include = "beets" },
     { include = "beetsplug" },
 ]
+include = ["test", "man/**/*"] # extra files to include in the sdist
 
 [tool.poetry.urls]
 Changelog = "https://github.com/beetbox/beets/blob/master/docs/changelog.rst"
@@ -149,6 +150,15 @@ build-backend = "poetry.core.masonry.api"
 poethepoet = ">=0.26"
 poetry = ">=1.8"
 
+[tool.poe.tasks.build]
+help = "Build the package"
+shell = """
+make -C docs man
+rm -rf man
+mv docs/_build/man .
+poetry build
+"""
+
 [tool.poe.tasks.bump]
 help = "Bump project version and update relevant files"
 cmd = "python ./extra/release.py bump $version"