# Improve release notes formatting / changelog conversion from rst to md
During our last release, we discovered issues with changelog formatting.
This PR improves and fixes several aspects:
## Changes
- Rewrite the changelog conversion logic to be more robust and
maintainable
- Fix indentation issues with nested bullet points
- Improve handling of long section headers
- Order bullet points alphabetically within sections for better
readability
- Use Sphinx `objects.inv` to resolve references and include links to
the documentation in _Markdown_
- Add tests to prevent formatting regressions
- Add pandoc as a dependency for Ubuntu CI builds
- Ensure documentation is built before generating changelog
## Problem
A regression was introduced when adjusting the track matching logic to
use `lapjv` instead of `munkres`. The `lapjv` algorithm returns `-1` for
unmatched items, which wasn't being handled correctly in the matching
logic. This caused incorrect track assignments when importing new music.
## Solution
- Modified the mapping creation to filter out unmatched items (where
index is `-1`)
- Updated test case to properly catch this scenario
These tests depend on certain `track_length_grace` and
`track_length_max` configuration which was set by other tests in this
module.
I discovered this issue when I tried to run
`test_order_works_when_track_names_are_entirely_wrong` test only
- I found that my local configuration was read and the test failed.
I had previously tested the `munkres` -> `lapjv` replacement
extensively, so I was today surprised to find that nothing gets matched
correctly when I tried importing some new tracks.
On the other hand I now remember making a small adjustment in the logic
to make autotagging tests pass which is when I introduced a bug: I did
not realize that `lapjv` returns index '-1' for each unmatched item.
This issue did not get caught by tests because this 'unmatched' item
index '-1' anecdotally ended up pointing to the last (expected) item in
the test making it pass.
This commit adjusts the aforementioned test to catch this issue and
fixes the logic to correctly identify unmatched tracks.
This utilises regex substitution in the substitute plugin. The previous
approach only used regex to match the pattern, then replaced it with a
static string. This change allows more complex substitutions, where the
output depends on the input.
### Example use case
Say we want to keep only the first artist of a multi-artist credit, as
in the following list:
```
Neil Young & Crazy Horse -> Neil Young
Michael Hurley, The Holy Modal Rounders, Jeffrey Frederick & The Clamtones -> Michael Hurley
James Yorkston and the Athletes -> James Yorkston
````
This would previously have required three separate rules, one for each
resulting artist. By using a regex substitution, we can get the desired
behaviour in a single rule:
```yaml
substitute:
^(.*?)(,| &| and).*: \1
```
(Capture the text until the first `,` ` &` or ` and`, then use that
capture group as the output)
### Notes
I've kept the previous behaviour of only applying the first matching
rule, but I'm not 100% sure it's the ideal approach.
I can imagine both cases where you want to apply several rules in
sequence and cases where you want to stop after the first match.
This PR refactors the test codebase by removing redundant functions and
simplifying item and album creation. Key changes include:
- Removed redundant `_item_ident` index tracker from `_common.py`.
- Removed `album` function from `_common.py` replacing it with direct
`library.Album` invocations.
- Removed `generate_album_info` and `generate_track_info` functions,
replacing them directly with `TrackInfo` and `AlbumInfo`.
- Updated `setup.cfg` to exclude test helper files from coverage
reports.
- Adjusted the tests regarding the changes, and simplified
`test_mbsync.py`.
These functions were used to generate mock data for tests but have been
replaced with direct instantiation of AlbumInfo and TrackInfo objects.
This change simplifies the test code and removes unnecessary helper
functions.
- Refactored Tekstowo backend to fetch lyrics directly from song pages.
- Added `encode` method to convert artist and title to their URL format,
where non-alphanumeric characters are replaced with underscores.
- Removed the now redundant search functionality and associated tests.
- Simplified `extract_lyrics` method to directly parse lyrics without
any checks.
