When we store paths in the database, we always use bytestrings for consistency.
But on Windows, these paths are converted back to Unicode before they reach the
FS API. This means that the codec used internally is immaterial.
However, we were naively using sys.getfilesystemencoding() for this internal
representation. On Windows, this is MBCS, a broken encoding that can't represent
all of Unicode. This change replaces that with UTF-8, a "real" codec.
The decoding bit now tries UTF-8 and falls back to MBCS for compatibility with
existing databases. The reality, however, is that existing databases may not
work with this change -- a byte string may represent something different in
UTF-8 from what it represents in MBCS. So users should recreated their DBs if
anything goes wrong.
The 'decode' call fails in what is already a unicode string. I'm not
sure under what circumstances the string is or isn't unicode (apparently
it varies), so I added a check. The test passes with the patch, at
least.
This allows matches to indicate both missing and unmatched tracks in their
candidates and solves some of the spaghetti tuples that were passed around
during autotagging.
This replaces order_items with assign_items, the first step to allowing unequal
numbers of items on either side of the equation (user files and canonical
tracks). Rather than returning a "holey" list and assuming that the TrackInfo
objects stay static, the function returns a dictionary mapping Item objects to
TrackInfo objects. To indicate unmatched objects, two sets are also returned.
For the moment, some temporary code is included to turn the result from this
new function into the old format (a holey Item list). This allowed me to test
this change in isolation before plunging ahead with the necessary refactoring to
expose all of this to the importer workflow, etc.
These new named tuples will represent candidates given by the autotagging
system. In the case of album matches, these are currently unwieldy,
verbosely-documented tuples that will only get more unwieldy with the addition
of "unmatched tracks".
This necessitated a slight refactoring in the plugin event handling mechanism.
Rather than loading all handlers up front and storing them in a module-scope
structure, we now scan for event handlers at every send(). This is probably
very slightly less efficient but allows for more flexible logic.
This is the crux of the matter. This new lock synchronizes accesses to the
database itself, allowing only one thread to have transactions active at a time.
In effect, this makes sure that beets only has one SQLite transaction active at
a time (and is very conservative in this effort; no read sharing is allowed
either). This prevents all contention in SQLite and hides the pathological
HAVE_SLEEP=0 behavior.
We now properly synchronize access to _tx_stacks and _connections, which can be
concurrently mutated by different threads. This way I don't have to worry about
GIL semantics: DRF => SC!
The Library object is now (almost) safe to share across threads. Per-thread
resources are now automatically managed internally. There is no longer any need
for the _reopen_lib hack to get multiple copies of the Library object.
