- adds another traversal through all plugins' template_fields for each
'evaluate_template' call.
- requires the following idiom (or equivalent):
@Plugin.template_field(field')
def _tmpl_field(album):
"""Return stuff.
"""
if isinstance(album, Album):
return stuff
Since both albums and items have an itime field, I'm putting itime into
ALBUM_KEYS_ITEM. This simplifies a lot of handling code. It also will make the
item and album itimes synchronize at some points, which is probably fine -- I
can't see a major use case for maintaining separate added dates for an album
and its tracks.
This isn't yet finished, it needs some input on how to organize the data, and actually where to implement the use of this data, but it already works in setting the date
Now the SELECT query for items only appears in Library.items(). All other
functions that retrieve items go through this method. This should make it
easier to change the way flexattrs are picked up.
With this change, we can get slightly closer to letting plugins extend the
query syntax with queries that don't pertain to a specific field. This will
likely need some more tweaking in the future, but it should allow for
some very interesting things.
The initial idea for this refactor was motivated by the need to make
PluginQuery.match() have the same method signature as the match() methods on
other queries. That is, it needed to take an *item*, not the pattern and
value. (The pattern is supplied when the query is constructed.) So it made
sense to move the value-to-pattern code to a class method.
But then I realized that all the other FieldQuery subclasses needed to do
essentially the same thing. So I eliminated PluginQuery altogether and
refactored FieldQuery to subsume its functionality. I then changed all the
other FieldQuery subclasses to conform to the same pattern.
This has the side effect of allowing different kinds of queries (even
non-field queries) down the road.
Here's another little experiment: to make flexattrs a little easier to use for
end users, you can now get and set them by using 'namespace-key' as the
argument to __getattr__ or __setattr__.
For example, try:
$ beet mod foo-bar=baz
$ beet ls -f '${foo-bar}'
baz
baz
baz
...
This uses two tables, item_attributes and album_attributes, as key/value
tables for the respective entities. Plugins register fields, at which point
they are magically materialized as properties on Items (Albums are not done).
This currently supports adding and modifying these fields but not retrieving
them (which will need some sort of join).
This message was being logged as an error every time MediaFile failed to parse a
file. But this is not actually an error -- the importer uses FileTypeErrors to
determine whether a file is music or not. This resulted in error logs for every
album art file, .m3u, etc. in the imported directory. Verbose output is a better
home for this message.
A user reported a problem with one of the logging statements where .format()
tried to convert a Unicode string to bytes because the log message was '', not
u''. As a rule, we should ensure that all logging statements use Unicode
literals.
With the new centralized print_obj function, we can greatly simplify the code
for the list command. This necessitated a couple of additional tweaks:
- For performance reasons, print_obj can now take a compiled template. (There's
still an issue with using the default/configured template, but we can cross
that bridge later).
- When listing albums, $path now expands to the album's item dir. So the format
string '$path' now exactly corresponds to passing the -p switch.
As an added bonus, we can now also reduce copypasta in the random plugin (which
behaves almost exactly the same as list).
This has been a long time coming, but we now finally keep track of ReplayGain
values in the database. This is an intermediate step toward a refactoring of the
RG plugin; at the moment, these values are not actually saved!
This should be used in *every* filename conversion, not just the destination
generation. Also required a change to sorted_walk (erroneously didn't use
syspath).
When we store paths in the database, we always use bytestrings for consistency.
But on Windows, these paths are converted back to Unicode before they reach the
FS API. This means that the codec used internally is immaterial.
However, we were naively using sys.getfilesystemencoding() for this internal
representation. On Windows, this is MBCS, a broken encoding that can't represent
all of Unicode. This change replaces that with UTF-8, a "real" codec.
The decoding bit now tries UTF-8 and falls back to MBCS for compatibility with
existing databases. The reality, however, is that existing databases may not
work with this change -- a byte string may represent something different in
UTF-8 from what it represents in MBCS. So users should recreated their DBs if
anything goes wrong.
This is the crux of the matter. This new lock synchronizes accesses to the
database itself, allowing only one thread to have transactions active at a time.
In effect, this makes sure that beets only has one SQLite transaction active at
a time (and is very conservative in this effort; no read sharing is allowed
either). This prevents all contention in SQLite and hides the pathological
HAVE_SLEEP=0 behavior.
We now properly synchronize access to _tx_stacks and _connections, which can be
concurrently mutated by different threads. This way I don't have to worry about
GIL semantics: DRF => SC!
The Library object is now (almost) safe to share across threads. Per-thread
resources are now automatically managed internally. There is no longer any need
for the _reopen_lib hack to get multiple copies of the Library object.
This is the first of several commits that will modernize the beets codebase for
Python 2.6 conventions. (Compatibility with Python 2.5 is hereby abandoned.)
In an attempt to finally address the longstanding SQLite locking issues, I'm
introducing a way to explicitly, lexically scope transactions. The Transaction
class is a context manager that always fully fetches after SELECTs and
automatically commits on exit. No direct access to the library is allowed, so
all changes will eventually be committed and all queries will be completed. This
will also provide a debugging mechanism to show where concurrent transactions
are beginning and ending.
To support composition (transaction reentrancy), an internal, per-Library stack
of transactions is maintained. Commits only happen when the outermost
transaction exits. This means that, while it's possible to introduce atomicity
bugs by invoking Library methods outside of a transaction, you can conveniently
call them *without* a currently-active transaction to get a single atomic
action.
Note that this "transaction stack" concepts assumes a single Library object per
thread. Because we need to duplicate Library objects for concurrent access due
to sqlite3 limitation already, this is fine for now. Later, the interface should
provide one transaction stack per thread for shared Library objects.
Instead of parsing the template at each call to destination(), it's now possible
to parse them *once*, a priori, and re-use the resulting template object. This
is analogous to the re module's compiled expressions.
Previously, all files would be moved/copied/deleted before the corresponding
Items and Albums were added to the database. Now, the in-place items are added
to the database; the files are moved; and then the new paths are saved to the
DB. The apply_choices coroutine now executes two database transactions per task.
This has a couple of benefits:
- %aunique{} requires album structures to be in place before the destination()
call, so this now works as expected.
- As an added bonus, the "in_album" parameter to move() and destination() --
along with its associated ugly hacks -- is no longer required.
For a less cumbersome uniquifying string, only a single field value is now used
instead of a prefix of a list of fields. The old semantics had two problems that
made it both unnecessary and insufficient:
- In the vast majority of cases, a single field suffices (year OR label OR
catalog number, for example) and forcing the string to include many identical
fields is unnecessary.
- If the albums are very similar, a prefix may be insufficient; a better
solution may be found with an arbitrary subset. (Of course, we can't afford to
search the whole power set.)
So we're going with a single field for now. This should cause far less
confusion.
The new fields are:
ALBUM: mb_releasegroupid asin catalognum script language country albumstatus
media albumdisambig
TRACK: disctitle encoder
These are not yet parsed from MusicBrainz responses (just added to MediaFile
and the database).
There's no longer a distinction between Unix and Windows substitutions. Enough
users reported problems with Windows-forbidden characters on Samba shares that
it seems appropriate to make all filenames Windows-safe, even on Unix. Users who
really want those additional characters (<>:"?*|\) can re-enable them via the
"replace" option. Nobody has complained about beets being *too* conservative.
This also adds sanitization of control characters, which is an all-around good
idea, and the substitution now runs in the Unicode (rather than byte) domain.
Previously, there was just an "artist sort name" field -- now there's a
corresponding sort name for both track artists and album artists. I also made
the names shorter (artist_sort and albumartist_sort).
Generates disambiguating strings to distinguish albums from one another. To be
used as the basis for a simpler path field, $unique, as a default disambiguator.
Using a class wrapper allows additional context to be provided to the functions.
Namely, we can now provide the Item itself to the function, so %foo{} could
conceivably do something useful even without arguments. This will be used for
the upcoming %unique function.
