mirror of
https://github.com/beetbox/beets.git
synced 2025-12-28 03:22:39 +01:00
switch to Kramdown for Markdown rendering
RDiscount's "smart" mode is not very smart at all. It renders -- as an emdash instead of an endash! Who ever heard of that? And apostrophes at the ends of words (as in "beets' scale") are not curlified, which seems like crime. Kramdown gets both of these right.
This commit is contained in:
parent
d16d877fcb
commit
d6d2a02e35
4 changed files with 17 additions and 17 deletions
|
|
@ -1,5 +1,5 @@
|
|||
markdown: rdiscount
|
||||
rdiscount:
|
||||
extensions: [smart]
|
||||
markdown: kramdown
|
||||
kramdown:
|
||||
entity_output: true
|
||||
url: http://beets.radbox.org
|
||||
permalink: /blog/:title.html
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ title: Tomahawk resolver
|
|||
layout: main
|
||||
section: blog
|
||||
---
|
||||
Beets is a music library manager--not, for the most part, a music player. It does include a [simple player plugin][bpd] and an [experimental Web-based player][web], but it generally leaves actual sound-reproduction to specialized tools.
|
||||
Beets is a music library manager---not, for the most part, a music player. It does include a [simple player plugin][bpd] and an [experimental Web-based player][web], but it generally leaves actual sound-reproduction to specialized tools.
|
||||
|
||||
[Tomahawk][] is one particularly exciting new open-source music player. The magic of Tomahawk lies in its ability to consolidate many sources of music into a single player interface. (It's also a very nicely-designed, cross-platform player even if you don't count the magic.) To integrate new sources of music with Tomahawk, you just have to provide a [resolver][]: a piece of code that searches for music and gives it to Tomahawk to display and play back.
|
||||
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ title: the road to 1.0
|
|||
section: blog
|
||||
layout: main
|
||||
---
|
||||
Beets has been in beta forever--for small values of *forever*, at least. I made [the first commit][firstcommit] to the original Subversion repository (for shame!) on May 14, 2008 and uploaded [version 1.0b1][b1] on June 17, 2010. I'm now working on beets' sixteenth beta. It's high time that the project hit the big 1.0. I'm excited to give beets the point-zero stamp of approval, but I have two big, backwards-incompatible changes I want to make before sprouting a stable maintenance branch. In this post, I'll describe my plans for beets 1.0b16 and b17--and beyond.
|
||||
Beets has been in beta forever---for small values of *forever*, at least. I made [the first commit][firstcommit] to the original Subversion repository (for shame!) on May 14, 2008 and uploaded [version 1.0b1][b1] on June 17, 2010. I'm now working on beets' sixteenth beta. It's high time that the project hit the big 1.0. I'm excited to give beets the point-zero stamp of approval, but I have two big, backwards-incompatible changes I want to make before sprouting a stable maintenance branch. In this post, I'll describe my plans for beets 1.0b16 and b17---and beyond.
|
||||
|
||||
## Beta 16
|
||||
|
||||
|
|
@ -15,7 +15,7 @@ As nerd-oriented software, beets has a *lot* of [configuration options][config].
|
|||
* [YAML][] syntax. The config file has grown a troubling number of half-working syntax hacks over the years. Lists of things have to be separated by whitespace, which means you can't use spaces in [regular expressions][replace]. Egregiously, I had to force users to substitute `_` characters for `:`s in query-based [path format][pathconfig] keys because ConfigParser splits on colons. Because it has a well-defined and nuanced [syntax specification][yamlspec], YAML-based config files will eschew these hacks: lists will look like lists and all characters will be treated as equals.
|
||||
* You'll have a ``~/.config/beets`` directory (or the equivalent on Windows). No more cluttering your home directory with the configuration file, the state file, the library database, and maybe even the import log.
|
||||
* ConfigParser does not play nice with Unicode. It's 2012, folks.
|
||||
* You'll be able to combine multiple configuration files--for example, using a global base configuration with per-location overrides.
|
||||
* You'll be able to combine multiple configuration files---for example, using a global base configuration with per-location overrides.
|
||||
* Most crucially, programming with the ConfigParser API is getting to be a nightmare. Look no further than the monstrous [import_files][] function signature to witness the pain of threading every config option through the UI to the business end of the code. Every time I add a new config option or command-line switch to ``beet import``, I have to touch at least five files to keep the frontend and backend in sync and update the unit tests. And that's just to parse the option: the real work for the feature begins *after* all this. The ConfigParser disaster discourages me from adding new features to beets.
|
||||
|
||||
These problems are so basic that I don't think I'm alone in growing uneasy with ConfigParser. So I'm writing a new configuration library called [Confit][] (that's pronounced [*con-FEE*][confitwiki]). I hope to make Confit into the best available library for configuring Python applications. I'll have more to say about Confit on this blog as work progresses.
|
||||
|
|
@ -47,7 +47,7 @@ And, finally, we'll drop [namespace packages][]. I didn't know this when I first
|
|||
[singleton]: http://en.wikipedia.org/wiki/Singleton_pattern
|
||||
[decorator]: http://www.python.org/dev/peps/pep-0318/
|
||||
|
||||
While these changes aren't particularly exciting for end users, it's important that I break all the plugins before 1.0 rather than after. The goal is to make beets' plugin system future-proof--to contain the [spaghetti][spaghetti code] before it spreads.
|
||||
While these changes aren't particularly exciting for end users, it's important that I break all the plugins before 1.0 rather than after. The goal is to make beets' plugin system future-proof---to contain the [spaghetti][spaghetti code] before it spreads.
|
||||
|
||||
[spaghetti code]: http://en.wikipedia.org/wiki/Spaghetti_code
|
||||
[plugins]: http://beets.readthedocs.org/en/latest/plugins/index.html#plugins-included-with-beets
|
||||
|
|
@ -55,8 +55,8 @@ While these changes aren't particularly exciting for end users, it's important t
|
|||
|
||||
## Beyond
|
||||
|
||||
With these two major changes out of the way, it will be time for some release candidates and then a massive party as we release 1.0. At this point, I plan on dividing beets development into a stable/[trunk][] development model: version 1.0 will see bug-fix-only releases while the new features go into a separate 1.1 branch. This will let me--and maybe other developers?--experiment with new stuff without rocking the boat for users who don't want to be bothered.
|
||||
With these two major changes out of the way, it will be time for some release candidates and then a massive party as we release 1.0. At this point, I plan on dividing beets development into a stable/[trunk][] development model: version 1.0 will see bug-fix-only releases while the new features go into a separate 1.1 branch. This will let me---and maybe other developers?---experiment with new stuff without rocking the boat for users who don't want to be bothered.
|
||||
|
||||
I have some exciting plans for new directions post-1.0. But, for now, I have two big betas to work on--we can talk about 1.1 and beyond a little later. Keep that dial right here.
|
||||
I have some exciting plans for new directions post-1.0. But, for now, I have two big betas to work on---we can talk about 1.1 and beyond a little later. Keep that dial right here.
|
||||
|
||||
[trunk]: http://en.wikipedia.org/wiki/Trunk_(software)
|
||||
|
|
|
|||
|
|
@ -21,13 +21,13 @@ A little bit of background: beets uses the amazing [SQLite][] database library t
|
|||
|
||||
[ACID guarantees]: http://en.wikipedia.org/wiki/ACID
|
||||
|
||||
But things can go wrong. If a transaction stays open too long, it can block other threads from accessing the database--or, in the worst case, several threads can deadlock while waiting for each other. For exactly this reason, SQLite has a lock timeout built in. If it ever sees that a thread has been waiting for a lock for more than five seconds (by default), it throws up its hands and the user sees the dreaded `database is locked` error.
|
||||
But things can go wrong. If a transaction stays open too long, it can block other threads from accessing the database---or, in the worst case, several threads can deadlock while waiting for each other. For exactly this reason, SQLite has a lock timeout built in. If it ever sees that a thread has been waiting for a lock for more than five seconds (by default), it throws up its hands and the user sees the dreaded `database is locked` error.
|
||||
|
||||
So the solution should be simple: somewhere, beets is holding a transaction open for more than five seconds, so we can either find the offending transaction or crank up that timeout. But herein lies the mystery: five seconds is a *long* time. That beets spends *5,000 milliseconds* manipulating the database in a single transaction is indicative of something dark and terrible. No amount of `SELECT`s and `INSERT`s at beets' scale should add up to five seconds, so turning up the timeout parameter is really just painting over the rot.
|
||||
|
||||
So I looked at every line in the source where a transaction could start. I made extra-double-sure that filesystem operations happened only outside of transactions. I fastidiously closed every [cursor][] after each `SELECT`. But all this was to no avail--the bug reports continued to pour in.
|
||||
So I looked at every line in the source where a transaction could start. I made extra-double-sure that filesystem operations happened only outside of transactions. I fastidiously closed every [cursor][] after each `SELECT`. But all this was to no avail---the bug reports continued to pour in.
|
||||
|
||||
At this point, I was almost certain that nothing was wrong with beets' transactions in themselves. I measured the length of each access and, on my machine, they each took a handful of milliseconds apiece--nowhere near a full five seconds.
|
||||
At this point, I was almost certain that nothing was wrong with beets' transactions in themselves. I measured the length of each access and, on my machine, they each took a handful of milliseconds apiece---nowhere near a full five seconds.
|
||||
|
||||
## The Real Problem
|
||||
|
||||
|
|
@ -64,13 +64,13 @@ Here's what it looks like. When a thread needs to access the database, it uses a
|
|||
with self.transaction() as tx:
|
||||
rows = tx.query('SELECT * FROM items WHERE id=?', (load_id,))
|
||||
|
||||
The only way to access the database is via methods on the [Transaction object][txn]. And creating a Transaction means acquiring a lock. Together, these two restrictions make it impossible for two different threads to access the database at the same time. This reduces the concurrency available in the DB (appropriate for beets but not for, say, a popular Web service) but eradicates the possibility of SQLite timeouts and will make it easy for beets to move to a different backend in the future--even one that doesn't support concurrency itself.
|
||||
The only way to access the database is via methods on the [Transaction object][txn]. And creating a Transaction means acquiring a lock. Together, these two restrictions make it impossible for two different threads to access the database at the same time. This reduces the concurrency available in the DB (appropriate for beets but not for, say, a popular Web service) but eradicates the possibility of SQLite timeouts and will make it easy for beets to move to a different backend in the future---even one that doesn't support concurrency itself.
|
||||
|
||||
[txn]: https://github.com/sampsyo/beets/blob/master/beets/library.py#L919
|
||||
|
||||
To make this explicit-transaction approach feasible, transactions need to be *composable:* it has to be possible to take two correctly-coded transactional functions and call them both together in a single transaction. For example, the beets Library has [a method that deletes a single track](https://github.com/sampsyo/beets/blob/master/beets/library.py#L1220). The ["beet remove" command][beet remove] needs to remove *many* tracks in one fell, atomic swoop.
|
||||
|
||||
The smaller method--`Library.remove`--uses a transaction internally so it can synchronize correctly when it's called alone. But the higher-level command has to call it many times in a single transaction, [like so](https://github.com/sampsyo/beets/blob/master/beets/ui/commands.py#L984):
|
||||
The smaller method---`Library.remove`---uses a transaction internally so it can synchronize correctly when it's called alone. But the higher-level command has to call it many times in a single transaction, [like so](https://github.com/sampsyo/beets/blob/master/beets/ui/commands.py#L984):
|
||||
|
||||
with lib.transaction():
|
||||
for item in items:
|
||||
|
|
@ -89,7 +89,7 @@ To accomplish this, each thread transparently maintains a *transaction stack* th
|
|||
|
||||
## Takeaway for Other Projects
|
||||
|
||||
What can we learn from the vanquishing of this monstrous bug--other than the [well-known fact][cbug classification] that [concurrency bugs are horrifying][heisenbug]? I think there are two lessons here: one for everybody who uses SQLite and one developers of any small-scale, desktop application that uses a database.
|
||||
What can we learn from the vanquishing of this monstrous bug---other than the [well-known fact][cbug classification] that [concurrency bugs are horrifying][heisenbug]? I think there are two lessons here: one for everybody who uses SQLite and one developers of any small-scale, desktop application that uses a database.
|
||||
|
||||
[heisenbug]: http://en.wiktionary.org/wiki/heisenbug
|
||||
[cbug classification]: http://www.cs.columbia.edu/~junfeng/09fa-e6998/papers/concurrency-bugs.pdf
|
||||
|
|
@ -104,13 +104,13 @@ I haven't seen this particular quirk documented elsewhere, but it should be comm
|
|||
|
||||
If you're writing a small-scale application that doesn't need highly concurrent access to a database, consider using explicit transactions based on a language-level construct (Python's [context managers][ctx] are a perfect example).
|
||||
|
||||
Without explicit transactions, it's hard--impossible, in some cases--to see where transactions begin and end. So it's easy to introduce bugs where transactions remain open much longer than they need to be. There are several advantages to marking the start and end of every transaction:
|
||||
Without explicit transactions, it's hard---impossible, in some cases---to see where transactions begin and end. So it's easy to introduce bugs where transactions remain open much longer than they need to be. There are several advantages to marking the start and end of every transaction:
|
||||
|
||||
* It's easy to verify that a transaction ends in a timely manner.
|
||||
* You can add synchronization to unsynchronized datastores like [LevelDB][] or flat files.
|
||||
* You can interpose on transactions for debugging purposes. For example, you might want to measure the time taken by each transaction. (This technique was instrumental to diagnosing this bug in beets.)
|
||||
|
||||
And if you're coding for SQLite in Python, feel free to [steal beets' Transaction implementation][txn]--it's open source!
|
||||
And if you're coding for SQLite in Python, feel free to [steal beets' Transaction implementation][txn]---it's open source!
|
||||
|
||||
[LevelDB]: http://code.google.com/p/leveldb/
|
||||
[cursor]: http://docs.python.org/library/sqlite3.html#cursor-objects
|
||||
|
|
|
|||
Loading…
Reference in a new issue