Commit graph

286 commits

Author SHA1 Message Date
David
e1ff258ce5 Move function to increase DRY 2014-11-16 21:39:58 -05:00
David
38f056a30c Adding tests for #1067 2014-11-16 21:33:03 -05:00
David
b901071e2c Implment -> Implement 2014-11-16 10:39:50 -05:00
Adrian Sampson
66d02bc289 Add link parameter to Item.move (#710) 2014-11-15 12:41:40 -08:00
Adrian Sampson
c158bb630c Fix mock function restoration in test :(
An argument for using decorators, context managers and stuff so this is
impossible to mess up.
2014-10-24 17:15:09 -07:00
Adrian Sampson
06b4730105 Re-import tests: stub out *_for_id functions
Currently, these just return no results. (For our testing purposes, the
matches we get from the ordinary metadata search suffice just fine.)
2014-09-28 11:39:04 -07:00
Adrian Sampson
a3eef35ba8 Test singleton re-imports 2014-09-28 11:16:38 -07:00
Adrian Sampson
f367a0971c Re-import test: check item-level data preserved 2014-09-28 11:04:25 -07:00
Adrian Sampson
e8d2ade167 Basic re-import tests 2014-09-28 10:58:43 -07:00
Thomas Scholtes
79d1203541 Remove unnecessary method on ImportTask 2014-09-18 19:32:29 +02:00
Thomas Scholtes
9a382eb581 Test importing unmatched tracks 2014-09-16 13:44:33 +02:00
Thomas Scholtes
e579db6f0a Hide task specific code from importer stage 2014-09-16 12:52:15 +02:00
Thomas Scholtes
3ce5dda0c9 Failing tests for empty import directories
Reproduces #932.
2014-09-06 18:52:57 +02:00
Thomas Scholtes
4835475fb7 Tags are not read while walking the import tree.
This makes skipping directories on incremental imports much faster and
fixes #158.
2014-08-25 18:40:56 +02:00
Thomas Scholtes
b1f670ada3 Move albums_in_dir() from autotag to import module
Has nothing to do with the former
2014-08-25 18:40:56 +02:00
Thomas Scholtes
225ce62a33 Catch all errors when loading state file.
A crash during the multi-threaded import process may leave the pickled
state invalid (see #913). We recover from all these errors.
2014-08-24 14:40:38 +02:00
Thomas Scholtes
0f2a9bdcdc Record singletons for incremental import
We still need to implement this for flat imports, archives and toppath
singletons. Fixes #860.
2014-07-31 11:09:16 +02:00
Thomas Scholtes
c5f23cc4f8 Don't write state to disk when nothing changes 2014-07-19 14:17:51 +02:00
Thomas Scholtes
46c9992633 Refactor resuming of imports
Instead of recording only the most recently imported path and assuming
that all previous path have also been imported, we record all imported
paths and skip recorded paths when resuming the import.

This fixes an issue where parallelism would screw up the import order
and reimport some files. It also allows us to record singleton items.

We can apply the same strategy can be applied for incremental imports.
2014-07-08 15:06:01 +02:00
Adrian Sampson
38eba4af31 Remove obsolete null checks
Along with their tests. Background:
https://github.com/sampsyo/beets/pull/720/files#r12027386
2014-04-26 20:07:26 -07:00
Thomas Scholtes
09ef565cfc Make _infer_album_fields a method 2014-04-19 17:02:36 +02:00
Thomas Scholtes
a09b2d4760 Test that duplicate files are removed 2014-04-19 17:02:20 +02:00
Thomas Scholtes
4677c55c4f Remove DuplicateCheckTest
Superseded by `ImportDuplicate*Test`.
2014-04-19 15:50:49 +02:00
Thomas Scholtes
efee81cda7 Refactor ImportDuplicateAlbumTest 2014-04-19 15:49:39 +02:00
Thomas Scholtes
52e3413c5b Test import singleton duplicates 2014-04-19 15:44:20 +02:00
Thomas Scholtes
0518d4e265 Add tests for duplicate album imports 2014-04-19 15:23:01 +02:00
Thomas Scholtes
bd047ec6c8 Add tests for resumed import 2014-04-17 21:56:15 +02:00
Thomas Scholtes
1f742130c4 Catch all archive extract errors and skip tests without unrar 2014-04-15 18:26:35 +02:00
Thomas Scholtes
68595ee09d Import rar archives 2014-04-15 18:24:09 +02:00
Thomas Scholtes
e3acdd0cc8 Import tar archives
Also refactors the importer code to make better use of ArchiveImportTask.
2014-04-15 18:24:08 +02:00
Thomas Scholtes
b783097329 Import zip archives
`beet import archive.zip` extracts the archive to a temporary directory and
imports the content.

The code is very hacky. To make it cleaner the `importer` module needs some
refactoring. One thing the code hints at is extending the `ImportTask` class.
2014-04-15 18:24:08 +02:00
Thomas Scholtes
4b012e5ddf Import tests use TestHelper
Removes duplicate code, increases speed and makes some files clake8 clean.
2014-04-15 14:57:01 +02:00
Adrian Sampson
ed8002bed5 tests: close databases to avoid errors on Windows
Fixes #655 and makes progress on #670.
2014-04-12 15:00:17 -07:00
Thomas Scholtes
2011156f8b Add test case for single file import 2014-04-03 22:13:31 +02:00
Adrian Sampson
03fa8d809b tests: properly install & restore some fixtures
Brought up while investigating #612.
2014-03-15 12:30:06 -07:00
Agatha
7f19194403 Updated test_importer for Various Artists
All py.test tests should pass now
2014-03-15 14:23:27 -04:00
Thomas Scholtes
e916609eee Save import history only when path associated
Fixes #570
2014-03-04 17:12:42 +01:00
Thomas Scholtes
3361d32a76 Add AutotagStub 2014-02-01 22:55:35 +01:00
Thomas Scholtes
50e384d48a Test choosing candidates during import 2014-02-01 22:55:35 +01:00
Thomas Scholtes
ba49675252 More tests and test documentation for Importer 2014-02-01 20:05:36 +01:00
Thomas Scholtes
9e105bd09b Add tests for TerminalImportSession 2014-02-01 19:19:48 +01:00
Thomas Scholtes
fb46571bf7 Set importer choices with add_choice
We can hook into this method in the next commit
2014-02-01 19:04:24 +01:00
Thomas Scholtes
24ffd0982e Add group albums stage and configuration 2014-02-01 15:02:00 +01:00
Thomas Scholtes
d12a4b20da Import multiple albums from single directory
If a directory contains multiple albums we can select the ALBUMS action to group
the tracks by album artist and album name and import those seperately.
2014-01-28 23:22:00 +01:00
Thomas Scholtes
432d6f4437 Removed akward tests 2014-01-28 20:51:00 +01:00
Thomas Scholtes
81b8d44814 Migrate outside file tests 2014-01-28 20:49:02 +01:00
Thomas Scholtes
9a3963b58e Remove duplicate test
This is already tested by `ImportExistingTest.test_asis_updated_moves_file`
2014-01-28 20:28:38 +01:00
Thomas Scholtes
ae208f5f53 Use ImportSession sublass for tests 2014-01-28 20:13:02 +01:00
Thomas Scholtes
bb6adbfa11 add compilation tests 2014-01-28 20:08:42 +01:00
Thomas Scholtes
f8b899091b Migrate ApplyExistingItemsTest 2014-01-28 20:08:42 +01:00
Thomas Scholtes
aaefd023eb Extract choose tracks function 2014-01-28 20:08:42 +01:00
Thomas Scholtes
96e2d128c0 Revert old test cases 2014-01-28 20:08:42 +01:00
Thomas Scholtes
2a4c6d5bca Fixed typo makes tests pass 2014-01-28 20:08:42 +01:00
Thomas Scholtes
69a40593a0 Remove duplicate tests 2014-01-28 20:08:42 +01:00
Thomas Scholtes
910dc6a647 Add ImportCompliationTest 2014-01-28 20:08:42 +01:00
Thomas Scholtes
39f1b8c3d8 Refactor MusicBrainz stubs into helper class 2014-01-28 20:08:42 +01:00
Thomas Scholtes
9f4e227f96 More ImportTest refactoring 2014-01-28 20:08:41 +01:00
Thomas Scholtes
5535eddfaa Refactor Import Test 2014-01-28 20:08:41 +01:00
Thomas Scholtes
9b12d488e2 New Import test case 2014-01-28 20:07:20 +01:00
Thomas Scholtes
d302081cca ...and some refactoring again 2014-01-28 20:07:20 +01:00
Thomas Scholtes
bfaa56e5bd Stupid typo 2014-01-28 20:07:20 +01:00
Thomas Scholtes
42f54e36bd These tests are already in ImportNonAutotaggedTest 2014-01-28 20:07:20 +01:00
Thomas Scholtes
0cdb896003 Move prune and file tasks 2014-01-28 20:07:20 +01:00
Thomas Scholtes
33edb9a878 So this is how python works! ;) 2014-01-28 20:07:20 +01:00
Thomas Scholtes
c6654c5acf Make test strings more verbose 2014-01-28 20:07:19 +01:00
Thomas Scholtes
03f1879980 Extract library file assertions 2014-01-28 20:07:19 +01:00
Thomas Scholtes
49693f715c Use _setup_library() helper 2014-01-28 20:07:19 +01:00
Thomas Scholtes
e63ac27d1c Extract _run_import and expose config in tests 2014-01-28 20:07:19 +01:00
Thomas Scholtes
d65595c70b Extract ImportHelper for tests 2014-01-28 20:07:19 +01:00
Thomas Scholtes
f900f1e0c8 Refactor import test 2014-01-28 20:07:19 +01:00
Thomas Scholtes
117a331091 Should infer *album artist* for albums from tag 2014-01-28 15:42:50 +01:00
Adrian Sampson
22d5e564c6 move move method to model objects 2013-09-16 22:12:26 -07:00
Adrian Sampson
c7fe017752 remove Library.{move,store} methods
These methods are now provided by LibModel, which makes dealing with items and
albums symmetric.
2013-08-21 15:34:45 -07:00
Adrian Sampson
9609e41cf8 store albums after moving art 2013-08-21 09:25:47 -07:00
Adrian Sampson
a7f0ad70ae test: move DummyIO initialization to harness
This makes it much harder to accidentally leave this I/O isolator installed.
2013-02-18 18:29:10 -08:00
Adrian Sampson
6c931faf1a tests: remove ad-hoc temporary directories
Now that we have a temporary directory in the shared harness, we can dispense
with the error-prone process of creating them for each test.
2013-02-18 18:23:32 -08:00
steini
10758c487f Configurable list of patterns which will be ignored when pruning empty directories.
* util.prune_dirs modified to accept glob patterns as clutter to determine emptiness.
* config option, 'clutter' (a list of filenames/glob patterns)
* ImportTask.prune passes this option's value to prune_dirs.
2013-02-04 23:41:38 +00:00
Adrian Sampson
99ed9e03de rename ImportTask.path to paths (#89) 2013-02-03 11:22:27 -08:00
Adrian Sampson
7a410f636b happy new year
For future reference, this command did the trick:
ack -l 'Copyright 201' | xargs perl -pi -E 's/Copyright 201./Copyright 2013/'
2013-01-11 10:43:41 -08:00
Adrian Sampson
cbd538de7b combine TempConfigTestCase and ExtraAsserts 2012-12-15 13:21:09 -08:00
Adrian Sampson
39cd1f6c63 switch from context manager to base class
Replaced temp_config context manager with TempConfigTestCase base class. This
lets us clean up even when a test fails.
2012-12-15 13:15:10 -08:00
Adrian Sampson
ed3a3d1937 confit-ify a bunch more tests 2012-12-14 14:21:06 -08:00
Adrian Sampson
ce166004cb use AlbumMatch/TrackMatch objects everywhere
This allows matches to indicate both missing and unmatched tracks in their
candidates and solves some of the spaghetti tuples that were passed around
during autotagging.
2012-07-01 16:33:48 -07:00
Adrian Sampson
11d4fb1abb move album art fetching to a plugin (fetchart) 2012-06-24 00:34:50 -07:00
Adrian Sampson
d88afbad11 library maintains per-thread DB conns. (GC-399) 2012-06-08 10:49:47 -07:00
Adrian Sampson
e00f15175a prune is a no-op when file exists
i.e., when moving a file that's already at its destination.
2012-05-30 16:39:47 -07:00
Adrian Sampson
86f513d4ab split apply_choice coroutine
This essential import pipeline stage is now two: one that applies metadata
changes and one that manipulates the filesystem. This will eventually allow
lastgenere to apply its changes before destinations are calculated.
2012-05-22 23:22:34 -07:00
Adrian Sampson
2b000c47a2 per_disc_numbering config option (GC-335) 2012-05-17 12:44:48 -07:00
Adrian Sampson
a6c1ad2235 reimporting with copying: copy external files 2012-05-17 11:42:58 -07:00
Adrian Sampson
429af42e14 use print_function __future__ import
All code should now use Python 3-style "print"s.
2012-05-13 21:08:27 -07:00
Adrian Sampson
b68e87b92c The Great Trailing Whitespace Purge of 2012
What can I say? I used to use TextMate!
2012-05-13 20:22:17 -07:00
Adrian Sampson
a28f930c52 transaction objects to control DB access
In an attempt to finally address the longstanding SQLite locking issues, I'm
introducing a way to explicitly, lexically scope transactions. The Transaction
class is a context manager that always fully fetches after SELECTs and
automatically commits on exit. No direct access to the library is allowed, so
all changes will eventually be committed and all queries will be completed. This
will also provide a debugging mechanism to show where concurrent transactions
are beginning and ending.

To support composition (transaction reentrancy), an internal, per-Library stack
of transactions is maintained. Commits only happen when the outermost
transaction exits. This means that, while it's possible to introduce atomicity
bugs by invoking Library methods outside of a transaction, you can conveniently
call them *without* a currently-active transaction to get a single atomic
action.

Note that this "transaction stack" concepts assumes a single Library object per
thread. Because we need to duplicate Library objects for concurrent access due
to sqlite3 limitation already, this is fine for now. Later, the interface should
provide one transaction stack per thread for shared Library objects.
2012-05-06 23:24:05 -07:00
Adrian Sampson
5ebb61bab7 moving import (#266): move album art files 2012-05-03 17:08:52 -07:00
Adrian Sampson
b327455fac prune empty directories when moving (#266) 2012-05-03 16:59:56 -07:00
Adrian Sampson
6b696c842f cleanup and docs for import_move (GH-26, GC-266)
- Copying and moving are mutually exclusive. Moving overrides copying so the
  user only has to add one line ("import_move: true") to disable copying and
  enable moving in its place.
- Deleting is only possible when copying.
- Deprecating the "delete" option (moving is almost always better).
- Removed command-line switch for moving. It's somewhat "unsafe", so this
  removes some potential for accidental irreversible changes.
- Changelog & thanks.
- Update docs to refer to import_move instead of import_delete as the
  correct solution for ending up with only one copy of the file.
2012-04-10 13:41:24 -07:00
Domen Kožar
1af4f86c17 support move action when importing 2012-04-05 01:14:17 +02:00
Adrian Sampson
24cdf2a72e duplicate trumping: remove items & delete files
Based on the "remove_duplicates" flag on ImportTask, the apply_choices coroutine
now looks for duplicates (using an extended version of the _duplicate_check
functions) and removes items from the library. It also *deletes* files
associated with those items when they are located inside the beets library
directory. Files outside of the directory are left on disk (but their DB entry
is still removed). This should "do the right thing" in most cases -- again, this
is something we can add a config option for if it comes up.
2012-03-20 14:23:44 -07:00
Adrian Sampson
19b08f8e99 duplicate resolution callback function (#164) 2012-03-19 15:32:53 -07:00
Adrian Sampson
6d2df0e4bf import_delete: delete local album art (#242) 2012-02-15 16:46:23 -08:00
Adrian Sampson
ea95fa1373 album art fetcher tests 2012-02-15 16:35:28 -08:00
Adrian Sampson
153f52a6eb import_delete prunes empty imported directories (#243) 2012-02-09 15:06:33 -08:00
Adrian Sampson
5a0105b12c fall back to unittest2 (#275) 2012-01-31 15:25:09 -08:00
Adrian Sampson
c40081808a fix double-removal when re-importing with deletion
With import_delete enabled and performing a re-import (which moves files), the
former location of the file would be "deleted". This would lead to a "file does
not exist" error.
2012-01-06 21:58:17 -08:00
Adrian Sampson
d73c133a53 query-conditioned path formats (#210)
Also, Library.path_formats is now a list of pairs instead of a dictionary. (I
would have used an OrderedDict, but that was added in 2.7.)
2011-12-28 19:01:13 -08:00
Adrian Sampson
08b539a80e fix field inference w/ null first item (closes #14 on GitHub)
When a partial match is found, its first item (task.items[0]) may be None, and
_infer_album_fields would crash in this case. This solution walks through the
items list and finds the first non-None item.
2011-12-16 16:45:50 -08:00
Adrian Sampson
5965b37f51 skip (configurable) clutter filenames when importing 2011-11-13 17:14:40 -08:00
Adrian Sampson
95f38dbe52 "info dictionaries" replaced with AlbumInfo and TrackInfo 2011-10-23 14:12:13 -07:00
Adrian Sampson
607757edf1 -i/import_incremental to only import new directories (#99) 2011-08-09 12:09:36 -07:00
Adrian Sampson
2c56fd22f2 fix replacement of in-library items 2011-08-04 16:14:07 -07:00
Adrian Sampson
3efeb9a133 -L flag to import lets you re-import items matching query (#69) 2011-08-04 15:29:59 -07:00
Adrian Sampson
248bccf951 move, rather than copying, when re-importing 2011-08-04 14:08:11 -07:00
Adrian Sampson
3e75d262a8 correctly detect item existence when copying 2011-08-04 13:35:44 -07:00
Adrian Sampson
f3130152b1 don't count existing items/albums as duplicates (allowing update) 2011-08-04 12:04:22 -07:00
Adrian Sampson
58fb4392ee refactor duplicate tests to take the whole task as an argument 2011-08-04 11:51:17 -07:00
Adrian Sampson
7f4f477c32 remove old items/albums from database when re-importing items 2011-08-04 11:16:12 -07:00
Adrian Sampson
951e4eec86 fix VA inference for small (1-track) albums 2011-07-07 09:19:33 -07:00
Adrian Sampson
45383eced2 fix VA inference (needs to run before move step in "apply") 2011-07-07 09:13:37 -07:00
Adrian Sampson
a448879ca9 infer album artist or VA for as-is imports (#161) 2011-06-29 10:36:07 -07:00
Adrian Sampson
da6ee13159 fix singleton quiet imports 2011-05-21 16:53:01 -07:00
Adrian Sampson
a0ef39aba5 duplicate detection on adjacent albums/items now works (#156) 2011-05-05 10:20:23 -07:00
Adrian Sampson
65dac30e4d break album art and finalization into new stages (#168) 2011-05-03 13:12:23 -07:00
Adrian Sampson
7e89282053 duplicate detection for items 2011-04-19 16:40:40 -07:00
Adrian Sampson
df6b1abfd8 clean up vestiges of TRACKS choice for album tasks 2011-04-19 13:21:27 -07:00
Adrian Sampson
db6eb60e23 update tests for timid mode 2011-04-19 13:13:16 -07:00
Adrian Sampson
d1b7c8dc55 display track changes; some scaffolding for user query 2011-04-14 09:07:06 -07:00
Adrian Sampson
45eef6e876 rename -i flag to -s ("singletons") and fix behavior 2011-04-12 23:31:44 -07:00
Adrian Sampson
12854ad2ff very first stab at a working individual-item importer flow
"beet import -i" now tags items instead of albums. There are many loose ends to
tie up (marked with TODOs in the source):
- What to do about applying non-track metadata to matched tracks? Currently it's
  just left in place.
- Plugin autotag candidates for tracks.
- No user querying yet.
- Non-autotagged -i import are unimplemented.
And, on top of those:
- Need to remove the action.TRACKS workflow and replace it with an option that
  lets you jump over to the individual-track interface from the album tagger.
2011-04-12 23:22:03 -07:00
Adrian Sampson
a39a5b5d66 extremely preliminary item importer skeleton
(I shouldn't have started on this yet; the autotagger functionality isn't in
place yet. Going back and doing that now...)
2011-04-12 20:52:39 -07:00
Adrian Sampson
6f9c460837 slight tweaks to tests to let them run under nose 2011-04-12 15:06:27 -07:00
Adrian Sampson
85ddfa4381 relocatable test rsrc directory 2011-04-12 14:26:48 -07:00
Adrian Sampson
c03ec1ee1c add a multithreaded import test case 2011-04-12 14:11:43 -07:00
Adrian Sampson
89f33466e0 move non-autotagged import test (wasn't testing UI) 2011-04-12 14:09:17 -07:00
Adrian Sampson
584cc74a5c fix bug when "applying" sentinel 2011-04-11 08:21:36 -07:00
Adrian Sampson
a675988eb2 add and use fancy enumeration module 2011-04-10 22:10:33 -07:00
Adrian Sampson
c0467c3724 encapsulate importer configuration in an object 2011-04-10 18:56:42 -07:00
Adrian Sampson
23392525ec The Great Importer Refactoring
I'm shuffling around the feature-creeping importer code to keep it as
interface-agnostic as possible. The "importer" module now takes care of the
basic, increasingly complicated workflow while the ui.commands module is
relegated to containing actual user-interface stuff.
2011-04-10 18:12:47 -07:00