Backup_Repos/beets
1
0
Fork 0
mirror of https://github.com/beetbox/beets.git synced 2025-12-08 09:34:23 +01:00
Code Issues Projects Releases Wiki Activity
10167 commits 43 branches 64 tags 60 MiB
Commit graph

277 commits

Author SHA1 Message Date
Adrian Sampson
8bb24e3134 lyrics: Set User-Agent header (fix #2357) 2016-12-30 10:55:24 -05:00
tigranl
dd115b1310 Add ui import 2016-12-11 00:35:51 +03:00
tigranl
5ca664e4aa Fix typos 2016-12-11 00:25:37 +03:00
tigranl
6ba5099034 Python version check for lyrics.py 2016-12-06 16:17:25 +03:00
Adrian Sampson
62e9a15f4d Fix a copy n' paste error found by flake8 2016-11-16 12:03:07 -05:00
Fabrice Laporte
7226624405 replace strip_part() by generate_alternatives() 
Delegate the update of titles and artists lists to the helper
generate_alternatives() function.
 2016-09-25 19:37:14 +02:00
Fabrice Laporte
e2703b9a7c always yield item artist and title first 
Rather than using an unordered set for storing pairs, append to a list
and build an OrderedDict from it to filter duplicated strings while
keeping order.
 2016-09-25 15:46:22 +02:00
Fabrice Laporte
8b4f39da42 lyrics: search for song title part preceding colon. fix #2205 2016-09-23 22:23:32 +02:00
Fabrice Laporte
4b702b338e lyrics: reduce code duplication in search_pairs() 2016-09-23 22:21:00 +02:00
Johnny Robeson
7a2bdf502f s/utf8/utf-8/ in all encoding/decoding contexts 
This matches up with the python documentation.
 2016-09-06 23:10:24 -04:00
Johnny Robeson
fcbfce3984 replace deprecated log.warn() with log.warning() 2016-08-09 00:33:38 -04:00
Johnny Robeson
be08d4b129 replace unichr with six.unichr in lyrics plugin 2016-07-02 02:36:05 -04:00
Adrian Sampson
5efd5b21c5 Use new as_str method 
Instead of `get(six.text_type)`, which was a surprisingly large portion of our
uses of six.
 2016-06-25 19:16:14 -07:00
Adrian Sampson
e16cc58cb9 Walk back some six.iter* uses 
In places where it doesn't much matter whether we use an iterator or the old
Python 2 list way, using the six name just hurts legibility.
 2016-06-25 18:29:55 -07:00
Johnny Robeson
78334876c3 treat HTMLParseError as a noop when missing 
Strict mode no longer exists in html.parser on python >= 3.5, and no longer means anything on python >= 3.3
 2016-06-24 05:53:56 -04:00
Johnny Robeson
edb1cbc5fc replace iter{items|values} with six.iter{items|values} 2016-06-24 05:53:55 -04:00
Johnny Robeson
e8afcbe7ec replace unicode with six.text_type 2016-06-24 05:53:49 -04:00
Johnny Robeson
4649226b9b use urllib from six.moves 2016-06-23 04:40:18 -04:00
Johnny Robeson
129e140015 use html_parser (really html.parser) from six.moves 2016-06-23 04:40:18 -04:00
Johnny Robeson
8fa71f78fe decode bytes from .encode() in lyrics plugin 2016-06-14 00:44:43 -04:00
Adrian Sampson
0051bdb506 lyrics: Avoid a spurious warning 2016-06-02 21:33:33 -07:00
Adrian Sampson
581fba6288 lyrics: Avoid crash when enabling google 
If you *both* haven't set an API key *and* BeautifulSoup wasn't
installed, the list.remove() call would crash. (This came up when
running the tests on a fresh machine without many dependencies.)
 2016-06-02 11:58:14 -07:00
wordofglass
1dd6739218 lyrics: fix a bug where the lyricswiki fetcher would try to unescape an empty (None) response and crash 2016-04-30 01:25:02 +02:00
wordofglass
c3c7da8061 lyrics: simplify source handling a little 2016-04-28 18:31:22 +02:00
wordofglass
2928a16bd5 lyrics: actually disable translation when there's no langdetect 2016-04-28 17:22:55 +02:00
wordofglass
c4b11f889f lyrics: clean up import handling and source removal 2016-04-28 17:15:25 +02:00
Jack Wilsdon
c5e2334fb5 Remove useless unescape 
Remove useless unescape as _scrape_script_cruft does it for us.
 2016-04-25 19:24:26 +01:00
Jack Wilsdon
1be9c3003e Use different method to remove junk from LyricsWiki 
Use `_scrape_strip_cruft` instead of `scrape_lyrics_from_html` so that
LyricsWiki does not depend on Beautiful Soup.
 2016-04-25 19:14:30 +01:00
wordofglass
607f41be43 Fix the previous fix... 2016-04-24 00:42:31 +02:00
wordofglass
4a5b886944 Fix two non-guarded import statements in the lyrics plugin 
These could make the import process crash with a traceback.
 2016-04-24 00:35:15 +02:00
Guilherme Danno
bf1b06f0c7 don't print entire lyrics during import 2016-04-22 17:30:06 -03:00
Fabrice Laporte
05970e8a93 re-query token when it has expired 2016-04-14 22:57:41 +02:00
Fabrice Laporte
56d7e5dfa0 send as little text as possible to bing api 
Bing API has a limit of 2M chars/month. It’s common to have repeating
sentences in lyrics so to reduce number of chars sent per song, store
sentences in a set and send it, instead of sending the whole lyrics.
 2016-04-14 22:57:17 +02:00
Fabrice Laporte
6cfc106b8a better docs and debug msg 2016-04-14 08:31:55 +02:00
Fabrice Laporte
58df77e2cb langdetect conditional import 2016-04-14 08:31:14 +02:00
Fabrice Laporte
e03c3af91f don't translate lyrics already in the target language 2016-04-14 01:11:14 +02:00
Fabrice Laporte
66a627fed8 restore module docstring 2016-04-14 00:58:42 +02:00
Fabrice Laporte
3c2479ab49 translate lyrics using Bing API 
By subscribing to Microsoft Translator API, one can now activate the
translation of lyrics from one set of source langages to a target
langage.
Translations are appended to each original sentence using ‘/‘ as
separator.
 2016-04-14 00:53:58 +02:00
Fabrice Laporte
d67950cdcc pep8 2016-04-14 00:45:55 +02:00
Adrian Sampson
d1753b341e lyrics: Some comments and better naming 2016-03-21 10:28:30 -07:00
Adrian Sampson
f684f29a25 lyrics: Tolerate pages without text (fix #1914) 2016-03-21 10:24:13 -07:00
Adrian Sampson
c9be5bc7d1 Merge pull request #1911 from jackwilsdon/fix-musixmatch-url 
Fix MusixMatch issues
 2016-03-18 11:51:27 -04:00
Jack Wilsdon
60148918d9 Fix LyricsWiki scraping code 
LyricsWiki now escapes song lyrics using HTML entities (presumably to
prevent scraping), so we now unescape these before parsing.

LyricsWiki has also added a script tag inside the div we are scraping,
so we have to remove this using `scrape_lyrics_from_html`.
 2016-03-17 17:49:41 +00:00
Jack Wilsdon
c417003184 Add missing newline 2016-03-16 21:07:28 +00:00
Jack Wilsdon
1ec06e14c5 Fix lyrics extraction from MusiXmatch 
Remove "lyrics_" prefix from extract_text_between arguments to reflect
changes made to the MusiXmatch website.
 2016-03-16 20:48:57 +00:00
Jack Wilsdon
44c799320f Improve URL generation in lyrics plugin 
Allow custom replacements to be defined in subclasses of
SymbolsReplaced.

Replace spaces with a hyphens when the source is MusiXmatch, instead of
(incorrectly) using underscores. This fixes #1880.
 2016-03-16 20:46:36 +00:00
Adrian Sampson
e54c7eec3d Standardize __future__ imports without parentheses 
Since the list is short enough now, we don't need parentheses for the line
wrap. This is a little less ugly.
 2016-02-28 15:03:51 -08:00
Adrian Sampson
d53019f9db Further whitespace fiddling 
Most commonly, this sticks with:

    log.debug(
        'some long message here'
    )

instead of placing the closing ) at the end of the string literal.
 2016-02-28 14:48:10 -08:00
Peter Kessen
f2fc1a78bf Removed import of unicode_literals from plugins 
* keyfinder
* lastimport
* lyrics
 2016-02-20 13:59:58 +01:00
Adrian Sampson
60888274c4 lyrics: Re-disable Genius backend 
As #1854 pointed out, the Genius API service is down *again*.
 2016-02-02 08:14:22 -08:00
Adrian Sampson
36ad0cf70d lyrics: Re-enable genius backend (#1770) 2016-01-06 14:29:51 -08:00
Jack Wilsdon
12cd5306b7 Update copyright dates to 2016 2015-12-30 15:42:06 +00:00
Adrian Sampson
a976c371f6 lyrics: Disable Genius backend by default (#1770) 2015-12-28 16:35:39 -08:00
Adrian Sampson
1a2ffd0426 lyrics: Catch errors in Genius backend (fix #1770) 
The HTTP requests had no error handling. Now they have lots.
 2015-12-28 16:32:28 -08:00
Adrian Sampson
5a285cc11f Fix #1673: Escape regex terms in lyrics 2015-12-12 18:09:24 -08:00
Adrian Sampson
d11fe020ce snake_case variable names 2015-12-12 18:03:12 -08:00
Peter Kessen
3eb8008b11 added encoding as comment in files 
added line like
# -*- coding: utf-8 -*-
to all files with correct license in header
 2015-11-19 18:41:01 +01:00
Adrian Sampson
9f7aa866bd Use ui.should_write everywhere 
There sure are a lot of plugins that want to write metadata!
 2015-11-07 13:20:09 -08:00
Joe LaSala
3ab9932059 Add Genius as a lyric source 
Makes use of the Genius API and genius-api.com, a separate resource that
provides readable lyrics.
 2015-10-13 19:14:08 -05:00
Adrian Sampson
b477f4a53f Merge pull request #1376 from tomjaspers/config-redacted-fields 
Config: new option -r redacts sensitive fields

Conflicts:
	beets/util/confit.py
 2015-03-27 22:11:24 -04:00
Tom Jaspers
944d38064d Config: implement redacted fields more idiomatic 
- `add_redacted_fields(self, *field_names)` to use argument unpacking
- foo =| bar instead of foo = foo | bar
 2015-03-25 14:55:29 +01:00
Tom Jaspers
f4ed3e16b1 Config: Add redacted fields from included plugins 2015-03-24 17:01:33 +01:00
Adrian Sampson
e7f8a627e9 Style fixes for pep8 1.6 2015-02-07 12:51:54 -08:00
Adrian Sampson
b8dab9cf9f Merge pull request #1247 from brunal/future 
Use all __future__ imports in beets core

Conflicts:
	beetsplug/web/__init__.py
	test/test_embedart.py
 2015-01-26 17:02:07 -08:00
Bruno Cauet
8165dec985 Delete manual formattings of album & item 2015-01-25 21:32:22 +01:00
Bruno Cauet
438044c6ae Fix bytes/str with unicode-nazi 
Further cleaning will require setting unicode_literals on plugins
 2015-01-20 15:25:12 +01:00
Bruno Cauet
02d8e88ef1 Replace all mentions of 'str' with 'bytes' 2015-01-20 12:48:15 +01:00
Bruno Cauet
90b388b775 Use __future__ imports but unicode_literals everywhere 
Include import of __future__ features division, absolute_imports and
print_function everywhere. Don't add unicode_literals yet for it is
harder to convert.

Goal is smoothing the transition to python 3.
 2015-01-19 12:25:16 +01:00
Bruno Cauet
b49cd3b73b Lyrics plugin: fix google backend 
Give config on backends init so the Google backend can get the API key
and Engine Id.
Fix #1226
 2015-01-15 11:28:04 +01:00
Bruno Cauet
4babc40fd8 Delete useless "config['mypluginname']" mentions 
Replace with self.config where this is painless.
More plugins would benefit from this update but that requires turning
functions into methods.
 2015-01-12 10:55:59 +01:00
Adrian Sampson
24317fd4c7 Rename BeetsPlugin._import_stages to import_stages 
For #1208. Restores backwards-compatibility and matches the development docs.
 2015-01-11 14:27:59 -08:00
Bruno Cauet
b27c5304d1 Merge branch 'master' into logging 
Conflicts:
	beetsplug/fetchart.py
	beetsplug/mpdstats.py
 2015-01-09 15:15:27 +01:00
Frederik “Freso” S. Olesen
4b1f0cbf48 Happy 2015. ;) 
See 7a410f636b

Command used:

    git grep -l 'Copyright 201'|xargs sed -i -E 's/Copyright 201./Copyright 2015/'`
 2015-01-08 21:37:09 +01:00
Bruno Cauet
00d5473187 Merge branch master into logging 
Conflicts:
	beetsplug/lyrics.py
 2015-01-06 21:46:04 +01:00
Bruno Cauet
63041736e3 Convert lyrics plugin, with OO rewrite of backends 2015-01-06 21:42:10 +01:00
Bruno Cauet
1afe82fb41 Make 2 plugins rely on auto log level mgmt 
ftintitle and title don't do manual management anymore.
 2015-01-06 21:27:15 +01:00
Bruno Cauet
427f7e7035 Automatic logger level changes on import 
Breaking changes: plugins should set set _import_stages instead of
import_stages. From outside the latter is replaced by import_stages().
This is because it is now wrapped with log level-getting & -setting
statements.
 2015-01-06 21:27:15 +01:00
Adrian Sampson
daba9e770e Fix #1204: insecure request warning in lyrics 2015-01-06 11:01:11 -08:00
Bruno Cauet
b8211a3c4c Every plugin uses its own logger 
logging.getLogger(__name__) everywhere!
Several loggers prefixed every log message with [logername], which we
delete here.
 2015-01-06 10:35:44 +01:00
Bruno Cauet
30f158a95e Move "from beets import logging" statements 
Move the import next to other beets-related imports
 2015-01-05 10:05:21 +01:00
Bruno Cauet
7df8bef8b7 Update logging imports: logging → beets.logging 2015-01-04 17:02:27 +01:00
Bruno Cauet
8cac47af2a Convert beets plugins to lazy logging 2015-01-04 17:02:27 +01:00
Fabrice Laporte
d4d5c085fa lyrics : remove empty divs before scraping 
it may result in \n being inserted that we will strip in
_scrape_strip_cruft
 2014-12-30 23:37:23 +01:00
Fabrice Laporte
802d1521ed lyrics: don't throw when extraction fails 2014-12-21 14:38:19 +01:00
Fabrice Laporte
dfc1aa20b3 lyrics: musixmatch, disable https verification 2014-12-21 14:35:16 +01:00
Fabrice Laporte
39584a8b69 fix flake8 2014-12-19 00:19:59 +01:00
Fabrice Laporte
c3f82c65a4 fix lyrics.com extraction markers 2014-12-18 23:56:33 +01:00
Adrian Sampson
b3bf70c11d Privatize global SOURCES in lyrics 2014-12-18 11:13:02 +00:00
Adrian Sampson
ac3f0824b0 Merge pull request #1148 from Kraymer/lyrics_musixmatch 
lyrics: add 'musixmatch' source

Conflicts:
	beetsplug/lyrics.py
 2014-12-18 10:36:16 +00:00
Adrian Sampson
20db9bb1a6 lyrics: Connect force option to CLI (#1150) 2014-12-18 04:03:15 +00:00
Fabrice Laporte
544d6dbe47 lyrics: add 'force' option 2014-12-17 22:40:44 +01:00
Fabrice Laporte
474adffe63 move helper functions from utils to plugins 2014-12-17 22:00:00 +01:00
Fabrice Laporte
9d0ca15ace lyrics: preserve default order of sources 2014-12-17 01:05:58 +01:00
Fabrice Laporte
e7a4b92de5 lyrics: add 'sources' option 2014-12-17 00:42:11 +01:00
Fabrice Laporte
0f2f43ca9b lyrics: add musixmatch source 2014-12-17 00:41:21 +01:00
Adrian Sampson
a984c1aa44 Use a non-capturing group by default (#1060) 
Now clients don't have to decide whether they need parentheses or not.
 2014-12-16 11:37:40 +00:00
Fabrice Laporte
829b623665 remove capturing parentheses 2014-12-15 22:48:01 +01:00
Fabrice Laporte
b62f15d9d9 feat_tokens: change argument name, fix regex flag 2014-12-14 22:46:51 +01:00
Fabrice Laporte
91a998df3c fix #1060 2014-12-13 23:34:50 +01:00
Adrian Sampson
165ee80f81 lyrics: Handle requests exceptions (#1136) 2014-12-11 16:03:49 -08:00
Fabrice Laporte
d31a7c6b28 remove str decoding as input sources are unicode 2014-12-11 00:14:43 +01:00
Fabrice Laporte
321f862f23 fix #1135 2014-12-09 23:37:42 +01:00
Alberto Leal
5883ee0b76 Default value for link title for page searches. 
Google API may not return results with a title attribute.
 2014-11-16 16:11:25 -05:00
Fabrice Laporte
b143ad7e3e fix #1035 do scraping tests on mock data 
don’t store scraped pages with licensed lyrics in repo
 2014-11-06 22:10:15 +01:00
Adrian Sampson
9137b5c2f3 Fix another lyrics scraper regression (#1034) 
Along with a test.
 2014-10-24 20:08:32 -07:00
Adrian Sampson
0325fe2225 lyrics: Remove script tags (fix #1034) 2014-10-24 17:33:11 -07:00
Fabrice Laporte
a6f0649c40 return no lyrics when HtmlParseError occured 2014-10-09 08:22:51 +02:00
Fabrice Laporte
c0c474b20f lyrics: strip title excerpt before matching 
improve the extraction of lyrics title from url title and increase the
matching threshold as a consequence.
 2014-10-08 14:49:09 +02:00
Fabrice Laporte
3ef52e8ead lyrics.py: remove unnecessary re compile step 2014-09-26 07:08:54 +02:00
Fabrice Laporte
a6a83be434 fix flake8 2014-09-24 23:30:38 +02:00
Fabrice Laporte
76b658b14a use beautiful soup strainer for a x20 performance gain! 2014-09-24 18:04:16 +02:00
Fabrice Laporte
8ef7837d22 merge strip_cruft() and _scrape_normalize_eol() into _scrape_strip_cruft 2014-09-24 16:51:54 +02:00
Fabrice Laporte
333591fd78 no html entities in _scrape_streamline_soup output 2014-09-24 00:25:50 +02:00
Fabrice Laporte
91a7eb249c add _scrape_merge_paragraphs lyrics scraping step + others scraping enh 2014-09-23 17:58:58 +02:00
Fabrice Laporte
a938e68c98 refactor scrape_lyrics_from_url into smaller functions 2014-09-23 13:21:31 +02:00
Fabrice L.
151ee87d8d remove a log.info() 2014-09-22 17:32:15 +02:00
Fabrice Laporte
aea640d241 lyrics.py: fix regexes used by strip_cruft (make them case insensitive) 
strip_cruft() should now correctly replace all <br> with \n thus making
insert_line_feeds() and sanitize_lyrics() functions superfluous (they have been
removed).
 2014-09-22 17:20:25 +02:00
e5e4eaeacd39c5cfba4d7c852c48277ae50331e6
816e4fb152 clean up after rebase 2014-09-09 11:53:44 +10:00
e5e4eaeacd39c5cfba4d7c852c48277ae50331e6
020ee2b1ed Fix Travis errors 
I was over zealous on the brackets for formatting
 2014-09-09 11:31:43 +10:00
e5e4eaeacd39c5cfba4d7c852c48277ae50331e6
65de93941d flake8 cleanup 
Cleanup after cleanup
 2014-09-09 11:28:43 +10:00
e5e4eaeacd39c5cfba4d7c852c48277ae50331e6
66aee8094f Clean up of logging messages as described here 
All logging now prefers the ' (single quote) over the " (double quote)

https://github.com/sampsyo/beets/wiki/Hacking
 2014-09-09 11:28:43 +10:00
Adrian Sampson
c0ce8c3e54 Changelog for #927 2014-09-02 21:45:35 -07:00
Padraic O'Donoghue
5b57032981 Remove scripts from lyrics 2014-09-03 03:15:34 +01:00
Thomas Scholtes
b512a0ce37 lyrics: Use multiple lyrics search strings. 
In particular we use the original artist and title before stripping
*and* and *featuring* suffixes.

Fixes #914.
 2014-08-24 16:17:21 +02:00
Fabrice Laporte
74898347c0 is_page_candidate() handle more languages 
Add translations in spanish and german for tokens to
ignore when comparing an url title with a song title.
 2014-04-26 19:25:25 +02:00
Fabrice Laporte
8ba91a49c6 change chars replacements done by slugify() 
adapt slugify() to make it return strings that can be used
as yaml keys (no spaces, etc.)
adapt is_page_candidate() accordingly
 2014-04-26 19:22:40 +02:00
Fabrice Laporte
567e6300fd fix flake8 2014-04-26 07:27:13 +02:00
Fabrice Laporte
117d16f2ad lyrics: add tests to track which websites can be scraped by our algo and be 
used as sources fot the google custom search engine.
 2014-04-26 07:26:50 +02:00
Adrian Sampson
e5d28e2171 lyrics is flake8-clean 2014-04-12 13:32:46 -07:00
Adrian Sampson
7fcd7daf7c lyrics: minor style/doc cleanup 2014-04-12 13:08:24 -07:00
Fabrice Laporte
42747797cd better handling of songs with featuring artists, songs variants versions (live 
etc), and songs combinations (lyrics are then appended)
 2014-04-12 12:29:20 +02:00
Thomas Scholtes
c3ea1ded30 Add item.try_write() to log errors 
Many commands and plugins use `item.write()` to update tags. Since the success
of the call is not critical to the functionality of most consumers we want to
catch any exceptions, log an error and continue with our task. The new method
encapsulates this logic.

This fixes #675.
 2014-04-10 15:26:05 +02:00
Filipe Fortes
0824ccb85f Don't try to parse None as HTML when fetching lyrics 
Fixes #479
 2013-12-18 07:37:01 -08:00
Adrian Sampson
38ecb35718 lyrics -f (#455, closes #414): style, changelog 2013-11-25 15:58:53 -08:00
Pedro Silva
9b75db8326 merge Bitdemon-master 
- minor style changes
- synchronize with documentation
 2013-11-17 12:34:53 +01:00
Bitdemon
f87606869c Update lyrics.py 2013-11-10 23:18:53 +01:00
Bitdemon
de0f0792ef Update lyrics.py 2013-11-10 23:13:35 +01:00
Bitdemon
f5f73be2b5 Update lyrics.py 2013-11-10 22:58:36 +01:00
Bitdemon
86d74f7d39 Update lyrics.py 
added -f option to lyrics plugin
first draft
 2013-11-10 22:44:37 +01:00
Adrian Sampson
c7fe017752 remove Library.{move,store} methods 
These methods are now provided by LibModel, which makes dealing with items and
albums symmetric.
 2013-08-21 15:34:45 -07:00
Fabrice Laporte
995d75f3f3 Logging: remove match ratio, add source website name 2013-06-29 14:24:41 +02:00
Fabrice Laporte
9780be270c Some tweaking to yield better results by not 
rejecting valid lyrics.
 2013-06-29 14:23:53 +02:00
Fabrice Laporte
c6f935ac4c Don't consider text between parentheses when 
matching url title with song title.
 2013-06-29 14:21:55 +02:00
Fabrice L.
6c8f45c7f7 Update lyrics.py 2013-06-12 01:20:08 +03:00
Fabrice Laporte
b3747189e5 lyrics: google backend should turn up more results 
bs4 scraping routine has been made more generic,
relying less on specific markup tags.
Better algorithm to detect which url titles match
song titles: domain names are now removed from url
titles.
Use regex to decimate \n in fetched lyrics.
 2013-06-12 00:07:01 +02:00
Fabrice Laporte
09e721efe6 rename "section" markup 2013-06-02 22:35:36 +02:00
Adrian Sampson
a0ef886801 lyrics: substitute more punctuation (fixes #270) 2013-05-12 12:45:49 -07:00
Adrian Sampson
a5cb34360d lyrics: fix encoding for Lyrics.com 2013-05-12 12:38:48 -07:00
Adrian Sampson
2a9afd3908 misc. style cleanup for #243 2013-04-15 10:52:17 -07:00
Fabrice Laporte
479b25bac3 Code style + fix doc typo 2013-04-08 18:35:02 +02:00