Backup_Repos/leech
1
0
Fork 0
mirror of https://github.com/kemayo/leech synced 2025-12-06 16:33:16 +01:00
Code Issues Projects Releases Wiki Activity
375 commits 3 branches 0 tags 1.3 MiB
Commit graph

58 commits

Author SHA1 Message Date
David Lynch
3fdbae5851 Pass through some more headers in the session 2024-12-17 16:34:57 -06:00
David Lynch
e3c63bce3c New config option: allow_spaces 
Determines whether spaces in filenames will be replaced with underscores
 2024-11-30 14:07:40 -06:00
David Lynch
2f21280d76 Adjust option loading so it's easier to override 2024-11-30 14:07:40 -06:00
David Lynch
91d2c4fd4b Fully cancel if the story extraction fails 2024-11-30 13:52:43 -06:00
David Lynch
ffb8e54e91 Better error for an Arbitrary story that fetches no content 2024-11-23 23:07:16 -06:00
David Lynch
6ecb1d8942 Make downloading images the default behavior 2024-11-23 16:33:01 -06:00
David Lynch
400c5cc801 Configurable whether to always convert images 2024-11-23 16:33:01 -06:00
David Lynch
e2bc6eba1c Change order of config loading so site-specific overrides of cover/image work 2024-11-23 16:33:01 -06:00
David Lynch
9508b00bcb Rearrange the image options to match cover options 2024-11-23 14:54:12 -06:00
David Lynch
542774543a Code style fixups, duplication cleanup, PIL textsize removal 2024-11-23 14:22:10 -06:00
Emmanuel Jemeni
34bf962df6 feat: Leech can now compress images to a specific target size 2024-11-23 13:22:54 -06:00
Emmanuel Jemeni
e6ad77a9fc fix: Completely fixes #2 ! 2024-11-23 13:22:54 -06:00
Emmanuel Jemeni
f933020357 refactor(leech.py): minor spelling error fixed 2024-11-23 13:22:54 -06:00
David Lynch
b8314341b2 Allow passing in multiple URLs 
Good trick: `./leech.py examples/practical{1,2,3,4,5,6,7}.json`
 2022-10-22 18:05:59 -05:00
David Lynch
bb9491cb96 Config option: output_dir 
Can be provided on the command line as `--output-dir`, or in leech.json
as `output_dir` (also in the `site_options` in leech.json).

Refs #67
 2021-09-04 15:46:16 -05:00
David Lynch
73df5d5cb3 Give nicer error messages when a SiteException is raised 2021-03-21 23:16:18 -05:00
David Lynch
0f176f5a7c session.cookies should stay a RequestsCookieJar 2021-02-12 15:50:18 -06:00
David Lynch
533c14f0d7 Normalize fancy unicode characters by default 
Kindle can't display the "𝖙𝖍𝖚𝖌 𝖑𝖎𝖋𝖊" mathematical bold fraktur codepoints
so NFKC normalize them (and anything else) into its plain equivalent.

Can be disabled by running with `--no-normalize` if needed.
 2021-02-05 01:59:20 -06:00
David Lynch
66576048da Fix flake8 errors 2019-05-25 20:03:17 -05:00
David Lynch
e78ffdb85b Method to get a site-key for config 
Means that things like XenForoIndex and AO3Series don't require separate
config entries.
 2018-10-11 15:42:59 -05:00
Alex Raubach
60084534a8 Create empty dict when leech.json not present 2018-09-15 11:03:52 -04:00
Alex Raubach
f2fc2c11db Capture cover options from leech.json and pass them to generate_epub() 2018-09-10 23:03:08 -04:00
Will Oursler
d1842e2bf1 Adds a system for site options to be included as click.options on commands. 2018-04-14 12:56:31 -04:00
Will Oursler
7c1702e6ff Fixes whitespace issues. 2018-04-13 18:18:37 -04:00
Will Oursler
379c087086 Merge a few things I missed in by hand. 2018-04-13 18:10:46 -04:00
Will Oursler
fd7998ea27 Clean up how verbose log output works... 2018-04-13 17:57:08 -04:00
Will Oursler
ecebf1de58 Merge branch 'master' into clickify 2018-04-13 17:52:37 -04:00
David Lynch
b8123e0b26 Explicitly VACUUM the cache on flush 2018-01-19 14:21:05 -06:00
David Lynch
6d52c72c99 Use logging instead of print 
Fixes #10
 2017-11-04 00:09:09 -05:00
Will Oursler
c702337040 Reworks how site-specific options work. 2017-10-13 19:37:13 -04:00
Will Oursler
db48233cf4 Switch from using raw argparser to using click. Preserves the existing 
interface, except leech --flush becomes leech flush
 2017-10-12 13:00:24 -04:00
David Lynch
8ac1aa8bb0 Add cover config to leech.json 2017-10-12 11:20:45 -05:00
Will Oursler
5bd07a5b90 Splits out ebook generation logic into a seperate module, in anticipation of maybe supporting multiple output formats. 2017-10-12 09:49:32 -04:00
David Lynch
6a782244fc Fix travis eclint call 2017-04-24 01:24:13 -05:00
David Lynch
5b4b9a0dc3 Canonicalize URLs 2017-02-23 15:03:23 -06:00
David Lynch
86e36715f1 Set up for Travis 2017-02-08 13:20:14 -06:00
David Lynch
24f7137929 Fix footnote backlinks, from structure change 2017-02-05 12:38:12 -06:00
David Lynch
e6343cb1c9 Stories are now made of nested sections/chapters 
This is prep-work for improving epub TOC generation a bit.
 2017-01-10 00:23:24 -08:00
David Lynch
24fa9aa22d Use a namedtuple for chapters 2016-09-23 13:11:52 -05:00
David Lynch
86f02812d2 Use requests-cache 2016-08-29 10:59:20 -05:00
David Lynch
921671f225 Add some front matter 2016-04-23 00:38:10 -05:00
David Lynch
05c98f28db Command to flush the cache 2016-03-18 08:46:25 -05:00
David Lynch
aa4ba528b7 Let sites define their own custom arguments 
Use this to let xenforo force the inclusion of the index-post
 2015-12-05 01:34:20 -06:00
David Lynch
c69eb1e33e Footnotes off in their own file 2015-11-30 20:10:58 -06:00
David Lynch
95e25dabd3 First pass at turning spoilers into footnotes for Xenforo 
This works as popup-footnotes in iBooks and on Kindle. It'd be a bit
better if I put the footnotes in their own file, so they won't be
dropped at the end of chapters on a Kindle. However, that requires
some flow restructuring, and this is an acceptable proof-of-concept
for now.
 2015-11-30 16:46:29 -06:00
David Lynch
bff375a8e7 Generate covers 2015-11-11 00:39:41 -08:00
David Lynch
8f198bae21 Allow logging in to sites, to view hidden things 2015-10-28 18:06:19 -05:00
David Lynch
2aba80be24 Change sites strategy to use classes and inheritance 2015-09-14 00:38:02 -05:00
David Lynch
81c348ade3 Make properly executable 2015-01-05 04:18:50 +00:00
David Lynch
d1a60dbb9c Add support for sta.sh 2014-04-28 18:23:15 -05:00