Zomega
82a76cd67b
Merge abd9acb2a7 into 5cb887f767
2025-03-26 21:21:52 +00:00
David Lynch
4d9c31b6ac
Make the parser used for BeautifulSoup configurable, still default lxml
...
Refs #98
2025-03-04 23:14:51 -06:00
David Lynch
3fdbae5851
Pass through some more headers in the session
2024-12-17 16:34:57 -06:00
David Lynch
e3c63bce3c
New config option: allow_spaces
...
Determines whether spaces in filenames will be replaced with underscores
2024-11-30 14:07:40 -06:00
David Lynch
2f21280d76
Adjust option loading so it's easier to override
2024-11-30 14:07:40 -06:00
David Lynch
91d2c4fd4b
Fully cancel if the story extraction fails
2024-11-30 13:52:43 -06:00
David Lynch
ffb8e54e91
Better error for an Arbitrary story that fetches no content
2024-11-23 23:07:16 -06:00
David Lynch
6ecb1d8942
Make downloading images the default behavior
2024-11-23 16:33:01 -06:00
David Lynch
400c5cc801
Configurable whether to always convert images
2024-11-23 16:33:01 -06:00
David Lynch
e2bc6eba1c
Change order of config loading so site-specific overrides of cover/image work
2024-11-23 16:33:01 -06:00
David Lynch
9508b00bcb
Rearrange the image options to match cover options
2024-11-23 14:54:12 -06:00
David Lynch
542774543a
Code style fixups, duplication cleanup, PIL textsize removal
2024-11-23 14:22:10 -06:00
Emmanuel Jemeni
34bf962df6
feat: Leech can now compress images to a specific target size
2024-11-23 13:22:54 -06:00
Emmanuel Jemeni
e6ad77a9fc
fix: Completely fixes #2 !
2024-11-23 13:22:54 -06:00
Emmanuel Jemeni
f933020357
refactor(leech.py): minor spelling error fixed
2024-11-23 13:22:54 -06:00
David Lynch
b8314341b2
Allow passing in multiple URLs
...
Good trick: `./leech.py examples/practical{1,2,3,4,5,6,7}.json`
2022-10-22 18:05:59 -05:00
David Lynch
bb9491cb96
Config option: output_dir
...
Can be provided on the command line as `--output-dir`, or in leech.json
as `output_dir` (also in the `site_options` in leech.json).
Refs #67
2021-09-04 15:46:16 -05:00
David Lynch
73df5d5cb3
Give nicer error messages when a SiteException is raised
2021-03-21 23:16:18 -05:00
David Lynch
0f176f5a7c
session.cookies should stay a RequestsCookieJar
2021-02-12 15:50:18 -06:00
David Lynch
533c14f0d7
Normalize fancy unicode characters by default
...
Kindle can't display the "𝖙𝖍𝖚𝖌 𝖑𝖎𝖋𝖊" mathematical bold fraktur codepoints
so NFKC normalize them (and anything else) into its plain equivalent.
Can be disabled by running with `--no-normalize` if needed.
2021-02-05 01:59:20 -06:00
David Lynch
66576048da
Fix flake8 errors
2019-05-25 20:03:17 -05:00
David Lynch
e78ffdb85b
Method to get a site-key for config
...
Means that things like XenForoIndex and AO3Series don't require separate
config entries.
2018-10-11 15:42:59 -05:00
Will Oursler
abd9acb2a7
Creates a read subcommand that allows for reading the story in terminal.
...
Finalize merge, a few things needed switching around.
Use site-specific options post merge...
2018-10-08 15:32:46 -07:00
Alex Raubach
60084534a8
Create empty dict when leech.json not present
2018-09-15 11:03:52 -04:00
Alex Raubach
f2fc2c11db
Capture cover options from leech.json and pass them to generate_epub()
2018-09-10 23:03:08 -04:00
Will Oursler
d1842e2bf1
Adds a system for site options to be included as click.options on commands.
2018-04-14 12:56:31 -04:00
Will Oursler
7c1702e6ff
Fixes whitespace issues.
2018-04-13 18:18:37 -04:00
Will Oursler
379c087086
Merge a few things I missed in by hand.
2018-04-13 18:10:46 -04:00
Will Oursler
fd7998ea27
Clean up how verbose log output works...
2018-04-13 17:57:08 -04:00
Will Oursler
ecebf1de58
Merge branch 'master' into clickify
2018-04-13 17:52:37 -04:00
David Lynch
b8123e0b26
Explicitly VACUUM the cache on flush
2018-01-19 14:21:05 -06:00
David Lynch
6d52c72c99
Use logging instead of print
...
Fixes #10
2017-11-04 00:09:09 -05:00
Will Oursler
c702337040
Reworks how site-specific options work.
2017-10-13 19:37:13 -04:00
Will Oursler
db48233cf4
Switch from using raw argparser to using click. Preserves the existing
...
interface, except leech --flush becomes leech flush
2017-10-12 13:00:24 -04:00
David Lynch
8ac1aa8bb0
Add cover config to leech.json
2017-10-12 11:20:45 -05:00
Will Oursler
5bd07a5b90
Splits out ebook generation logic into a seperate module, in anticipation of maybe supporting multiple output formats.
2017-10-12 09:49:32 -04:00
David Lynch
6a782244fc
Fix travis eclint call
2017-04-24 01:24:13 -05:00
David Lynch
5b4b9a0dc3
Canonicalize URLs
2017-02-23 15:03:23 -06:00
David Lynch
86e36715f1
Set up for Travis
2017-02-08 13:20:14 -06:00
David Lynch
24f7137929
Fix footnote backlinks, from structure change
2017-02-05 12:38:12 -06:00
David Lynch
e6343cb1c9
Stories are now made of nested sections/chapters
...
This is prep-work for improving epub TOC generation a bit.
2017-01-10 00:23:24 -08:00
David Lynch
24fa9aa22d
Use a namedtuple for chapters
2016-09-23 13:11:52 -05:00
David Lynch
86f02812d2
Use requests-cache
2016-08-29 10:59:20 -05:00
David Lynch
921671f225
Add some front matter
2016-04-23 00:38:10 -05:00
David Lynch
05c98f28db
Command to flush the cache
2016-03-18 08:46:25 -05:00
David Lynch
aa4ba528b7
Let sites define their own custom arguments
...
Use this to let xenforo force the inclusion of the index-post
2015-12-05 01:34:20 -06:00
David Lynch
c69eb1e33e
Footnotes off in their own file
2015-11-30 20:10:58 -06:00
David Lynch
95e25dabd3
First pass at turning spoilers into footnotes for Xenforo
...
This works as popup-footnotes in iBooks and on Kindle. It'd be a bit
better if I put the footnotes in their own file, so they won't be
dropped at the end of chapters on a Kindle. However, that requires
some flow restructuring, and this is an acceptable proof-of-concept
for now.
2015-11-30 16:46:29 -06:00
David Lynch
bff375a8e7
Generate covers
2015-11-11 00:39:41 -08:00
David Lynch
8f198bae21
Allow logging in to sites, to view hidden things
2015-10-28 18:06:19 -05:00
