Backup_Repos/leech
1
0
Fork 0
mirror of https://github.com/kemayo/leech synced 2025-12-15 12:56:41 +01:00
Code Issues Projects Releases Wiki Activity
269 commits 2 branches 0 tags 1.3 MiB
Commit graph

269 commits

This branch
This branch
All branches
Author SHA1 Message Date
David Lynch
bb9491cb96 Config option: output_dir 
Can be provided on the command line as `--output-dir`, or in leech.json
as `output_dir` (also in the `site_options` in leech.json).

Refs #67
 2021-09-04 15:46:16 -05:00
David Lynch
312fbc03a1 Requires Python 3.7 since namedtuple defaults are needed 
Fixes #70
 2021-09-04 15:12:17 -05:00
David Lynch
186fbd406d
Merge pull request #69 from KeinNiemand/master 
Include files in ebook and sites in poetry build
 2021-09-04 14:59:15 -05:00
KeinNiemand
4d580f483e
include files in ebook and sites in poetry build 2021-09-03 20:28:15 +02:00
David Lynch
f05bfb51ef AO3: work if www is present in the URL 2021-08-10 17:15:18 -05:00
David Lynch
25312736d4 Use namedtuple in the epub generator so it's easier to understand 2021-07-21 10:32:55 -05:00
David Lynch
f1bd28e942 Fanfiction.net: experiment with falling back to the wayback machine 2021-07-19 15:17:39 -05:00
David Lynch
41bbe46399
Merge pull request #62 from atamis/master 
Add Dockerfile
 2021-06-10 09:15:48 -05:00
Azrea Amis
674a5b7e6b Simplify Dockerfile 
Replace intervenining container with just disabling virtualenvs
installing via poetry.
 2021-05-09 06:19:53 +00:00
Azrea Amis
98b4622bd5 Add Docker info to README 2021-05-08 21:38:22 +00:00
Azrea Amis
1a984c685b Add Dockerfile 2021-05-08 21:38:06 +00:00
David Lynch
9b80a112d0 Output summary and tags in the Front Matter 2021-05-01 16:36:08 -05:00
David Lynch
d1caf85883 Extract tags when present 
Supported currently on Xenforo and AO3
 2021-05-01 16:35:49 -05:00
David Lynch
37cb0332b7 AO3: fix issue that could occur if the work had gaps in chapter numbers 2021-04-05 19:55:46 -05:00
David Lynch
77cc334bcf
Merge pull request #60 from ClaasJG/master 
Stable seed generation for Sections
 2021-03-27 19:16:11 -05:00
ClaasJG
5b39c73904 Add stable Section id based on URL 
Remove Chapter id
 2021-03-28 00:41:03 +01:00
David Lynch
bf315d06fe Grab the much more-pythonic CF email decode from #37 2021-03-27 11:20:01 -05:00
David Lynch
966bd2c120 Include Ward in examples 
It has some very noticeable encoding issues, which probably require more
thought on how to sanitize weird inputs.

Fixes #37
 2021-03-27 11:16:44 -05:00
David Lynch
f25befc237 Decode cloudflare email address protection 
Makes a generic _clean function on Site that can be called. Will
probably want to migrate some other generic bits into there after
analysis of what's *really* generic.
 2021-03-27 10:46:39 -05:00
David Lynch
dfa298dd3b Better error message for restricted AO3 stories 2021-03-21 23:17:29 -05:00
David Lynch
73df5d5cb3 Give nicer error messages when a SiteException is raised 2021-03-21 23:16:18 -05:00
David Lynch
d561d1d80b
Merge pull request #58 from ClaasJG/master 
Detect loop when using 'next_selector'
 2021-03-19 09:57:31 -05:00
claasjg
d4f3986515
Detect URL loop with next selector 2021-03-19 14:49:38 +01:00
David Lynch
ce998c84c3 Extract spoilers to footnotes on royalroad 2021-03-07 11:28:49 -06:00
David Lynch
0d0bdf470e Escape chapter titles when building templates 
Unescaped ampersands cause validation errors...

TODO: should move away from string substitution to build XHTML

Refs #56
 2021-03-02 02:30:05 -06:00
David Lynch
d50f23d07b Special exception for hitting a cloudflare captcha page 
Fanfiction.net is currently doing this, so let's at least acknowledge it

Refs #53
 2021-02-12 16:02:55 -06:00
David Lynch
0f176f5a7c session.cookies should stay a RequestsCookieJar 2021-02-12 15:50:18 -06:00
David Lynch
e66adc171f Add example for getting all of Practical Guide in one ebook 
...this has been sitting uncommitted since I did all the `next` stuff,
so I think I just forgot to commit it then.
 2021-02-12 10:46:16 -06:00
David Lynch
406e88acfa Add an example for Heretical Edge 2, to show how to exclude chapters 
Refs #46
 2021-02-12 10:43:54 -06:00
David Lynch
5e8d42e256 JSON indentation should be spaces 2021-02-12 10:37:14 -06:00
David Lynch
28cc1fbcc7 Arbitrary should store contents as a string, not a bs4 Tag 
It coincidentally works by being string-like for previous uses, but it's
not string-like enough for the new unicode stuff.

Fixes #54
 2021-02-05 19:58:47 -06:00
David Lynch
533c14f0d7 Normalize fancy unicode characters by default 
Kindle can't display the "𝖙𝖍𝖚𝖌 𝖑𝖎𝖋𝖊" mathematical bold fraktur codepoints
so NFKC normalize them (and anything else) into its plain equivalent.

Can be disabled by running with `--no-normalize` if needed.
 2021-02-05 01:59:20 -06:00
David Lynch
ba77390f27 Remove .travis.yml 2021-01-26 14:43:11 -06:00
David Lynch
3a8137a099 eclint restrictions 2021-01-26 14:42:07 -06:00
David Lynch
3ec075cf2a cover.py isn't executable so different call needed 2021-01-26 14:37:29 -06:00
David Lynch
74a07cb2f8 But without the typo in the YAML this time 2021-01-26 13:53:37 -06:00
David Lynch
112565099c Tweak action 2021-01-26 13:53:00 -06:00
David Lynch
b1e9350a23
Create python-package-poetry.yml 2021-01-26 13:49:26 -06:00
David Lynch
7c7d536019 Bump lock packages 2021-01-26 13:12:39 -06:00
David Lynch
ae1b77da2f Wattpad: use API instead 
Their on-page HTML sometimes uses JS to load parts of the story
 2021-01-26 13:11:56 -06:00
David Lynch
23c7a1496c Quick take on wattpad 2021-01-26 01:56:41 -06:00
David Lynch
7c040c08a0
Merge pull request #49 from keturn/feat/travis-build 
have Travis CI test more things
 2021-01-23 13:00:29 -06:00
David Lynch
7af94af17b
Merge pull request #48 from keturn/feat/leech-script 
provide "leech" as a runnable script
 2021-01-23 12:59:18 -06:00
David Lynch
2465a64fe3
Merge pull request #51 from IdanDor/master 
Made arbitrary sites no longer leak memory and fixed worm epub.
 2021-01-23 12:57:45 -06:00
IdanDor
6d7b5ffcf0 Removed trailing whitespace. 2021-01-23 13:30:03 +02:00
IdanDor
1afac50437 Made arbitrary sites no longer leak memory and fixed worm epub. 
Each `Chapter` object had a reference to the entire page tree, meaning that the program rose in RAM usage by a lot.

Transformed Worm to be with next_selector so the chapters are correctly ordered, E.2 is not skipped and the download does not crush due to `?share=twitter` url matched before.

Fixed Worm titles.
 2021-01-23 12:12:48 +02:00
Kevin Turner
dc13946a77 have Travis CI test more things 
- run the --help command
- try making a cover
- build the distribution and show what it made
 2021-01-14 21:32:27 -08:00
Kevin Turner
f909472509 provide "leech" as a runnable script 2021-01-14 21:21:31 -08:00
David Lynch
377e6ff76c Update dependencies on attrs and Pillow to latest versions 2020-11-01 21:13:20 -06:00
David Lynch
4429049ade Fix the Vacant Throne example as well 2020-09-08 23:07:21 -05:00