Backup_Repos/leech
1
0
Fork 0
mirror of https://github.com/kemayo/leech synced 2026-01-01 21:23:28 +01:00
Code Issues Projects Releases Wiki Activity
294 commits 2 branches 0 tags 1.3 MiB
Commit graph

294 commits

This branch
This branch
All branches
Author SHA1 Message Date
Idan Dor
422360de4e Fixed whitespacing for flake8. 2022-11-04 16:10:58 +02:00
Idan Dor
d3e603a028 Added image embedding support for epub 
Specifically, added image_selector for arbitrary sites that allows
selecting img tags from chapters, downloading them
and embedding them within the resulting epub.

In the case of Pale, this means that the character banners and
extra materials do not require an internet connection to view.

Also made the two pale.json's more consistent (pale.json now correctly
includes the title of the chapters).
 2022-11-04 16:04:18 +02:00
David Lynch
b8314341b2 Allow passing in multiple URLs 
Good trick: `./leech.py examples/practical{1,2,3,4,5,6,7}.json`
 2022-10-22 18:05:59 -05:00
David Lynch
c3b28d6169
Merge pull request #82 from 7h0m4s/Update_PracticalGuide_Example_Files 
Update Practical Guide example json
 2022-09-03 14:52:38 -05:00
Thomas Pegoraro
a3496c73b5 -Remove " .entry-wrapper" class from content_selector as the class doesn't exist on the Practical Guide website anymore. 
- Add Practical Guide book 7 json file.
 2022-09-03 13:14:54 +10:00
David Lynch
d81eefa7f3 AO3: use new form helper so this shouldn't break again if fields change 2022-05-13 11:04:25 -05:00
David Lynch
f57db3e1a8 Helper for extracting form data from a soup 2022-05-13 11:04:05 -05:00
David Lynch
e9f704716a Xenforo: change some of the style-removal 
It was causing some formatting issues, particularly on Worm fics which
did forum-style sections. (Also, indented text done via margin-left on
divs, which entirely removed the div and ran lines together.)
 2022-04-27 11:07:16 -05:00
David Lynch
56bc2b941c AO3: utf8 field no longer in login form 2022-04-16 18:26:26 -05:00
David Lynch
08abe54e79 Switch out use of :=, forgot I wasn't requiring 3.8 yet 2022-03-06 10:46:13 -06:00
David Lynch
172877410b Xenforo: if fetching a specific threadmark category, add it to the title 
Unless it's 1, since that's always "threadmarks" and the main story.

Refs #79
 2022-03-06 10:42:39 -06:00
David Lynch
64d97b09d0
Merge pull request #78 from codetheweb/feat-unsong 
Add Unsong
 2022-03-02 11:53:33 -06:00
Max Isom
2414edc1b1
Add Unsong 2022-02-25 21:39:23 -05:00
David Lynch
7eae5dee07 When downloading covers errors don't incorrectly concat a string 
Refs #77
 2022-02-22 12:24:50 -06:00
David Lynch
29589a0886 RoyalRoad: don't error when covers are relative URLs 
Only happens when the work has no set cover, because it gets a /dist/
URL rather than a CDN URL.

Fixes #77
 2022-02-22 12:19:58 -06:00
David Lynch
7eb48c872d Clean up epub file list generation 2022-02-13 12:25:01 -06:00
David Lynch
ba7f694ec4
Merge pull request #76 from kemayo/dependabot/pip/urllib3-1.26.5 
Bump urllib3 from 1.26.2 to 1.26.5
 2022-02-13 11:50:40 -06:00
dependabot[bot]
ebb543e727
Bump urllib3 from 1.26.2 to 1.26.5 
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.2 to 1.26.5.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.2...1.26.5)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-13 17:49:20 +00:00
David Lynch
c74cbac820
Merge pull request #73 from kemayo/dependabot/pip/pillow-9.0.0 
Bump pillow from 8.1.0 to 9.0.0
 2022-02-13 11:48:39 -06:00
David Lynch
f204dcd928 Add a class to generated spoiler divs 2022-02-13 11:44:36 -06:00
David Lynch
697e4c0bf9 Royalroad: don't crash on malformed spoiler tags 
Fixes #74
 2022-02-03 11:08:40 -06:00
dependabot[bot]
9949126676
Bump pillow from 8.1.0 to 9.0.0 
Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.1.0 to 9.0.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/8.1.0...9.0.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-01-13 03:54:53 +00:00
David Lynch
dc9c9dbe57 Pull summary and tags for royalroad 2021-11-07 13:16:59 -06:00
David Lynch
4242aa6f63 Strip colors on all sites, not just xenforo 2021-11-07 11:16:26 -06:00
David Lynch
a3ea8fbb8c Examples for Pale 
Refs #66
 2021-09-04 19:38:20 -05:00
David Lynch
bb9491cb96 Config option: output_dir 
Can be provided on the command line as `--output-dir`, or in leech.json
as `output_dir` (also in the `site_options` in leech.json).

Refs #67
 2021-09-04 15:46:16 -05:00
David Lynch
312fbc03a1 Requires Python 3.7 since namedtuple defaults are needed 
Fixes #70
 2021-09-04 15:12:17 -05:00
David Lynch
186fbd406d
Merge pull request #69 from KeinNiemand/master 
Include files in ebook and sites in poetry build
 2021-09-04 14:59:15 -05:00
KeinNiemand
4d580f483e
include files in ebook and sites in poetry build 2021-09-03 20:28:15 +02:00
David Lynch
f05bfb51ef AO3: work if www is present in the URL 2021-08-10 17:15:18 -05:00
David Lynch
25312736d4 Use namedtuple in the epub generator so it's easier to understand 2021-07-21 10:32:55 -05:00
David Lynch
f1bd28e942 Fanfiction.net: experiment with falling back to the wayback machine 2021-07-19 15:17:39 -05:00
David Lynch
41bbe46399
Merge pull request #62 from atamis/master 
Add Dockerfile
 2021-06-10 09:15:48 -05:00
Azrea Amis
674a5b7e6b Simplify Dockerfile 
Replace intervenining container with just disabling virtualenvs
installing via poetry.
 2021-05-09 06:19:53 +00:00
Azrea Amis
98b4622bd5 Add Docker info to README 2021-05-08 21:38:22 +00:00
Azrea Amis
1a984c685b Add Dockerfile 2021-05-08 21:38:06 +00:00
David Lynch
9b80a112d0 Output summary and tags in the Front Matter 2021-05-01 16:36:08 -05:00
David Lynch
d1caf85883 Extract tags when present 
Supported currently on Xenforo and AO3
 2021-05-01 16:35:49 -05:00
David Lynch
37cb0332b7 AO3: fix issue that could occur if the work had gaps in chapter numbers 2021-04-05 19:55:46 -05:00
David Lynch
77cc334bcf
Merge pull request #60 from ClaasJG/master 
Stable seed generation for Sections
 2021-03-27 19:16:11 -05:00
ClaasJG
5b39c73904 Add stable Section id based on URL 
Remove Chapter id
 2021-03-28 00:41:03 +01:00
David Lynch
bf315d06fe Grab the much more-pythonic CF email decode from #37 2021-03-27 11:20:01 -05:00
David Lynch
966bd2c120 Include Ward in examples 
It has some very noticeable encoding issues, which probably require more
thought on how to sanitize weird inputs.

Fixes #37
 2021-03-27 11:16:44 -05:00
David Lynch
f25befc237 Decode cloudflare email address protection 
Makes a generic _clean function on Site that can be called. Will
probably want to migrate some other generic bits into there after
analysis of what's *really* generic.
 2021-03-27 10:46:39 -05:00
David Lynch
dfa298dd3b Better error message for restricted AO3 stories 2021-03-21 23:17:29 -05:00
David Lynch
73df5d5cb3 Give nicer error messages when a SiteException is raised 2021-03-21 23:16:18 -05:00
David Lynch
d561d1d80b
Merge pull request #58 from ClaasJG/master 
Detect loop when using 'next_selector'
 2021-03-19 09:57:31 -05:00
claasjg
d4f3986515
Detect URL loop with next selector 2021-03-19 14:49:38 +01:00
David Lynch
ce998c84c3 Extract spoilers to footnotes on royalroad 2021-03-07 11:28:49 -06:00
David Lynch
0d0bdf470e Escape chapter titles when building templates 
Unescaped ampersands cause validation errors...

TODO: should move away from string substitution to build XHTML

Refs #56
 2021-03-02 02:30:05 -06:00