Backup_Repos/leech
1
0
Fork 0
mirror of https://github.com/kemayo/leech synced 2026-04-20 03:20:47 +02:00
Code Issues Projects Releases Wiki Activity
320 commits 4 branches 0 tags 1.7 MiB
Commit graph

176 commits

Author SHA1 Message Date
Idan Dor
1edde92a9d Fixed whitespacing for flake8. 2024-11-23 13:22:53 -06:00
Idan Dor
31f663c6e0 Added image embedding support for epub 
Specifically, added image_selector for arbitrary sites that allows
selecting img tags from chapters, downloading them
and embedding them within the resulting epub.

In the case of Pale, this means that the character banners and
extra materials do not require an internet connection to view.

Also made the two pale.json's more consistent (pale.json now correctly
includes the title of the chapters).
 2024-11-23 13:22:53 -06:00
David Lynch
7967c59636 Support 2fa for xenforo logins 2024-10-13 00:52:50 -05:00
David Lynch
249221f5d7 Fix questionable questing, which has moved to xenforo2 2024-05-14 22:08:22 -05:00
David Lynch
1f57cd6f07 Basic success-testing on logins 2024-05-14 22:07:05 -05:00
David Lynch
ef9309eb66 Fix xenforo login 2024-05-14 22:06:09 -05:00
David Lynch
cc423f62bb Fix the royalroad stolen-content removal 
They added speak:none to the CSS, and I was strictly checking for a rule
that only contained display:none.
 2024-02-10 20:05:49 -06:00
David Lynch
64d77b62db Improve cloudflare email decoding 
New format for the protected emails, wrapping a span in an a.
 2024-01-28 13:26:34 -06:00
David Lynch
d30e56a518 Strip out the new stolen-content warnings on royalroad 
They might make these harder to work out in the future, but for now...
 2024-01-19 21:34:39 -06:00
David Lynch
6c692968a4 Use isinstance rather than direct type comparison 2023-08-06 17:56:13 -05:00
David Lynch
03e9d3844f Add the-sietch.com to xenforo sites 2023-08-06 17:43:51 -05:00
David Lynch
5ddbb310b3 Let xenforo sites cope with index.php URLs 2023-08-06 17:43:28 -05:00
David Lynch
7230f65a68 Add offset/limit options to royalroad 2023-05-04 09:42:13 -05:00
KeinNiemand
356bae9a7a Don't prettify royalroad soup, Fixes #92 2023-05-04 13:17:28 +02:00
David Lynch
6895a0eb61 AO3 single-chapter story bugs 2023-03-31 23:51:27 -05:00
David Lynch
fe5ca86d87 Royalroad's markup has changed slightly, fix so title and summary work 2023-03-17 16:06:52 -05:00
David Lynch
d81eefa7f3 AO3: use new form helper so this shouldn't break again if fields change 2022-05-13 11:04:25 -05:00
David Lynch
f57db3e1a8 Helper for extracting form data from a soup 2022-05-13 11:04:05 -05:00
David Lynch
e9f704716a Xenforo: change some of the style-removal 
It was causing some formatting issues, particularly on Worm fics which
did forum-style sections. (Also, indented text done via margin-left on
divs, which entirely removed the div and ran lines together.)
 2022-04-27 11:07:16 -05:00
David Lynch
56bc2b941c AO3: utf8 field no longer in login form 2022-04-16 18:26:26 -05:00
David Lynch
08abe54e79 Switch out use of :=, forgot I wasn't requiring 3.8 yet 2022-03-06 10:46:13 -06:00
David Lynch
172877410b Xenforo: if fetching a specific threadmark category, add it to the title 
Unless it's 1, since that's always "threadmarks" and the main story.

Refs #79
 2022-03-06 10:42:39 -06:00
David Lynch
29589a0886 RoyalRoad: don't error when covers are relative URLs 
Only happens when the work has no set cover, because it gets a /dist/
URL rather than a CDN URL.

Fixes #77
 2022-02-22 12:19:58 -06:00
David Lynch
f204dcd928 Add a class to generated spoiler divs 2022-02-13 11:44:36 -06:00
David Lynch
697e4c0bf9 Royalroad: don't crash on malformed spoiler tags 
Fixes #74
 2022-02-03 11:08:40 -06:00
David Lynch
dc9c9dbe57 Pull summary and tags for royalroad 2021-11-07 13:16:59 -06:00
David Lynch
4242aa6f63 Strip colors on all sites, not just xenforo 2021-11-07 11:16:26 -06:00
David Lynch
f05bfb51ef AO3: work if www is present in the URL 2021-08-10 17:15:18 -05:00
David Lynch
f1bd28e942 Fanfiction.net: experiment with falling back to the wayback machine 2021-07-19 15:17:39 -05:00
David Lynch
d1caf85883 Extract tags when present 
Supported currently on Xenforo and AO3
 2021-05-01 16:35:49 -05:00
David Lynch
37cb0332b7 AO3: fix issue that could occur if the work had gaps in chapter numbers 2021-04-05 19:55:46 -05:00
David Lynch
77cc334bcf
Merge pull request #60 from ClaasJG/master 
Stable seed generation for Sections
 2021-03-27 19:16:11 -05:00
ClaasJG
5b39c73904 Add stable Section id based on URL 
Remove Chapter id
 2021-03-28 00:41:03 +01:00
David Lynch
bf315d06fe Grab the much more-pythonic CF email decode from #37 2021-03-27 11:20:01 -05:00
David Lynch
f25befc237 Decode cloudflare email address protection 
Makes a generic _clean function on Site that can be called. Will
probably want to migrate some other generic bits into there after
analysis of what's *really* generic.
 2021-03-27 10:46:39 -05:00
David Lynch
dfa298dd3b Better error message for restricted AO3 stories 2021-03-21 23:17:29 -05:00
claasjg
d4f3986515
Detect URL loop with next selector 2021-03-19 14:49:38 +01:00
David Lynch
ce998c84c3 Extract spoilers to footnotes on royalroad 2021-03-07 11:28:49 -06:00
David Lynch
d50f23d07b Special exception for hitting a cloudflare captcha page 
Fanfiction.net is currently doing this, so let's at least acknowledge it

Refs #53
 2021-02-12 16:02:55 -06:00
David Lynch
28cc1fbcc7 Arbitrary should store contents as a string, not a bs4 Tag 
It coincidentally works by being string-like for previous uses, but it's
not string-like enough for the new unicode stuff.

Fixes #54
 2021-02-05 19:58:47 -06:00
David Lynch
ae1b77da2f Wattpad: use API instead 
Their on-page HTML sometimes uses JS to load parts of the story
 2021-01-26 13:11:56 -06:00
David Lynch
23c7a1496c Quick take on wattpad 2021-01-26 01:56:41 -06:00
IdanDor
6d7b5ffcf0 Removed trailing whitespace. 2021-01-23 13:30:03 +02:00
IdanDor
1afac50437 Made arbitrary sites no longer leak memory and fixed worm epub. 
Each `Chapter` object had a reference to the entire page tree, meaning that the program rose in RAM usage by a lot.

Transformed Worm to be with next_selector so the chapters are correctly ordered, E.2 is not skipped and the download does not crush due to `?share=twitter` url matched before.

Fixed Worm titles.
 2021-01-23 12:12:48 +02:00
David Lynch
c208e33752 Arbitrary: strip all namespaced elements 
This is `fb:like` and similar, which break some epub readers.

Refs: #41, #43
 2020-09-08 23:04:47 -05:00
David Lynch
988368bb66 Better xenforo blockquote chrome removal 2020-08-18 13:21:01 -05:00
David Lynch
2103f37cfb AO3: fallback for single-chapter works 2020-05-04 00:31:19 -05:00
David Lynch
6fbdc8843d Make arbitrary site chapter-title selectors more resilient 2020-04-29 17:55:20 -05:00
David Lynch
6631095726 Fiction.live: niche URLs 
* occasional stories with "Sci-fi" in the URL instead of "stories"
* rare cases of `-` in the work id

Fixes #31
 2019-11-14 14:45:19 -06:00
David Lynch
a856f9d0f8 Fiction.live: account for a weird rare bug/possibility in votes 
Also, add a bunch of error handling / logging to the section-parsing to
avoid this in the future.

Fixes #30
 2019-11-07 09:34:39 -06:00