Backup_Repos/leech
1
0
Fork 0
mirror of https://github.com/kemayo/leech synced 2026-02-16 11:43:38 +01:00
Code Issues Projects Releases Wiki Activity
190 commits 3 branches 0 tags 1.3 MiB
Commit graph

115 commits

Author SHA1 Message Date
David Lynch
f002064352 Xenforo2 title labels 2019-07-24 23:29:12 -05:00
David Lynch
a148fa8c43 Flake8 errors 2019-07-13 13:17:54 -05:00
David Lynch
3443304ab1 XenForo: handle SV's XenForo2 changes 2019-07-13 11:42:22 -05:00
David Lynch
b1b51bdc8f Xenforo: clean out title prefixes 2019-06-17 16:13:09 -05:00
David Lynch
c8f5b3f8d8 XenForo should use reader-view if available 
Much like 40b4856 greatly sped up AO3, this greatly speeds up XenForo
 2019-05-29 01:56:39 -05:00
David Lynch
2bd5d77715 Helper for URL-joining 2019-05-29 01:55:35 -05:00
David Lynch
66576048da Fix flake8 errors 2019-05-25 20:03:17 -05:00
David Lynch
40b4856a14 Optimize AO3: use full_work URL 2019-05-25 15:31:39 -05:00
David Lynch
f64fce0286 AO3: login form changed 2018-12-29 21:00:02 -06:00
David Lynch
0a81069d24 Slightly more verbose logging of load failures 2018-12-29 20:46:55 -06:00
David Lynch
e78ffdb85b Method to get a site-key for config 
Means that things like XenForoIndex and AO3Series don't require separate
config entries.
 2018-10-11 15:42:59 -05:00
David Lynch
cdcd110c50 AO3: change title detection for logged-in only 2018-10-11 15:42:36 -05:00
David Lynch
0c771ee767
Merge pull request #17 from AlexRaubach/rr_notes 
Place post-chapter RR author notes at the end of the chapter
 2018-10-01 12:19:35 -05:00
David Lynch
02bd6ae0c6
Merge pull request #16 from AlexRaubach/covers 
Download cover art from RR and arbitrary sites
 2018-10-01 12:18:39 -05:00
David Lynch
929284b67d New features for arbitrary sites 
* next_selector: find next content page, if not using chapter selector
* content_title_selector: pull a chapter title from the content
* content_text_selector: pull specific text from the content element

`content_selector` will now fetch all content elements on the page, each
as a Chapter, not just the first one that matches.
 2018-10-01 11:18:39 -05:00
David Lynch
f17b040f64 Fix spacing 2018-09-29 14:32:38 -05:00
David Lynch
b3f4e720d0 Include AlternateHistory as a xenforo site 
Fixes #18. Well, makes it unnecessary. Strictly, it'd maybe still be useful to
show how to do a XenForo site via `arbitary`.
 2018-09-29 12:00:41 -05:00
Alex Raubach
1ff009f893 Improve Prechapter author note detection 2018-09-27 13:48:33 -04:00
Alex Raubach
a9dfdb5dd3 Add a null check to RR author note placement 2018-09-17 20:22:13 -04:00
Alex Raubach
cf62faf5dd Support two RR author notes in one chapter 2018-09-17 20:03:01 -04:00
Alex Raubach
94900cb126 Simplify Royal Road chapter scraper 2018-09-17 00:05:47 -04:00
Alex Raubach
d71184ae8b Place post-chapter RR author notes at the end of the chapter 2018-09-16 17:36:07 -04:00
David Lynch
18c9d68617 Xenforo: cope with ThreadmarksPro's fetchers 2018-09-15 00:18:07 -05:00
Alex Raubach
ff568eef10 Allow arbitrary sites to include a cover url 2018-09-02 22:08:36 -04:00
Alex Raubach
571e262735 Find RR cover img src and assign to cover_url 2018-09-02 22:08:36 -04:00
Alex Raubach
fe76b5427b Add cover_url attribute 2018-09-02 22:08:36 -04:00
David Lynch
8273ca1a77 Fix spacing 2018-08-29 23:39:39 -05:00
David Lynch
17cd0ea4e2 Royalroad domain name fiddliness 2018-08-29 23:07:06 -05:00
David Lynch
6c8ac39d64
fromtimestamp still needed 
My bad.
 2018-08-29 23:04:17 -05:00
David Lynch
a151f02c84
Fix spacing 
...I'm bad at the web interface.
 2018-08-29 23:01:43 -05:00
David Lynch
69c9c21f47
Avoid double-fetching the chapter contents 
Doesn't matter hugely if caching is enabled, but it's still suboptimal.
 2018-08-29 23:00:45 -05:00
random human
23b76d2aac Fix royalroadl.com chapter dates 
Since the timestamp provided with the chapter list is approximate, fetch
the actual chapter in order to get unixtime.
 2018-08-30 04:03:29 +05:30
Alex Raubach
1bfc9b75f7
Remove unneeded whitespace 2018-08-28 23:24:59 -04:00
Alex Raubach
2019616505
Check that the chapter has content before parsing 
Trying to select the first element in line 87 will throw a list index out of range error if there is no content matching the selector.
 2018-08-28 21:59:16 -04:00
David Lynch
fb8d6cf0d6
Merge pull request #9 from Zomega/clickify 
Switch from using raw argparser to using click.
 2018-08-17 21:33:23 -05:00
David Lynch
499530993c Royalroad seems to need www now 2018-07-11 21:24:22 -05:00
Will Oursler
d1842e2bf1 Adds a system for site options to be included as click.options on commands. 2018-04-14 12:56:31 -04:00
Will Oursler
ecebf1de58 Merge branch 'master' into clickify 2018-04-13 17:52:37 -04:00
David Lynch
868ef4b157 Handle mobile links for FFN 2018-03-30 15:18:57 -05:00
David Lynch
7d2c1647e2 Safer check on retry-after 2018-02-28 20:54:37 -06:00
David Lynch
2042f813d0 Allow AO3 logins for member-only stories 2018-01-19 14:15:43 -06:00
David Lynch
f8d494283c Proper URL normalization for AO3 chapters 2018-01-19 13:19:45 -06:00
David Lynch
e9dab9ab7d Fix linting on royalroad 2017-11-17 22:57:54 -06:00
David Lynch
e099f47e66 Support: RoyalRoad 2017-11-17 21:37:13 -06:00
David Lynch
7bb6da382c Oh hey, another missing Section URL 2017-11-04 00:30:59 -05:00
David Lynch
6d52c72c99 Use logging instead of print 
Fixes #10
 2017-11-04 00:09:09 -05:00
David Lynch
43599aceb5
Merge branch 'master' into clickify 2017-11-03 15:21:44 -05:00
David Lynch
f1ac7c8bda Retry failed site-requests 2017-10-31 00:27:54 -05:00
David Lynch
27b677a444 Fix no-threadmarks autodetect 2017-10-29 19:50:19 -05:00
David Lynch
257ab69394 Arbitrary handler: canonicalize URLs 2017-10-22 17:31:10 -05:00