Commit graph

4124 commits

Author SHA1 Message Date
Jim Miller
018bd04305 Fetcher comments & tweaks 2021-01-28 21:48:49 -06:00
Jim Miller
7c6817bc4f Don't use _method names in new code. 2021-01-28 21:16:23 -06:00
Jim Miller
f8d976f42e Refactor fetcher cache into a dynamic subclass of Fetcher impls. 2021-01-28 21:14:15 -06:00
Jim Miller
d237ac849c Plugin: Only run one BG download at time. 2021-01-28 20:23:37 -06:00
Jim Miller
e1f9de264f Get cloudscraper 1.2.56 changes, but keep py2 compat. 2021-01-28 20:23:37 -06:00
Jim Miller
eb63b8bae5 Refactor cookiejar into Fetcher. 2021-01-28 17:41:33 -06:00
Jim Miller
aa5706f372 Fix Referer header code. 2021-01-27 13:35:42 -06:00
Jim Miller
75999010f0 Refactor CloudScraperFetcher subclass 2021-01-27 12:29:47 -06:00
Jim Miller
a906d8f26b Partial refactoring of cache code. 2021-01-27 11:50:55 -06:00
Jim Miller
67d9eb92f4 More refactoring and consolidation of fetch code. 2021-01-26 18:44:36 -06:00
Jim Miller
59a19a7510 Refactor requests code into own fetcher subclass. 2021-01-26 16:28:40 -06:00
Jim Miller
8894b87212 Consolidate http header code 2021-01-26 12:33:39 -06:00
Jim Miller
682b3ba325 Tweak ffnet plugin sleep setting. 2021-01-26 11:17:02 -06:00
Jim Miller
3c67c4bf13 Remove httplib._MAXHEADERS workaround for royalroad.com 2021-01-26 10:42:02 -06:00
Jim Miller
c8aec09a0e Problem with lushstories.com cookies not happeneing anymore. 2021-01-26 10:29:11 -06:00
Jim Miller
ec13618224 Add requests-file to CLI dependencies. 2021-01-26 10:09:15 -06:00
Jim Miller
dfc68fd0ed HTTPError->HTTPErrorFFF all for trekfanfiction.net's server misconfiguration? 2021-01-26 10:08:28 -06:00
Jim Miller
12a7caa667 Bump Test Version 3.99.0 2021-01-25 21:29:28 -06:00
Jim Miller
80a131b555 Remove unused imports . 2021-01-25 21:27:21 -06:00
Jim Miller
10993a4fe2 Remove unused imports calibre-plugin 2021-01-25 21:09:24 -06:00
Jim Miller
00d9c42e57 Remove unused imports writers 2021-01-25 20:54:11 -06:00
Jim Miller
66520e236c Remove unused imports fanficfare 2021-01-25 20:53:31 -06:00
Jim Miller
edd089237e Remove unused imports adapters 2021-01-25 20:53:04 -06:00
Jim Miller
1d8c10b168 Re-impl use_ssl_unverified_context option. 2021-01-25 20:30:08 -06:00
Jim Miller
ca9ea7ef99 Need User-Agent for POST too. Still need to integrate get/post. 2021-01-25 19:28:19 -06:00
Jim Miller
ea29473239 Consolidate 404->StoryDoesNotExist checks in one place. 2021-01-25 18:13:17 -06:00
Jim Miller
8175361275 Update and keep kludge for 500 trekfanfiction.net server error. 2021-01-25 14:08:39 -06:00
Jim Miller
4959b6eb4f Refactor out some unneeded methods. 2021-01-25 13:20:57 -06:00
Jim Miller
bd3fb5dfe1 Fix file:// fetch mostly for default_cover_image 2021-01-25 12:54:15 -06:00
Jim Miller
88bf48ce44 Remove unused copy of six.py 2021-01-25 12:49:01 -06:00
Jim Miller
b9998abc48 Replace own retry system with urllib3.util.retry.Retry. 2021-01-25 12:34:49 -06:00
Jim Miller
b0cbb7da0d Fix INI _filelist by not using same fetcher anymore. 2021-01-24 17:12:07 -06:00
Jim Miller
8e58e90e84 Refactor Requestable class from Configurable and move decode and zalgo there -- INI _filelist broken? 2021-01-24 15:55:21 -06:00
Jim Miller
75b1cc23b5 Refactor _fetchUrl() to get_request() 2021-01-24 14:12:41 -06:00
Jim Miller
3ba65f922b Refactor _fetchUrlRaw() to get_request_raw() 2021-01-24 14:07:04 -06:00
Jim Miller
38a9c7db05 Refactor _fetchUrlOpened() to get_request_redirected() and remove FakeOpened. 2021-01-24 13:44:35 -06:00
Jim Miller
2e905841e2 Always use requests, no parameters on GETs, still FakeOpened and now fake HTTPError 2021-01-24 13:15:32 -06:00
Jim Miller
7c262e71fa Refactor _fetchUrl()s implicit POST to explicit post_request()s 2021-01-23 15:02:48 -06:00
Jim Miller
b948591389 Refactor: rename _postUrl to post_request 2021-01-23 14:04:50 -06:00
Jim Miller
0822212bcb Remove overly complicated _customized_fetch_url() that was complicating refactoring. 2021-01-23 14:03:07 -06:00
Jim Miller
22b1bca6cd Working on refactor network fetch code. 2021-01-23 13:09:11 -06:00
Jim Miller
f9a1fef55d Bump Test Version 3.29.2 2021-01-23 10:24:56 -06:00
Jim Miller
df016a5e36 Fix for chapter error report. Closes #641 2021-01-23 10:24:38 -06:00
Jim Miller
96d0167538 Bump Test Version 3.29.1 2021-01-20 12:27:50 -06:00
Jim Miller
8b1da6f6ec adapter_fanfictionnet: Start keeping story title part of storyUrl. 2021-01-20 12:27:42 -06:00
Jim Miller
0479e418b2 Bump Release Version 3.29.0 Closes #622 2021-01-20 09:52:08 -06:00
Jim Miller
88f1b9c44d Update translations. 2021-01-20 09:46:40 -06:00
Jim Miller
ef98363abb Bump Test Version 3.28.8 2021-01-18 11:45:45 -06:00
Jim Miller
1342e87c14 New strings for translation. 2021-01-18 11:45:38 -06:00
Jim Miller
03c19c10a3 Add better chapter error reporting, refactor proceed_question code. 2021-01-18 11:42:33 -06:00