Commit graph

4415 commits

Author SHA1 Message Date
Jim Miller
f99810a1ca Remove a stray space from ini. 2021-01-31 15:28:16 -06:00
Jim Miller
2ee5d71821 Py2/py3 conflict with 'what is a class'. 2021-01-31 15:13:06 -06:00
Jim Miller
6bcf0f5499 Bump Test Version 3.99.6 2021-01-31 14:34:06 -06:00
Jim Miller
5d00f16003 Tweak fetcher debug output. 2021-01-31 14:34:06 -06:00
Jim Miller
86dee0081d Convert adapter_spikeluvercom to base_efiction after site changes. 2021-01-31 13:08:52 -06:00
Jim Miller
623158bb01 Enable pagecache for base_efiction. 2021-01-31 13:07:50 -06:00
Jim Miller
e3217dfed6 Bump Test Version 3.99.5 2021-01-31 11:45:30 -06:00
Jim Miller
7aa451a3c1 Clean up debug output a little. 2021-01-31 11:42:33 -06:00
Jim Miller
f02b854343 Remove 'extrasleep' feature--it was ugly--increase slow_down_sleep_time for the couple sites that used it. We can trust the users, right? 2021-01-31 11:34:38 -06:00
Jim Miller
fad21498d2 Bump Test Version 3.99.4 2021-01-31 10:48:35 -06:00
Jim Miller
e1c27f8841 Merge branch 'fetch_refactor' into master 2021-01-31 10:47:06 -06:00
Jim Miller
e380560cb3 'Remove' unsupported --unverified_ssl CLI option. Use use_ssl_unverified_context:true 2021-01-31 10:38:29 -06:00
Jim Miller
03ebc65f6b Refactor autosave cookie/cache in CLI, etc. 2021-01-31 10:32:42 -06:00
Jim Miller
b1c2fe6885 Bump Test Version 3.99.3 2021-01-31 09:55:07 -06:00
Jim Miller
3c20a4c247 BasicCache save/load working. 2021-01-31 09:53:42 -06:00
Jim Miller
e4d81f0dff Busy cursor while building RejectListDialog 2021-01-31 09:50:56 -06:00
Jim Miller
2191498ef6 Bump Test Version 3.29.6 2021-01-30 17:24:35 -06:00
Jim Miller
8ef5dcc1b3 Handle case of story URLs from removed sites in rejects more gracefully. 2021-01-30 17:24:12 -06:00
Jim Miller
9fb72efa15 Fix adapter_fictionpresscom url pattern properly. 2021-01-30 17:22:30 -06:00
Jim Miller
773f83bb06 Bump Test Version 3.29.5 2021-01-30 16:20:39 -06:00
Jim Miller
e2b632c99a Merge branch 'get_section_url' 2021-01-30 16:18:34 -06:00
Jim Miller
586ddce59f Make get_section_url classmethod for performance. 2021-01-30 16:06:20 -06:00
Jim Miller
6695b9a846 get_section_url for rejects 2021-01-30 15:01:46 -06:00
Jim Miller
6ffdf768bb Comment out out debug output. 2021-01-30 15:00:31 -06:00
Jim Miller
1e38646026 Comment out out debug output. 2021-01-30 14:57:15 -06:00
Jim Miller
ef6ceaf8b0 Fix so existing [.../1/] INI sections will work for ffnet, refactor _section_url to get_section_url 2021-01-30 14:27:51 -06:00
Jim Miller
0c0534ea74 Refactor set_sleep(_override) a bit better. 2021-01-29 18:06:09 -06:00
Jim Miller
8ba5d2c423 Refactor use_pagecache into an INI setting and a sharable, thread safe cache impl. 2021-01-29 17:31:30 -06:00
Jim Miller
ddf82749af Bump Test Version 3.99.1 2021-01-29 13:41:11 -06:00
Jim Miller
3f6793b301 Refactor sleep and progressbar into FetcherDecorators 2021-01-29 13:30:49 -06:00
Jim Miller
67d4eb46ee Fix for intermediate fetcher/cache for plugin. 2021-01-29 12:07:47 -06:00
Jim Miller
ff9db222b3 Repackage brotlidecpy to not use _brotlidecpy 2021-01-29 11:06:40 -06:00
Jim Miller
166a7795d6 Switch bundled brotli to @sidney's py2/py3 brotlidecpy 2021-01-29 10:26:32 -06:00
Jim Miller
dd261dec96 Re-refactor cache code to not be a fetcher and instead dynamically 'decorate' fetcher do_request with caching code. 2021-01-28 22:10:54 -06:00
Jim Miller
018bd04305 Fetcher comments & tweaks 2021-01-28 21:48:49 -06:00
Jim Miller
7c6817bc4f Don't use _method names in new code. 2021-01-28 21:16:23 -06:00
Jim Miller
f8d976f42e Refactor fetcher cache into a dynamic subclass of Fetcher impls. 2021-01-28 21:14:15 -06:00
Jim Miller
d237ac849c Plugin: Only run one BG download at time. 2021-01-28 20:23:37 -06:00
Jim Miller
e1f9de264f Get cloudscraper 1.2.56 changes, but keep py2 compat. 2021-01-28 20:23:37 -06:00
Jim Miller
1d55f4778d Bump Test Version 3.29.4 2021-01-28 20:03:55 -06:00
Jim Miller
20b5b8fb95 Plugin: Only run one BG download at time. 2021-01-28 20:03:42 -06:00
Jim Miller
eb63b8bae5 Refactor cookiejar into Fetcher. 2021-01-28 17:41:33 -06:00
Jim Miller
5922d027b7 Bump Test Version 3.29.3 2021-01-28 09:40:28 -06:00
Jim Miller
a46edf092d Get cloudscraper 1.2.56 changes, but keep py2 compat. 2021-01-28 09:38:32 -06:00
Jim Miller
aa5706f372 Fix Referer header code. 2021-01-27 13:35:42 -06:00
Jim Miller
75999010f0 Refactor CloudScraperFetcher subclass 2021-01-27 12:29:47 -06:00
Jim Miller
a906d8f26b Partial refactoring of cache code. 2021-01-27 11:50:55 -06:00
Jim Miller
67d9eb92f4 More refactoring and consolidation of fetch code. 2021-01-26 18:44:36 -06:00
Jim Miller
59a19a7510 Refactor requests code into own fetcher subclass. 2021-01-26 16:28:40 -06:00
Jim Miller
8894b87212 Consolidate http header code 2021-01-26 12:33:39 -06:00