Commit graph

220 commits

Author SHA1 Message Date
WithoutPants
05488d59c3
Find scrapers in subdirectories (#554) 2020-05-19 08:44:33 +10:00
bnkai
0fc57ce1e0
Fix xpath comments text (#550) 2020-05-18 12:26:20 +10:00
WithoutPants
215c4e3bde
Change builtin freeones scraper to community yml (#542) 2020-05-15 20:10:20 +10:00
bnkai
0b50e83dbf
freeones scraper tweaks (#509) 2020-05-04 14:11:49 +10:00
WithoutPants
3d22d5a742
Refactor build (#493)
* Add lint/format checks to build
* Make travis get full repo to get tags
* Run packr2 once in cross-compile
* Fix quotes in package.json
* Fix linting issues
* Formatting
* Fix vet issue
* Fix go lint issues
* Show start of each platform compilation
* Add validate target
* Set gitattributes for go fmt and mod vendor
* Fix tag name
* Add fmt-ui target
2020-04-29 12:13:08 +10:00
WithoutPants
82201e23e0
Make ethnicity freetext and fix freeones ethnicity panic (#431)
* Make ethnicity free text

* Fix panic in freeones scraper for other ethnicity
2020-04-02 08:25:39 +11:00
WithoutPants
abf2b49803
Configurable scraper user agent string (#409)
* Add debug scrape option.

Co-authored-by: HiddenPants255 <>
2020-03-21 08:55:15 +11:00
WithoutPants
34d829338d
Add image scraping support (#370)
* Add sub-scraper functionality
* Add scraping of performer image
* Add scene cover image scraping
* Port UI changes to v2.5
* Fix v2.5 dialog suggest color
* Don't convert eol of UI to support pretty
2020-03-11 11:41:55 +11:00
caustico
5fb8bbf768
Movies Section (#338)
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2020-03-10 14:28:15 +11:00
WithoutPants
03c07a429d
Add Xpath post processing and performer name query (#333)
* Extend xpath configuration. Support concatenation

* Add parseDate parsing option

* Add regex replacements

* Add xpath query performer by name

* Fix loading spinner on scrape performer

* Change ReplaceAll to Replace
2020-01-31 17:17:40 -05:00
WithoutPants
78eb527ec4 Scraper fixes (#332)
* Fix panic on invalid xpath

* Add missing attrs to scraped performer fragment
2020-01-24 22:36:24 -05:00
WithoutPants
7fdaccf669 Xpath scraping from URL (#285)
* Add xpath performer and scene scraping

* Add studio scraping

* Refactor code

* Fix compile error

* Don't overwrite performer URL during a scrape
2020-01-04 11:39:33 -05:00
WithoutPants
f52db4f58b Add stash scraper type (#269)
* Add stash scraper type

* Add graphql client to vendor

* Embed stash credentials in URL

* Fill URL from scraped scene

* Nil IDs returned from remote stash

* Nil check
2019-12-20 19:13:23 -05:00
WithoutPants
92837fe1f7 Add scene metadata scraping functionality (#236)
* Add scene scraping functionality

* Adapt to changed scraper config
2019-12-15 20:35:34 -05:00
WithoutPants
50784025f2 Change scraper config to yaml (#256) 2019-12-12 14:27:44 -05:00
WithoutPants
17247060b6 Generic performer scrapers (#203)
* Generalise scraper API

* Add script performer scraper

* Fixes from testing

* Add context to scrapers and generalise

* Add scraping performer from URL

* Add error handling

* Move log to debug

* Add supported scrape types
2019-11-18 21:49:05 -05:00
bill
9f6888a3d6 fix freeones scraper bugs 2019-10-16 02:05:49 +03:00
Friendly C
7c94262020 Freeones Scrape: Fix scraping by alias 2019-10-10 23:56:06 +02:00
bnkai
bcc70af7e5 Fix minor freeones scraper bug (#41)
Fix minor freeones scraper bug
2019-04-11 11:54:38 -07:00
Stash Dev
b488c1ed7d Reorg 2019-02-14 15:42:52 -08:00