stash/pkg/manager
SmallCoccinelle 09c4c4173d
Support a minModTime input on metadata scans. (#1951)
* Support a maxAge input on metadata scans.

Extend the GraphQL world with a Duration scalar. It is parsed as a
typical Go duration, i.e., "4h" is 4 hours. Alternatively, one can
pass an integer which is interpreted as seconds.

Extend Mutation.metadataScan(input: $input) to support a new optional
value, maxAge. If set, the scanner will exit early if the file it
is looking at has an mtime older than the cutOff point generated by

now() - maxAge

This speeds up scanning in the case where the user knows how old the
changes on disk are, by exiting the scan early if that is the case.

* Change maxAge into minModTime

Introduce a `Timestamp` scalar, so we have a scalar we control. Let
it accept three formats:

* RFC3339Nano
* @UNIX where UNIX is a unix-timestamp: seconds after 01-01-1970
* '<4h': a timestamp relative to the current server time

This scalar parses to a time.Time.

Use MinModTime in the scanner to filter out a large number of scan
analyzes by exiting the scan operation early.

* Heed the linter, perform errcheck

* Rename test vars for consistency.

* Code review: move minModTime into queuefiles

* Remove the ability to input Unix timestamps

Test failures on the CI-system explains why this is undesirable. It is
not clear what timezone one is operating in when entering a unix
timestamp. We could go with UTC, but it is so much easier to require an
RFC3339 timestamp, which avoids this problem entirely.

* Move the minModTime field into filters

Create a new filter input object for metadata scans, and push the
minModTime field in there. If we come up with new filters, they can
be added to that input object rather than cluttering the main input
object.

* Use utils.ParseDateStringAsTime

Replace time.Parse with utils.ParseDateStringAsTime

While here, add some more test cases for that parser.
2021-11-26 11:48:31 +11:00
..
config Tasks page refactor (#1949) 2021-11-24 09:09:47 +11:00
jsonschema Add stash_ids to performer, scene, studio import/export (#1916) 2021-10-29 11:12:39 +11:00
paths refactor: move from io/ioutil to io and os package (#1772) 2021-09-27 10:55:23 +10:00
apikey.go Migrate to golang-jwt (#1757) 2021-09-24 11:57:49 +10:00
checksum.go Toward better context handling (#1835) 2021-10-14 15:32:41 +11:00
downloads.go Enable gocritic (#1848) 2021-10-18 14:12:40 +11:00
exclude_files.go Fixups + enable the commentFormatting linter (#1866) 2021-10-20 16:10:46 +11:00
exclude_files_test.go Fixups + enable the commentFormatting linter (#1866) 2021-10-20 16:10:46 +11:00
filename_parser.go Show duration and filesize in results (#1776) 2021-10-25 11:40:13 +11:00
gallery.go Images section (#813) 2020-10-13 10:12:46 +11:00
generator.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
generator_phash.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
generator_preview.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
generator_sprite.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
image.go Errcheck phase 1 (#1715) 2021-09-21 09:34:25 +10:00
import.go Errorlint sweep + minor linter tweaks (#1796) 2021-10-12 14:03:08 +11:00
json_utils.go Images section (#813) 2020-10-13 10:12:46 +11:00
manager.go Improve flag and environment config overrides (#1898) 2021-11-08 10:14:11 +11:00
manager_tasks.go Refactor generate task (#1871) 2021-10-23 10:48:42 +11:00
post_migrate.go Toward better context handling (#1835) 2021-10-14 15:32:41 +11:00
running_streams.go Errcheck phase 1 (#1715) 2021-09-21 09:34:25 +10:00
scene.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
screenshot.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
studio.go Errorlint sweep + minor linter tweaks (#1796) 2021-10-12 14:03:08 +11:00
subscribe.go Job queueing (#1379) 2021-05-24 14:24:18 +10:00
task.go Toward better context handling (#1835) 2021-10-14 15:32:41 +11:00
task_autotag.go Identify task (#1839) 2021-10-28 14:25:17 +11:00
task_clean.go Show duration and filesize in results (#1776) 2021-10-25 11:40:13 +11:00
task_export.go refactor: move from io/ioutil to io and os package (#1772) 2021-09-27 10:55:23 +10:00
task_generate.go Fix race condition in generate task (#1888) 2021-10-26 08:41:40 +11:00
task_generate_markers.go Refactor generate task (#1871) 2021-10-23 10:48:42 +11:00
task_generate_phash.go Refactor generate task (#1871) 2021-10-23 10:48:42 +11:00
task_generate_preview.go Refactor generate task (#1871) 2021-10-23 10:48:42 +11:00
task_generate_screenshot.go Identify task (#1839) 2021-10-28 14:25:17 +11:00
task_generate_sprite.go Refactor generate task (#1871) 2021-10-23 10:48:42 +11:00
task_identify.go Scraper refactor middle (#2043) 2021-11-26 11:20:06 +11:00
task_import.go Toward better context handling (#1835) 2021-10-14 15:32:41 +11:00
task_migrate_hash.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
task_plugin.go Plugin hooks (#1452) 2021-06-11 17:24:58 +10:00
task_scan.go Support a minModTime input on metadata scans. (#1951) 2021-11-26 11:48:31 +11:00
task_scan_gallery.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
task_scan_image.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
task_scan_scene.go Scan refactor (#1816) 2021-10-15 10:39:48 +11:00
task_stash_box_tag.go Scraper refactor middle (#2043) 2021-11-26 11:20:06 +11:00
task_transcode.go Refactor generate task (#1871) 2021-10-23 10:48:42 +11:00