Commit graph

6 commits

Author SHA1 Message Date
abdusalam.dihan
93a05fce8f Use sync.Map instead of LRU for the per-job regexp cache
The preceding commit added lru.Cache for the compiled-regexp cache to
match the style in pkg/sqlite/regex.go. That file's use case is
different: a small bounded cache serving a read-dominated workload. The
auto-tag regexp cache is job-scoped (so eviction buys us nothing) and
hit by every worker on every candidate (so the LRU's per-Get mutex
becomes contention, measurable under the parallel worker pool).

sync.Map's read-optimised path avoids the contention without changing
any observable behavior. Kept as a separate commit so it can be
reverted independently if upstream prefers the LRU approach — the
first commit stands on its own either way.
2026-04-19 22:22:37 +01:00
abdusalam.dihan
cd64433dc5 Speed up file-based auto-tag
Replaces the per-file SQL QueryForAutoTag prefilter with an in-memory
2-rune prefix index over performers/studios/tags, preloaded once at job
start. Also:

  - runs file processing through job.TaskQueue so scenes/images/
    galleries tag in parallel instead of one file at a time
  - keyset-paginates the query loop so batch N+1 doesn't pay the
    O(offset) scan past large tables
  - bulk-loads studio/tag aliases via a new optional AllAliasLoader
    interface, avoiding N+1 GetAliases calls during preload
  - caches compiled name regexps (same candidate names repeat across
    thousands of files)
  - hoists strings.ToLower(path) and allASCII(path) out of the per-
    candidate match loop
  - opens a fresh write txn per applied match instead of holding one
    for every tagger phase

Tagger gains *AtPath methods that own the cache + txn manager, letting
the task code stay slim.
2026-04-19 22:22:37 +01:00
WithoutPants
e843c890fb
Add related object filter criteria to various filter types in graphql schema (#4861)
* Move filter criterion handlers into separate file
* Add related filters for image filter
* Add related filters for scene filter
* Add related filters to gallery filter
* Add related filters to movie filter
* Add related filters to performer filter
* Add related filters to studio filter
* Add related filters to tag filter
* Add scene filter to scene marker filter
2024-06-11 11:34:38 +10:00
DingDongSoLong4
c364346a59
Model refactor (#3915)
* Add mockery config file
* Move basic file/folder structs to models
* Fix hack due to import loop
* Move file interfaces to models
* Move folder interfaces to models
* Move scene interfaces to models
* Move scene marker interfaces to models
* Move image interfaces to models
* Move gallery interfaces to models
* Move gallery chapter interfaces to models
* Move studio interfaces to models
* Move movie interfaces to models
* Move performer interfaces to models
* Move tag interfaces to models
* Move autotag interfaces to models
* Regenerate mocks
2023-09-01 10:39:29 +10:00
WithoutPants
964b559309 Restructure data layer (#2532)
* Add new txn manager interface
* Add txn management to sqlite
* Rename get to getByID
* Add contexts to repository methods
* Update query builders
* Add context to reader writer interfaces
* Use repository in resolver
* Tighten interfaces
* Tighten interfaces in dlna
* Tighten interfaces in match package
* Tighten interfaces in scraper package
* Tighten interfaces in scan code
* Tighten interfaces on autotag package
* Remove ReaderWriter usage
* Merge database package into sqlite
2022-09-06 07:03:40 +00:00
WithoutPants
d88515abcd
Autotag optimisation (#2368)
* Add duration to autotag finish message
* No sorting scene/image/gallery where not specified
* Use an LRU cache for sqlite regexp function
* Compile path separator regex once
* Cache objects with single letter first names
* Move finished auto-tag log
* Add more verbose logging
* Add new changelog
2022-03-09 12:01:56 +11:00