stash/pkg
notsafeforgit 34821a5d40 perf: eliminate O(N^2) bottlenecks in image and scene duplicate detection
This update resolves major performance regressions when processing large libraries:

1. Optimized FindMany in both Image and Scene stores to use map-based ID lookups. Previously, this function used slices.Index in a loop, resulting in O(N^2) complexity. On a library with 300k items, this was causing the server to hang indefinitely.

2. Refined the exact image duplicate SQL query to match the scene checker's level of optimization. It now joins the files table and orders results by total duplicate file size, ensuring that the most impactful duplicates are shown first.

3. Removed the temporary LIMIT 1000 from the image duplicate query now that the algorithmic bottlenecks have been resolved.
2026-03-29 23:47:36 -07:00
..
exec Fix/ffprobe unmarshalling error (#2685) 2022-06-22 10:49:14 +10:00
ffmpeg [hwaccel] add envvar for /dev/dri device (#6728) 2026-03-23 16:10:22 +11:00
file Add option to ignore zip contents during clean (#6700) 2026-03-18 15:58:32 +11:00
fsutil Allow unicode characters when stripping filenames for json export (#6748) 2026-03-30 11:38:20 +11:00
gallery Make gallery/scene association during scan more consistent (#6705) 2026-03-19 08:54:44 +11:00
group Backend support for Group custom fields (#6596) 2026-02-23 11:53:12 +11:00
hash Use ffmpeg as a general fallback when generating phash (#6641) 2026-03-05 07:58:51 +11:00
image Make gallery/scene association during scan more consistent (#6705) 2026-03-19 08:54:44 +11:00
javascript Add package docs and project vision/goals (#5169) 2024-08-28 09:01:39 +10:00
job Replace ValueOnlyContext with context.WithoutCancel (#6379) 2025-12-08 07:59:42 +11:00
logger Log errors returned from graphql (#3562) 2023-04-17 15:27:25 +10:00
match Add parent tag hierarchy support to tag tagger (#6620) 2026-03-16 11:34:57 +11:00
models feat: Implement Image Duplicate Checker 2026-03-29 23:47:36 -07:00
performer Convert career length fields to dates (#6682) 2026-03-17 15:48:30 +11:00
pkg Add mutex to repositoryCache for thread safety (#6741) 2026-03-30 09:09:28 +11:00
plugin [packaging] switch to pnpm (#6186) 2025-11-10 09:17:25 +11:00
python Add package docs and project vision/goals (#5169) 2024-08-28 09:01:39 +10:00
savedfilter Maintain saved filters in full export/import (#5465) 2024-11-12 16:59:28 +11:00
scene Make gallery/scene association during scan more consistent (#6705) 2026-03-19 08:54:44 +11:00
scraper add {phash} argument to queryURLParameters (#6701) 2026-03-18 15:46:52 +11:00
session [Feature] Reveal file in system file manager from file info panel (#6587) 2026-02-23 12:51:35 +11:00
sliceutil feat: auto-remove duplicate aliases (#6514) 2026-02-04 10:37:15 +11:00
sqlite perf: eliminate O(N^2) bottlenecks in image and scene duplicate detection 2026-03-29 23:47:36 -07:00
stashbox Fix: Remove padFuzzyDate From Performer (#6757) 2026-03-30 09:07:54 +11:00
studio Feature Request: Add organized flag to studios (#6303) 2026-02-19 09:05:17 +11:00
tag Merge tag values dialog (#6552) 2026-02-11 11:27:57 +11:00
txn Database connection pool refactor (#5274) 2024-09-20 12:56:26 +10:00
utils perf: further optimize image duplicate detection 2026-03-29 23:47:36 -07:00