Backup_Repos/stash
1
0
Fork 0
mirror of https://github.com/stashapp/stash.git synced 2026-03-28 08:01:57 +01:00
Code Issues Projects Releases Wiki Activity
stash/pkg/sqlite
History
notsafeforgit f2f299307a perf: eliminate O(N^2) bottlenecks in image and scene duplicate detection 
This update resolves major performance regressions when processing large libraries:

1. Optimized FindMany in both Image and Scene stores to use map-based ID lookups. Previously, this function used slices.Index in a loop, resulting in O(N^2) complexity. On a library with 300k items, this was causing the server to hang indefinitely.

2. Refined the exact image duplicate SQL query to match the scene checker's level of optimization. It now joins the files table and orders results by total duplicate file size, ensuring that the most impactful duplicates are shown first.

3. Removed the temporary LIMIT 1000 from the image duplicate query now that the algorithmic bottlenecks have been resolved.
2026-03-20 15:29:04 -07:00
..
blob Developer option: extra blob paths (#4566) 2024-02-16 12:39:45 +11:00
migrations update logs and fix UNIQUE constraint failure (#6617) 2026-02-28 09:11:13 +11:00
anonymise.go Backend support for Group custom fields (#6596) 2026-02-23 11:53:12 +11:00
anonymise_test.go Add anonymise database task (#3186) 2022-12-23 09:15:27 +11:00
batch.go Add findFiles and findFile graphql queries (#5941) 2025-06-24 13:05:17 +10:00
blob.go Add related object filter criteria to various filter types in graphql schema (#4861) 2024-06-11 11:34:38 +10:00
blob_migrate.go Add filesystem based blob storage (#3187) 2023-03-17 10:52:49 +11:00
blob_test.go Movie group renames (#5039) 2024-07-04 09:10:26 +10:00
common.go File storage rewrite (#2676) 2022-09-06 07:03:42 +00:00
criterion_handlers.go Fix: Custom Field Filtering (#6614) 2026-02-28 11:05:13 +11:00
custom_fields.go Backend support for image custom fields (#6598) 2026-02-24 07:41:40 +11:00
custom_fields_test.go Backend support for image custom fields (#6598) 2026-02-24 07:41:40 +11:00
custom_migrations.go File storage rewrite (#2676) 2022-09-06 07:03:42 +00:00
database.go Add basename and parent_folders fields to Folder graphql interface (#6494) 2026-02-27 10:58:11 +11:00
date.go Date precision (#6359) 2025-12-08 09:11:40 +11:00
doc.go Add package docs and project vision/goals (#5169) 2024-08-28 09:01:39 +10:00
driver.go Sort special characters before numbers (#3829) 2023-06-22 09:51:21 +10:00
file.go Fix: Custom Field Filtering (#6614) 2026-02-28 11:05:13 +11:00
file_filter.go fix: support string-based fingerprints in hashes filter (#6654) 2026-03-10 15:07:46 +11:00
file_filter_test.go fix: support string-based fingerprints in hashes filter (#6654) 2026-03-10 15:07:46 +11:00
file_test.go fix: support string-based fingerprints in hashes filter (#6654) 2026-03-10 15:07:46 +11:00
filter.go SQL performance improvements (#6378) 2025-12-08 08:08:31 +11:00
filter_hierarchical.go Containing Group/Sub-Group relationships (#5105) 2024-08-30 11:43:44 +10:00
filter_internal_test.go Restructure data layer (#2532) 2022-09-06 07:03:40 +00:00
fingerprint.go Support setting file fingerprints (#4376) 2023-12-22 14:07:10 +11:00
folder.go Add folder criteria to scenes, images and galleries and sidebars (#6636) 2026-03-05 08:02:13 +11:00
folder_filter.go Add basename and parent_folders fields to Folder graphql interface (#6494) 2026-02-27 10:58:11 +11:00
folder_filter_test.go Add basename and parent_folders fields to Folder graphql interface (#6494) 2026-02-27 10:58:11 +11:00
folder_test.go Add basename and parent_folders fields to Folder graphql interface (#6494) 2026-02-27 10:58:11 +11:00
functions.go Correct title ordering for objects without titles (#3244) 2022-12-09 08:29:34 +11:00
gallery.go Backend support for gallery custom fields (#6592) 2026-02-23 07:39:28 +11:00
gallery_chapter.go Use slices package from the stdlib when possible (#5360) 2024-10-29 11:26:23 +11:00
gallery_chapter_test.go SQLite model refactoring (#3791) 2023-06-15 12:46:09 +10:00
gallery_filter.go Add folder criteria to scenes, images and galleries and sidebars (#6636) 2026-03-05 08:02:13 +11:00
gallery_test.go Backend support for Group custom fields (#6596) 2026-02-23 11:53:12 +11:00
group.go Backend support for Group custom fields (#6596) 2026-02-23 11:53:12 +11:00
group_filter.go FR: Add Missing is-missing Filter Options Across all Object Types (#6565) 2026-02-26 16:36:54 +11:00
group_relationships.go Containing Group/Sub-Group relationships (#5105) 2024-08-30 11:43:44 +10:00
group_test.go Backend support for Group custom fields (#6596) 2026-02-23 11:53:12 +11:00
history.go Scene play and o-counter history view and editing (#4532) 2024-02-22 11:28:18 +11:00
image.go perf: eliminate O(N^2) bottlenecks in image and scene duplicate detection 2026-03-20 15:29:04 -07:00
image_filter.go fix: update image duplicate checker UI and API handling 2026-03-16 04:15:28 -07:00
image_test.go fix: resolve image duplicate finder issues 2026-03-13 18:39:34 -07:00
migrate.go Fix various migration issues (#5723) 2025-03-19 08:04:21 +11:00
performer.go FR: Change Career Length to Career Start and Career End (#6449) 2026-02-17 13:44:03 +11:00
performer_filter.go FR: Add Missing is-missing Filter Options Across all Object Types (#6565) 2026-02-26 16:36:54 +11:00
performer_test.go FR: Change Career Length to Career Start and Career End (#6449) 2026-02-17 13:44:03 +11:00
phash.go Phash distance filter (#3596) 2023-04-17 15:36:51 +10:00
query.go Fix: Custom Field Filtering (#6614) 2026-02-28 11:05:13 +11:00
record.go Date precision (#6359) 2025-12-08 09:11:40 +11:00
regex.go Update a number of dependencies (incl. CVE fixes) (#4107) 2023-10-26 16:24:32 +11:00
relationships.go Movie/Group tags (#4969) 2024-06-18 11:24:15 +10:00
repository.go SQL performance improvements (#6378) 2025-12-08 08:08:31 +11:00
saved_filter.go Use slices package from the stdlib when possible (#5360) 2024-10-29 11:26:23 +11:00
saved_filter_test.go Default view filters (#4962) 2024-06-18 10:51:52 +10:00
scene.go perf: eliminate O(N^2) bottlenecks in image and scene duplicate detection 2026-03-20 15:29:04 -07:00
scene_filter.go FR: Add Missing is-missing Filter Options Across all Object Types (#6565) 2026-02-26 16:36:54 +11:00
scene_marker.go SQL performance improvements (#6378) 2025-12-08 08:08:31 +11:00
scene_marker_filter.go Scene Marker duration filter and sort (#5472) 2024-11-29 16:28:10 +11:00
scene_marker_test.go Scene Marker duration filter and sort (#5472) 2024-11-29 16:28:10 +11:00
scene_test.go Scene custom field backend support (#6584) 2026-02-18 16:50:32 +11:00
setup_test.go feat: Implement Image Duplicate Checker 2026-03-13 15:23:02 -07:00
sql.go FR: Add Missing is-missing Filter Options Across all Object Types (#6565) 2026-02-26 16:36:54 +11:00
stash_id_test.go Feature: Tag StashID support (#6255) 2025-11-13 14:24:09 +11:00
studio.go Feature Request: Add organized flag to studios (#6303) 2026-02-19 09:05:17 +11:00
studio_filter.go FR: Add Missing is-missing Filter Options Across all Object Types (#6565) 2026-02-26 16:36:54 +11:00
studio_test.go Feature Request: Add organized flag to studios (#6303) 2026-02-19 09:05:17 +11:00
table.go Initialise UpdatedAt for stash ids (#5680) 2025-02-26 08:03:35 +11:00
tables.go Backend support for image custom fields (#6598) 2026-02-24 07:41:40 +11:00
tag.go FR: Tags Tagger (#6559) 2026-02-25 11:39:14 +11:00
tag_filter.go FR: Add Missing is-missing Filter Options Across all Object Types (#6565) 2026-02-26 16:36:54 +11:00
tag_test.go Fix: Custom Field Filtering (#6614) 2026-02-28 11:05:13 +11:00
timestamp.go Scene play and o-counter history view and editing (#4532) 2024-02-22 11:28:18 +11:00
transaction.go Database connection pool refactor (#5274) 2024-09-20 12:56:26 +10:00
transaction_test.go Upgrade gqlgenc (#5901) 2025-06-03 08:55:51 +10:00
tx.go SQL performance improvements (#6378) 2025-12-08 08:08:31 +11:00
values.go Model refactor (#3915) 2023-09-01 10:39:29 +10:00