This fixes a bug where identical image duplicates were not being detected.
The implementation was incorrectly scanning the phash BLOB into a string and then attempting to parse it as a hex string. Since phashes are stored as 64-bit integers, they were being converted to decimal strings. For phashes with the MSB set (negative when treated as int64), the resulting decimal string started with a '-', which caused the hex parser to fail and skip the image entirely.
Additionally, even for non-negative phashes, parsing a decimal string as hex yielded incorrect hash values.
Scanning directly into the utils.Phash struct (which uses int64) matches how Scene phashes are handled and ensures the hash values are correct.
- Wrap FindDuplicateImages query in r.withReadTxn() to ensure a database transaction in context.
- Use queryFunc instead of queryStruct for fetching multiple hashes, preventing runtime errors.
- Fix N+1 query issue in duplicate grouping by using qb.FindMany() instead of qb.Find() for each duplicate image.
- Revert searchColumns array to exclude "images.details" which was from another PR and remove related failing test.
- Removed unused `strconv` import from `pkg/sqlite/image.go`.
- Added missing `github.com/stashapp/stash/pkg/utils` import to resolve the undefined `utils` reference.
- Fixed pagination prop in ImageDuplicateChecker component.
- Formatted modified go files using gofmt.
- Ran prettier over the UI codebase to resolve the formatting check CI failure.
This change unifies the duplicate detection logic by leveraging the shared phash utility. It also enhances the UI with:
- Pagination for large result sets.
- Sorting duplicate groups by total file size.
- A more detailed table view with image thumbnails, paths, and dimensions.
- Consistency with the existing Scene Duplicate Checker tool.
This change introduces a new tool to identify duplicate images based on their perceptual hash (phash). It includes:
- Backend implementation for phash distance comparison and grouping.
- GraphQL schema updates and API resolvers.
- Frontend UI for the Image Duplicate Checker tool.
- Unit tests for the image search and duplicate detection logic.
* fix: support string-based fingerprints in hashes filter
* Fix tests and add phash test
File fingerprints weren't using correct types. Filter test wasn't using correct types. Add phash to general files.
---------
Co-authored-by: hyper440 <hyper440@users.noreply.github.com>
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
* Add useDebouncedState hook
* Add basename to folder sort whitelist
* Add parent_folder criterion to gallery
* Add selection on enter if single result
* Fix stale thumbnails after file content changes
When a file's content changed (e.g. after renaming files in a gallery),
the scan handler updated fingerprints but did not bump the entity's
updated_at timestamp. Since thumbnail URLs use updated_at as a cache
buster and are served with immutable/1-year cache headers, browsers
would indefinitely serve the old cached thumbnail.
Update image, scene, and gallery scan handlers to call UpdatePartial
(which sets updated_at to now) whenever file content changes, not only
when a new file association is created.
* Skip scanning zip contents when fingerprint is unchanged
When a zip-based gallery's modification time changes but its content
hash (oshash/md5) remains the same, skip walking and rescanning every
file inside the zip. This avoids expensive per-file fingerprint
recalculation when zip metadata changes without actual content changes.
Closes#6512
* Log a debug message when skipping a zip scan due to unchanged
fingerprint
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
* Replace panic with warning if creating a folder hierarchy where parent is equal to current
* Clean stash paths so that comparison works correctly when creating folder hierarchies
* Add basename field to folder
* Add parent_folders field to folder
* Add basename column to folder table
* Add basename filter field
* Create missing folder hierarchies during migration
* Treat files/folders in zips where path can't be made relative as not found
Addresses an issue during clean where corrupt folder entries in zip files could not be removed due to an error during the call to Rel.
* Add root paths parameter to GetOrCreateFolderHierarchy
Ensures that folders are only created up to the root library paths.
* Create full folder hierarchy when scanning a new folder
During a recursive scan, folders should be created as they are encountered (folders are handled in a single thread). This change applies only during a selective scan. Creates up to the root library folder.
* Create folder hierarchy on new file scan
This should only apply when scanning a specific file, as parent folders should be been created during a recursive scan.
* Fix existing folders with missing parents during scan
* Add reveal in file manager button to file info panel
Adds a folder icon button next to the path field in the Scene, Image,
and Gallery file info panels. Clicking it calls a new GraphQL mutation
that opens the file's enclosing directory in the system file manager
(Finder on macOS, Explorer on Windows, xdg-open on Linux).
Also fixes the existing revealInFileManager implementations which were
constructing exec.Command but never calling Run(), making them no-ops:
- darwin: add Run() to open -R
- windows: add Run() and fix flag from \select to /select,<path>
- linux: implement with xdg-open on the parent directory
- desktop.go: use os.Stat instead of FileExists so folders work too
* Disallow reveal operation if request not from loopback
---------
Co-authored-by: 1509x <1509x@users.noreply.github.com>
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
* Add group filter criteria to tag and studio
* Add sidebar to groups list
* Refactor ListOperations to accept buttons
* Move create new button back to navbar
Having the create new button with a plus icon conflicted with the add sub-group button in the sub-groups view.
* Simplify group-sub-groups view
* Fix custom field import/export for studio
* Update studio unit tests
* Add tag create and update unit tests
* Add custom fields to tag filter graphql
* Add unit tests for tag filtering
* Add filter unit tests for studio
* Remove reflection from mapped value processing
* AI generated unit tests
* Move mappedConfig to separate file
* Rename group to configScraper
* Separate mapped post-processing code into separate file
* Update test after group rename
* Check map entry when returning scraper
* Refactor config into definition
* Support single string for string slice translation
* Rename config.go to definition.go
* Rename configScraper to definedScraper
* Rename config_scraper.go to defined_scraper.go
