Commit graph

33 commits

Author SHA1 Message Date
WithoutPants
4017c42fe2
Handle modified files where the case of the filename changed on case-insensitive filesystems (#6327)
* Find existing files with case insensitivity if filesystem is case insensitive
* Handle case change in folders
* Optimise to only test file system case sensitivity if the first query found nothing

This limits the overhead to new paths, and adds an extra query for new paths to windows installs
2025-12-02 12:53:37 +11:00
gregpetersonanon
914bbfc164
Prevent scanner from failing when reading file info (#6123) 2025-10-20 10:54:26 +11:00
WithoutPants
602f95dd29
Fix video files with identical phashes being merged during scan (#5461)
* Change Fingerprints.Remove to return new instead of mutate current
* Match only by oshash and md5 when merging scenes during scan
2024-11-07 14:29:26 +11:00
WithoutPants
a17199ba21
Handle symlink zip files (#5249) 2024-09-11 13:58:02 +10:00
CJ
0fa71be697
Add scan option to force gallery zip rescan (#4113)
* Add scan option to force rescan
---------
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2024-05-20 13:52:36 +10:00
WithoutPants
141f60f8fb
Fix interactive speed being lost when file is moved (#4799) 2024-05-03 13:10:05 +10:00
WithoutPants
76e5598876
Improve handling of moved and added video files (#4598)
* If old file path is not in library, treat as move
* Use existing phash if file with same oshash exists
2024-02-20 14:04:31 +11:00
WithoutPants
403f7c54ef
Move zip files while moving folders (#4374) 2024-01-16 14:48:28 +11:00
DingDongSoLong4
33f2ebf2a3
Model refactor, part 3 (#4152)
* Remove manager.Repository
* Refactor other repositories
* Fix tests and add database mock
* Add AssertExpectations method
* Refactor routes
* Move default movie image to internal/static and add convenience methods
* Refactor default performer image boxes
2023-10-16 14:26:34 +11:00
WithoutPants
0dbe3e6ea8
Fix data corruption when moving folders (#4169)
* Add data correction migration
* Correct folder hierarchy after folder move
2023-10-03 17:05:08 +11:00
DingDongSoLong4
c364346a59
Model refactor (#3915)
* Add mockery config file
* Move basic file/folder structs to models
* Fix hack due to import loop
* Move file interfaces to models
* Move folder interfaces to models
* Move scene interfaces to models
* Move scene marker interfaces to models
* Move image interfaces to models
* Move gallery interfaces to models
* Move gallery chapter interfaces to models
* Move studio interfaces to models
* Move movie interfaces to models
* Move performer interfaces to models
* Move tag interfaces to models
* Move autotag interfaces to models
* Regenerate mocks
2023-09-01 10:39:29 +10:00
WithoutPants
93b41fb650
Add folder rename detection (#3817) 2023-07-11 11:53:49 +10:00
DingDongSoLong4
90683bd263
Fix zip gallery moving (#3610)
* Fix folder ZipFileID synchronization
* Fix zip gallery moving
2023-04-19 13:06:53 +10:00
WithoutPants
7cff71c35f
Add filesystem based blob storage (#3187)
* Refactor transaction hooks. Add preCommit
* Add BlobStore
* Use blobStore for tag images
* Use blobStore for studio images
* Use blobStore for performer images
* Use blobStore for scene covers
* Don't generate screenshots in legacy directory
* Run post-hooks outside original transaction
* Use blobStore for movie images
* Remove unnecessary DestroyImage methods
* Add missing filter for scene cover
* Add covers to generate options
* Add generate cover option to UI
* Add screenshot migration
* Delete thumb files as part of screenshot migration
2023-03-17 10:52:49 +11:00
xWTF
6a5a2060bf
Fix gallery zip scan context (#3433)
* fix zip scan context
* move ValueOnlyContext to utils, use it for zipCtx
2023-02-16 10:07:52 +11:00
WithoutPants
f39fa416a9
Fix database locked errors (#3153)
* Make read-only operations use WithReadTxn
* Allow one database write thread
* Add unit test for concurrent transactions
* Perform some actions after commit to release txn
* Suppress some errors from cancelled context
2022-11-21 06:49:10 +11:00
WithoutPants
479ebfc88d
Reimplement case-insensitivity move bug fix (#3047)
* Use eq for FindByPath for case sensitivity
* Handle case sensitive moves
2022-10-25 11:37:54 +11:00
WithoutPants
1c92336798
Fix symlink size calculation (#3046) 2022-10-25 10:57:37 +11:00
DingDongSoLong4
4db0e48f73
Fix zip gallery renaming (#3036) 2022-10-24 09:38:02 +11:00
WithoutPants
6ce409cd56
Prevent errors from stopping scan 2022-10-11 14:20:24 +11:00
WithoutPants
b160c3bb97
Associate funscript files on scan (#2978) 2022-10-06 11:17:01 +11:00
WithoutPants
dce90a3ed9
Handle file rescan (#2951)
* Fire handlers when file updated or moved
* Create galleries as needed
* Clean empty galleries
* Handle cleaning zip folders when path changed
* Fix gallery association on duplicate images
* Re-create missing folder-based galleries
2022-09-28 16:08:00 +10:00
WithoutPants
0848b02e93
Various bug fixes (#2945)
* Only update fingerprints if changed
* Fix panic when loading primary file fails
* Fix gallery/scene association
* Fix display of scene gallery in card
* Use natural_cs collation with paths for title sorting
2022-09-25 12:07:55 +10:00
WithoutPants
b74428cb42
Various bug fixes (#2938)
* Don't recalculate MD5 if not enabled

Remove MD5 if oshash has changed and MD5 was not calculated.

* Fix panic in paged DLNA
* Prevent identical hashes in stash-box drafts
2022-09-21 15:39:41 +10:00
WithoutPants
cffcd9f4b8
Various bug fixes (#2935)
* Sort scene/image/gallery tags by name
* Calculate md5 if missing
* Prevent multiple folder create logs
2022-09-20 17:02:14 +10:00
WithoutPants
9e08edc76f
[Files Refactor] Don't require fingerprint calculation post-migrate (#2892) 2022-09-07 14:21:10 +10:00
WithoutPants
273cf0383d [Files Refactor] Performance tuning (#2865)
* Don't load image files by default
* Don't load gallery files by default
* Don't load scene files by default
* Retry locked transactions forever
* Don't show release notes if config not loaded
* Don't translate path slashes in export
2022-09-06 07:04:52 +00:00
WithoutPants
9b31b20fed [Files Refactor] Performance tuning (#2813)
* Do database txn in same thread. Retry on locked db
* Remove captions from slimscenedata
* Fix tracing
* Use where in instead of individual selects
* Remove scenes_query view
* Remove image query view
* Remove gallery query view
* Use where in for FindMany
* Don't interrupt scanning zip files
* Fix image filesize sort
2022-09-06 07:03:42 +00:00
WithoutPants
abb574205a Files refactor fixes (#2743)
* Fix destroy gallery not destroying file
* Re-add minModTime functionality
* Deprecate useFileMetadata and stripFileExtension
* Optimise files post migration
* Decorate moved files. Use first missing file in move
* Include path in thumbnail generation error log
* Fix stash-box draft submission
* Don't destroy files unless deleting
* Call handler for files with no associated objects
* Fix moved zips causing error on scan
2022-09-06 07:03:42 +00:00
WithoutPants
5495d72849 File storage rewrite (#2676)
* Restructure data layer part 2 (#2599)
* Refactor and separate image model
* Refactor image query builder
* Handle relationships in image query builder
* Remove relationship management methods
* Refactor gallery model/query builder
* Add scenes to gallery model
* Convert scene model
* Refactor scene models
* Remove unused methods
* Add unit tests for gallery
* Add image tests
* Add scene tests
* Convert unnecessary scene value pointers to values
* Convert unnecessary pointer values to values
* Refactor scene partial
* Add scene partial tests
* Refactor ImagePartial
* Add image partial tests
* Refactor gallery partial update
* Add partial gallery update tests
* Use zero/null package for null values
* Add files and scan system
* Add sqlite implementation for files/folders
* Add unit tests for files/folders
* Image refactors
* Update image data layer
* Refactor gallery model and creation
* Refactor scene model
* Refactor scenes
* Don't set title from filename
* Allow galleries to freely add/remove images
* Add multiple scene file support to graphql and UI
* Add multiple file support for images in graphql/UI
* Add multiple file for galleries in graphql/UI
* Remove use of some deprecated fields
* Remove scene path usage
* Remove gallery path usage
* Remove path from image
* Move funscript to video file
* Refactor caption detection
* Migrate existing data
* Add post commit/rollback hook system
* Lint. Comment out import/export tests
* Add WithDatabase read only wrapper
* Prepend tasks to list
* Add 32 pre-migration
* Add warnings in release and migration notes
2022-09-06 07:03:42 +00:00
bnkai
0c0bdd4e21
Fix oshash calculation for symlinks (#2198) 2022-01-04 11:30:42 +11:00
WithoutPants
602183cca9
File size scanning fixes (#1944) 2021-11-04 10:50:03 +11:00
WithoutPants
39fdde273d
Scan refactor (#1816)
* Add file scanner
* Scan scene changes
* Split scan files
* Generalise scan
* Refactor ffprobe
* Refactor ffmpeg encoder
* Move scene scan code to scene package
* Move matchExtension to utils
* Refactor gallery scanning
* Refactor image scanning
* Prevent race conditions on identical hashes
* Refactor image thumbnail generation
* Perform count concurrently
* Allow progress increment before total set
* Make progress updates more frequent
2021-10-15 10:39:48 +11:00