* update tag hierarchy validation
* refactor MergeHierarchy
* update tag hierarchy error message
* rename tag hierarchy function
* add tag path to error message
* Rename EnsureHierarchy to ValidateHierarchy
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
* Don't capitalize local variables
ValidCodecs -> validCodecs
* Capitalize deprecation markers
A deprecated marker should be capitalized.
* Use re.MustCompile for static regexes
If the regex fails to compile, it's a programmer error, and should be
treated as such. The regex is entirely static.
* Simplify else-if constructions
Rewrite
else { if cond {}}
to
else if cond {}
* Use a switch statement to analyze formats
Break an if-else chain. While here, simplify code flow.
Also introduce a proper static error for unsupported image formats,
paving the way for being able to check against the error.
* Rewrite ifElse chains into switch statements
The "Effective Go" https://golang.org/doc/effective_go#switch document
mentions it is more idiomatic to write if-else chains as switches when
it is possible.
Find all the plain rewrite occurrences in the code base and rewrite.
In some cases, the if-else chains are replaced by a switch scrutinizer.
That is, the code sequence
if x == 1 {
..
} else if x == 2 {
..
} else if x == 3 {
...
}
can be rewritten into
switch x {
case 1:
..
case 2:
..
case 3:
..
}
which is clearer for the compiler: it can decide if the switch is
better served by a jump-table then a branch-chain.
* Rewrite switches, introduce static errors
Introduce two new static errors:
* `ErrNotImplmented`
* `ErrNotSupported`
And use these rather than forming new generative errors whenever the
code is called. Code can now test on the errors (since they are static
and the pointers to them wont change).
Also rewrite ifElse chains into switches in this part of the code base.
* Introduce a StashBoxError in configuration
Since all stashbox errors are the same, treat them as such in the code
base. While here, rewrite an ifElse chain.
In the future, it might be beneifical to refactor configuration errors
into one error which can handle missing fields, which context the error
occurs in and so on. But for now, try to get an overview of the error
categories by hoisting them into static errors.
* Get rid of an else-block in transaction handling
If we succesfully `recover()`, we then always `panic()`. This means the
rest of the code is not reachable, so we can avoid having an else-block
here.
It also solves an ifElse-chain style check in the code base.
* Use strings.ReplaceAll
Rewrite
strings.Replace(s, o, n, -1)
into
strings.ReplaceAll(s, o, n)
To make it consistent and clear that we are doing an all-replace in the
string rather than replacing parts of it. It's more of a nitpick since
there are no implementation differences: the stdlib implementation is
just to supply -1.
* Rewrite via gocritic's assignOp
Statements of the form
x = x + e
is rewritten into
x += e
where applicable.
* Formatting
* Review comments handled
Stash-box is a proper noun.
Rewrite a switch into an if-chain which returns on the first error
encountered.
* Use context.TODO() over context.Background()
Patch in the same vein as everything else: use the TODO() marker so we
can search for it later and link it into the context tree/tentacle once
it reaches down to this level in the code base.
* Tell the linter to ignore a section in manager_tasks.go
The section is less readable, so mark it with a nolint for now. Because
the rewrite enables a ifElseChain, also mark that as nolint for now.
* Use strings.ReplaceAll over strings.Replace
* Apply an ifElse rewrite
else { if .. { .. } } rewrite into else if { .. }
* Use switch-statements over ifElseChains
Rewrite chains of if-else into switch statements. Where applicable,
add an early nil-guard to simplify case analysis. Also, in
ScanTask's Start(..), invert the logic to outdent the whole block, and
help the reader: if it's not a scene, the function flow is now far more
local to the top of the function, and it's clear that the rest of the
function has to do with scene management.
* Enable gocritic on the code base.
Disable appendAssign for now since we aren't passing that check yet.
* Document the nolint additions
* Document StashBoxBatchPerformerTagInput
* Add movies and movie_count properties to Performer type
Extend the GraphQL API to allow getting the movies and movie count by
performer.
* Add movies count to performer card
* Add movies and movie_count properties to Studio type
Extend the GraphQL API to allow getting the movies and movie count by
studio.
* Add movies count to studio card
* Log 3 unchecked errors
Rather than ignore errors, log them at
the WARNING log level.
The server has been functioning without these, so assume they are not at
the ERROR level.
* Log errors in concurrency test
If we can't initialize the configuration, treat the test as a failure.
* Undo the errcheck on configurations for now.
* Handle unchecked errors in pkg/manager
* Resolve unchecked errors
* Handle DLNA/DMS unchecked errors
* Handle error checking in concurrency test
Generalize config initialization, so we can initialize a configuration
without writing it to disk.
Use this in the test case, since otherwise the test fails to write.
* Handle the remaining unchecked errors
* Heed gosimple in update test
* Use one-line if-initializer statements
While here, fix a wrong variable capture error.
* testing.T doesn't support %w
use %v instead which is supported.
* Remove unused query builder functions
The Int/String criterion handler functions are now generalized.
Thus, there's no need to keep these functions around anymore.
* Mark filterBuilder.addRecursiveWith nolint
The function is useful in the future and no other refactors are looking
nice.
Keep the function around, but tell the linter to ignore it.
* Remove utils.Btoi
There are no users of this utility function
* Return error on scan failure
If we fail to scan the row when looking for the
unique checksum index, then report the error upwards.
* Fix comments on exported functions
* Fix typos
* Fix startup error
* Add migration to create studio aliases table
* Refactor studioQueryBuilder.Query to use filterBuilder
* Expand GraphQL API with aliases support for studio
* Add aliases support for studios to the UI
* List aliases in details panel
* Allow editing aliases in edit panel
* Add 'aliases' filter when searching
* Find studios by alias in filter / select
* Add auto-tagging based on studio aliases
* Support studio aliases for filename parsing
* Support importing and exporting of studio aliases
* Search for studio alias as well during scraping
* Add migration script for tag relations table
* Expand hierarchical filter features
Expand the features of the hierarchical multi input filter with support
for using a relations table, which only has parent_id and child_id
columns, and support adding an additional intermediate table to join on,
for example for scenes and tags which are linked by the scenes_tags
table as well.
* Add hierarchical filtering for tags
* Add hierarchical tags support to scene markers
Refactor filtering of scene markers to filterBuilder and in the process
add support for hierarchical tags as well.
* List parent and child tags on tag details page
* Support setting parent and child tags
Add support for setting parent and child tags during tag creation and
tag updates.
* Validate no loops are created in tags hierarchy
* Update tag merging to support tag hierarcy
* Add unit tests for tags.EnsureUniqueHierarchy
* Fix applying recursive to with clause
The SQL `RECURSIVE` of a `WITH` clause only needs to be applied once,
imediately after the `WITH`. So this fixes the query building to do just
that, automatically applying the `RECURSIVE` keyword when any added with
clause is added as recursive.
* Rename hierarchical root id column
* Rewrite hierarchical filtering for performance
Completely rewrite the hierarchical filtering to optimize for
performance. Doing the recursive query in combination with a complex
query seems to break SQLite optimizing some things which means that the
recursive part might be 2,5 second slower than adding a static
`VALUES()` list. This is mostly noticable in case of the tag hierarchy
where setting an exclusion with any depth (or depth: all) being applied
has this performance impact of 2,5 second. "Include" also suffered this
issue, but some rewritten query by joining in the *_tags table in one
pass and applying a `WHERE x IS NOT NULL` filter did seem to optimize
that case. But that optimization isn't applied to the `IS NULL` filter
of "exclude". Running a simple query beforehand to get all (recursive)
items and then applying them to the query doesn't have this performance
penalty.
* Remove UI references to child studios and tags
* Add parents to tag export
* Support importing of parent relationship for tags
* Assign stable ids to parent / child badges
* Silence Apollo warning on parents/children fields on tags
Silence warning triggered by Apollo GraphQL by explicitly instructing it
to use the incoming parents/children values. By default it already does
this, but it triggers a warning as it might be unintended that it uses
the incoming values (instead of for example merging both arrays).
Setting merge to false still applies the same behaviour (use only
incoming values) but silences the warning as it's explicitly configured
to work like this.
* Rework detecting unique tag hierarchy
Completely rework the unique tag hierarchy to detect invalid hierarchies
for which a tag is "added in the middle". So when there are tags A <- B
and A <- C, you could previously edit tag B and add tag C as a sub tag
without it being noticed as parent A being applied twice (to tag C).
While afterwards saving tag C would fail as tag A was applied as parent
twice. The updated code correctly detects this scenario as well.
Furthermore the error messaging has been reworked a bit and the message
now mentions both the direct parent / sub tag as well as the tag which
would results in the error. So in aboves example it would now show the
message that tag C can't be applied because tag A already is a parent.
* Update relations on cached tags when needed
Update the relations on cached tags when a tag is created / updated /
deleted so these always reflect the correct state. Otherwise (re)opening
a tag might still show the old relations untill the page is fully
reloaded or the list is navigated. But this obviously is strange when
you for example have tag A, create or update tag B to have a relation to
tag A, and from tags B page click through to tag A and it doesn't show
that it is linked to tag B.
* Unify scraped types
* Make name fields optional
* Unify single scrape queries
* Change UI to use new interfaces
* Add multi scrape interfaces
* Use images instead of image
* Fix width in database test setup
* Added more filters on resolution field
* added test to verify resolution range is defined for every resolution
* Refactor UI code
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
* Add API to merge tags
Add new API endpoint, `tagsMerge(source, destination)` to merge multiple
tags into a single one. The "sources" must be provided as a list of ids
and the destination as a single id. All usages of the source tags
(scenes, markers (primary and additional), images, galleries and
performers) will be updated to the destination tag, all aliases of the
source tags will be updated to the destination, and the name of the
source will be added as alias to the destination as well.
* Add merge tag UI
* Add unit tests
* Update test mocks
* Update internationalisation
* Add changelog entry
Co-authored-by: gitgiggety <gitgiggety@outlook.com>
* Add Tag Update/UpdateFull
* Tag alias implementation
* Refactor tag page
* Add aliases in UI
* Include tag aliases in q filter
* Include aliases in tag select
* Add aliases to auto-tagger
* Use aliases in scraper
* Add tag aliases for filename parser
* Add funscript route to scenes
Adds a /scene/:id/funscript route which serves a funscript file, if present.
Current convention is that these are files stored with the same path, but with the extension ".funscript".
* Look for funscript during scan
This is stored in the Scene record and used to drive UI changes for funscript support.
Currently, that's limited to a funscript link in the Scene's file info.
* Add filtering and sorting for interactive
* Add Handy connection key to interface config
* Add Handy client and placeholder component.
Uses defucilis/thehandy, but not thehandy-react as I had difficulty integrating the context with the existing components.
Instead, the expensive calculation for the server time offset is put in localStorage for reuse.
A debounce was added when scrubbing the video, as otherwise it spammed the Handy API with updates to the current offset.
* Add tools settings page
* Add tools and move dupe checker
* Make negative number get all
* Show missing phashes
* Add multi-edit button
* Show scene details
* Add scraping support for performer tags
* Add performer count to tag cards
* Refactor sqlite test setup
* Add performer tag filtering in gallery and image
* Add bulk update performer
* Add Performers tab to tag page
* Add count filters and sort bys for tags
* Move scene count to icon in performer card #1148
* Add resolution enum extension
* Add filter builder
* Use filterBuilder for scene query
* Optimise joins
* Add binary operators to scene query
* Use Query for auto-tag
* Add regex string filter criterion
* Use query interface for auto tagging
* Use Query interface for filename parser
* Remove query regex interfaces
* Add selective auto tag
* Use page size 0 as no limit
* extend resolutions
- Simplifies logic
- Adds more options including 540p, 1440p, and resolutions common to VR such as 1920p
- Supports vertical/portrait videos and images
* implement new resolution filters
* Fix integer overflow for scene size on 32bit systems
* Cast to double in sqlite to prevent potential overflow
* Add migration to reset scene sizes and scan logic to repopulate if empty
* Add organized boolean to scene model (#729)
* Add organized button to scene page
* Add flag to galleries and images
* Import/export changes
* Make organized flag not null
* Ignore organized scenes for autotag
Co-authored-by: com1234 <com1234@notarealemail.com>
* Added natural sort for scene and images
* Use natural sort for movie names
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
* Fix potential image errors
* Fix issue preventing favoriting of tagged performers
* Add error handling in case of network issues
* Show individual search errors
* Unset scene results if query fails
* Don't abort scene submission if scene id isn't found
* Include studios in movie export
* Generalise cards
* Add selection and export for movies
* Refactor gallery card
* Refactor export dialogs
* Add performer selection and export
* Add selection and export for studios
* Add selection and export of tags
* Include movie scenes and gallery images
* Add gql client generation files
* Update dependencies
* Add stash-box client generation to the makefile
* Move scraped scene object matchers to models
* Add stash-box to scrape with dropdown
* Add scrape scene from fingerprint in UI
* Allow clearing of tag images
* Allow clearing of studio images
* Allow clearing of performer images
* Allow clearing of movie images
* Add filtering for missing images
* api/urlbuilders/movie: Auto format.
* graphql+pkg+ui: Implement scraping movies by URL.
This patch implements the missing required boilerplate for scraping
movies by URL, using performers and scenes as a reference.
Although this patch contains a big chunck of ground work for enabling
scraping movies by fragment, the feature would require additional
changes to be completely implemented and was not tested.
* graphql+pkg+ui: Scrape movie studio.
Extends and corrects the movie model for the ability to store and
dereference studio IDs with received studio string from the scraper.
This was done with Scenes as a reference. For simplicity the duplication
of having `ScrapedMovieStudio` and `ScrapedSceneStudio` was kept, which
should probably be refactored to be the same type in the model in the
future.
* ui/movies: Add movie scrape dialog.
Adds possibility to update existing movie entries with the URL scraper.
For this the MovieScrapeDialog.tsx was implemented with Performers and
Scenes as a reference. In addition DurationUtils needs to be called one
time for converting seconds from the model to the string that is
displayed in the component. This seemed the least intrusive to me as it
kept a ScrapeResult<string> type compatible with ScrapedInputGroupRow.
* Change scrape matching (studio, movies, tag, performers) to case insensitive
* * fix collate order
* * make filename parser findbyname calls case insensitive
* * add unit testing for Tags GetFindbyName/s
* Add lint/format checks to build
* Make travis get full repo to get tags
* Run packr2 once in cross-compile
* Fix quotes in package.json
* Fix linting issues
* Formatting
* Fix vet issue
* Fix go lint issues
* Show start of each platform compilation
* Add validate target
* Set gitattributes for go fmt and mod vendor
* Fix tag name
* Add fmt-ui target
* Add movie migration
* Update server and UI code for type changes
* Add studio to movies
* Movie blobs to end
* Document movie duration
* Add filtering on movie studio
* Add sub-scraper functionality
* Add scraping of performer image
* Add scene cover image scraping
* Port UI changes to v2.5
* Fix v2.5 dialog suggest color
* Don't convert eol of UI to support pretty
* Add stash scraper type
* Add graphql client to vendor
* Embed stash credentials in URL
* Fill URL from scraped scene
* Nil IDs returned from remote stash
* Nil check
* Add auto tagging for performers, studios and tags
* Fix scene title logging
* Fix golang 1.11 compile error
* Improve regex pattern. Only log when added
* Remove paging from parse query
* Add integration test
* Fix prefix detection on paths
* Make regex matching case-insensitive
* Port filename parser code to backend
* Add performers to scene filename parser UI
* Finish porting parser to backend
* Add performer, studio and tag parsing
* Hide fields not being parsed
* Don't query for empty performer/studio/tag
* Use exact matches
* Fix panic
* Fix arrays changed false positive. Fix layout
* Initial UI prototype
* Add backend support to update multiple scenes
* Fix title editing issues
* Add query regex support. UI improvements
* Rewrite parser. Add fields button and page size
* Add helper text for escaping {} characters
* Validate date
* Only set values if different from original
* Only update scenes that have something changed
* Add built in parser input recipes
* Make pattern matching case-insensistive
* Added flags to customize the host and port
* Start up just one server rather than a server for HTTP and HTTPS. HTTPS server will only start if a cert and key are found
* Allow configuration of generated file path and database path. Closes#33
* Added checkboxes to choose what gets generated for the generate task. Closes#32
