Commit graph

102 commits

Author SHA1 Message Date
SmallCoccinelle
655d3ae969
Toward better context handling (#1835)
* Use the request context

The code uses context.Background() in a flow where there is a
http.Request. Use the requests context instead.

* Use a true context in the plugin example

Let AddTag/RemoveTag take a context and use that context throughout
the example.

* Avoid the use of context.Background

Prefer context.TODO over context.Background deep in the call chain.

This marks the site as something which we need to context-handle
later, and also makes it clear to the reader that the context is
sort-of temporary in the code base.

While here, be consistent in handling the `act` variable in each
branch of the if .. { .. } .. check.

* Prefer context.TODO over context.Background

For the different scraping operations here, there is a context
higher up the call chain, which we ought to use. Mark the call-sites
as TODO for now, so we can come back later on a sweep of which parts
can be context-lifted.

* Thread context upwards

Initialization requires context for transactions. Thread the context
upward the call chain.

At the intialization call, add a context.TODO since we can't break this
yet. The singleton assumption prevents us from pulling it up into main for
now.

* make tasks context-aware

Change the task interface to understand contexts.

Pass the context down in some of the branches where it is needed.

* Make QueryStashBoxScene context-aware

This call naturally sits inside the request-context. Use it.

* Introduce a context in the JS plugin code

This allows us to use a context for HTTP calls inside the system.

Mark the context with a TODO at top level for now.

* Nitpick error formatting

Use %v rather than %s for error interfaces.
Do not begin an error strong with a capital letter.

* Avoid the use of http.Get in FFMPEG download chain

Since http.Get has no context, it isn't possible to break out or have
policy induced. The call will block until the GET completes. Rewrite
to use a http Request and provide a context.

Thread the context through the call chain for now. provide
context.TODO() at the top level of the initialization chain.

* Make getRemoteCDPWSAddress aware of contexts

Eliminate a call to http.Get and replace it with a context-aware
variant.

Push the context upwards in the call chain, but plug it before the
scraper interface so we don't have to rewrite said interface yet.

Plugged with context.TODO()

* Scraper: make the getImage function context-aware

Use a context, and pass it upwards. Plug it with context.TODO()
up the chain before the rewrite gets too much out of hand for now.

Minor tweaks along the way, remove a call to context.Background()
deep in the call chain.

* Make NOTIFY request context-aware

The call sits inside a Request-handler. So it's natural to use the
requests context as the context for the outgoing HTTP request.

* Use a context in the url scraper code

We are sitting in code which has a context, so utilize it for the
request as well.

* Use a context when checking versions

When we check the version of stash on Github, use a context. Thread
the context up to the initialization routine of the HTTP/GraphQL
server and plug it with a context.TODO() for now.

This paves the way for providing a context to the HTTP server code in a
future patch.

* Make utils func ReadImage context-aware

In almost all of the cases, there is a context in the call chain which
is a natural use. This is true for all the GraphQL mutations.

The exception is in task_stash_box_tag, so plug that task with
context.TODO() for now.

* Make stash-box get context-aware

Thread a context through the call chain until we hit the Client API.
Plug it with context.TODO() there for now.

* Enable the noctx linter

The code is now free of any uncontexted HTTP request. This means we
pass the noctx linter, and we can enable it in the code base.
2021-10-14 15:32:41 +11:00
Eng Zer Jun
62af723017
refactor: move from io/ioutil to io and os package (#1772)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-09-27 10:55:23 +10:00
SmallCoccinelle
a9e2a590b2
Lint checks phase 2 (#1747)
* Log 3 unchecked errors

Rather than ignore errors, log them at
the WARNING log level.

The server has been functioning without these, so assume they are not at
the ERROR level.

* Log errors in concurrency test

If we can't initialize the configuration, treat the test as a failure.

* Undo the errcheck on configurations for now.

* Handle unchecked errors in pkg/manager

* Resolve unchecked errors

* Handle DLNA/DMS unchecked errors

* Handle error checking in concurrency test

Generalize config initialization, so we can initialize a configuration
without writing it to disk.

Use this in the test case, since otherwise the test fails to write.

* Handle the remaining unchecked errors

* Heed gosimple in update test

* Use one-line if-initializer statements

While here, fix a wrong variable capture error.

* testing.T doesn't support %w

use %v instead which is supported.

* Remove unused query builder functions

The Int/String criterion handler functions are now generalized.

Thus, there's no need to keep these functions around anymore.

* Mark filterBuilder.addRecursiveWith nolint

The function is useful in the future and no other refactors are looking
nice.

Keep the function around, but tell the linter to ignore it.

* Remove utils.Btoi

There are no users of this utility function

* Return error on scan failure

If we fail to scan the row when looking for the
unique checksum index, then report the error upwards.

* Fix comments on exported functions

* Fix typos

* Fix startup error
2021-09-23 17:15:50 +10:00
SmallCoccinelle
87709fd018
Errcheck phase 1 (#1715)
* Avoid redundant logging in migrations

Return the error and let the caller handle the logging of the error if
needed.

While here, defer m.Close() to the function boundary.

* Treat errors as values

Use %v rather than %s and pass the errors directly.

* Generate a wrapped error on stat-failure

* Log 3 unchecked errors

Rather than ignore errors, log them at
the WARNING log level.

The server has been functioning without these, so assume they are not at
the ERROR level.

* Propagate errors upward

Failure in path generation was ignored. Propagate the errors upward the
call stack, so it can be handled at the level of orchestration.

* Warn on errors

Log errors rather than quenching them.

Errors are logged at the Warn-level for now.

* Check error when creating test databases

Use the builtin log package and stop the program fatally on error.

* Add warnings to uncheck task errors

Focus on the task system in a single commit, logging unchecked
errors as warnings.

* Warn-on-error in API routes

Look through the API routes, and make sure errors are being logged if
they occur. Prefer the Warn-log-level because none of these has proven
to be fatal in the system up until now.

* Propagate error when adding Util API

* Propagate error on adding util API

* Return unhandled error

* JS log API: propagate and log errors

* JS Plugins: log GQL addition failures.

* Warn on failure to write to stdin

* Warn on failure to stop task

* Wrap viper.BindEnv

The current viper code only errors if no name is provided, so it should
never fail. Rewrite the code flow to factor through a panic-function.

This removes error warnings from this part of the code.

* Log errors in concurrency test

If we can't initialize the configuration, treat the test as a failure.

* Warn on errors in configuration code

* Plug an unchecked error in gallery zip walking

* Warn on screenshot serving failure

* Warn on encoder screenshot failure

* Warn on errors in path-handling code

* Undo the errcheck on configurations for now.

* Use one-line initializers where applicable

rather than using

  err := f()
  if err!= nil { ..

prefer the shorter

  if err := f(); err != nil { ..

If f() isn't too long of a name, or wraps a function with a body.
2021-09-21 09:34:25 +10:00
SmallCoccinelle
4b00d24248
Remove unused (#1709)
* Remove stuff which isn't being used

Some fields, functions and structs aren't in use by the project. Remove
them for janitorial reasons.

* Remove more unused code

All of these functions are currently not in use. Clean up the code by
removal, since the version control has the code if need be.

* Remove unused functions

There's a large set of unused functions and variables in the code base.
Remove these, so it clearer what code to support going forward.

Dead code has been eliminated.

Where applicable, comment const-sections in tests, so reserved
identifiers are still known.

* Fix use-def of tsURL

The first def of tsURL doesn't matter because there's no use before
we hit the 2nd def.

* Remove dead code assignment

Setting logFile = "" is effectively dead code, because there's no use
of it later.

* Comment out found

The variable 'found' is dead in the function (because no post-process
action is following it). Comment it for now.

* Comment dead code in tests

These might provide hints as to what isn't covered at the moment.

* Dead code removal

In the case of constants where iota is involved, move the iota so it
matches the current key values.

This avoids problems with persistently stored key IDs.
2021-09-09 14:10:08 +10:00
SmallCoccinelle
e7f6cb22b7
Error strings noncapitalized (#1704)
* Fix error string capitalization

Error strings often follow another string. Hence, they should not be
capitalized, unless referencing a name.

* Uncapitalize more error strings

While here, use %v on the error directly, which makes it easier to wrap
the error later with %w if need be.

* Uncapitalize more error strings

While here, rename Url to URL as a nitpick.
2021-09-08 11:23:10 +10:00
WithoutPants
709d7ce1cc
Load TLS config files from config path before stash home (#1678)
* Load tls files from config or home directory
* Update README
* Require both ssl files if either present
2021-08-31 19:37:45 +10:00
bnkai
f1786ad871
Make audio stream optional for preview generation (#1454) 2021-06-11 15:01:32 +10:00
EnameEtavir
d6ada23616
Fix: unhandled errors (#1419)
As these errors where not explicitly ignored with _, I made changes to make sure they bubble up.
2021-05-25 18:40:51 +10:00
WithoutPants
22a2fc3177
Make ffmpeg download location more portable (#1384)
* Download ffmpeg to config path
* Add setup message for ffmpeg download
* Handle missing ffmpeg in tasks
2021-05-18 09:14:25 +10:00
InfiniteTF
31981d4116
Add in-memory screenshot generation for sprites and phash (#1316) 2021-05-05 13:22:05 +10:00
bnkai
1882b44951
Strip file extension from scene title when scanning (#1022) 2021-01-07 11:38:30 +11:00
JoeSmithStarkers
71c814c116
Added streaming quality options (#790) 2020-10-22 15:02:27 +11:00
WithoutPants
98dda782aa
Prevent invalid date tag in video file from aborting scan (#836)
* Give more context when ffprobe fails
* Suppress JSONTime unmarshal error
* Tidy scan logging
2020-10-11 12:02:41 +11:00
bnkai
94dc74f4a8
Fix ffmpeg/ffprobe downloads (#824) 2020-10-03 17:28:02 +10:00
JoeSmithStarkers
30e88b96ee
Added ffmpeg/ffprobe permission correction code for linux/osx (#814) 2020-10-03 16:59:23 +10:00
JoeSmithStarkers
85aa1d8790
Replace os.Rename with util.SafeMove to allow cross device moving to not fail. (#745)
Fixed annoyingly noisy transcoding progress log messages.
Fixed minor minor issue with directories than are named "blah.mp4" being detected as files to be scanned.
2020-08-21 17:57:07 +10:00
JoeSmithStarkers
ecc42e4e24
Preview generation fallback (#725)
* Added preview generation fallback feature.
When a preview generation fails (often for wmv/avi files), the new code tries with less stricted (no xerror) and more time consuming options (slow+fast seek).
Fix a minor issue when stash downloads ffmpeg/ffprobe, but doesn't re-detect their paths.
2020-08-17 09:21:58 +10:00
WithoutPants
a2341f0819
Allow customisation of preview generation (#673)
* Add generate-specific options
* Include no-cache in preview response
2020-07-23 12:51:35 +10:00
WithoutPants
37be146a9d
Transcode stream refactor (#609)
* Remove forceMkv and forceHEVC
* Add HLS support and refactor
* Add new streaming endpoints
2020-07-23 11:56:08 +10:00
InfiniteTF
4ec6d62e01
Selectable wall preview type (#510)
* Add optional image preview generation
* Add setting for video preview encoding preset
2020-05-27 09:33:49 +10:00
WithoutPants
41c6d9e681
Revert preview generation change 2020-05-13 17:23:23 +10:00
bnkai
4829b4b214
Fix edge case preview generation, tweak preview generation preset (#528) 2020-05-11 17:24:52 +10:00
WithoutPants
3d22d5a742
Refactor build (#493)
* Add lint/format checks to build
* Make travis get full repo to get tags
* Run packr2 once in cross-compile
* Fix quotes in package.json
* Fix linting issues
* Formatting
* Fix vet issue
* Fix go lint issues
* Show start of each platform compilation
* Add validate target
* Set gitattributes for go fmt and mod vendor
* Fix tag name
* Add fmt-ui target
2020-04-29 12:13:08 +10:00
bnkai
d5617307f1
Add detection of container/video_codec/audio_codec compatibility for live file streaming or transcoding (#384)
* add forceMKV, forceHEVC config options
* drop audio stream instead of trying to transcode for ffmpeg unsupported/unknown audio codecs
2020-04-10 08:38:34 +10:00
WithoutPants
3de6955a9e
Generate cover image (#376)
* Make mutating metadata ops mutation
* Implement scene generate screenshot
* Remove fetch policy on metadata mutations
* Port UI changes to v2.5
* Set generated image in database
2020-03-12 08:34:04 +11:00
bnkai
3f511e48e8 Fix previews generation bug (#231) 2019-11-30 08:58:15 -08:00
WithoutPants
49c9b1aa9a Fix race condition in running encoders (#199) 2019-11-14 10:27:13 -08:00
WithoutPants
d1ea2fffa5 Add configurable transcode sizes (#178) 2019-11-04 16:38:33 -05:00
WithoutPants
be12a9f5a1 Fix ffmpeg error output (#176) 2019-11-04 16:34:57 -05:00
WithoutPants
1ac6703d7f Merge from master 2019-10-18 08:15:45 +11:00
WithoutPants
a401a7880e Close streams/encodes before deleting file 2019-10-18 07:42:12 +11:00
Leopere
9ed941dd35
Merge pull request #84 from WithoutPants/live_transcode_seeking
Add seeking for live transcoding via video.js
2019-10-14 11:39:12 -04:00
Leopere
b408782134
Merge pull request #131 from ExceptionalError/ffmpeg
Ffmpeg changes
2019-10-13 21:27:38 -04:00
ExceptionalError
4eb843d83e
revert changes #117 2019-10-12 16:04:00 +02:00
WithoutPants
afcadd941b Make title from file metadata optional 2019-10-12 19:20:27 +11:00
Leopere
e317fd934b
partial reversion of PR #117 2019-10-11 16:51:11 -04:00
ExceptionalError
10af75a670
Added output of error message 2019-10-09 06:16:17 +02:00
ExceptionalError
d082580ee0
modified args for screenshot 2019-10-09 06:15:00 +02:00
ExceptionalError
1d4feab478
Added quotes to path
fixes #48
2019-08-31 07:11:01 +02:00
WithoutPants
79f2902e26 Parse title, details, date from file metadata 2019-08-16 08:47:35 +10:00
WithoutPants
aeef01a64c Add row-based multithreading for live transcodes 2019-08-01 11:36:29 +10:00
WithoutPants
8ed3c5f71d Add seeking for live transcodes via video.js 2019-07-29 13:58:04 +10:00
WithoutPants
22577918fb Add basic live transcoding to webm 2019-07-25 19:05:53 +10:00
rudi123github
b8f1cf9d53
Update encoder_scene_preview_chunk.go
Apply ffmeg max_uxing_queue_size option fix to avoid problems with some videos
2019-06-21 23:03:00 +02:00
Stash Dev
29a40c5bdd Configuration
* Added flags to customize the host and port
* Start up just one server rather than a server for HTTP and HTTPS.  HTTPS server will only start if a cert and key are found
2019-04-11 10:55:58 -07:00
Stash Dev
182afad695 Added -pix_fmt yuv420p to H264 encodes 2019-03-29 08:23:21 -07:00
Stash Dev
71f8fc9dd5 Fixes FFMPEG failure edge case 2019-03-28 16:29:45 -07:00
Stash Dev
abbc2fa230 Another tweak to fix sprite generation 2019-03-28 15:58:13 -07:00
Stash Dev
2e57c2a17a Fixes
Various fixes from Discord discussions.
2019-03-27 12:26:50 -07:00
Stash Dev
c4d45db30c Fix json time when unmarshalling
https://github.com/stashapp/stash/issues/25
2019-03-09 10:14:55 -08:00
Stash Dev
b488c1ed7d Reorg 2019-02-14 15:42:52 -08:00