asonix/pict-rs
2
2
Fork 1
Code Issues 3 Pull requests Actions Projects Releases 139 Wiki Activity
552 commits 23 branches 310 tags 14 MiB
Commit graph

246 commits

Author SHA1 Message Date
Aode (Lion) 6cdae7b318 Add 'cache' endpoint for ingesting ephemeral media
All checks were successful
continuous-integration/drone/push Build is passing
Details 
By default, cached media should only stick around for 7 days, however
The timeout is reset every time media is accessed, so only obscure
cached media will be flushed from the cache. This '7 days' number is
configurable through the commandline run options as --media-cache-duration
and in the pict-rs.toml file as [media] cache_duration
 2022-04-05 20:29:30 -05:00
Aode (lion) b0670c2f66 Move migrate to old
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-04-03 15:07:31 -05:00
Aode (lion) 409b7d4c54 Remove unused conversions 2022-04-03 12:51:21 -05:00
Aode (lion) 29f0774331 Add ability to claim uploads
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-04-02 21:15:39 -05:00
Aode (lion) 8734dfbdc7 Add backgrounding routes
Some checks failed
continuous-integration/drone/push Build is failing
Details 
- Accept backgrounded uploads
- Allow backgrounded processing

Still TODO:
- Endpoint for waiting on/claiming an upload
 2022-04-02 20:56:29 -05:00
Aode (Lion) c4d014597e Better job recovery
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-04-02 18:53:03 -05:00
Aode (Lion) 77a400c7ca Implement UploadRepo
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-04-02 17:41:00 -05:00
Aode (Lion) 132e395e5c Multiple items: 
- Reduce duplicate work in generate job
- Use hash instead of identifier for unique processing
- Move motion ident generation behind concurrent processor lock
 2022-04-02 17:40:08 -05:00
Aode (Lion) 09f53b9ce6 Move away from UploadManager to direct repo & store actions
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-04-02 16:44:03 -05:00
Aode (lion) 6ed592c432 In Progress: process jobs
Some checks failed
continuous-integration/drone/push Build is failing
Details 
- Is this a good idea? it definitely will make interacting with pict-rs harder. Maybe it's best not to do this
 2022-04-01 16:51:12 -05:00
Aode (lion) c0d8e0e8e3 Support multiple queues for job processor
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-04-01 11:51:46 -05:00
Aode (lion) 09281d9ae8 Move stream timeout to all response bodies
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 16:48:26 -05:00
Aode (lion) 63d66050c8 Enable buffering iterator items for iterstream
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 16:18:00 -05:00
Aode (lion) 5adb3fde89 Move some Stream implementations into 'stream' module
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 15:59:17 -05:00
Aode (lion) 0e490ff54a Remove custom BytesFreezer stream
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 14:28:08 -05:00
Aode (lion) 25f4480809 Remove custom stream error mapping types
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 14:17:20 -05:00
Aode (lion) 8226a3571d Enable multi-threaded job processing
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 13:18:47 -05:00
Aode (lion) 602d1ea935 Begin implementing queue, put cleanups in it
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 12:51:16 -05:00
Aode (lion) e25a4781a8 Remove unneeded variable
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 11:09:23 -05:00
Aode (lion) 99d042f36e Allow disabling gif/mp4
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-29 11:04:56 -05:00
Aode (Lion) eb5e39c634 Use color-eyre 2022-03-28 20:47:46 -05:00
Aode (lion) 1291bf8beb Divorce reqwest from main application's spans
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-28 18:42:22 -05:00
Aode (lion) 1662f153ce Numerous changes:
All checks were successful
continuous-integration/drone/push Build is passing
Details 
- Improve error printing (display chain in addition to spantrace)
- Fix migration (read main identifier from identifier tree, not filename tree)
- Ensure uniqueness for processed images in ConcurrentProcessor (use source identifier in addition to thumbnail path, include extension in thumbnail path)
- Update default log levels (make pict-rs quieter)
- Add timeout for serving images from object storage (5 seconds)
 2022-03-28 15:34:36 -05:00
Aode (Lion) 3792a8923a Finish redoing config probably
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-27 23:27:07 -05:00
Aode (lion) ca28f68ef5 Start redoing config again
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-03-27 19:10:06 -05:00
Aode (lion) 37e6b21b55 Fix Range, consolidate errors, test object storage
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-03-26 20:45:12 -05:00
Aode (lion) 15b52ba6ec Get it compiling again
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-03-26 16:49:23 -05:00
Aode (lion) 323016f994 Build out migration path
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-03-25 18:47:50 -05:00
Aode (lion) 750ce4782e Rework configuration
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-03-24 22:06:29 -05:00
Aode (lion) d6567fbbbd Implement sled repo
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-03-24 17:09:15 -05:00
Aode (lion) a0c99d05eb Create initial trait representation for database
Some checks failed
continuous-integration/drone/push Build is failing
Details
2022-03-24 13:16:41 -05:00
Aode (Lion) 2ad536db17 Promote console from compile flag to runtime flag
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
2022-03-21 22:41:51 -05:00
Aode (Lion) f13f870a92 Only use console config when console feature is enabled
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-21 22:05:32 -05:00
Aode (Lion) 62615be371 Allow configuring console buffer capacity
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/tag Build is passing
Details
2022-03-21 21:56:19 -05:00
Aode (Lion) d30415bb49 Don't debug PathStore
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-21 21:43:38 -05:00
Aode (Lion) 9a8dc0e44f Fix sled cache capacity help text
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
2022-03-21 21:18:51 -05:00
Aode (Lion) da159c69cb Try setting a lower default cache capacity
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
2022-03-21 21:15:43 -05:00
Aode (lion) 21b6e1dc08 Use actix-web's Range header
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-03-01 11:23:15 -06:00
Aode (Lion) 73137ede21 Release candidate 1
Some checks failed
continuous-integration/drone/push Build is failing
Details
continuous-integration/drone/tag Build is failing
Details
2022-02-26 12:22:30 -06:00
Aode (lion) 0e948a2bf5 Add flag for console
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/tag Build is passing
Details
2022-02-01 10:10:52 -06:00
Aode (Lion) 445c99dbf1 Re-use existing BytesMut when streaming file
All checks were successful
continuous-integration/drone/push Build is passing
Details
2022-01-13 18:11:46 -06:00
Aode (lion) f79d38df8d Update rust-runner image, improve tokio-console support
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/tag Build is passing
Details
2021-12-19 12:59:51 -06:00
Aode (lion) c48808c1e1 Update deps
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-18 20:24:22 -06:00
Aode (lion) 44d171252b console
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-16 20:46:46 -06:00
Aode (lion) c445615c47 Update to latest betas
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-12 14:48:37 -06:00
Aode (lion) e209a282d9 No more follows-from for process
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/tag Build is passing
Details
2021-12-06 21:11:43 -06:00
Aode (lion) 11739cd42c Remove fancy span following code
All checks were successful
continuous-integration/drone/tag Build is passing
Details
continuous-integration/drone/push Build is passing
Details
2021-12-06 20:35:22 -06:00
Aode (lion) a47f2a3428 Log on new span, no pretty
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/tag Build is passing
Details
2021-12-06 18:48:07 -06:00
Aode (lion) 920f1f154c Improve follows-from and parent span relationships
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-12-06 12:43:03 -06:00
Aode (lion) 42aed5165c Update tracing-awc, add drone
All checks were successful
continuous-integration/drone Build is passing
Details
2021-12-03 14:21:31 -06:00
Aode (lion) 7a3d1bef5a Only log closing spans, not opening spans 2021-11-24 21:54:27 -06:00
Aode (lion) 5f5c1d3530 Merge branch 'main' of git.asonix.dog:asonix/pict-rs into main 2021-11-23 16:31:19 -06:00
Aode (lion) 5b44de2070 clippy 2021-11-23 16:31:15 -06:00
Aode (Lion) 79afd0e8ea Update deps 2021-11-23 12:29:42 -06:00
Aode (lion) 0d153ca239 Make stores take filename into account when saving files 2021-11-16 09:45:16 -06:00
Aode (lion) c5d6610276 Properly handle out-of-bounds range requests 2021-11-03 12:04:59 -05:00
Aode (lion) 4c8da2b414 Keep a reqwest client around for sharing an object storage connection pool 2021-11-02 17:21:00 -05:00
Aode (lion) ad7c323985 Update readme, cleanup migration key on completion 2021-11-01 10:44:49 -05:00
Aode (lion) 93c268fc58 Flush at every migrated file 2021-11-01 10:38:11 -05:00
Aode (lion) ccb9f49b8b Implement storage migration 2021-10-31 21:11:35 -05:00
Aode (lion) cf50da30ec Use rust-s3 fork with asyncwrite support 2021-10-31 12:35:11 -05:00
Aode (lion) 76e3d37009 Clean up ObjectStore's Debug impl 2021-10-28 21:07:31 -05:00
Aode (lion) 366db1e1fe Make object storage with minio work 2021-10-28 20:59:11 -05:00
Aode (lion) 657152e1a8 Don't println 2021-10-28 00:18:18 -05:00
Aode (lion) f596edc363 Improve configuration documentation 2021-10-28 00:17:37 -05:00
Aode (lion) f9f4fc63d6 Optionally support s3-compatible storage (untested) 2021-10-27 23:06:03 -05:00
Aode (lion) 26a2401027 Use tmp files for mp4s 2021-10-23 14:14:12 -05:00
Aode (lion) 6f04595c3b Remove unneeded mime conversions 2021-10-23 12:35:07 -05:00
Aode (lion) 36b83f114a Merge branch 'main' into asonix/generic-pictrs-storage 2021-10-22 23:49:47 -05:00
Aode (lion) 48557bc2ea Make pict-rs generic over file storage 2021-10-22 23:48:56 -05:00
Aode (Lion) 2295837ea4 2021 2021-10-21 16:36:31 -05:00
Aode (Lion) f67aeb92fa Keep better track of gif/mp4 thumbnail 2021-10-20 21:36:18 -05:00
Aode (Lion) 1767942e31 Remove recursion 2021-10-20 20:36:10 -05:00
Aode (Lion) d69d261999 Make it actually compilable 2021-10-20 20:30:21 -05:00
Aode (Lion) 4aefc0403a Remove more boxes 2021-10-20 20:13:39 -05:00
Aode (Lion) 877390878b Remove more boxes 2021-10-20 19:28:40 -05:00
Aode (Lion) e7c3e2c96c Rename stream module 2021-10-20 19:00:41 -05:00
Aode (Lion) e7b4e4d1cc Remove a few boxes 2021-10-20 18:58:32 -05:00
Aode (Lion) 922200673e Update to latest betas 2021-10-20 18:16:53 -05:00
Aode (Lion) 9559a8b202 Update cli flags 2021-10-18 23:51:04 -05:00
Aode (Lion) 3fe66129be Update opentelemetry url info 2021-10-18 23:44:56 -05:00
Aode (Lion) 34038a0f16 Fix restructure, upload 2021-10-18 23:37:11 -05:00
Aode (Lion) 195f614a4b Restructure attempt 1 2021-10-18 20:29:06 -05:00
Aode (Lion) 0f3be6566e Split upload_manager files 2021-10-18 18:44:05 -05:00
Aode (Lion) 2fb8d3e39a Reduce required Unpin bounds with pin-project-lite 2021-10-18 18:02:33 -05:00
Aode (lion) 4c5482b3a8 Add more instrumentation 2021-10-13 20:31:40 -05:00
Aode (lion) 786d8469ee It works 2021-10-13 19:50:07 -05:00
Aode (Lion) 09cb2a53b0 Rewrite to avoid direct AsyncX impls 2021-10-13 19:06:53 -05:00
Aode (lion) 2318fd9dca Enable io-uring 2021-10-12 23:16:31 -05:00
Aode (Lion) 23e6657208 Re-enable skipping validation on import, strip when converting to prescribed format 2021-09-26 11:02:19 -05:00
Aode (Lion) 15dc79d10e Instrument a couple more functions 2021-09-25 17:09:55 -05:00
Aode (lion) f390c62cbf Wrap BytesFreezer in span 2021-09-25 16:39:10 -05:00
Aode (lion) ba68bcbde3 Add better span information to commands, spawned tasks 2021-09-25 15:23:05 -05:00
Aode (lion) 7cb6a15acd Wire up cross-application propagation 2021-09-19 14:36:53 -05:00
Aode (lion) 79a0fa32ee Remove tracing-error workaround 2021-09-18 16:34:23 -05:00
Aode (lion) 5e9d923d03 Instrument for OpenTelemetry 2021-09-18 16:29:30 -05:00
Aode (lion) 06e402ec72 Don't enter across await points 2021-09-16 17:51:20 -05:00
Aode (Lion) acba660c37 Update to latest actix-form-data 2021-09-13 20:50:51 -05:00
Aode (Lion) 5d3e6f50b3 Instrument better with Tracing 2021-09-13 20:22:42 -05:00
Aode (lion) 79bd562d8e Clippy lints 2021-09-12 10:42:44 -05:00