102 commits

Author	SHA1	Message	Date
asonix	bd3975f455	Simplify some repo functions, remove 0.3 migration	2023-07-25 20:08:18 -05:00
asonix	f221887599	Clippy	2023-07-23 16:04:17 -05:00
asonix	aac1bb7bc4	Finish media proxy implementation	2023-07-23 15:45:52 -05:00
asonix	4cca7d0f86	Prepare for proxying media	2023-07-23 12:10:36 -05:00
asonix	75bf425c6e	Update alias query, process query to accept proxy url	2023-07-23 11:45:10 -05:00
asonix	7dc8ede8eb	Gate cleanup behind read-only config	2023-07-22 20:26:28 -05:00
asonix	1f3298363f	Attempt to mark unmarked variants as accessed on launch	2023-07-22 19:55:50 -05:00
asonix	fe1f7c869f	Enable cleanup of variants ... Still TODO: doing a first pass to mark variants as accessed on launch	2023-07-22 19:41:50 -05:00
asonix	47e13ec04e	Mark variants as accessed on fetch	2023-07-22 18:50:04 -05:00
asonix	ce0df080f4	Add prometheus metrics	2023-07-22 16:47:59 -05:00
asonix	d661732fb4	Tidy imports	2023-07-22 12:47:32 -05:00
asonix	a390b21df1	Make running pict-rs fluent	2023-07-22 12:39:39 -05:00
asonix	3ca994ee31	Replace global config with passed-around config ... This will enable spinning up pict-rs multiple times in the same process with different configurations	2023-07-22 12:31:01 -05:00
asonix	ac48003f45	Extract ProcessMap out of static	2023-07-22 11:16:19 -05:00
asonix	2f0a3618d8	Replace awc with reqwest	2023-07-21 16:58:31 -05:00
asonix	127dd6cdad	Make max file count configurable	2023-07-18 20:32:17 -05:00
asonix	eeac900d7e	Add quality value to process method, move -coalesce earlier in pipelines	2023-07-18 16:18:01 -05:00
asonix	c4c920191f	Add read-only mode	2023-07-17 14:24:49 -05:00
asonix	dd1d509bb1	Enable configuring download client timeout	2023-07-17 13:44:31 -05:00
asonix	2961aae6e3	Port migration changes from 0.4.1	2023-07-16 22:07:42 -05:00
asonix	9871ad2ee0	Clippy	2023-07-14 23:16:57 -05:00
asonix	786f583d98	Add public_url to Store for optional redirects to public object store urls ... Add details generation to file migration to set the content-type for uploads	2023-07-14 14:53:37 -05:00
asonix	2c22f2ee3a	It compiles and runs, but doesn't work	2023-07-13 17:42:21 -05:00
asonix	33b83f97f2	It compiles ... Need to reinstate media limits	2023-07-13 13:48:59 -05:00
asonix	ad1837f9dd	VERY BROKEN: start replacing parts of pict-rs	2023-07-12 22:12:21 -05:00
asonix	58d9765594	WIP: Rework format detection and conversions ... - Add a bunch of types & methods that aren't used yet. The idea is creating the output from the known input parameters is pure, and works for any uploaded media - Introduce a new kind of media "animation" which is neither an image nor a video. It is likely that animations can be processed properly at the /image/process.{ext} endpoint with a little massaging TODO: - Integrate this into exiftool, ffmpeg, magick commands - Detect apng with ffmpeg (imagemagick treats it as a still png unless it's given a hint) - Infallible conversion from Details to OutputFile - this might be impossible given that we don't currently store more than a mime type and a frame count to give any sort of indication of real type - Try not to break API	2023-07-11 23:11:25 -05:00
asonix	691bca286c	Make object storage timeouts configurable	2023-07-11 13:01:58 -05:00
asonix	e10cb1401d	Bump version	2023-07-09 15:16:04 -05:00
asonix	7dadbcf3ed	Replace UnsupportedFormat error with more helpful errors	2023-07-09 15:07:49 -05:00
asonix	7768d4e58e	Add database exports ... This should help admins running pict-rs on traditional filesystems to keep valid backups of pict-rs' sled repo	2023-07-08 17:36:01 -05:00
asonix	d51c348a3f	Run health checks on old and new stores before migrating	2023-07-07 14:05:13 -05:00
asonix	3ccb8ecd8c	Validate alias represents real file before setting not found	2023-07-07 13:33:27 -05:00
asonix	9e7376d411	Remove is_missing error, make Identifier and DeleteToken return Option on fetch	2023-07-07 13:17:26 -05:00
asonix	7274538b70	Try adding store healthcheck	2023-07-07 12:05:42 -05:00
asonix	c150903980	Expand store differentiation	2023-07-07 11:19:39 -05:00
asonix	75be876d3c	Differentiate missing files in old vs new store for migration	2023-07-07 11:03:10 -05:00
asonix	508dd4340b	Fix panic when migrating < 100 files, improve resumed migration messaging	2023-07-06 16:06:37 -05:00
asonix	3487cb0e30	Add ability to set 404 image ... Fix imagemagick blur arguments	2023-07-05 19:07:03 -05:00
asonix	e7e4876908	Explicitly handle cases of invalid aliases	2023-07-05 16:46:44 -05:00
asonix	23e67b9697	Enhance migration handling of missing identifiers ... Queue a cleanup task for hashes that don't have original file identifiers Be more specific about what field is missing when a field is missing	2023-07-05 09:52:19 -05:00
asonix	c8e84b8914	Greatly improve error information during migration	2023-07-02 21:07:15 -05:00
asonix	e6f17cbac6	Log on successful migration	2023-06-29 11:45:06 -05:00
asonix	01d3610f46	Add basic progress indication for store migration	2023-06-29 11:39:47 -05:00
asonix	a1d48cbb42	Share client between object store and request extractor ... Add config to limit connection pool size	2023-06-23 11:20:20 -05:00
asonix	554d852e68	Create unique errors for Repo and Store, separate from UploadError ... Add .is_not_found() for Store errors, wire up Object and File storage to properly set NotFound Allow skipping files that are not found during store migration	2023-06-20 15:59:08 -05:00
asonix	281ac43dff	Remove cache functionality	2023-02-25 11:34:48 -06:00
asonix	f0f40db8c3	Add healthcheck for db	2023-01-29 11:36:09 -06:00
asonix	5e5dd27a05	Retry migrate_file, remove variant before relating new one	2023-01-04 18:58:05 -06:00
asonix	28f7a139a0	Retry in-process, and rate-limit self	2023-01-04 17:26:05 -06:00
asonix	c55b1115d1	Increase client timeout, attempt to keep track of migration progress for resuming	2023-01-04 15:58:32 -06:00
asonix	3c844d86c5	Allow configuring pict-rs through serializable types	2022-12-26 16:35:25 -06:00
asonix	2c1a37fb62	Extract pict-rs as library	2022-12-23 12:56:15 -06:00