330 commits

Author	SHA1	Message	Date
asonix	0aab38b40a	Add timeout configuration for external validation	2023-09-18 19:50:17 -05:00
asonix	8f3f1dcc6f	Clippy	2023-09-15 15:31:01 -04:00
asonix	b81bbb9b2d	Add external validation check	2023-09-06 18:53:52 -05:00
asonix	d32f61b85e	Consider timeout errors client errors	2023-08-05 15:15:20 -05:00
asonix	04bf9aade7	clippy	2023-08-05 11:58:46 -05:00
asonix	35c280a08d	Add process timeout	2023-08-05 11:51:52 -05:00
asonix	a240480b94	Add content-length header when completing multipart requests	2023-07-21 19:33:24 -05:00
asonix	a56fa1f188	Add content-length	2023-07-21 19:17:26 -05:00
asonix	a30b1ae540	Aggregate bytes list into single bytes for object storage ... This is a workaround for reqwest overriding the content-length header	2023-07-21 18:54:29 -05:00
asonix	e1ba1c13e5	Replace awc with reqwest	2023-07-21 16:21:24 -05:00
asonix	b32e1cc6f6	Fix dangling unprocessed uploads ... Adds error boundary around backgrounded ingest	2023-07-17 22:30:10 -05:00
asonix	be73907a0a	Fix resuming migrations, update version number	2023-07-16 20:13:53 -05:00
asonix	ee0ce5bc38	Return Root Cause error from backgrounded endpoint	2023-07-16 12:21:46 -05:00
asonix	f9b8e817e6	Drain joinset before completing migration	2023-07-16 10:02:05 -05:00
asonix	37e2012509	Move store migration to separate file	2023-07-15 15:04:32 -05:00
asonix	6a05d3588c	Add permissions error on Process	2023-07-15 14:41:33 -05:00
asonix	25c717771c	Attempt to implement concurrent hash migrations	2023-07-15 14:37:17 -05:00
asonix	e23b33e245	Enable setting content type when uploading objects	2023-07-15 13:06:34 -05:00
asonix	4a407afd9f	Fix alias cleanup ordering	2023-07-13 19:59:39 -05:00
asonix	943c4d5b58	Guard against failed commands not erroring	2023-07-11 17:52:46 -05:00
asonix	691bca286c	Make object storage timeouts configurable	2023-07-11 13:01:58 -05:00
asonix	f5c39f9be5	Typo, error formatting	2023-07-10 17:15:43 -05:00
asonix	4be2eb67d1	Increase object storage signature expiry to 15 seconds	2023-07-10 16:28:51 -05:00
asonix	9004ecaadf	Better classify process related errors	2023-07-10 15:29:41 -05:00
asonix	e10cb1401d	Bump version	2023-07-09 15:16:04 -05:00
asonix	7dadbcf3ed	Replace UnsupportedFormat error with more helpful errors	2023-07-09 15:07:49 -05:00
asonix	1694f49436	Use json for ffmpeg, imagemagick details parsing	2023-07-09 14:50:58 -05:00
asonix	686be1d93b	Turn export_path into flag for sled	2023-07-08 19:56:26 -05:00
asonix	7768d4e58e	Add database exports ... This should help admins running pict-rs on traditional filesystems to keep valid backups of pict-rs' sled repo	2023-07-08 17:36:01 -05:00
asonix	4680f9c3aa	Instrument repo and store build methods	2023-07-08 10:42:03 -05:00
asonix	d51c348a3f	Run health checks on old and new stores before migrating	2023-07-07 14:05:13 -05:00
asonix	3ccb8ecd8c	Validate alias represents real file before setting not found	2023-07-07 13:33:27 -05:00
asonix	9e7376d411	Remove is_missing error, make Identifier and DeleteToken return Option on fetch	2023-07-07 13:17:26 -05:00
asonix	7274538b70	Try adding store healthcheck	2023-07-07 12:05:42 -05:00
asonix	c150903980	Expand store differentiation	2023-07-07 11:19:39 -05:00
asonix	75be876d3c	Differentiate missing files in old vs new store for migration	2023-07-07 11:03:10 -05:00
asonix	508dd4340b	Fix panic when migrating < 100 files, improve resumed migration messaging	2023-07-06 16:06:37 -05:00
asonix	3487cb0e30	Add ability to set 404 image ... Fix imagemagick blur arguments	2023-07-05 19:07:03 -05:00
asonix	e7e4876908	Explicitly handle cases of invalid aliases	2023-07-05 16:46:44 -05:00
asonix	23e67b9697	Enhance migration handling of missing identifiers ... Queue a cleanup task for hashes that don't have original file identifiers Be more specific about what field is missing when a field is missing	2023-07-05 09:52:19 -05:00
asonix	2274dfecb4	Fix io-uring - read_to_end was polling Stream mapped through StreamReader after completion	2023-07-05 00:35:07 -05:00
asonix	c8e84b8914	Greatly improve error information during migration	2023-07-02 21:07:15 -05:00
asonix	e6f17cbac6	Log on successful migration	2023-06-29 11:45:06 -05:00
asonix	01d3610f46	Add basic progress indication for store migration	2023-06-29 11:39:47 -05:00
asonix	0d1df5ebc2	Avoid failed identifier cleanup on failed duplicate upload	2023-06-23 11:43:15 -05:00
asonix	7ba2fc9b26	Fix duplicate uploads potentially purging existing uploads on failure	2023-06-23 11:39:43 -05:00
asonix	58f0c328d9	Fix typo in error message	2023-06-23 11:20:55 -05:00
asonix	a1d48cbb42	Share client between object store and request extractor ... Add config to limit connection pool size	2023-06-23 11:20:20 -05:00
asonix	071d788349	Include original error in object storage sendrequest error	2023-06-21 17:07:14 -05:00
asonix	a3ff072350	Remove unused cache duration parameter	2023-06-21 17:05:53 -05:00