62 commits

Author	SHA1	Message	Date
asonix	24812afeba	Add paged hash access	2023-08-28 16:02:11 -05:00
asonix	f95e61e402	Expose repo migrations	2023-08-16 16:32:19 -05:00
asonix	ba27a1a223	Build out repo-repo migration	2023-08-16 16:09:40 -05:00
asonix	1559d57f0a	Don't overwrite existing variants	2023-08-16 15:12:16 -05:00
asonix	4d909ba32a	Make migration concurrent	2023-08-16 11:47:36 -05:00
asonix	1ee77b19d8	Make Repo object safe. Create migration	2023-08-15 19:19:03 -05:00
asonix	09236d731d	Fix poll after completion, misused compare_and_swap	2023-08-14 21:17:57 -05:00
asonix	26ca3a7195	Remove associated Bytes type	2023-08-14 20:00:00 -05:00
asonix	da876fd553	Make it compile	2023-08-14 14:25:19 -05:00
asonix	3129f7844e	BROKEN: start work on hash discriminant	2023-08-13 22:06:42 -05:00
asonix	e4e93cddf9	Return JobId from push	2023-08-13 19:52:27 -05:00
asonix	5d4486fdf4	Finish queue implementation update for sled	2023-08-13 19:47:20 -05:00
asonix	35d0c065e9	Start work on 0.5 db format - initial queuerepo changes	2023-08-13 14:12:38 -05:00
asonix	bd3975f455	Simplify some repo functions, remove 0.3 migration	2023-07-25 20:08:18 -05:00
asonix	e59483c12c	Add ProxyRepo	2023-07-23 10:23:46 -05:00
asonix	fe1f7c869f	Enable cleanup of variants ... Still TODO: doing a first pass to mark variants as accessed on launch	2023-07-22 19:41:50 -05:00
asonix	47e13ec04e	Mark variants as accessed on fetch	2023-07-22 18:50:04 -05:00
asonix	b786406ad0	Change IdentifierAccess to VariantAccess	2023-07-22 17:57:52 -05:00
asonix	3d93300ba9	Add access repos for alias & identifier	2023-07-18 21:56:13 -05:00
asonix	2961aae6e3	Port migration changes from 0.4.1	2023-07-16 22:07:42 -05:00
asonix	33b83f97f2	It compiles ... Need to reinstate media limits	2023-07-13 13:48:59 -05:00
asonix	7768d4e58e	Add database exports ... This should help admins running pict-rs on traditional filesystems to keep valid backups of pict-rs' sled repo	2023-07-08 17:36:01 -05:00
asonix	4680f9c3aa	Instrument repo and store build methods	2023-07-08 10:42:03 -05:00
asonix	3ccb8ecd8c	Validate alias represents real file before setting not found	2023-07-07 13:33:27 -05:00
asonix	9e7376d411	Remove is_missing error, make Identifier and DeleteToken return Option on fetch	2023-07-07 13:17:26 -05:00
asonix	e7e4876908	Explicitly handle cases of invalid aliases	2023-07-05 16:46:44 -05:00
asonix	23e67b9697	Enhance migration handling of missing identifiers ... Queue a cleanup task for hashes that don't have original file identifiers Be more specific about what field is missing when a field is missing	2023-07-05 09:52:19 -05:00
asonix	01d3610f46	Add basic progress indication for store migration	2023-06-29 11:39:47 -05:00
asonix	554d852e68	Create unique errors for Repo and Store, separate from UploadError ... Add .is_not_found() for Store errors, wire up Object and File storage to properly set NotFound Allow skipping files that are not found during store migration	2023-06-20 15:59:08 -05:00
asonix	281ac43dff	Remove cache functionality	2023-02-25 11:34:48 -06:00
asonix	0aa3f574a5	Clippy	2023-01-29 11:57:59 -06:00
asonix	f6d6d54b88	Update deps	2023-01-29 11:47:28 -06:00
asonix	f0f40db8c3	Add healthcheck for db	2023-01-29 11:36:09 -06:00
asonix	718f09c43a	Clean tracing, simplify validation, rename InputFormat -> VideoFormat	2022-10-01 21:17:18 -05:00
asonix	656b0225af	Clippy	2022-09-27 23:23:06 -05:00
asonix	29cab025c6	Improve 0.3 migration code, repo traces	2022-09-27 23:19:52 -05:00
asonix	9db057fdc5	Add admin endpoint to fetch identifier from an alias	2022-09-25 20:59:57 -05:00
asonix	c5b107eb4c	clippy	2022-09-11 10:04:37 -05:00
asonix	7f49af58fe	Update actix-form-data, impl Store & Repo for Data<T>	2022-09-10 19:29:22 -05:00
Aode (lion)	547934071a	Don't migrate from the old database if it doesn't exist	2022-08-14 11:49:20 -05:00
Aode (Lion)	4ac20546ce	Improve migration from 0.3 formats	2022-04-24 17:10:15 -05:00
Aode (Lion)	b3cbda1337	Add internal endpoint for cleaning variants	2022-04-11 16:56:39 -05:00
Aode (Lion)	05e2cf5e08	Cache on Alias rather than Hash ... Make sled repo blocking tasks share parent span	2022-04-06 12:14:12 -05:00
Aode (Lion)	865f4ee09c	Caching works: ... - Insert new bucket for current access time before removing previous access time record. - Add instrument to more methods for more precise tracing	2022-04-05 21:47:35 -05:00
Aode (Lion)	6cdae7b318	Add 'cache' endpoint for ingesting ephemeral media ... By default, cached media should only stick around for 7 days, however The timeout is reset every time media is accessed, so only obscure cached media will be flushed from the cache. This '7 days' number is configurable through the commandline run options as --media-cache-duration and in the pict-rs.toml file as [media] cache_duration	2022-04-05 20:29:30 -05:00
Aode (lion)	b0670c2f66	Move migrate to old	2022-04-03 15:07:31 -05:00
Aode (lion)	409b7d4c54	Remove unused conversions	2022-04-03 12:51:21 -05:00
Aode (lion)	8734dfbdc7	Add backgrounding routes ... - Accept backgrounded uploads - Allow backgrounded processing Still TODO: - Endpoint for waiting on/claiming an upload	2022-04-02 20:56:29 -05:00
Aode (Lion)	c4d014597e	Better job recovery	2022-04-02 18:53:03 -05:00
Aode (Lion)	77a400c7ca	Implement UploadRepo	2022-04-02 17:41:00 -05:00