Commit graph

84 commits

Author SHA1 Message Date
asonix e7357cbdaf Count spawned tasks in cleanup, debug log queue length 2024-01-30 14:57:48 -06:00
asonix fe1132aec1 Enable at-most-once queueing for some cleanup jobs 2024-01-24 17:14:31 -06:00
asonix 19147e2035 postgres: allow connecting to TLS-enabled databases 2024-01-15 18:11:08 -05:00
asonix bfc2410552 Add trace-level logs to every loop (paranoid) 2023-12-28 11:58:38 -06:00
asonix 6c921817e1 Enable thumbnailing animations, use dynamic dispatch for a number of async readers 2023-09-24 11:54:16 -05:00
asonix b2674f06d0 More streme 2023-09-10 20:08:01 -04:00
asonix 858899b943 Add metrics & tracing to dropped uploads, turn pop into a single query 2023-09-07 19:20:41 -05:00
asonix a4b1ab7dfb Instrument postgres db calls 2023-09-04 21:51:27 -05:00
asonix 37448722de postgres: Rework job & upload notifications (more)
postgres: Add metrics to job push & pop, upload wait
sled: add upload wait metrics
2023-09-04 16:20:26 -05:00
asonix 31caea438e Fix slow connection pool access 2023-09-03 20:05:29 -05:00
asonix 8a892ba622 it might work 2023-09-03 18:21:46 -05:00
asonix a2933dbebc Implement all the todos 2023-09-03 17:11:34 -05:00
asonix 94cb2a9ef3 Pass clippy 2023-09-03 16:59:41 -05:00
asonix 443d327edf Implement a couple more repo traits 2023-09-03 12:47:06 -05:00
asonix 8921f57a21 Remove Identifier trait, Replace with Arc<str> 2023-09-02 18:38:02 -05:00
asonix 8eb4cda256 Implement first query 2023-09-02 13:52:15 -05:00
asonix 8c532c97e6 Initial postgres work 2023-09-02 11:52:55 -05:00
asonix ba3a23ed43 Add error codes 2023-09-01 20:51:01 -05:00
asonix fd74161c61 Rename some repo methods, generate UploadId in repo 2023-09-01 18:41:04 -05:00
asonix c0f6b1e988 Enable searching hashes by date 2023-08-29 13:27:18 -05:00
asonix 64950bfe0e Shorten pagination slugs 2023-08-29 12:53:29 -05:00
asonix 2b6ea97773 Enable setting timestamp on hash creation, improve paging api 2023-08-28 18:43:24 -05:00
asonix 24812afeba Add paged hash access 2023-08-28 16:02:11 -05:00
asonix f95e61e402 Expose repo migrations 2023-08-16 16:32:19 -05:00
asonix ba27a1a223 Build out repo-repo migration 2023-08-16 16:09:40 -05:00
asonix 1559d57f0a Don't overwrite existing variants 2023-08-16 15:12:16 -05:00
asonix 4d909ba32a Make migration concurrent 2023-08-16 11:47:36 -05:00
asonix 1ee77b19d8 Make Repo object safe. Create migration 2023-08-15 19:19:03 -05:00
asonix 09236d731d Fix poll after completion, misused compare_and_swap 2023-08-14 21:17:57 -05:00
asonix 26ca3a7195 Remove associated Bytes type 2023-08-14 20:00:00 -05:00
asonix da876fd553 Make it compile 2023-08-14 14:25:19 -05:00
asonix 3129f7844e BROKEN: start work on hash discriminant 2023-08-13 22:06:42 -05:00
asonix e4e93cddf9 Return JobId from push 2023-08-13 19:52:27 -05:00
asonix 5d4486fdf4 Finish queue implementation update for sled 2023-08-13 19:47:20 -05:00
asonix 35d0c065e9 Start work on 0.5 db format - initial queuerepo changes 2023-08-13 14:12:38 -05:00
asonix bd3975f455 Simplify some repo functions, remove 0.3 migration 2023-07-25 20:08:18 -05:00
asonix e59483c12c Add ProxyRepo 2023-07-23 10:23:46 -05:00
asonix fe1f7c869f Enable cleanup of variants
Still TODO: doing a first pass to mark variants as accessed on launch
2023-07-22 19:41:50 -05:00
asonix 47e13ec04e Mark variants as accessed on fetch 2023-07-22 18:50:04 -05:00
asonix b786406ad0 Change IdentifierAccess to VariantAccess 2023-07-22 17:57:52 -05:00
asonix 3d93300ba9 Add access repos for alias & identifier 2023-07-18 21:56:13 -05:00
asonix 2961aae6e3 Port migration changes from 0.4.1 2023-07-16 22:07:42 -05:00
asonix 33b83f97f2 It compiles
Need to reinstate media limits
2023-07-13 13:48:59 -05:00
asonix 7768d4e58e Add database exports
This should help admins running pict-rs on traditional filesystems to
keep valid backups of pict-rs' sled repo
2023-07-08 17:36:01 -05:00
asonix 4680f9c3aa Instrument repo and store build methods 2023-07-08 10:42:03 -05:00
asonix 3ccb8ecd8c Validate alias represents real file before setting not found 2023-07-07 13:33:27 -05:00
asonix 9e7376d411 Remove is_missing error, make Identifier and DeleteToken return Option on fetch 2023-07-07 13:17:26 -05:00
asonix e7e4876908 Explicitly handle cases of invalid aliases 2023-07-05 16:46:44 -05:00
asonix 23e67b9697 Enhance migration handling of missing identifiers
Queue a cleanup task for hashes that don't have original file identifiers
Be more specific about what field is missing when a field is missing
2023-07-05 09:52:19 -05:00
asonix 01d3610f46 Add basic progress indication for store migration 2023-06-29 11:39:47 -05:00