1508 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
asonix	38d5f18577	Fix copy+paste bug in alias access ... This manifests as proxied images not having correct access times after migrating from sled to postgres. This could lead to high resource use as all proxied media will be marked as 'accessed' at the time of migration, causing cleanup of all proxied media to happen at around the same time, which is 7 days after last access by default.	2025-04-26 13:59:14 -05:00
asonix	ca7c6f7ba0	Prepare 0.5.17 release	2025-04-21 13:56:16 -05:00
asonix	e62a0769cf	metrics-exporter-prometheus 0.17	2025-04-21 12:55:52 -05:00
asonix	f6b87ebb46	Update dependencies (minor & point)	2025-04-21 12:54:37 -05:00
asonix	31463434ca	Fix deny	2025-04-17 15:47:14 -05:00
asonix	3e29dea11a	Opentelemetry 0.29	2025-04-17 15:46:00 -05:00
asonix	7f0756099d	Update dependencies (minor & point)	2025-04-17 15:38:41 -05:00
asonix	c76554a378	v0.5.17-pre.11	2025-04-04 19:14:30 -05:00
asonix	e01c0af292	Merge pull request 'Fix UTF-8 Errors in Repo Migrations' (#85) from asonix/repo-migration-fixes into main ... Reviewed-on: #85	2025-04-04 23:31:06 +00:00
asonix	805e3b5ca5	Fix UTF8 errors in repo migrations ... As it turns out, sometimes UUIDs are made entirely of bytes that are valid utf-8. This causes problems if a repo migration takes place (and in some other scenarios, such as deleting aliases). Here is an example of the problem 1. A UUID is generated which is valid utf8 for an Alias and stored in sled. 2. A migration is triggered, and this Alias is read out of sled as a Name, rather than a UUID. 3. This alias is inserted into postgres with a representation that isn't formatted like a UUID. 4. That alias is requested by a user, and doesn't match because the representation is different. Another example of the problem is the following 1. A Delete Token is generated which is valid utf8 and contains a null byte. Rust says this is fine. This delete token is stored in sled. 2. A migration is triggered, and this Delete Token is read out of sled as a Name, rather than a UUID. 3. This Delete Token is not inserted into postgres, because postgres does not permit null bytes in utf8 strings. Of these scenarios, only the first can be fixed retroactively. This commit includes a script which runs on pict-rs' first launch to find and fix any aliases or tokens that have improper representations in the postgres database. Any aliases lost to null bytes in past migrations cannot be recovered without running the migration a second time, but it's likely that a given postgres install has diverged enough from it's previous database that a new migration would cause more problems than it would solve. In order to fix this going forward, pict-rs always represents aliases and delete tokens that are 16 bytes long as UUIDs in memory, which matches the expected format in the repos. This means any future migrations will not experience this problem.	2025-04-04 17:57:39 -05:00
asonix	aa6b4b47e6	Add postgres to flake	2025-04-04 17:18:43 -05:00
asonix	c9fae6d36e	try adding concurrency to repo migration	2025-04-04 13:59:59 -05:00
asonix	0633784f8b	Remove generator key from migration. It isn't required anymore	2025-04-04 13:18:45 -05:00
asonix	0856c2eb1b	Include service.version in tracing context	2025-04-02 11:40:06 -05:00
asonix	e9ab28a962	Move integration tests into script	2025-03-29 21:10:01 -05:00
asonix	73afada7e7	Fix string trimming for bash	2025-03-29 17:33:51 -05:00
asonix	5e670710eb	Test each suite individually	2025-03-29 17:26:48 -05:00
asonix	85ac9ec5aa	Use same rustflags for all tests	2025-03-29 17:09:45 -05:00
asonix	65fd38682b	Split tests into individual steps	2025-03-29 17:08:42 -05:00
asonix	d7e83c5d6e	Enable integration tests in publish workflow	2025-03-29 16:58:13 -05:00
asonix	5c9d419cc8	Add basic integration tests for process endpoint	2025-03-29 16:57:07 -05:00
asonix	b81eb306c0	Use install-imagemagick action	2025-03-29 15:16:18 -05:00
asonix	2f24476ae3	re-enable cargo cache. link problem was OOM	2025-03-29 14:55:27 -05:00
asonix	c60b0f727a	No cache in tests	2025-03-29 12:24:25 -05:00
asonix	f9b1cf961c	Add details integration test	2025-03-29 12:16:18 -05:00
asonix	3adec92cff	Run integration tests in nix derivation	2025-03-28 22:04:48 -05:00
asonix	76a8097920	Consistency in release notes	2025-03-28 21:44:19 -05:00
asonix	13d506d7eb	Update dependencies (minor & point)	2025-03-28 21:40:50 -05:00
asonix	b3aadbef8a	Prepare 0.5.17-pre.10 release	2025-03-28 21:40:04 -05:00
asonix	99c260f69d	Enable comma-separated filters in environment variables	2025-03-28 19:48:36 -05:00
asonix	fd5ee28224	Merge pull request 'Disable chunked transfer encoding of responses' (#82) from Nothing4You/pict-rs:content-length into main ... Reviewed-on: #82 Reviewed-by: asonix <asonix@asonix.dog>	2025-03-29 00:23:14 +00:00
asonix	8833951b56	Merge pull request 'Add tests with system deps' (#84) from asonix/integration-tests into main ... Reviewed-on: #84	2025-03-29 00:18:49 +00:00
Richard Schwab	6063347b0d	add tests for content-length and content-range headers, as well as returned body size ... this uses the content-length header returned by a non-ranged HEAD request as a reference value	2025-03-29 00:13:29 +01:00
Richard Schwab	4fcb878f2b	Implement dummy stream for HEAD responses to allow actix to calculate correct content-length headers	2025-03-28 22:12:21 +01:00
Richard Schwab	3063346b9c	Disable chunked transfer encoding of responses ... This also implicitly adds the content-length header, which improves compatibility with CDN caches such as Cloudflare when using range requests.	2025-03-28 22:12:21 +01:00
asonix	a71812930e	Add tests with system deps ... These new tests each spin up their own instance of pict-rs. This allows for configuring pict-rs differently depending on the test that needs to be run. It also opens the possibility of writing tests with object storage and postgres requirements in the future. So far there's pretty minimal test coverage. Just uploading, downloading, deleting, and checking validation errors for images and animations. In the future, tests need to be added for videos, many variations of the process endpoint, and the admin endpoints. The new tests are locked behind a configuration option (not a feature) called `system_deps`. They can be enabled by passing `RUSTFLAGS='--cfg system_deps'` This is to prevent tests from running and failing in environments where the exiftool, imagemagick, and ffmpeg binaries are not present. This builds imagemagick from source for a few reasons 1. binaries for imagemagick 7 are not available for debian 12 2. the imagemagick appimage does not support avif files 3. the imagemagick appimage is limited to x86_64	2025-03-28 10:30:03 -05:00
asonix	742677fff6	Fix blurhash computation for some animations ... In the pure-rust blurhash path, I recently added byte streaming for the image reader, which for some formats (like gif) can produce and operate on the first frame of an animation without downloading the whole file. This means that a blurhash could be calculated and the reader dropped before the writer had written all the bytes to the channel. Erroring in this case is incorrect.	2025-03-27 11:47:24 -05:00
asonix	7483ca43f9	object_store 0.12	2025-03-25 15:21:49 -05:00
asonix	d506f38b78	Opentelemetry 0.28	2025-03-25 15:18:09 -05:00
asonix	2059d7aaf5	Update dependencies (minor & point)	2025-03-25 15:12:09 -05:00
asonix	cfaf6d70ef	update rustls-channel-resolver	2025-02-09 13:34:53 -06:00
asonix	6eea4bcf2e	Update deny.toml	2025-02-08 16:11:50 -06:00
asonix	2c84bb2092	Add release notes	2025-02-08 16:03:23 -06:00
asonix	e4d213fb49	rand 0.9.0	2025-02-08 15:53:37 -06:00
asonix	6473e37f27	config 0.15	2025-02-08 15:49:17 -06:00
asonix	9000152fe5	0.5.17-pre.9	2025-02-08 15:42:56 -06:00
asonix	071a541300	Update nixpkgs	2025-02-08 14:55:19 -06:00
asonix	ae785fa8fc	BytesReader: Remove unneeded try_recv	2025-02-08 14:39:21 -06:00
asonix	c8817db997	BytesReader: implement in terms of BufReader	2025-02-08 14:36:51 -06:00
asonix	939cd7955c	Add BytesReader for streaming image decoding for blurhashes	2025-02-08 14:21:52 -06:00