Merge pull request 'v4.2.9' (#67) from asonix/changes into asonix/downstream

Reviewed-on: #67
2024-06-19 17:06:20 +00:00 · 2024-06-19 17:06:20 +00:00 · 16af4f4c8a
parent ba9c861116 8a73ead5c7
commit 16af4f4c8a
67 changed files with 771 additions and 321 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -2,6 +2,61 @@

 All notable changes to this project will be documented in this file.

+## [4.2.9] - 2024-05-30
+
+### Security
+
+- Update dependencies
+- Fix private mention filtering ([GHSA-5fq7-3p3j-9vrf](https://github.com/mastodon/mastodon/security/advisories/GHSA-5fq7-3p3j-9vrf))
+- Fix password change endpoint not being rate-limited ([GHSA-q3rg-xx5v-4mxh](https://github.com/mastodon/mastodon/security/advisories/GHSA-q3rg-xx5v-4mxh))
+- Add hardening around rate-limit bypass ([GHSA-c2r5-cfqr-c553](https://github.com/mastodon/mastodon/security/advisories/GHSA-c2r5-cfqr-c553))
+
+### Added
+
+- Add rate-limit on OAuth application registration ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/30316))
+- Add fallback redirection when getting a webfinger query `WEB_DOMAIN@WEB_DOMAIN` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/28592))
+- Add `digest` attribute to `Admin::DomainBlock` entity in REST API ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/29092))
+
+### Removed
+
+- Remove superfluous application-level caching in some controllers ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29862))
+- Remove aggressive OAuth application vacuuming ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/30316))
+
+### Fixed
+
+- Fix leaking Elasticsearch connections in Sidekiq processes ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30450))
+- Fix language of remote posts not being recognized when using unusual casing ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30403))
+- Fix off-by-one in `tootctl media` commands ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30306))
+- Fix removal of allowed domains (in `LIMITED_FEDERATION_MODE`) not being recorded in the audit log ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/30125))
+- Fix not being able to block a subdomain of an already-blocked domain through the API ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30119))
+- Fix `Idempotency-Key` being ignored when scheduling a post ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30084))
+- Fix crash when supplying the `FFMPEG_BINARY` environment variable ([timothyjrogers](https://github.com/mastodon/mastodon/pull/30022))
+- Fix improper email address validation ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29838))
+- Fix results/query in `api/v1/featured_tags/suggestions` ([mjankowski](https://github.com/mastodon/mastodon/pull/29597))
+- Fix unblocking internationalized domain names under certain conditions ([tribela](https://github.com/mastodon/mastodon/pull/29530))
+- Fix admin account created by `mastodon:setup` not being auto-approved ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29379))
+- Fix reference to non-existent var in CLI maintenance command ([mjankowski](https://github.com/mastodon/mastodon/pull/28363))
+
+## [4.2.8] - 2024-02-23
+
+### Added
+
+- Add hourly task to automatically require approval for new registrations in the absence of moderators ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29318), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/29355))
+  In order to prevent future abandoned Mastodon servers from being used for spam, harassment and other malicious activity, Mastodon will now automatically switch new user registrations to require moderator approval whenever they are left open and no activity (including non-moderation actions from apps) from any logged-in user with permission to access moderation reports has been detected in a full week.
+  When this happens, users with the permission to change server settings will receive an email notification.
+  This feature is disabled when `EMAIL_DOMAIN_ALLOWLIST` is used, and can also be disabled with `DISABLE_AUTOMATIC_SWITCHING_TO_APPROVED_REGISTRATIONS=true`.
+
+### Changed
+
+- Change registrations to be closed by default on new installations ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29280))
+  If you are running a server and never changed your registrations mode from the default, updating will automatically close your registrations.
+  Simply re-enable them through the administration interface or using `tootctl settings registrations open` if you want to enable them again.
+
+### Fixed
+
+- Fix processing of remote ActivityPub actors making use of `Link` objects as `Image` `url` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29335))
+- Fix link verifications when page size exceeds 1MB ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29358))
+
 ## [4.2.7] - 2024-02-16

 ### Fixed
--- a/2
+++ b/2
@ -204,3 +204,5 @@ gem 'net-http', '~> 0.3.2'
 gem 'rubyzip', '~> 2.3'

 gem 'hcaptcha', '~> 7.1'
+
+gem 'mail', '~> 2.8'
--- a/Gemfile.lock
+++ b/Gemfile.lock
@ -28,47 +28,47 @@ GIT
 GEM
  remote: https://rubygems.org/
  specs:
-    actioncable (7.0.8)
-      actionpack (= 7.0.8)
-      activesupport (= 7.0.8)
+    actioncable (7.0.8.1)
+      actionpack (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
      nio4r (~> 2.0)
      websocket-driver (>= 0.6.1)
-    actionmailbox (7.0.8)
-      actionpack (= 7.0.8)
-      activejob (= 7.0.8)
-      activerecord (= 7.0.8)
-      activestorage (= 7.0.8)
-      activesupport (= 7.0.8)
+    actionmailbox (7.0.8.1)
+      actionpack (= 7.0.8.1)
+      activejob (= 7.0.8.1)
+      activerecord (= 7.0.8.1)
+      activestorage (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
      mail (>= 2.7.1)
      net-imap
      net-pop
      net-smtp
-    actionmailer (7.0.8)
-      actionpack (= 7.0.8)
-      actionview (= 7.0.8)
-      activejob (= 7.0.8)
-      activesupport (= 7.0.8)
+    actionmailer (7.0.8.1)
+      actionpack (= 7.0.8.1)
+      actionview (= 7.0.8.1)
+      activejob (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
      mail (~> 2.5, >= 2.5.4)
      net-imap
      net-pop
      net-smtp
      rails-dom-testing (~> 2.0)
-    actionpack (7.0.8)
-      actionview (= 7.0.8)
-      activesupport (= 7.0.8)
+    actionpack (7.0.8.1)
+      actionview (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
      rack (~> 2.0, >= 2.2.4)
      rack-test (>= 0.6.3)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.0, >= 1.2.0)
-    actiontext (7.0.8)
-      actionpack (= 7.0.8)
-      activerecord (= 7.0.8)
-      activestorage (= 7.0.8)
-      activesupport (= 7.0.8)
+    actiontext (7.0.8.1)
+      actionpack (= 7.0.8.1)
+      activerecord (= 7.0.8.1)
+      activestorage (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
      globalid (>= 0.6.0)
      nokogiri (>= 1.8.5)
-    actionview (7.0.8)
-      activesupport (= 7.0.8)
+    actionview (7.0.8.1)
+      activesupport (= 7.0.8.1)
      builder (~> 3.1)
      erubi (~> 1.4)
      rails-dom-testing (~> 2.0)
@ -78,22 +78,22 @@ GEM
      activemodel (>= 4.1, < 7.1)
      case_transform (>= 0.2)
      jsonapi-renderer (>= 0.1.1.beta1, < 0.3)
-    activejob (7.0.8)
-      activesupport (= 7.0.8)
+    activejob (7.0.8.1)
+      activesupport (= 7.0.8.1)
      globalid (>= 0.3.6)
-    activemodel (7.0.8)
-      activesupport (= 7.0.8)
-    activerecord (7.0.8)
-      activemodel (= 7.0.8)
-      activesupport (= 7.0.8)
-    activestorage (7.0.8)
-      actionpack (= 7.0.8)
-      activejob (= 7.0.8)
-      activerecord (= 7.0.8)
-      activesupport (= 7.0.8)
+    activemodel (7.0.8.1)
+      activesupport (= 7.0.8.1)
+    activerecord (7.0.8.1)
+      activemodel (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
+    activestorage (7.0.8.1)
+      actionpack (= 7.0.8.1)
+      activejob (= 7.0.8.1)
+      activerecord (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
      marcel (~> 1.0)
      mini_mime (>= 1.1.0)
-    activesupport (7.0.8)
+    activesupport (7.0.8.1)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (>= 1.6, < 2)
      minitest (>= 5.1)
@ -215,7 +215,7 @@ GEM
      activerecord (>= 5.a)
      database_cleaner-core (~> 2.0.0)
    database_cleaner-core (2.0.1)
-    date (3.3.3)
+    date (3.3.4)
    debug_inspector (1.1.0)
    devise (4.9.2)
      bcrypt (~> 3.0)
@ -288,7 +288,7 @@ GEM
    faraday_middleware (1.2.0)
      faraday (~> 1.0)
    fast_blank (1.0.1)
-    fastimage (2.2.7)
+    fastimage (2.3.1)
    ffi (1.15.5)
    ffi-compiler (1.0.1)
      ffi (>= 1.0.0)
@ -368,7 +368,7 @@ GEM
    jmespath (1.6.2)
    json (2.6.3)
    json-canonicalization (1.0.0)
-    json-jwt (1.15.3)
+    json-jwt (1.15.3.1)
      activesupport (>= 4.2)
      aes_key_wrap
      bindata
@ -424,7 +424,7 @@ GEM
      activesupport (>= 4)
      railties (>= 4)
      request_store (~> 1.0)
-    loofah (2.21.3)
+    loofah (2.21.4)
      crass (~> 1.0.2)
      nokogiri (>= 1.12.0)
    mail (2.8.1)
@ -461,15 +461,15 @@ GEM
    net-ldap (0.18.0)
    net-pop (0.1.2)
      net-protocol
-    net-protocol (0.2.1)
+    net-protocol (0.2.2)
      timeout
    net-scp (4.0.0)
      net-ssh (>= 2.6.5, < 8.0.0)
-    net-smtp (0.3.3)
+    net-smtp (0.3.4)
      net-protocol
    net-ssh (7.1.0)
    nio4r (2.7.0)
-    nokogiri (1.16.2)
+    nokogiri (1.16.5)
      mini_portile2 (~> 2.8.2)
      racc (~> 1.4)
    nsa (0.3.0)
@ -534,10 +534,10 @@ GEM
      activesupport (>= 3.0.0)
    raabro (1.4.0)
    racc (1.7.3)
-    rack (2.2.8)
+    rack (2.2.8.1)
    rack-attack (6.7.0)
      rack (>= 1.0, < 4)
-    rack-cors (2.0.1)
+    rack-cors (2.0.2)
      rack (>= 2.0.0)
    rack-oauth2 (1.21.3)
      activesupport
@ -551,20 +551,20 @@ GEM
      rack
    rack-test (2.1.0)
      rack (>= 1.3)
-    rails (7.0.8)
-      actioncable (= 7.0.8)
-      actionmailbox (= 7.0.8)
-      actionmailer (= 7.0.8)
-      actionpack (= 7.0.8)
-      actiontext (= 7.0.8)
-      actionview (= 7.0.8)
-      activejob (= 7.0.8)
-      activemodel (= 7.0.8)
-      activerecord (= 7.0.8)
-      activestorage (= 7.0.8)
-      activesupport (= 7.0.8)
+    rails (7.0.8.1)
+      actioncable (= 7.0.8.1)
+      actionmailbox (= 7.0.8.1)
+      actionmailer (= 7.0.8.1)
+      actionpack (= 7.0.8.1)
+      actiontext (= 7.0.8.1)
+      actionview (= 7.0.8.1)
+      activejob (= 7.0.8.1)
+      activemodel (= 7.0.8.1)
+      activerecord (= 7.0.8.1)
+      activestorage (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
      bundler (>= 1.15.0)
-      railties (= 7.0.8)
+      railties (= 7.0.8.1)
    rails-controller-testing (1.0.5)
      actionpack (>= 5.0.1.rc1)
      actionview (>= 5.0.1.rc1)
@ -579,9 +579,9 @@ GEM
    rails-i18n (7.0.7)
      i18n (>= 0.7, < 2)
      railties (>= 6.0.0, < 8)
-    railties (7.0.8)
-      actionpack (= 7.0.8)
-      activesupport (= 7.0.8)
+    railties (7.0.8.1)
+      actionpack (= 7.0.8.1)
+      activesupport (= 7.0.8.1)
      method_source
      rake (>= 12.2)
      thor (~> 1.0)
@ -605,8 +605,9 @@ GEM
    responders (3.1.0)
      actionpack (>= 5.2)
      railties (>= 5.2)
-    rexml (3.2.6)
-    rotp (6.2.2)
+    rexml (3.2.8)
+      strscan (>= 3.0.9)
+    rotp (6.3.0)
    rouge (4.1.2)
    rpam2 (4.0.2)
    rqrcode (2.2.0)
@ -731,6 +732,7 @@ GEM
      redlock (~> 1.0)
    strong_migrations (0.8.0)
      activerecord (>= 5.2)
+    strscan (3.0.9)
    swd (1.3.0)
      activesupport (>= 3)
      attr_required (>= 0.0.5)
@ -744,7 +746,7 @@ GEM
    test-prof (1.2.3)
    thor (1.3.0)
    tilt (2.2.0)
-    timeout (0.4.0)
+    timeout (0.4.1)
    tpm-key_attestation (0.12.0)
      bindata (~> 2.4)
      openssl (> 2.0)
@ -808,7 +810,7 @@ GEM
    xorcist (1.1.3)
    xpath (3.2.0)
      nokogiri (~> 1.8)
-    zeitwerk (2.6.11)
+    zeitwerk (2.6.13)

 PLATFORMS
  ruby
@ -871,6 +873,7 @@ DEPENDENCIES
  letter_opener_web (~> 2.0)
  link_header (~> 0.0)
  lograge (~> 0.12)
+  mail (~> 2.8)
  mario-redis-lock (~> 1.2)
  md-paperclip-azure (~> 2.2)
  memory_profiler
--- a/SECURITY.md
+++ b/SECURITY.md
@ -13,8 +13,8 @@ A "vulnerability in Mastodon" is a vulnerability in the code distributed through

 ## Supported Versions

-| Version | Supported        |
-| ------- | ---------------- |
-| 4.2.x   | Yes              |
-| 4.1.x   | Yes              |
-| < 4.1   | No               |
+| Version | Supported |
+| ------- | --------- |
+| 4.2.x   | Yes       |
+| 4.1.x   | Yes       |
+| < 4.1   | No        |
--- a/app/controllers/admin/domain_allows_controller.rb
+++ b/app/controllers/admin/domain_allows_controller.rb
@ -25,6 +25,8 @@ class Admin::DomainAllowsController < Admin::BaseController
  def destroy
    authorize @domain_allow, :destroy?
    UnallowDomainService.new.call(@domain_allow)
+    log_action :destroy, @domain_allow
+
    redirect_to admin_instances_path, notice: I18n.t('admin.domain_allows.destroyed_msg')
  end

--- a/app/controllers/api/v1/admin/domain_blocks_controller.rb
+++ b/app/controllers/api/v1/admin/domain_blocks_controller.rb
@ -29,10 +29,11 @@ class Api::V1::Admin::DomainBlocksController < Api::BaseController
  def create
    authorize :domain_block, :create?

+    @domain_block = DomainBlock.new(resource_params)
    existing_domain_block = resource_params[:domain].present? ? DomainBlock.rule_for(resource_params[:domain]) : nil
-    return render json: existing_domain_block, serializer: REST::Admin::ExistingDomainBlockErrorSerializer, status: 422 if existing_domain_block.present?
+    return render json: existing_domain_block, serializer: REST::Admin::ExistingDomainBlockErrorSerializer, status: 422 if conflicts_with_existing_block?(@domain_block, existing_domain_block)

-    @domain_block = DomainBlock.create!(resource_params)
+    @domain_block.save!
    DomainBlockWorker.perform_async(@domain_block.id)
    log_action :create, @domain_block
    render json: @domain_block, serializer: REST::Admin::DomainBlockSerializer
@ -55,6 +56,10 @@ class Api::V1::Admin::DomainBlocksController < Api::BaseController

  private

+  def conflicts_with_existing_block?(domain_block, existing_domain_block)
+    existing_domain_block.present? && (existing_domain_block.domain == TagManager.instance.normalize_domain(domain_block.domain) || !domain_block.stricter_than?(existing_domain_block))
+  end
+
  def set_domain_blocks
    @domain_blocks = filtered_domain_blocks.order(id: :desc).to_a_paginated_by_id(limit_param(LIMIT), params_slice(:max_id, :since_id, :min_id))
  end
--- a/app/controllers/api/v1/featured_tags/suggestions_controller.rb
+++ b/app/controllers/api/v1/featured_tags/suggestions_controller.rb
@ -12,6 +12,10 @@ class Api::V1::FeaturedTags::SuggestionsController < Api::BaseController
  private

  def set_recently_used_tags
-    @recently_used_tags = Tag.recently_used(current_account).where.not(id: current_account.featured_tags).limit(10)
+    @recently_used_tags = Tag.recently_used(current_account).where.not(id: featured_tag_ids).limit(10)
+  end
+
+  def featured_tag_ids
+    current_account.featured_tags.pluck(:tag_id)
  end
 end
--- a/app/controllers/concerns/cache_concern.rb
+++ b/app/controllers/concerns/cache_concern.rb
@ -198,34 +198,19 @@ module CacheConcern
    end
  end

+  # TODO: Rename this method, as it does not perform any caching anymore.
  def cache_collection(raw, klass)
-    return raw unless klass.respond_to?(:with_includes)
+    return raw unless klass.respond_to?(:preload_cacheable_associations)

-    raw = raw.cache_ids.to_a if raw.is_a?(ActiveRecord::Relation)
-    return [] if raw.empty?
+    records = raw.to_a

-    cached_keys_with_value = begin
-      Rails.cache.read_multi(*raw).transform_keys(&:id).transform_values { |r| ActiveRecordCoder.load(r) }
-    rescue ActiveRecordCoder::Error
-      {} # The serialization format may have changed, let's pretend it's a cache miss.
-    end
+    klass.preload_cacheable_associations(records)

-    uncached_ids = raw.map(&:id) - cached_keys_with_value.keys
-
-    klass.reload_stale_associations!(cached_keys_with_value.values) if klass.respond_to?(:reload_stale_associations!)
-
-    unless uncached_ids.empty?
-      uncached = klass.where(id: uncached_ids).with_includes.index_by(&:id)
-
-      uncached.each_value do |item|
-        Rails.cache.write(item, ActiveRecordCoder.dump(item))
-      end
-    end
-
-    raw.filter_map { |item| cached_keys_with_value[item.id] || uncached[item.id] }
+    records
  end

+  # TODO: Rename this method, as it does not perform any caching anymore.
  def cache_collection_paginated_by_id(raw, klass, limit, options)
-    cache_collection raw.cache_ids.to_a_paginated_by_id(limit, options), klass
+    cache_collection raw.to_a_paginated_by_id(limit, options), klass
  end
 end
--- a/app/controllers/well_known/webfinger_controller.rb
+++ b/app/controllers/well_known/webfinger_controller.rb
@ -21,7 +21,7 @@ module WellKnown
      username = username_from_resource

      @account = begin
-        if username == Rails.configuration.x.local_domain
+        if username == Rails.configuration.x.local_domain || username == Rails.configuration.x.web_domain
          Account.representative
        else
          Account.find_local!(username)
--- a/app/javascript/packs/admin.jsx
+++ b/app/javascript/packs/admin.jsx
@ -145,6 +145,10 @@ delegate(document, '#form_admin_settings_enable_bootstrap_timeline_accounts', 'c
 const onChangeRegistrationMode = (target) => {
  const enabled = target.value === 'approved';

+  [].forEach.call(document.querySelectorAll('.form_admin_settings_registrations_mode .warning-hint'), (warning_hint) => {
+    warning_hint.style.display = target.value === 'open' ? 'inline' : 'none';
+  });
+
  [].forEach.call(document.querySelectorAll('#form_admin_settings_require_invite_text'), (input) => {
    input.disabled = !enabled;
    if (enabled) {
--- a/app/lib/activitypub/parser/status_parser.rb
+++ b/app/lib/activitypub/parser/status_parser.rb
@ -3,6 +3,8 @@
 class ActivityPub::Parser::StatusParser
  include JsonLdHelper

+  NORMALIZED_LOCALE_NAMES = LanguagesHelper::SUPPORTED_LOCALES.keys.index_by(&:downcase).freeze
+
  # @param [Hash] json
  # @param [Hash] magic_values
  # @option magic_values [String] :followers_collection
@ -86,6 +88,13 @@ class ActivityPub::Parser::StatusParser
  end

  def language
+    lang = raw_language_code
+    lang.presence && NORMALIZED_LOCALE_NAMES.fetch(lang.downcase.to_sym, lang)
+  end
+
+  private
+
+  def raw_language_code
    if content_language_map?
      @object['contentMap'].keys.first
    elsif name_language_map?
@ -95,8 +104,6 @@ class ActivityPub::Parser::StatusParser
    end
  end

-  private
-
  def audience_to
    as_array(@object['to'] || @json['to']).map { |x| value_or_id(x) }
  end
--- a/app/lib/vacuum/applications_vacuum.rb
+++ b/app/lib/vacuum/applications_vacuum.rb
@ -1,10 +0,0 @@
-# frozen_string_literal: true
-
-class Vacuum::ApplicationsVacuum
-  def perform
-    Doorkeeper::Application.where(owner_id: nil)
-                           .where.missing(:created_users, :access_tokens, :access_grants)
-                           .where(created_at: ...1.day.ago)
-                           .in_batches.delete_all
-  end
-end
--- a/app/lib/video_metadata_extractor.rb
+++ b/app/lib/video_metadata_extractor.rb
@ -22,7 +22,7 @@ class VideoMetadataExtractor
  private

  def ffmpeg_command_output
-    command = Terrapin::CommandLine.new('ffprobe', '-i :path -print_format :format -show_format -show_streams -show_error -loglevel :loglevel')
+    command = Terrapin::CommandLine.new(Rails.configuration.x.ffprobe_binary, '-i :path -print_format :format -show_format -show_streams -show_error -loglevel :loglevel')
    command.run(path: @path, format: 'json', loglevel: 'fatal')
  end

--- a/app/mailers/admin_mailer.rb
+++ b/app/mailers/admin_mailer.rb
@ -61,6 +61,12 @@ class AdminMailer < ApplicationMailer
    end
  end

+  def auto_close_registrations
+    locale_for_account(@me) do
+      mail subject: default_i18n_subject(instance: @instance)
+    end
+  end
+
  private

  def process_params
--- a/app/models/concerns/account_interactions.rb
+++ b/app/models/concerns/account_interactions.rb
@ -185,7 +185,7 @@ module AccountInteractions
  end

  def unblock_domain!(other_domain)
-    block = domain_blocks.find_by(domain: other_domain)
+    block = domain_blocks.find_by(domain: normalized_domain(other_domain))
    block&.destroy
  end

@ -313,4 +313,8 @@ module AccountInteractions
  def remove_potential_friendship(other_account)
    PotentialFriendshipTracker.remove(id, other_account.id)
  end
+
+  def normalized_domain(domain)
+    TagManager.instance.normalize_domain(domain)
+  end
 end
--- a/app/models/concerns/cacheable.rb
+++ b/app/models/concerns/cacheable.rb
@ -14,6 +14,10 @@ module Cacheable
      includes(@cache_associated)
    end

+    def preload_cacheable_associations(records)
+      ActiveRecord::Associations::Preloader.new(records: records, associations: @cache_associated).call
+    end
+
    def cache_ids
      select(:id, :updated_at)
    end
--- a/app/models/feed.rb
+++ b/app/models/feed.rb
@ -28,7 +28,7 @@ class Feed
      unhydrated = redis.zrangebyscore(key, "(#{min_id}", "(#{max_id}", limit: [0, limit], with_scores: true).map(&:first).map(&:to_i)
    end

-    Status.where(id: unhydrated).cache_ids
+    Status.where(id: unhydrated)
  end

  def key
--- a/app/models/public_feed.rb
+++ b/app/models/public_feed.rb
@ -29,7 +29,7 @@ class PublicFeed
    scope.merge!(media_only_scope) if media_only?
    scope.merge!(language_scope) if account&.chosen_languages.present?

-    scope.cache_ids.to_a_paginated_by_id(limit, max_id: max_id, since_id: since_id, min_id: min_id)
+    scope.to_a_paginated_by_id(limit, max_id: max_id, since_id: since_id, min_id: min_id)
  end

  private
--- a/app/models/status.rb
+++ b/app/models/status.rb
@ -338,38 +338,6 @@ class Status < ApplicationRecord
      StatusPin.select('status_id').where(status_id: status_ids).where(account_id: account_id).each_with_object({}) { |p, h| h[p.status_id] = true }
    end

-    def reload_stale_associations!(cached_items)
-      account_ids = []
-
-      cached_items.each do |item|
-        account_ids << item.account_id
-        account_ids << item.reblog.account_id if item.reblog?
-      end
-
-      account_ids.uniq!
-
-      status_ids = cached_items.map { |item| item.reblog? ? item.reblog_of_id : item.id }.uniq
-
-      return if account_ids.empty?
-
-      accounts = Account.where(id: account_ids).includes(:account_stat, :user).index_by(&:id)
-
-      status_stats = StatusStat.where(status_id: status_ids).index_by(&:status_id)
-
-      cached_items.each do |item|
-        item.account = accounts[item.account_id]
-        item.reblog.account = accounts[item.reblog.account_id] if item.reblog?
-
-        if item.reblog?
-          status_stat = status_stats[item.reblog.id]
-          item.reblog.status_stat = status_stat if status_stat.present?
-        else
-          status_stat = status_stats[item.id]
-          item.status_stat = status_stat if status_stat.present?
-        end
-      end
-    end
-
    def from_text(text)
      return [] if text.blank?

--- a/app/models/tag_feed.rb
+++ b/app/models/tag_feed.rb
@ -33,7 +33,7 @@ class TagFeed < PublicFeed
    scope.merge!(account_filters_scope) if account?
    scope.merge!(media_only_scope) if media_only?

-    scope.cache_ids.to_a_paginated_by_id(limit, max_id: max_id, since_id: since_id, min_id: min_id)
+    scope.to_a_paginated_by_id(limit, max_id: max_id, since_id: since_id, min_id: min_id)
  end

  private
--- a/app/models/user.rb
+++ b/app/models/user.rb
@ -96,6 +96,8 @@ class User < ApplicationRecord
  accepts_nested_attributes_for :invite_request, reject_if: ->(attributes) { attributes['text'].blank? && !Setting.require_invite_text }
  validates :invite_request, presence: true, on: :create, if: :invite_text_required?

+  validates :email, presence: true, email_address: true
+
  validates_with BlacklistedEmailValidator, if: -> { ENV['EMAIL_DOMAIN_LISTS_APPLY_AFTER_CONFIRMATION'] == 'true' || !confirmed? }
  validates_with EmailMxValidator, if: :validate_email_dns?
  validates :agreement, acceptance: { allow_nil: false, accept: [true, 'true', '1'] }, on: :create
--- a/app/serializers/rest/admin/domain_block_serializer.rb
+++ b/app/serializers/rest/admin/domain_block_serializer.rb
@ -1,11 +1,15 @@
 # frozen_string_literal: true

 class REST::Admin::DomainBlockSerializer < ActiveModel::Serializer
-  attributes :id, :domain, :created_at, :severity,
+  attributes :id, :domain, :digest, :created_at, :severity,
             :reject_media, :reject_reports,
             :private_comment, :public_comment, :obfuscate

  def id
    object.id.to_s
  end
+
+  def digest
+    object.domain_digest
+  end
 end
--- a/app/services/activitypub/process_account_service.rb
+++ b/app/services/activitypub/process_account_service.rb
@ -201,10 +201,15 @@ class ActivityPub::ProcessAccountService < BaseService
    value = first_of_value(@json[key])

    return if value.nil?
-    return value['url'] if value.is_a?(Hash)

-    image = fetch_resource_without_id_validation(value)
-    image['url'] if image
+    if value.is_a?(String)
+      value = fetch_resource_without_id_validation(value)
+      return if value.nil?
+    end
+
+    value = first_of_value(value['url']) if value.is_a?(Hash) && value['type'] == 'Image'
+    value = value['href'] if value.is_a?(Hash)
+    value if value.is_a?(String)
  end

  def public_key
--- a/app/services/notify_service.rb
+++ b/app/services/notify_service.rb
@ -71,16 +71,17 @@ class NotifyService < BaseService
          LEFT JOIN mentions m ON m.silent = FALSE AND m.account_id = :sender_id AND m.status_id = s.id
          WHERE s.id = :id
        UNION ALL
-          SELECT s.id, s.in_reply_to_id, m.id, st.path || s.id, st.depth + 1
-          FROM ancestors st
-          JOIN statuses s ON s.id = st.in_reply_to_id
-          LEFT JOIN mentions m ON m.silent = FALSE AND m.account_id = :sender_id AND m.status_id = s.id
-          WHERE st.mention_id IS NULL AND NOT s.id = ANY(path) AND st.depth < :depth_limit
+          SELECT s.id, s.in_reply_to_id, m.id, ancestors.path || s.id, ancestors.depth + 1
+          FROM ancestors
+          JOIN statuses s ON s.id = ancestors.in_reply_to_id
+          /* early exit if we already have a mention matching our requirements */
+          LEFT JOIN mentions m ON m.silent = FALSE AND m.account_id = :sender_id AND m.status_id = s.id AND s.account_id = :recipient_id
+          WHERE ancestors.mention_id IS NULL AND NOT s.id = ANY(path) AND ancestors.depth < :depth_limit
      )
      SELECT COUNT(*)
-      FROM ancestors st
-      JOIN statuses s ON s.id = st.id
-      WHERE st.mention_id IS NOT NULL AND s.visibility = 3
+      FROM ancestors
+      JOIN statuses s ON s.id = ancestors.id
+      WHERE ancestors.mention_id IS NOT NULL AND s.account_id = :recipient_id AND s.visibility = 3
    SQL
  end

--- a/app/services/post_status_service.rb
+++ b/app/services/post_status_service.rb
@ -160,7 +160,7 @@ class PostStatusService < BaseService

  def idempotency_duplicate
    if scheduled?
-      @account.schedule_statuses.find(@idempotency_duplicate)
+      @account.scheduled_statuses.find(@idempotency_duplicate)
    else
      @account.statuses.find(@idempotency_duplicate)
    end
@ -214,7 +214,7 @@ class PostStatusService < BaseService
  end

  def scheduled_options
-    @options.tap do |options_hash|
+    @options.dup.tap do |options_hash|
      options_hash[:in_reply_to_id]  = options_hash.delete(:thread)&.id
      options_hash[:application_id]  = options_hash.delete(:application)&.id
      options_hash[:scheduled_at]    = nil
--- a/app/services/verify_link_service.rb
+++ b/app/services/verify_link_service.rb
@ -19,7 +19,7 @@ class VerifyLinkService < BaseService

  def perform_request!
    @body = Request.new(:get, @url).add_headers('Accept' => 'text/html').perform do |res|
-      res.code == 200 ? res.body_with_limit : nil
+      res.code == 200 ? res.truncated_body : nil
    end
  end

--- a/app/validators/email_address_validator.rb
+++ b/app/validators/email_address_validator.rb
@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+# NOTE: I initially wrote this as `EmailValidator` but it ended up clashing
+# with an indirect dependency of ours, `validate_email`, which, turns out,
+# has the same approach as we do, but with an extra check disallowing
+# single-label domains. Decided to not switch to `validate_email` because
+# we do want to allow at least `localhost`.
+
+class EmailAddressValidator < ActiveModel::EachValidator
+  def validate_each(record, attribute, value)
+    value = value.strip
+
+    address = Mail::Address.new(value)
+    record.errors.add(attribute, :invalid) if address.address != value
+  rescue Mail::Field::FieldError
+    record.errors.add(attribute, :invalid)
+  end
+end
--- a/app/views/admin/settings/registrations/show.html.haml
+++ b/app/views/admin/settings/registrations/show.html.haml
@ -10,9 +10,11 @@

  %p.lead= t('admin.settings.registrations.preamble')

+  .flash-message= t('admin.settings.registrations.moderation_recommandation')
+
  .fields-row
    .fields-row__column.fields-row__column-6.fields-group
-      = f.input :registrations_mode, collection: %w(open approved none), wrapper: :with_label, include_blank: false, label_method: ->(mode) { I18n.t("admin.settings.registrations_mode.modes.#{mode}") }
+      = f.input :registrations_mode, collection: %w(open approved none), wrapper: :with_label, include_blank: false, label_method: ->(mode) { I18n.t("admin.settings.registrations_mode.modes.#{mode}") }, warning_hint: I18n.t('admin.settings.registrations_mode.warning_hint')

    .fields-row__column.fields-row__column-6.fields-group
      = f.input :require_invite_text, as: :boolean, wrapper: :with_label, disabled: !approved_registrations?
--- a/app/views/admin_mailer/auto_close_registrations.text.erb
+++ b/app/views/admin_mailer/auto_close_registrations.text.erb
@ -0,0 +1,3 @@
+<%= raw t('admin_mailer.auto_close_registrations.body', instance: @instance) %>
+
+<%= raw t('application_mailer.view')%> <%= admin_settings_registrations_url %>
--- a/app/workers/scheduler/auto_close_registrations_scheduler.rb
+++ b/app/workers/scheduler/auto_close_registrations_scheduler.rb
@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+class Scheduler::AutoCloseRegistrationsScheduler
+  include Sidekiq::Worker
+  include Redisable
+
+  sidekiq_options retry: 0
+
+  # Automatically switch away from open registrations if no
+  # moderator had any activity in that period of time
+  OPEN_REGISTRATIONS_MODERATOR_THRESHOLD = 1.week + UserTrackingConcern::SIGN_IN_UPDATE_FREQUENCY
+
+  def perform
+    return if Rails.configuration.x.email_domains_whitelist.present? || ENV['DISABLE_AUTOMATIC_SWITCHING_TO_APPROVED_REGISTRATIONS'] == 'true'
+    return unless Setting.registrations_mode == 'open'
+
+    switch_to_approval_mode! unless active_moderators?
+  end
+
+  private
+
+  def active_moderators?
+    User.those_who_can(:manage_reports).exists?(current_sign_in_at: OPEN_REGISTRATIONS_MODERATOR_THRESHOLD.ago...)
+  end
+
+  def switch_to_approval_mode!
+    Setting.registrations_mode = 'approved'
+
+    User.those_who_can(:manage_settings).includes(:account).find_each do |user|
+      AdminMailer.with(recipient: user.account).auto_close_registrations.deliver_later
+    end
+  end
+end
--- a/app/workers/scheduler/vacuum_scheduler.rb
+++ b/app/workers/scheduler/vacuum_scheduler.rb
@ -22,7 +22,6 @@ class Scheduler::VacuumScheduler
      preview_cards_vacuum,
      backups_vacuum,
      access_tokens_vacuum,
-      applications_vacuum,
      feeds_vacuum,
      imports_vacuum,
    ]
@ -56,10 +55,6 @@ class Scheduler::VacuumScheduler
    Vacuum::ImportsVacuum.new
  end

-  def applications_vacuum
-    Vacuum::ApplicationsVacuum.new
-  end
-
  def content_retention_policy
    ContentRetentionPolicy.current
  end
--- a/config/application.rb
+++ b/config/application.rb
@ -48,6 +48,7 @@ require_relative '../lib/chewy/strategy/bypass_with_warning'
 require_relative '../lib/webpacker/manifest_extensions'
 require_relative '../lib/webpacker/helper_extensions'
 require_relative '../lib/rails/engine_extensions'
+require_relative '../lib/action_dispatch/remote_ip_extensions'
 require_relative '../lib/active_record/database_tasks_extensions'
 require_relative '../lib/active_record/batches'
 require_relative '../lib/simple_navigation/item_extensions'
--- a/config/initializers/ffmpeg.rb
+++ b/config/initializers/ffmpeg.rb
@ -1,5 +1,6 @@
 # frozen_string_literal: true

-if ENV['FFMPEG_BINARY'].present?
-  FFMPEG.ffmpeg_binary = ENV['FFMPEG_BINARY']
+Rails.application.configure do
+  config.x.ffmpeg_binary = ENV['FFMPEG_BINARY'] || 'ffmpeg'
+  config.x.ffprobe_binary = ENV['FFPROBE_BINARY'] || 'ffprobe'
 end
--- a/config/initializers/rack_attack.rb
+++ b/config/initializers/rack_attack.rb
@ -37,6 +37,10 @@ class Rack::Attack
      authenticated_token&.id
    end

+    def warden_user_id
+      @env['warden']&.user&.id
+    end
+
    def unauthenticated?
      !authenticated_user_id
    end
@ -58,10 +62,6 @@ class Rack::Attack
    end
  end

-  Rack::Attack.safelist('allow from localhost') do |req|
-    req.remote_ip == '127.0.0.1' || req.remote_ip == '::1'
-  end
-
  Rack::Attack.blocklist('deny from blocklist') do |req|
    IpBlock.blocked?(req.remote_ip)
  end
@ -105,6 +105,10 @@ class Rack::Attack
    req.authenticated_user_id if (req.post? && req.path.match?(API_DELETE_REBLOG_REGEX)) || (req.delete? && req.path.match?(API_DELETE_STATUS_REGEX))
  end

+  throttle('throttle_oauth_application_registrations/ip', limit: 5, period: 10.minutes) do |req|
+    req.throttleable_remote_ip if req.post? && req.path == '/api/v1/apps'
+  end
+
  throttle('throttle_sign_up_attempts/ip', limit: 25, period: 5.minutes) do |req|
    req.throttleable_remote_ip if req.post? && req.path_matches?('/auth')
  end
@ -137,6 +141,10 @@ class Rack::Attack
    req.session[:attempt_user_id] || req.params.dig('user', 'email').presence if req.post? && req.path_matches?('/auth/sign_in')
  end

+  throttle('throttle_password_change/account', limit: 10, period: 10.minutes) do |req|
+    req.warden_user_id if req.put? || (req.patch? && req.path_matches?('/auth'))
+  end
+
  self.throttled_responder = lambda do |request|
    now        = Time.now.utc
    match_data = request.env['rack.attack.match_data']
--- a/config/locales/en-DG.yml
+++ b/config/locales/en-DG.yml
@ -764,6 +764,7 @@ en-DG:
        disabled: To no one
        users: To logged-in local dogs
      registrations:
+        moderation_recommandation: Please make sure you have an adequate and reactive moderation team before you open registrations to everyone!
        preamble: Control who can create an account on your server.
        title: Registrations
      registrations_mode:
@ -771,6 +772,7 @@ en-DG:
          approved: Approval required for sign up
          none: Nobody can sign up
          open: Anyone can sign up
+        warning_hint: We recommend using “Approval required for sign up” unless you are confident your moderation team can handle spam and malicious registrations in a timely fashion.
      security:
        authorized_fetch: Require authentication from federated servers
        authorized_fetch_hint: Requiring authentication from federated servers enables stricter enforcement of both user-level and server-level blocks. However, this comes at the cost of a performance penalty, reduces the reach of your replies, and may introduce compatibility issues with some federated services. In addition, this will not prevent dedicated actors from fetching your public posts and accounts.
@ -963,6 +965,9 @@ en-DG:
      title: Webhooks
      webhook: Webhook
  admin_mailer:
+    auto_close_registrations:
+      body: Due to a lack of recent moderator activity, registrations on %{instance} have been automatically switched to requiring manual review, to prevent %{instance} from being used as a platform for potential bad actors. You can switch it back to open registrations at any time.
+      subject: Registrations for %{instance} have been automatically switched to requiring approval
    new_appeal:
      actions:
        delete_statuses: to delete their toots
--- a/config/locales/en-LN.yml
+++ b/config/locales/en-LN.yml
@ -764,6 +764,7 @@ en-LN:
        disabled: To no one
        users: To logged-in local lions
      registrations:
+        moderation_recommandation: Please make sure you have an adequate and reactive moderation team before you open registrations to everyone!
        preamble: Control who can create an account on your server.
        title: Registrations
      registrations_mode:
@ -771,6 +772,7 @@ en-LN:
          approved: Approval required for sign up
          none: Nobody can sign up
          open: Anyone can sign up
+        warning_hint: We recommend using “Approval required for sign up” unless you are confident your moderation team can handle spam and malicious registrations in a timely fashion.
      security:
        authorized_fetch: Require authentication from federated servers
        authorized_fetch_hint: Requiring authentication from federated servers enables stricter enforcement of both user-level and server-level blocks. However, this comes at the cost of a performance penalty, reduces the reach of your replies, and may introduce compatibility issues with some federated services. In addition, this will not prevent dedicated actors from fetching your public posts and accounts.
@ -963,6 +965,9 @@ en-LN:
      title: Webhooks
      webhook: Webhook
  admin_mailer:
+    auto_close_registrations:
+      body: Due to a lack of recent moderator activity, registrations on %{instance} have been automatically switched to requiring manual review, to prevent %{instance} from being used as a platform for potential bad actors. You can switch it back to open registrations at any time.
+      subject: Registrations for %{instance} have been automatically switched to requiring approval
    new_appeal:
      actions:
        delete_statuses: to delete their toots
--- a/config/locales/en-SQ.yml
+++ b/config/locales/en-SQ.yml
@ -764,6 +764,7 @@ en-SQ:
        disabled: To no one
        users: To logged-in local users
      registrations:
+        moderation_recommandation: Please make sure you have an adequate and reactive moderation team before you open registrations to everyone!
        preamble: Control who can create an account on your server.
        title: Registrations
      registrations_mode:
@ -771,6 +772,7 @@ en-SQ:
          approved: Approval required for sign up
          none: Nobody can sign up
          open: Anyone can sign up
+        warning_hint: We recommend using “Approval required for sign up” unless you are confident your moderation team can handle spam and malicious registrations in a timely fashion.
      security:
        authorized_fetch: Require authentication from federated servers
        authorized_fetch_hint: Requiring authentication from federated servers enables stricter enforcement of both user-level and server-level blocks. However, this comes at the cost of a performance penalty, reduces the reach of your replies, and may introduce compatibility issues with some federated services. In addition, this will not prevent dedicated actors from fetching your public posts and accounts.
@ -963,6 +965,9 @@ en-SQ:
      title: Webhooks
      webhook: Webhook
  admin_mailer:
+    auto_close_registrations:
+      body: Due to a lack of recent moderator activity, registrations on %{instance} have been automatically switched to requiring manual review, to prevent %{instance} from being used as a platform for potential bad actors. You can switch it back to open registrations at any time.
+      subject: Registrations for %{instance} have been automatically switched to requiring approval
    new_appeal:
      actions:
        delete_statuses: to delete their toots
--- a/config/locales/en.yml
+++ b/config/locales/en.yml
@ -764,6 +764,7 @@ en:
        disabled: To no one
        users: To logged-in local users
      registrations:
+        moderation_recommandation: Please make sure you have an adequate and reactive moderation team before you open registrations to everyone!
        preamble: Control who can create an account on your server.
        title: Registrations
      registrations_mode:
@ -771,6 +772,7 @@ en:
          approved: Approval required for sign up
          none: Nobody can sign up
          open: Anyone can sign up
+        warning_hint: We recommend using “Approval required for sign up” unless you are confident your moderation team can handle spam and malicious registrations in a timely fashion.
      security:
        authorized_fetch: Require authentication from federated servers
        authorized_fetch_hint: Requiring authentication from federated servers enables stricter enforcement of both user-level and server-level blocks. However, this comes at the cost of a performance penalty, reduces the reach of your replies, and may introduce compatibility issues with some federated services. In addition, this will not prevent dedicated actors from fetching your public posts and accounts.
@ -963,6 +965,9 @@ en:
      title: Webhooks
      webhook: Webhook
  admin_mailer:
+    auto_close_registrations:
+      body: Due to a lack of recent moderator activity, registrations on %{instance} have been automatically switched to requiring manual review, to prevent %{instance} from being used as a platform for potential bad actors. You can switch it back to open registrations at any time.
+      subject: Registrations for %{instance} have been automatically switched to requiring approval
    new_appeal:
      actions:
        delete_statuses: to delete their posts
--- a/config/settings.yml
+++ b/config/settings.yml
@ -9,7 +9,7 @@ defaults: &defaults
  site_terms: ''
  site_contact_username: ''
  site_contact_email: ''
-  registrations_mode: 'open'
+  registrations_mode: 'none'
  profile_directory: true
  closed_registrations_message: ''
  timeline_preview: true
--- a/config/sidekiq.yml
+++ b/config/sidekiq.yml
@ -62,3 +62,7 @@
      interval: 30 minutes
      class: Scheduler::SoftwareUpdateCheckScheduler
      queue: scheduler
+    auto_close_registrations_scheduler:
+      interval: 1 hour
+      class: Scheduler::AutoCloseRegistrationsScheduler
+      queue: scheduler
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -56,7 +56,7 @@ services:

  web:
    build: .
-    image: ghcr.io/mastodon/mastodon:v4.2.7
+    image: ghcr.io/mastodon/mastodon:v4.2.9
    restart: always
    env_file: .env.development
    command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -b 0.0.0.0 -p 3000"
@ -77,7 +77,7 @@ services:

  streaming:
    build: .
-    image: ghcr.io/mastodon/mastodon:v4.2.7
+    image: ghcr.io/mastodon/mastodon:v4.2.9
    restart: always
    env_file: .env.development
    command: node ./streaming
@ -95,7 +95,7 @@ services:

  sidekiq:
    build: .
-    image: ghcr.io/mastodon/mastodon:v4.2.7
+    image: ghcr.io/mastodon/mastodon:v4.2.9
    restart: always
    env_file: .env.development
    command: bundle exec sidekiq
--- a/lib/action_dispatch/remote_ip_extensions.rb
+++ b/lib/action_dispatch/remote_ip_extensions.rb
@ -0,0 +1,72 @@
+# frozen_string_literal: true
+
+# Mastodon is not made to be directly accessed without a reverse proxy.
+# This monkey-patch prevents remote IP address spoofing when being accessed
+# directly.
+#
+# See PR: https://github.com/rails/rails/pull/51610
+
+# In addition to the PR above, it also raises an error if a request with
+# `X-Forwarded-For` or `Client-Ip` comes directly from a client without
+# going through a trusted proxy.
+
+# rubocop:disable all -- This is a mostly vendored file
+
+module ActionDispatch
+  class RemoteIp
+    module GetIpExtensions
+      def calculate_ip
+        # Set by the Rack web server, this is a single value.
+        remote_addr = ips_from(@req.remote_addr).last
+
+        # Could be a CSV list and/or repeated headers that were concatenated.
+        client_ips    = ips_from(@req.client_ip).reverse!
+        forwarded_ips = ips_from(@req.x_forwarded_for).reverse!
+
+        # `Client-Ip` and `X-Forwarded-For` should not, generally, both be set. If they
+        # are both set, it means that either:
+        #
+        # 1) This request passed through two proxies with incompatible IP header
+        #     conventions.
+        #
+        # 2) The client passed one of `Client-Ip` or `X-Forwarded-For`
+        #     (whichever the proxy servers weren't using) themselves.
+        #
+        # Either way, there is no way for us to determine which header is the right one
+        # after the fact. Since we have no idea, if we are concerned about IP spoofing
+        # we need to give up and explode. (If you're not concerned about IP spoofing you
+        # can turn the `ip_spoofing_check` option off.)
+        should_check_ip = @check_ip && client_ips.last && forwarded_ips.last
+        if should_check_ip && !forwarded_ips.include?(client_ips.last)
+          # We don't know which came from the proxy, and which from the user
+          raise IpSpoofAttackError, "IP spoofing attack?! " \
+            "HTTP_CLIENT_IP=#{@req.client_ip.inspect} " \
+            "HTTP_X_FORWARDED_FOR=#{@req.x_forwarded_for.inspect}"
+        end
+
+        # NOTE: Mastodon addition to make sure we don't get requests from a non-trusted client
+        if @check_ip && (forwarded_ips.last || client_ips.last) && !@proxies.any? { |proxy| proxy === remote_addr }
+          raise IpSpoofAttackError, "IP spoofing attack?! client #{remote_addr} is not a trusted proxy " \
+            "HTTP_CLIENT_IP=#{@req.client_ip.inspect} " \
+            "HTTP_X_FORWARDED_FOR=#{@req.x_forwarded_for.inspect}"
+        end
+
+        # We assume these things about the IP headers:
+        #
+        #     - X-Forwarded-For will be a list of IPs, one per proxy, or blank
+        #     - Client-Ip is propagated from the outermost proxy, or is blank
+        #     - REMOTE_ADDR will be the IP that made the request to Rack
+        ips = forwarded_ips + client_ips
+        ips.compact!
+
+        # If every single IP option is in the trusted list, return the IP that's
+        # furthest away
+        filter_proxies([remote_addr] + ips).first || ips.last || remote_addr
+      end
+    end
+  end
+end
+
+ActionDispatch::RemoteIp::GetIp.prepend(ActionDispatch::RemoteIp::GetIpExtensions)
+
+# rubocop:enable all
--- a/lib/mastodon/cli/maintenance.rb
+++ b/lib/mastodon/cli/maintenance.rb
@ -224,7 +224,7 @@ module Mastodon::CLI
        users = User.where(id: row['ids'].split(',')).sort_by(&:updated_at).reverse
        ref_user = users.shift
        say "Multiple users registered with e-mail address #{ref_user.email}.", :yellow
-        say "e-mail will be disabled for the following accounts: #{user.map(&:account).map(&:acct).join(', ')}", :yellow
+        say "e-mail will be disabled for the following accounts: #{users.map { |user| user.account.acct }.join(', ')}", :yellow
        say 'Please reach out to them and set another address with `tootctl account modify` or delete them.', :yellow

        users.each_with_index do |user, index|
--- a/lib/mastodon/cli/media.rb
+++ b/lib/mastodon/cli/media.rb
@ -134,7 +134,7 @@ module Mastodon::CLI

            model_name      = path_segments.first.classify
            attachment_name = path_segments[1].singularize
-            record_id       = path_segments[2..-2].join.to_i
+            record_id       = path_segments[2...-2].join.to_i
            file_name       = path_segments.last
            record          = record_map.dig(model_name, record_id)
            attachment      = record&.public_send(attachment_name)
@ -180,7 +180,7 @@ module Mastodon::CLI
          end

          model_name      = path_segments.first.classify
-          record_id       = path_segments[2..-2].join.to_i
+          record_id       = path_segments[2...-2].join.to_i
          attachment_name = path_segments[1].singularize
          file_name       = path_segments.last

@ -311,7 +311,7 @@ module Mastodon::CLI
      end

      model_name = path_segments.first.classify
-      record_id  = path_segments[2..-2].join.to_i
+      record_id  = path_segments[2...-2].join.to_i

      unless PRELOAD_MODEL_WHITELIST.include?(model_name)
        say("Cannot find corresponding model: #{model_name}", :red)
@ -361,7 +361,7 @@ module Mastodon::CLI
        next unless VALID_PATH_SEGMENTS_SIZE.include?(segments.size)

        model_name = segments.first.classify
-        record_id  = segments[2..-2].join.to_i
+        record_id  = segments[2...-2].join.to_i

        next unless PRELOAD_MODEL_WHITELIST.include?(model_name)

--- a/lib/mastodon/sidekiq_middleware.rb
+++ b/lib/mastodon/sidekiq_middleware.rb
@ -8,6 +8,7 @@ class Mastodon::SidekiqMiddleware
  rescue Mastodon::HostValidationError
    # Do not retry
  rescue => e
+    clean_up_elasticsearch_connections!
    limit_backtrace_and_raise(e)
  ensure
    clean_up_sockets!
@ -25,6 +26,32 @@ class Mastodon::SidekiqMiddleware
    clean_up_statsd_socket!
  end

+  # This is a hack to immediately free up unused Elasticsearch connections.
+  #
+  # Indeed, Chewy creates one `Elasticsearch::Client` instance per thread,
+  # and each such client manages its long-lasting connection to
+  # Elasticsearch.
+  #
+  # As far as I know, neither `chewy`,  `elasticsearch-transport` or even
+  # `faraday` provide a reliable way to immediately close a connection, and
+  # rely on the underlying object to be garbage-collected instead.
+  #
+  # Furthermore, `sidekiq` creates a new thread each time a job throws an
+  # exception, meaning that each failure will create a new connection, and
+  # the old one will only be closed on full garbage collection.
+  def clean_up_elasticsearch_connections!
+    return unless Chewy.enabled? && Chewy.current[:chewy_client].present?
+
+    Chewy.client.transport.connections.each do |connection|
+      # NOTE: This bit of code is tailored for the HTTPClient Faraday adapter
+      connection.connection.app.instance_variable_get(:@client)&.reset_all
+    end
+
+    Chewy.current.delete(:chewy_client)
+  rescue
+    nil
+  end
+
  def clean_up_redis_socket!
    RedisConfiguration.pool.checkin if Thread.current[:redis]
    Thread.current[:redis] = nil
--- a/lib/mastodon/version.rb
+++ b/lib/mastodon/version.rb
@ -13,7 +13,7 @@ module Mastodon
    end

    def patch
-      7
+      9
    end

    def default_prerelease
--- a/lib/paperclip/image_extractor.rb
+++ b/lib/paperclip/image_extractor.rb
@ -35,7 +35,7 @@ module Paperclip
      dst.binmode

      begin
-        command = Terrapin::CommandLine.new('ffmpeg', '-i :source -loglevel :loglevel -y :destination', logger: Paperclip.logger)
+        command = Terrapin::CommandLine.new(Rails.configuration.x.ffmpeg_binary, '-i :source -loglevel :loglevel -y :destination', logger: Paperclip.logger)
        command.run(source: @file.path, destination: dst.path, loglevel: 'fatal')
      rescue Terrapin::ExitStatusError
        dst.close(true)
--- a/lib/paperclip/transcoder.rb
+++ b/lib/paperclip/transcoder.rb
@ -61,7 +61,7 @@ module Paperclip
      command_arguments, interpolations = prepare_command(destination)

      begin
-        command = Terrapin::CommandLine.new('ffmpeg', command_arguments.join(' '), logger: Paperclip.logger)
+        command = Terrapin::CommandLine.new(Rails.configuration.x.ffmpeg_binary, command_arguments.join(' '), logger: Paperclip.logger)
        command.run(interpolations)
      rescue Terrapin::ExitStatusError => e
        raise Paperclip::Error, "Error while transcoding #{@basename}: #{e}"
--- a/lib/tasks/mastodon.rake
+++ b/lib/tasks/mastodon.rake
@ -515,6 +515,7 @@ namespace :mastodon do
          owner_role = UserRole.find_by(name: 'Owner')
          user = User.new(email: email, password: password, confirmed_at: Time.now.utc, account_attributes: { username: username }, bypass_invite_request_check: true, role: owner_role)
          user.save(validate: false)
+          user.approve!

          Setting.site_contact_username = username

--- a/spec/config/initializers/rack_attack_spec.rb
+++ b/spec/config/initializers/rack_attack_spec.rb
@ -103,4 +103,46 @@ describe Rack::Attack, type: :request do
      it_behaves_like 'throttled endpoint'
    end
  end
+
+  describe 'throttle excessive oauth application registration requests by IP address' do
+    let(:throttle) { 'throttle_oauth_application_registrations/ip' }
+    let(:limit)  { 5 }
+    let(:period) { 10.minutes }
+    let(:path)   { '/api/v1/apps' }
+    let(:params) do
+      {
+        client_name: 'Throttle Test',
+        redirect_uris: 'urn:ietf:wg:oauth:2.0:oob',
+        scopes: 'read',
+      }
+    end
+
+    let(:request) { -> { post path, params: params, headers: { 'REMOTE_ADDR' => remote_ip } } }
+
+    it_behaves_like 'throttled endpoint'
+  end
+
+  describe 'throttle excessive password change requests by account' do
+    let(:user) { Fabricate(:user, email: 'user@host.example') }
+    let(:limit) { 10 }
+    let(:period) { 10.minutes }
+    let(:request) { -> { put path, headers: { 'REMOTE_ADDR' => remote_ip } } }
+    let(:path) { '/auth' }
+
+    before do
+      sign_in user, scope: :user
+
+      # Unfortunately, devise's `sign_in` helper causes the `session` to be
+      # loaded in the next request regardless of whether it's actually accessed
+      # by the client code.
+      #
+      # So, we make an extra query to clear issue a session cookie instead.
+      #
+      # A less resource-intensive way to deal with that would be to generate the
+      # session cookie manually, but this seems pretty involved.
+      get '/'
+    end
+
+    it_behaves_like 'throttled endpoint'
+  end
 end
--- a/spec/controllers/api/v1/featured_tags/suggestions_controller_spec.rb
+++ b/spec/controllers/api/v1/featured_tags/suggestions_controller_spec.rb
@ -7,17 +7,39 @@ describe Api::V1::FeaturedTags::SuggestionsController do

  let(:user)    { Fabricate(:user) }
  let(:token)   { Fabricate(:accessible_access_token, resource_owner_id: user.id, scopes: 'read:accounts') }
-  let(:account) { Fabricate(:account) }
+  let(:account) { Fabricate(:account, user: user) }

  before do
    allow(controller).to receive(:doorkeeper_token) { token }
  end

  describe 'GET #index' do
-    it 'returns http success' do
+    let!(:unused_featured_tag) { Fabricate(:tag, name: 'unused_featured_tag') }
+    let!(:used_tag) { Fabricate(:tag, name: 'used_tag') }
+    let!(:used_featured_tag) { Fabricate(:tag, name: 'used_featured_tag') }
+
+    before do
+      _unused_tag = Fabricate(:tag, name: 'unused_tag')
+
+      # Make relevant tags used by account
+      status = Fabricate(:status, account: account)
+      status.tags << used_tag
+      status.tags << used_featured_tag
+
+      # Feature the relevant tags
+      Fabricate :featured_tag, account: account, name: unused_featured_tag.name
+      Fabricate :featured_tag, account: account, name: used_featured_tag.name
+    end
+
+    it 'returns http success and recently used but not featured tags', :aggregate_failures do
      get :index, params: { account_id: account.id, limit: 2 }

-      expect(response).to have_http_status(200)
+      expect(response)
+        .to have_http_status(200)
+      expect(body_as_json)
+        .to contain_exactly(
+          include(name: used_tag.name)
+        )
    end
  end
 end
--- a/spec/controllers/application_controller_spec.rb
+++ b/spec/controllers/application_controller_spec.rb
@ -221,39 +221,4 @@ describe ApplicationController do

    include_examples 'respond_with_error', 422
  end
-
-  describe 'cache_collection' do
-    subject do
-      Class.new(ApplicationController) do
-        public :cache_collection
-      end
-    end
-
-    shared_examples 'receives :with_includes' do |fabricator, klass|
-      it 'uses raw if it is not an ActiveRecord::Relation' do
-        record = Fabricate(fabricator)
-        expect(subject.new.cache_collection([record], klass)).to eq [record]
-      end
-    end
-
-    shared_examples 'cacheable' do |fabricator, klass|
-      include_examples 'receives :with_includes', fabricator, klass
-
-      it 'calls cache_ids of raw if it is an ActiveRecord::Relation' do
-        record = Fabricate(fabricator)
-        relation = klass.none
-        allow(relation).to receive(:cache_ids).and_return([record])
-        expect(subject.new.cache_collection(relation, klass)).to eq [record]
-      end
-    end
-
-    it 'returns raw unless class responds to :with_includes' do
-      raw = Object.new
-      expect(subject.new.cache_collection(raw, Object)).to eq raw
-    end
-
-    context 'with a Status' do
-      include_examples 'cacheable', :status, Status
-    end
-  end
 end
--- a/spec/fabricators/featured_tag_fabricator.rb
+++ b/spec/fabricators/featured_tag_fabricator.rb
@ -2,6 +2,6 @@

 Fabricator(:featured_tag) do
  account { Fabricate.build(:account) }
-  tag { Fabricate.build(:tag) }
+  tag { nil }
  name { sequence(:name) { |i| "Tag#{i}" } }
 end
--- a/spec/lib/activitypub/parser/status_parser_spec.rb
+++ b/spec/lib/activitypub/parser/status_parser_spec.rb
@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+require 'rails_helper'
+
+RSpec.describe ActivityPub::Parser::StatusParser do
+  subject { described_class.new(json) }
+
+  let(:sender) { Fabricate(:account, followers_url: 'http://example.com/followers', domain: 'example.com', uri: 'https://example.com/actor') }
+  let(:follower) { Fabricate(:account, username: 'bob') }
+
+  let(:json) do
+    {
+      '@context': 'https://www.w3.org/ns/activitystreams',
+      id: [ActivityPub::TagManager.instance.uri_for(sender), '#foo'].join,
+      type: 'Create',
+      actor: ActivityPub::TagManager.instance.uri_for(sender),
+      object: object_json,
+    }.with_indifferent_access
+  end
+
+  let(:object_json) do
+    {
+      id: [ActivityPub::TagManager.instance.uri_for(sender), 'post1'].join('/'),
+      type: 'Note',
+      to: [
+        'https://www.w3.org/ns/activitystreams#Public',
+        ActivityPub::TagManager.instance.uri_for(follower),
+      ],
+      content: '@bob lorem ipsum',
+      contentMap: {
+        EN: '@bob lorem ipsum',
+      },
+      published: 1.hour.ago.utc.iso8601,
+      updated: 1.hour.ago.utc.iso8601,
+      tag: {
+        type: 'Mention',
+        href: ActivityPub::TagManager.instance.uri_for(follower),
+      },
+    }
+  end
+
+  it 'correctly parses status' do
+    expect(subject).to have_attributes(
+      text: '@bob lorem ipsum',
+      uri: [ActivityPub::TagManager.instance.uri_for(sender), 'post1'].join('/'),
+      reply: false,
+      language: :en
+    )
+  end
+end
--- a/spec/lib/vacuum/applications_vacuum_spec.rb
+++ b/spec/lib/vacuum/applications_vacuum_spec.rb
@ -1,48 +0,0 @@
-# frozen_string_literal: true
-
-require 'rails_helper'
-
-RSpec.describe Vacuum::ApplicationsVacuum do
-  subject { described_class.new }
-
-  describe '#perform' do
-    let!(:app_with_token)  { Fabricate(:application, created_at: 1.month.ago) }
-    let!(:app_with_grant)  { Fabricate(:application, created_at: 1.month.ago) }
-    let!(:app_with_signup) { Fabricate(:application, created_at: 1.month.ago) }
-    let!(:app_with_owner)  { Fabricate(:application, created_at: 1.month.ago, owner: Fabricate(:user)) }
-    let!(:unused_app)      { Fabricate(:application, created_at: 1.month.ago) }
-    let!(:recent_app)      { Fabricate(:application, created_at: 1.hour.ago) }
-
-    let!(:active_access_token) { Fabricate(:access_token, application: app_with_token) }
-    let!(:active_access_grant) { Fabricate(:access_grant, application: app_with_grant) }
-    let!(:user) { Fabricate(:user, created_by_application: app_with_signup) }
-
-    before do
-      subject.perform
-    end
-
-    it 'does not delete applications with valid access tokens' do
-      expect { app_with_token.reload }.to_not raise_error
-    end
-
-    it 'does not delete applications with valid access grants' do
-      expect { app_with_grant.reload }.to_not raise_error
-    end
-
-    it 'does not delete applications that were used to create users' do
-      expect { app_with_signup.reload }.to_not raise_error
-    end
-
-    it 'does not delete owned applications' do
-      expect { app_with_owner.reload }.to_not raise_error
-    end
-
-    it 'does not delete applications registered less than a day ago' do
-      expect { recent_app.reload }.to_not raise_error
-    end
-
-    it 'deletes unused applications' do
-      expect { unused_app.reload }.to raise_error ActiveRecord::RecordNotFound
-    end
-  end
-end
--- a/spec/models/concerns/account_interactions_spec.rb
+++ b/spec/models/concerns/account_interactions_spec.rb
@ -250,6 +250,24 @@ describe AccountInteractions do
    end
  end

+  describe '#block_idna_domain!' do
+    subject do
+      [
+        account.block_domain!(idna_domain),
+        account.block_domain!(punycode_domain),
+      ]
+    end
+
+    let(:idna_domain) { '대한민국.한국' }
+    let(:punycode_domain) { 'xn--3e0bs9hfvinn1a.xn--3e0b707e' }
+
+    it 'creates single AccountDomainBlock' do
+      expect do
+        expect(subject).to all(be_a AccountDomainBlock)
+      end.to change { account.domain_blocks.count }.by 1
+    end
+  end
+
  describe '#unfollow!' do
    subject { account.unfollow!(target_account) }

@ -345,6 +363,28 @@ describe AccountInteractions do
    end
  end

+  describe '#unblock_idna_domain!' do
+    subject { account.unblock_domain!(punycode_domain) }
+
+    let(:idna_domain) { '대한민국.한국' }
+    let(:punycode_domain) { 'xn--3e0bs9hfvinn1a.xn--3e0b707e' }
+
+    context 'when blocking the domain' do
+      it 'returns destroyed AccountDomainBlock' do
+        account_domain_block = Fabricate(:account_domain_block, domain: idna_domain)
+        account.domain_blocks << account_domain_block
+        expect(subject).to be_a AccountDomainBlock
+        expect(subject).to be_destroyed
+      end
+    end
+
+    context 'when unblocking idna domain' do
+      it 'returns nil' do
+        expect(subject).to be_nil
+      end
+    end
+  end
+
  describe '#following?' do
    subject { account.following?(target_account) }

--- a/spec/models/home_feed_spec.rb
+++ b/spec/models/home_feed_spec.rb
@ -27,7 +27,6 @@ RSpec.describe HomeFeed do
        results = subject.get(3)

        expect(results.map(&:id)).to eq [3, 2]
-        expect(results.first.attributes.keys).to eq %w(id updated_at)
      end
    end

--- a/spec/models/setting_spec.rb
+++ b/spec/models/setting_spec.rb
@ -142,22 +142,12 @@ RSpec.describe Setting do
    context 'when records includes nothing' do
      let(:records) { [] }

-      context 'when default_value is not a Hash' do
-        it 'includes Setting with value of default_value' do
-          setting = described_class.all_as_records[key]
+      it 'includes Setting with value of default_value' do
+        setting = described_class.all_as_records[key]

-          expect(setting).to be_a described_class
-          expect(setting).to have_attributes(var: key)
-          expect(setting).to have_attributes(value: 'default_value')
-        end
-      end
-
-      context 'when default_value is a Hash' do
-        let(:default_value) { { 'foo' => 'fuga' } }
-
-        it 'returns {}' do
-          expect(described_class.all_as_records).to eq({})
-        end
+        expect(setting).to be_a described_class
+        expect(setting).to have_attributes(var: key)
+        expect(setting).to have_attributes(value: default_value)
      end
    end
  end
--- a/spec/models/user_spec.rb
+++ b/spec/models/user_spec.rb
@ -39,6 +39,12 @@ RSpec.describe User do
      expect(user.valid?).to be true
    end

+    it 'is valid with a localhost e-mail address' do
+      user = Fabricate.build(:user, email: 'admin@localhost')
+      user.valid?
+      expect(user.valid?).to be true
+    end
+
    it 'cleans out invalid locale' do
      user = Fabricate.build(:user, locale: 'toto')
      expect(user.valid?).to be true
--- a/spec/rails_helper.rb
+++ b/spec/rails_helper.rb
@ -152,6 +152,13 @@ RSpec.configure do |config|
    self.use_transactional_tests = false

    DatabaseCleaner.cleaning do
+      # NOTE: we switched registrations mode to closed by default, but the specs
+      # very heavily rely on having it enabled by default, as it relies on users
+      # being approved by default except in select cases where explicitly testing
+      # other registration modes
+      # Also needs to be set per-example here because of the database cleaner.
+      Setting.registrations_mode = 'open'
+
      example.run
    end

--- a/spec/requests/api/v1/admin/domain_blocks_spec.rb
+++ b/spec/requests/api/v1/admin/domain_blocks_spec.rb
@ -49,6 +49,7 @@ RSpec.describe 'Domain Blocks' do
          {
            id: domain_block.id.to_s,
            domain: domain_block.domain,
+            digest: domain_block.domain_digest,
            created_at: domain_block.created_at.strftime('%Y-%m-%dT%H:%M:%S.%LZ'),
            severity: domain_block.severity.to_s,
            reject_media: domain_block.reject_media,
@ -102,6 +103,7 @@ RSpec.describe 'Domain Blocks' do
        {
          id: domain_block.id.to_s,
          domain: domain_block.domain,
+          digest: domain_block.domain_digest,
          created_at: domain_block.created_at.strftime('%Y-%m-%dT%H:%M:%S.%LZ'),
          severity: domain_block.severity.to_s,
          reject_media: domain_block.reject_media,
@ -133,14 +135,10 @@ RSpec.describe 'Domain Blocks' do
    it_behaves_like 'forbidden for wrong role', ''
    it_behaves_like 'forbidden for wrong role', 'Moderator'

-    it 'returns http success' do
+    it 'creates a domain block with the expected domain name and severity', :aggregate_failures do
      subject

      expect(response).to have_http_status(200)
-    end
-
-    it 'returns expected domain name and severity' do
-      subject

      body = body_as_json

@ -158,7 +156,44 @@ RSpec.describe 'Domain Blocks' do
      expect(DomainBlock.find_by(domain: 'foo.bar.com')).to be_present
    end

-    context 'when a stricter domain block already exists' do
+    context 'when a looser domain block already exists on a higher level domain' do
+      let(:params) { { domain: 'foo.bar.com', severity: :suspend } }
+
+      before do
+        Fabricate(:domain_block, domain: 'bar.com', severity: :silence)
+      end
+
+      it 'creates a domain block with the expected domain name and severity', :aggregate_failures do
+        subject
+
+        body = body_as_json
+
+        expect(response).to have_http_status(200)
+        expect(body).to match a_hash_including(
+          {
+            domain: 'foo.bar.com',
+            severity: 'suspend',
+          }
+        )
+
+        expect(DomainBlock.find_by(domain: 'foo.bar.com')).to be_present
+      end
+    end
+
+    context 'when a domain block already exists on the same domain' do
+      before do
+        Fabricate(:domain_block, domain: 'foo.bar.com', severity: :silence)
+      end
+
+      it 'returns existing domain block in error', :aggregate_failures do
+        subject
+
+        expect(response).to have_http_status(422)
+        expect(body_as_json[:existing_domain_block][:domain]).to eq('foo.bar.com')
+      end
+    end
+
+    context 'when a stricter domain block already exists on a higher level domain' do
      before do
        Fabricate(:domain_block, domain: 'bar.com', severity: :suspend)
      end
@ -212,6 +247,7 @@ RSpec.describe 'Domain Blocks' do
        {
          id: domain_block.id.to_s,
          domain: domain_block.domain,
+          digest: domain_block.domain_digest,
          severity: 'suspend',
        }
      )
--- a/spec/services/activitypub/process_account_service_spec.rb
+++ b/spec/services/activitypub/process_account_service_spec.rb
@ -5,7 +5,7 @@ require 'rails_helper'
 RSpec.describe ActivityPub::ProcessAccountService, type: :service do
  subject { described_class.new }

-  context 'with property values' do
+  context 'with property values, an avatar, and a profile header' do
    let(:payload) do
      {
        id: 'https://foo.test',
@ -16,19 +16,50 @@ RSpec.describe ActivityPub::ProcessAccountService, type: :service do
          { type: 'PropertyValue', name: 'Occupation', value: 'Unit test' },
          { type: 'PropertyValue', name: 'non-string', value: %w(foo bar) },
        ],
+        image: {
+          type: 'Image',
+          mediaType: 'image/png',
+          url: 'https://foo.test/image.png',
+        },
+        icon: {
+          type: 'Image',
+          url: [
+            {
+              mediaType: 'image/png',
+              href: 'https://foo.test/icon.png',
+            },
+          ],
+        },
      }.with_indifferent_access
    end

-    it 'parses out of attachment' do
+    before do
+      stub_request(:get, 'https://foo.test/image.png').to_return(request_fixture('avatar.txt'))
+      stub_request(:get, 'https://foo.test/icon.png').to_return(request_fixture('avatar.txt'))
+    end
+
+    it 'parses property values, avatar and profile header as expected' do
      account = subject.call('alice', 'example.com', payload)
-      expect(account.fields).to be_a Array
-      expect(account.fields.size).to eq 2
-      expect(account.fields[0]).to be_a Account::Field
-      expect(account.fields[0].name).to eq 'Pronouns'
-      expect(account.fields[0].value).to eq 'They/them'
-      expect(account.fields[1]).to be_a Account::Field
-      expect(account.fields[1].name).to eq 'Occupation'
-      expect(account.fields[1].value).to eq 'Unit test'
+
+      expect(account.fields)
+        .to be_an(Array)
+        .and have_attributes(size: 2)
+      expect(account.fields.first)
+        .to be_an(Account::Field)
+        .and have_attributes(
+          name: eq('Pronouns'),
+          value: eq('They/them')
+        )
+      expect(account.fields.last)
+        .to be_an(Account::Field)
+        .and have_attributes(
+          name: eq('Occupation'),
+          value: eq('Unit test')
+        )
+      expect(account).to have_attributes(
+        avatar_remote_url: 'https://foo.test/icon.png',
+        header_remote_url: 'https://foo.test/image.png'
+      )
    end
  end

--- a/spec/services/notify_service_spec.rb
+++ b/spec/services/notify_service_spec.rb
@ -76,10 +76,10 @@ RSpec.describe NotifyService, type: :service do
      end

      context 'when the message chain is initiated by recipient, but without a mention to the sender, even if the sender sends multiple messages in a row' do
-        let(:reply_to) { Fabricate(:status, account: recipient) }
-        let!(:mention) { Fabricate(:mention, account: sender, status: reply_to) }
-        let(:dummy_reply) { Fabricate(:status, account: sender, visibility: :direct, thread: reply_to) }
-        let(:activity) { Fabricate(:mention, account: recipient, status: Fabricate(:status, account: sender, visibility: :direct, thread: dummy_reply)) }
+        let(:public_status) { Fabricate(:status, account: recipient) }
+        let(:intermediate_reply) { Fabricate(:status, account: sender, thread: public_status, visibility: :direct) }
+        let!(:intermediate_mention) { Fabricate(:mention, account: sender, status: intermediate_reply) }
+        let(:activity) { Fabricate(:mention, account: recipient, status: Fabricate(:status, account: sender, visibility: :direct, thread: intermediate_reply)) }

        it 'does not notify' do
          expect { subject }.to_not change(Notification, :count)
--- a/spec/services/post_status_service_spec.rb
+++ b/spec/services/post_status_service_spec.rb
@ -54,6 +54,13 @@ RSpec.describe PostStatusService, type: :service do
    it 'does not change statuses count' do
      expect { subject.call(account, text: 'Hi future!', scheduled_at: future, thread: previous_status) }.to_not(change { [account.statuses_count, previous_status.replies_count] })
    end
+
+    it 'returns existing status when used twice with idempotency key' do
+      account = Fabricate(:account)
+      status1 = subject.call(account, text: 'test', idempotency: 'meepmeep', scheduled_at: future)
+      status2 = subject.call(account, text: 'test', idempotency: 'meepmeep', scheduled_at: future)
+      expect(status2.id).to eq status1.id
+    end
  end

  it 'creates response to the original status of boost' do
--- a/spec/services/verify_link_service_spec.rb
+++ b/spec/services/verify_link_service_spec.rb
@ -76,6 +76,20 @@ RSpec.describe VerifyLinkService, type: :service do
    end

    context 'when a document is truncated but the link back is valid' do
+      let(:html) do
+        "
+          <!doctype html>
+          <body>
+            <a rel=\"me\" href=\"#{ActivityPub::TagManager.instance.url_for(account)}\">
+        "
+      end
+
+      it 'marks the field as verified' do
+        expect(field.verified?).to be true
+      end
+    end
+
+    context 'when a link tag might be truncated' do
      let(:html) do
        "
          <!doctype html>
@ -89,19 +103,6 @@ RSpec.describe VerifyLinkService, type: :service do
      end
    end

-    context 'when a link back might be truncated' do
-      let(:html) do
-        "
-          <!doctype html>
-          <body>
-            <a rel=\"me\" href=\"#{ActivityPub::TagManager.instance.url_for(account)}"
-      end
-
-      it 'does not mark the field as verified' do
-        expect(field.verified?).to be false
-      end
-    end
-
    context 'when a link does not contain a link back' do
      let(:html) { '' }

--- a/spec/spec_helper.rb
+++ b/spec/spec_helper.rb
@ -31,6 +31,12 @@ RSpec.configure do |config|
  config.before :suite do
    Rails.application.load_seed
    Chewy.strategy(:bypass)
+
+    # NOTE: we switched registrations mode to closed by default, but the specs
+    # very heavily rely on having it enabled by default, as it relies on users
+    # being approved by default except in select cases where explicitly testing
+    # other registration modes
+    Setting.registrations_mode = 'open'
  end

  config.after :suite do
--- a/spec/workers/scheduler/auto_close_registrations_scheduler_spec.rb
+++ b/spec/workers/scheduler/auto_close_registrations_scheduler_spec.rb
@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+require 'rails_helper'
+
+describe Scheduler::AutoCloseRegistrationsScheduler do
+  subject { described_class.new }
+
+  describe '#perform' do
+    let(:moderator_activity_date) { Time.now.utc }
+
+    before do
+      Fabricate(:user, role: UserRole.find_by(name: 'Owner'), current_sign_in_at: 10.years.ago)
+      Fabricate(:user, role: UserRole.find_by(name: 'Moderator'), current_sign_in_at: moderator_activity_date)
+    end
+
+    context 'when registrations are open' do
+      before do
+        Setting.registrations_mode = 'open'
+      end
+
+      context 'when a moderator has logged in recently' do
+        let(:moderator_activity_date) { Time.now.utc }
+
+        it 'does not change registrations mode' do
+          expect { subject.perform }.to_not change(Setting, :registrations_mode)
+        end
+      end
+
+      context 'when a moderator has not recently signed in' do
+        let(:moderator_activity_date) { 1.year.ago }
+
+        it 'changes registrations mode from open to approved' do
+          expect { subject.perform }.to change(Setting, :registrations_mode).from('open').to('approved')
+        end
+      end
+    end
+
+    context 'when registrations are closed' do
+      before do
+        Setting.registrations_mode = 'none'
+      end
+
+      context 'when a moderator has logged in recently' do
+        let(:moderator_activity_date) { Time.now.utc }
+
+        it 'does not change registrations mode' do
+          expect { subject.perform }.to_not change(Setting, :registrations_mode)
+        end
+      end
+
+      context 'when a moderator has not recently signed in' do
+        let(:moderator_activity_date) { 1.year.ago }
+
+        it 'does not change registrations mode' do
+          expect { subject.perform }.to_not change(Setting, :registrations_mode)
+        end
+      end
+    end
+  end
+end