Commit graph

502 commits

Author SHA1 Message Date
Claire f5f17e897b Fix tootctl accounts migrate error due to typo (#23567) 2023-03-13 18:40:18 +01:00
Claire f432db7b9f
Fix sidekiq jobs not triggering Elasticsearch index updates (#24046) 2023-03-12 23:47:55 +01:00
9p4 b715bd8e53
Add refreshing many accounts at once with "tootctl accounts refresh" (#23304) 2023-03-08 17:06:53 +01:00
Jean byroot Boussier 922837dc96
Upgrade to latest redis-rb 4.x and fix deprecations (#23616)
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-03-04 16:38:28 +01:00
Claire 8784498ebf
Fix tootctl accounts migrate error due to typo (#23567) 2023-03-03 20:45:12 +01:00
Nick Schonning 8fd3fc404d
Autofix Rubocop Rails/RootPathnameMethods (#23760) 2023-02-22 09:57:15 +09:00
Nick Schonning 0cfdd1a401
Enable Rubocop Style/StringConcatenation defaults (#23792) 2023-02-22 09:54:36 +09:00
Nick Schonning 59c8d43d94
Autofix Rubocop Style/RescueStandardError (#23745) 2023-02-20 11:01:20 +01:00
Nick Schonning af4c95100c
Autofix Rubocop Style/FormatString (#23743) 2023-02-20 07:58:33 +01:00
Nick Schonning 717683d1c3
Autofix Rubocop remaining Layout rules (#23679) 2023-02-20 06:58:28 +01:00
Nick Schonning bf785df9fe
Audofix Rubocop Style/WordArray (#23739) 2023-02-20 06:14:10 +01:00
Nick Schonning 81ad6c2e39
Autofix Rubocop Style/StringLiterals (#23695) 2023-02-19 07:38:14 +09:00
Nick Schonning 2177daeae9
Autofix Rubocop Style/RedundantBegin (#23703) 2023-02-19 07:09:40 +09:00
Nick Schonning ab7816a414
Autofix Rubocop Style/Lambda (#23696) 2023-02-18 12:39:00 +01:00
Nick Schonning e2a3ebb271
Autofix Rubocop Style/IfUnlessModifier (#23697) 2023-02-18 12:37:47 +01:00
Nick Schonning e2567df860
Enable Lint/RedundantCopDisableDirective (#23687) 2023-02-18 04:30:14 +01:00
Nick Schonning ac59d6f19f
Enable Rubocop Style/NumericLiterals (#23647) 2023-02-18 11:05:57 +09:00
asonix cc7499ed8b Merge branch 'asonix/changes' into v4.1.0-branch 2023-02-11 11:56:18 -06:00
Claire 70c0d754a6
Bump version to 4.1.0 (#23471)
* Bump version to 4.1.0

* Editorialize changelog some more and highlight API changes

* Update changelog
2023-02-10 22:21:23 +01:00
Nick Schonning 11557d1c5a
Apply Rubocop Rails/RootPublicPath (#23447) 2023-02-08 10:38:07 +01:00
Nick Schonning 203739dd3a
Apply Rubocop Performance/StringIdentifierArgument (#23444) 2023-02-08 02:36:20 +01:00
Claire 79ca19e9b2
Bump version to 4.1.0rc3 (#23384) 2023-02-03 16:39:38 +01:00
Claire 2f112432e6
Bump version to 4.1.0rc2 (#23220) 2023-01-25 16:20:54 +01:00
Claire 8180f7ba19
Bump version to 4.1.0rc1 (#23112) 2023-01-20 14:19:12 +01:00
Claire cb4e28f405
Add tootctl domains purge options to select subdomains and keep domain blocks (#22063)
* Add --include-subdomains option to tootctl domains purge

* Add support for '*.' subdomain wildcard patterns in `tootctl domains purge`

* Fix custom emojis deletion not following subdomain and URI options

* Change `tootctl domains purge` to not purge domain blocks unless --purge-domain-blocks is passed

* Refactor `tootctl domains purge`

* Add feedback on deleted domain blocks
2023-01-18 16:50:50 +01:00
Jeong Arm 0e8f8a1a1c
Implement tootctl accounts prune (#18397)
* Implement tootctl accounts prune

* Optimise query

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-01-13 22:34:16 +01:00
Claire 745bdb11a0
Add tootctl accounts migrate (#22330)
* Add tootctl accounts replay-migration

Fixes #22281

* Change `tootctl accounts replay-migration` to `tootctl accounts migrate`
2023-01-13 17:00:23 +01:00
Claire f239d31f23
Add --email and --dry-run options to tootctl accounts delete (#22328) 2022-12-15 14:52:50 +01:00
Evan 78ef635980
Add command to remove avatar and header images of inactive remote accounts from the local database (#22149)
* Add tootctl subcommand media remove-profile-media

* Trigger workflows

* Correcting external linting

* External linting error

* External linting fix

* Merging with remove command

* Linting

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Remove saving a list of purged accounts

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-12-14 19:50:07 +01:00
Claire 55b210b3e5
Fix crash and incorrect behavior in tootctl domains crawl (#19004) 2022-12-13 20:02:32 +01:00
Claire c52263f6f8
Fix deprecation warning in tootctl accounts rotate (#22120) 2022-12-07 14:13:10 +01:00
Claire d587a268fd
Add logging for Rails cache timeouts (#21667)
* Reduce redis cache store connect timeout from default 20 seconds to 5 seconds

* Log cache store errors
2022-11-27 20:37:37 +01:00
asonix 2d681831ce Merge branch 'asonix/changes' into v4.0.2-branch 2022-11-15 09:14:19 -06:00
Eugen Rochko 03b0f3ac83
Bump version to 4.0.2 (#20725) 2022-11-15 03:57:18 +01:00
asonix 023d3beb7e Merge branch 'asonix/changes' into v4.0.1-branch 2022-11-14 17:13:45 -06:00
Eugen Rochko 4415dd6036
Bump version to 4.0.1 (#20696) 2022-11-14 22:21:14 +01:00
asonix 22ec78b226 Merge branch 'asonix/changes' into v4.0.0-branch 2022-11-14 15:18:19 -06:00
Eugen Rochko fb389bd73c
Bump version to 4.0.0 (#20636) 2022-11-14 20:27:12 +01:00
asonix b1ca2423f6 Merge branch 'asonix/changes' into v4.0.0rc4-branch 2022-11-14 10:50:31 -06:00
Eugen Rochko 75299a042c
Bump version to 4.0.0rc4 (#20634) 2022-11-14 08:50:14 +01:00
Claire 457c37e47a
Fix index name in fix-duplicates task (#20632) 2022-11-14 08:33:48 +01:00
Claire bd806a3090
Update fix-duplicates (#20502)
Fixes #19133
2022-11-13 21:01:38 +01:00
asonix 4dd7eca9d4 Merge branch 'asonix/changes' into v4.0.0rc3-branch 2022-11-11 15:29:01 -06:00
Eugen Rochko 53028af10e
Bump version to 4.0.0rc3 (#20378) 2022-11-11 08:39:38 +01:00
asonix 7cfb5498f4 Merge asonix/changes 2022-11-06 12:29:17 -06:00
Eugen Rochko 5187e4e758
Bump version to 4.0.0rc2 (#19831) 2022-11-06 06:59:56 +01:00
Eugen Rochko 8ae0936ddd
Bump version to 4.0.0rc1 (#19473) 2022-10-28 00:26:02 +02:00
Eugen Rochko d7595adbf4
Add --remove-role option to tootctl accounts modify (#19477)
Fix #19152
2022-10-27 14:31:10 +02:00
Jeong Arm 882e54c786
Fix Ambiguous SQL error on tootctl media refresh (#19206) 2022-09-20 23:50:19 +02:00
luzpaz 4aa3b9bd01
Fix typos (#18604)
* Fix typos

Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,pixelx,ro`

* Follow-up typo fix
2022-08-28 17:44:34 +02:00
Eugen Rochko c556c3a0d1
Add admin API for managing canonical e-mail blocks (#19067) 2022-08-28 03:31:54 +02:00
Jeong Arm e682975afd
Add '--days' option to tootctl media refresh (#18425)
* Add '--days' option to tootctl media refresh

* Fix undefined scope
2022-08-25 04:40:17 +02:00
Brayd fc46fa8f99
Minimal adjustments to the short description (#18001)
Minimal adjustments have been made to the short description so that it logically follows the long description
2022-08-13 15:41:12 +02:00
Eugen Rochko 44b2ee3485
Add customizable user roles (#18641)
* Add customizable user roles

* Various fixes and improvements

* Add migration for old settings and fix tootctl role management
2022-07-05 02:41:40 +02:00
Eugen Rochko fbcbf7898f
Bump version to 3.5.3 (#18530) 2022-05-26 23:26:15 +02:00
Eugen Rochko a9b64b24d6
Change algorithm of tootctl search deploy to improve performance (#18463) 2022-05-22 22:16:43 +02:00
Aode (Lion) 75ab44662c Merge branch 'asonix/changes' into v3.5.2-branch 2022-05-13 12:21:10 -05:00
Claire 014065913c
Bump version to 3.5.2 (#18295)
* Bump version to 3.5.2

* Change some entries to be more clear

* Add some extra notes

* Fix line wrap

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-05-04 00:57:42 +02:00
Claire bc19c083ce
Add ability to set approval-based registration through tootctl (#18248)
Fixes #18235

Add `tootctl settings registrations approved` with
optional `--require-reason` switch.
2022-05-02 17:41:34 +02:00
Eugen Rochko 7b0fe4aef9
Fix opening and closing Redis connections instead of using a pool (#18171)
* Fix opening and closing Redis connections instead of using a pool

* Fix Redis connections not being returned to the pool in CLI commands
2022-04-29 22:43:07 +02:00
Eugen Rochko 3917353645
Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Eugen Rochko ed5491e5de
Bump version to 3.5.1 (#18000) 2022-04-08 21:57:24 +02:00
Claire cb45c04d26
Fix migration error handling (#17991) 2022-04-07 20:46:30 +02:00
Claire 5f0fc639da
Fix error re-running some migrations if they get interrupted at the wrong moment (#17989) 2022-04-07 20:17:49 +02:00
Eugen Rochko 6221b36b27
Remove sign-in token authentication, instead send e-mail about new sign-in (#17970) 2022-04-06 20:58:12 +02:00
Aode (Lion) a0142bc315 Merge branch 'asonix/changes' into v3.5.0-branch 2022-03-30 18:55:28 -05:00
Eugen Rochko 8c7223f4ea
Bump version to 3.5.0 (#17911) 2022-03-30 14:52:37 +02:00
Eugen Rochko d7d049aab7
Bump version to 3.5.0rc3 (#17876) 2022-03-26 04:29:36 +01:00
Eugen Rochko 07f8b4d1b1
Bump version to 3.5.0rc2 (#17855) 2022-03-26 02:54:11 +01:00
Claire 3afd59df0f
Fix tootctl email_domain_blocks add (#17842)
Fixes #17831
2022-03-21 19:10:09 +01:00
Claire b07906bdb0
Fix wrong language code for Kurdish languages (#17812) 2022-03-17 01:37:03 +01:00
Eugen Rochko 4bdce2c513
Bump version to 3.5.0rc1 (#17618)
* Bump version to 3.5.0rc1

* Various fixes and improvements

* Update AUTHORS.md

* Various fixes and improvements

* Update README.md
2022-03-15 08:16:45 +01:00
Claire 642528f455
Update fix-duplicates maintenance task (#17731)
* Update fix-duplicates task to 2022_02_10_153119

Also add support for Appeal to AccountMerging#merge_with!

* Update fix-duplicates task to 2022_03_07_094650

* Update fix-duplicates task to 2022_03_09_213005

* Update fix-duplicates task to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060626

* Update fix-duplicates script to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060706

* Update fix-duplicates task to 2022_03_10_060959

* Silence CodeClimate
2022-03-12 08:33:11 +01:00
Eugen Rochko 75e33fd08f
Fix null values being included in some indexes (#17711)
* Fix null values being included in some indexes

* Update lib/mastodon/migration_helpers.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add documentation link to corruption error message

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-12 08:12:57 +01:00
Rens Groothuijsen c439e13e12
Enable importing GIF emojis in CLI (#17706) 2022-03-06 23:41:44 +01:00
Josh Soref b5329e0035
Spelling (#17705)
* spelling: account

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: affiliated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: appearance

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: autosuggest

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: cacheable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: component

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: conversations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: domain.example

Clarify what's distinct and use RFC friendly domain space.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: environment

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeds

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: functional

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: inefficiency

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: not

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: notifications

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: occurring

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: position

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: progress

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: promotable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: reblogging

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: repetitive

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: resolve

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: saturated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: similar

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: strategies

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: success

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: targeting

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnails

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unauthorized

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unsensitizes

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: validations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: various

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
2022-03-06 22:51:40 +01:00
Jeong Arm 3e12abc1fe
Calculate max_id without random vector (#17623) 2022-02-23 16:44:59 +01:00
Eugen Rochko 3413f1c44b
Forward-port version bump to 3.4.6 (#17434) 2022-02-03 14:21:38 +01:00
Claire 54581d43e7
Bump version to 3.4.5 (#17402) 2022-01-31 21:27:40 +01:00
Claire a99adeaad3
Fix edge case in migration helpers that caused crash because of PostgreSQL quirks (#17398) 2022-01-30 22:34:54 +01:00
Claire 8a07ecd377
Remove leftover database columns from Devise::Models::Rememberable (#17191)
* Remove leftover database columns from Devise::Models::Rememberable

* Update fix-duplication maintenance script

* Improve errors/warnings in the fix-duplicates maintenance script
2022-01-23 15:46:30 +01:00
Claire 76761d5fc0
Add ability for admins to delete canonical email blocks (#16644)
* Add admin option to remove canonical email blocks from a deleted account

* Add tootctl canonical_email_blocks to inspect and remove canonical email blocks
2021-12-17 23:02:14 +01:00
Takeshi Umeda 6c8c031bcd
Add remove orphans to tootctl statuses remove (#17067)
* Add remove orphans to tootctl statuses remove

* Add REINDEX and change option from vacuum to compression-database

* Changed to extract the deletion target of conversations to a temporary table

* Support progress bar and exceptions when media remove

* Add continue option

* Fix compression to compress

* Remove skip_remove_orphans
2021-12-12 06:09:14 +01:00
Takeshi Umeda 0ac7efdc50
Fix performance of tootctl statuses remove (#17052)
* Fix performance of tootctl statuses remove

* Fix model class
2021-11-26 22:08:47 +01:00
Takeshi Umeda 06631fdc53
Fix ElasticSearch to Elasticsearch (#17050) 2021-11-26 08:30:02 +01:00
OSAMU SATO 53aca8aecf
Add batch_size option to bin/tootctl search deploy (#17049) 2021-11-26 08:29:53 +01:00
Eugen Rochko 6e50134a42
Add trending links (#16917)
* Add trending links

* Add overriding specific links trendability

* Add link type to preview cards and only trend articles

Change trends review notifications from being sent every 5 minutes to being sent every 2 hours

Change threshold from 5 unique accounts to 15 unique accounts

* Fix tests
2021-11-25 13:07:38 +01:00
Takeshi Umeda 3419d3ec84
Bump chewy from 5.2.0 to 7.2.3 (supports Elasticsearch 7.x) (#16915)
* Bump chewy from 5.2.0 to 7.2.2

* fix style (codeclimate)

* fix style

* fix style

* Bump chewy from 7.2.2 to 7.2.3
2021-11-18 22:02:08 +01:00
Eugen Rochko 2251db42ec
Forward port version bumps to 3.4.2 and 3.4.3 (#16945)
* Bump version to 3.4.2

* Bump version to 3.4.3
2021-11-06 05:32:14 +01:00
Jeong Arm 884c60002e
Skip blocked domains media on tootctl media refresh (#16914) 2021-10-28 19:30:44 +02:00
Jeong Arm 3f5f4273b3
Add optional domain restrict to tootctl accounts cull (#16511)
* Add optional domain restrict to accounts cull

* Use "unless" - codeclimate
2021-10-14 21:09:56 +02:00
Claire a8ef6d24d8
Fix tootctl accounts cull not excluding domains on timeouts and certificate issues (#16433)
Fixes #16410
2021-10-14 21:08:37 +02:00
Claire 4ac78e2a06
Add feature to automatically delete old toots (#16529)
* Add account statuses cleanup policy model

* Record last inspected toot to delete to speed up successive calls to statuses_to_delete

* Add service to cleanup a given account's statuses within a budget

* Add worker to go through account policies and delete old toots

* Fix last inspected status id logic

All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.

* Add tests

* Refactor scheduler and add tests

* Add user interface

* Add support for discriminating based on boosts/favs

* Add UI support for min_reblogs and min_favs, rework UI

* Address first round of review comments

* Replace Snowflake#id_at_start with with_random parameter

* Add tests

* Add tests for StatusesCleanupController

* Rework settings page

* Adjust load-avoiding mechanisms

* Please CodeClimate
2021-08-09 23:11:50 +02:00
Claire 5a1e072517
Change references to tootsuite/mastodon to mastodon/mastodon (#16491)
* Change references to tootsuite/mastodon to mastodon/mastodon

* Remove obsolete test fixture

* Replace occurrences of tootsuite/mastodon with mastodon/mastodon in CHANGELOG

And a few other places
2021-07-13 15:46:20 +02:00
Eugen Rochko 771c9d4ba8
Add ability to skip sign-in token authentication for specific users (#16427)
Remove "active within last two weeks" exception for sign in token requirement

Change admin reset password to lock access until the password is reset
2021-07-08 05:31:28 +02:00
Claire 2e0eac71dd
Add --by-uri option to tootctl domains purge (#16434)
Fixes #16410
2021-07-07 21:17:00 +02:00
asonix 4159889a39 Merge branch 'asonix/changes' into v3.4.1-branch 2021-06-19 16:56:41 -05:00
Eugen Rochko d6486c969f
Bump version to 3.4.1 (#16350) 2021-06-03 04:26:02 +02:00
Claire 11d3c065a5
Fix migration script not being able to run if it fails midway (#16312)
* Fix migration script not being able to run if it fails midway

* Fix old migration script

* Fix old migration script

* Refactor CorruptionError
2021-06-02 19:15:17 +02:00
Claire 526332c545
Fix account deletion sometimes failing because of optimistic locks (#16317)
* Fix account deletion sometimes failing because of optimistic locks

In some rare occasions[1], deleting accounts would fail with a
`StaleObjectError` exception.

Indeed, account deletion manually sets the `AccountStat` values without
handling cases where the optimistic locking on `AccountStat` would fail.

To my knowledge, with the rewrite of account counters in #15913, the
`DeleteAccountService` is now the only place that changes the counters in
a way that is not atomic.

Since in this specific case, we do not care about the previous values of the
account counters, it appears we don't need locking at all for this table
anymore.

[1]: https://discourse.joinmastodon.org/t/account-cant-be-deleted/3602

* Bump MAX_SUPPORTED_VERSION in maintenance script
2021-06-02 17:41:25 +02:00
Claire abf4c2ab21
Fix tootctl search deploy on Ruby 3 (#16346)
Fixes #16344
2021-06-01 17:31:49 +02:00
Eugen Rochko 4c7efdba40
Bump version to 3.4.0 (#16239) 2021-05-16 23:55:07 +02:00
Claire 76064e6608
Update fix-duplicates maintenance script to support latest migrations (#16231)
* Update maintenance script to support latest database migrations

* Update Account#merge_with!
2021-05-12 23:19:44 +02:00
Eugen Rochko 7bd2b54a46
Bump version to 3.4.0rc2 (#16206) 2021-05-10 22:23:48 +02:00
Takeshi Umeda 9b18914c35
Add a Redis environment variable for sidekiq (#16188) 2021-05-09 10:40:17 +02:00
Eugen Rochko d1442a06c3
Bump version to 3.4.0rc1 (#16053) 2021-05-08 17:15:06 +02:00
Claire a5f91a11d0
Fix older migrations on Ruby 3 (#16174) 2021-05-07 15:56:45 +02:00
Takeshi Umeda 25345c90ff
Fix how to change connection pool for rails 6 (#16158)
* Fix how to change connection pool for rails 6

* Fix to match the code elsewhere
2021-05-05 04:41:10 +02:00
Takeshi Umeda 422df9d670
Fix cache redis not being used (#16131) 2021-04-29 15:43:49 +02:00
abcang ddabbbf5a6
Fix DB connection pool settings in CLI (#15983) 2021-03-31 17:46:17 +02:00
Stanislas d33351af3c
tootctl emoji import: case insensitive duplicate check (#15738) 2021-03-24 10:55:16 +01:00
Claire c31c95ffe4
Remove MySQL-specific code from Mastodon::MigrationHelpers (#15924)
Mastodon::MigrationHelpers has been forked from Gitlab a long time ago, but
Mastodon has never supported using a MySQL database.

Removing MySQL support from Mastodon::MigrationHelpers makes it a little easier
to maintain. In particular, it removes code that would need updating with
Rails 6.
2021-03-19 13:14:40 +01:00
Claire 82caed594c
Change deduplication order of tootctl maintenance fix-duplicates (#15923)
Hopefully fixes #15922

Also update support up to latest database schema version
2021-03-19 11:07:56 +01:00
Claire b358229834
Further preparation for Rails 6 (#15916)
* Use ActiveRecord::Result#to_ary instead of deprecated to_hash

They do the same thing, and to_hash has been removed from Rails 6.1

* Explicitly name polymorphic indexes to workaround a bug in Rails 6.1

cf. https://github.com/rails/rails/issues/41693

* Fix incorrect usage of “foreign_key” in migration script

* Use `ActiveModel::Errors#delete` instead of deprecated clear method

* Fix link headers tests on Rails 6.1

Rails 6.1 adds values to the Link header by default, thus it is not a
LinkHeader object anymore. Fix the test to parse the Link header instead
of assuming it is a LinkHeader.
2021-03-19 02:45:34 +01:00
Claire 43eff898a0
Prepare Mastodon for Rails 6 (#15911)
* Fix misuse of foreign_type

* Fix use of removed "add_template_helper"

* Use response.media_type instead of response.content_type in tests

* Fix CSV export controller test on Rails 6

Rails 6 sets a "filename*" field in the Content-Disposition header to
explicitly encode the filename as UTF-8.

This changes checks the first part of the Content-Disposition header so
it matches in both Rails 5 and Rails 6.

* Fix emoji formatting with Rails 6

* Make emoji output more idiomatic and robust

* Switch from redis-rails gem to built-in Rails redis cache storage
2021-03-17 10:09:55 +01:00
Shubhendra Singh Chauhan c8d11b8bdb
Fixed code quality issues (#15541)
* Added .deepsource.toml

* Removed bad use of `alias`

* Fixed operand order in the binary expression

* Prefixed unused method arguments with an underscore

* Replaced the old OpenSSL algorithmic constants with the newer strings initializers.

* Removed unnecessary UTF-8 encoding comment
2021-01-31 21:26:09 +01:00
ThibG 13d5b81579
Fix “tootctl accounts unfollow” (#15639)
Fixes #15635

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-29 18:38:56 +01:00
luigi eb51e43fb4
Optimize some regex matching (#15528)
* Use Regex#match?

* Replace =~ too

* Avoid to call match? from Nil

* Keep value of Regexp.last_match
2021-01-22 10:09:08 +01:00
ThibG 54d4e5252b
Use Rails' index_by where it makes sense (#15542)
* Use Rails' index_by where it makes sense

* Fix tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-12 09:27:38 +01:00
ThibG 9395143126
Fix maintenance script not re-indexing some indexes on textual values (#15515)
* Fix maintenance script not re-indexing some indexes on textual values

Fixes #15475

* Refresh instance view at the end of the maintenance script run

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-09 03:51:32 +01:00
Sophie Parker c7262d12b2
Improve Emoji import (fix #15429) (#15430)
* Improve Emoji import

Skip macOS '._' shadow files in tar archive to speed up import

* Fix codeclimate format issue with whitespace

* Update lib/mastodon/emoji_cli.rb

suggestions from Gargron to improve comment

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Update emoji_cli.rb

Remove extraneous comment (macOS-specific comment now with correct line)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2020-12-30 23:19:55 +01:00
Eugen Rochko 633d175146
Bump version to 3.3.0 (#15433) 2020-12-27 06:00:55 +01:00
ThibG f1f96ebf02
Fix being able to import more than allowed number of follows (#15384)
* Fix being able to import more than allowed number of follows

Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.

This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).

* Catch validation errors to not re-queue failed follows

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-26 23:52:46 +01:00
Eugen Rochko 444b21b55f
Bump version to 3.3.0rc3 (#15411) 2020-12-23 01:36:13 +01:00
ThibG 1cf2c3a810
Fix external user creation failing when invite request text is required (#15405)
* Fix external user creation failing when invite request text is required

Also fixes tootctl-based user creation.

* Add test about invites when invite request text is otherwise required

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-22 17:14:32 +01:00
ThibG c4e860277d
Fix "tootctl accounts fix-duplicates" (#15373)
- `pluck_each` cannot be used this way with `group`
- typo

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-19 13:34:16 +01:00
Eugen Rochko c6598b17d9
Bump version to 3.3.0rc2 (#15365) 2020-12-19 00:28:38 +01:00
ThibG a60d9335d8
Fix resolving accounts sometimes creating duplicate records for a given AP id (#15364)
* Fix ResolveAccountService accepting mismatching acct: URI

* Set attributes that should be updated regardless of suspension

* Fix key fetching

* Automatically merge remote accounts with duplicate `uri`

* Add tests

* Add "tootctl accounts fix-duplicates"

Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and
merge them under the canonical `acct:` URI.

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 23:26:26 +01:00
ThibG 75944a2f2d
Bump max supported schema version in maintenance script (#15359)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 09:46:59 +01:00
ThibG 8485c436d5
Bump max supported schema version in maintenance script (#15349)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-15 12:57:24 +01:00
Eugen Rochko 216b85b053
Fix performance on instances list in admin UI (#15282)
- Reduce duplicate queries
- Remove n+1 queries
- Add accounts count to detailed view
- Add separate action log entry for updating existing domain blocks
2020-12-14 09:06:34 +01:00
ThibG fbe7519243
Fix account merging in maintenance script (#15264)
Also include AccountNote and AccountDeletionRequest to the list of classes
needing to be reassigned the merged account.
2020-12-04 02:23:51 +01:00
ThibG fbff89be03
Fix tootctl ip_blocks add failing if not given a comment (#15263)
Fixes #15261
2020-12-02 21:19:41 +01:00
Eugen Rochko 8e0b98556f
Bump version to 3.3.0rc1 (#15230) 2020-12-01 06:38:05 +01:00
Eugen Rochko e7f1439b97
Bump version to 3.2.1 (#15227) 2020-11-27 18:29:22 +01:00
Eugen Rochko f844386809
Add tootctl accounts merge (#15201)
* Add `tootctl accounts merge`

* Update lib/mastodon/accounts_cli.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-11-23 17:50:16 +01:00
ThibG 1242e57c27
Deal with collation-related index corruption (#14860)
* Add tootctl maintenance fix-duplicates

This tool goes through the database to detect and fix duplicates.
This operation is very slow and may cause data loss (of data that would be
inaccessible without intervention because of the existing index corruptions).
It tries its best to make sensible decisions, and asks the user in some cases.

* Add warning message in db:migrate hook

* Clear Rails cache after being done with database deduplication

Avoids followers hash cache being incorrect, among other things
2020-11-19 17:37:49 +01:00
Eugen Rochko 3134691948
Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
Eugen Rochko 5e1364c448
Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Eugen Rochko ed099d8bdc
Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
Y.Yamashiro abc91e5fa6
Change S3 ACL in "tootctl media remove-orphans --fix-permissions" from fixed value to environment file's value. (#14715) 2020-09-02 00:17:58 +02:00
Eugen Rochko 71085987e8
Add --fix-permissions option to tootctl media remove-orphans (#14383) 2020-09-01 03:33:21 +02:00
dependabot[bot] 8972e5f7f6
Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-09-01 03:04:00 +02:00
Takeshi Umeda 175cd4f8ef
Fix an error when file_file_size is nil in tootctl media remove (#14657) 2020-08-24 18:09:46 +02:00
ThibG 8d217d7231
Improve email address validation (#14565)
* Increase DNS timeout from 1 second to 5 seconds for MX check

1 seconds is rather short when using a recursive DNS resolver which
hasn't got a cached result already available. Use 5 seconds instead,
which is the timeout value we use for outgoing HTTP queries.

* Add more precise error messages for invalid e-mail addresses
2020-08-12 12:40:25 +02:00
ThibG decc5b9a78
Fix tootctl media commands not handling snowflake ids for media_attachments (#14536) 2020-08-10 01:51:06 +02:00
Eugen Rochko 93dd413a47
Bump version to 3.2.0 (#14395) 2020-07-27 00:07:26 +02:00
Eugen Rochko d1a6f8d5a8
Bump version to 3.2.0rc2 (#14382) 2020-07-24 18:47:25 +02:00
Eugen Rochko 6f798eb574
Bump version to 3.2.0rc1 (#14312) 2020-07-15 21:07:27 +02:00
Eugen Rochko 4abe3be321
Change tootctl search deploy algorithm (#14300) 2020-07-14 18:10:35 +02:00