Commit graph

32 commits

Author SHA1 Message Date
KMY
aa2cdc898a Merge remote-tracking branch 'parent/main' into upstream-20240618 2024-06-18 07:43:33 +09:00
Matt Jankowski
665f6f09a0
Add expired/revoked scopes for doorkeeper models via extension modules (#29936) 2024-06-11 08:50:51 +00:00
KMY
13ad383039 Merge remote-tracking branch 'parent/main' into upstream-20240531 2024-05-31 08:27:09 +09:00
Emelia Smith
d20a5c3ec9
Fix: remove broken OAuth Application vacuuming & throttle OAuth Application registrations (#30316)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2024-05-29 14:00:05 +00:00
KMY
dc7778cb8b Fix rubocop 2024-05-24 08:38:58 +09:00
KMY
c546939a40 Merge remote-tracking branch 'parent/main' into upstream-20240524 2024-05-24 08:33:49 +09:00
Matt Jankowski
def6b686ff
Fix Rails/WhereRange cop (#30343) 2024-05-20 09:37:36 +00:00
KMY
44f4a93430 Merge remote-tracking branch 'parent/main' into upstream-20240422 2024-04-22 08:58:37 +09:00
Matt Jankowski
11e0049b08
Use enum-generated scopes/queries for BulkImport (#29975) 2024-04-18 10:13:35 +00:00
KMY
52c5784910 Merge remote-tracking branch 'parent/main' into upstream-20240409 2024-04-09 12:15:51 +09:00
Matt Jankowski
285f63c02e
Use composable query in User.active scope (#29775) 2024-04-08 13:53:49 +00:00
KMY(雪あすか)
b2acc7dbb8
Add: #581 NGワード検出履歴を定期的に削除する処理 (#619)
* Add: #581 NGワード検出履歴を定期的に削除する処理

* マイグレーションコードを修正
2024-02-28 08:34:47 +09:00
KMY(雪あすか)
7d96d5828e
Add: #600 NGルール (#602)
* Wip

* Wip

* Wip: History

* Wip: テストコード作成

* Fix test

* Wip

* Wip

* Wip

* Fix test

* Wip

* Wip

* Wip

* Wip

* なんとか完成、これから動作確認

* spell miss

* Change ng rule timings

* Fix test

* Wip

* Fix test

* Wip

* Fix form

* 表示まわりの改善
2024-02-26 17:45:41 +09:00
KMY(雪あすか)
f2fb829e63
Add: #518 コンテンツキャッシュについて、リアクションのあった投稿は削除しないオプション (#535)
* Add: #518 コンテンツキャッシュについて、リアクションのあった投稿は削除しないオプション

* Fix: 新規登録のテストが特定時刻で落ちる問題
2024-02-13 08:56:55 +09:00
KMY
9fa938eb0f Merge remote-tracking branch 'parent/main' into upstream-20240125 2024-01-25 18:15:21 +09:00
Matt Jankowski
7a1f087659
Add created_before and updated_before scopes to MediaAttachment (#28869) 2024-01-24 10:32:54 +00:00
KMY(雪あすか)
98753287ac
Fix: アンテナに登録された投稿がアンテナ削除時Redisから削除されない問題 (#417)
* Fix: アンテナに登録された投稿がRedisから削除されない問題

* Fix test

* Tootctlに変更

* 処理を共通化
2024-01-04 15:28:22 +09:00
KMY
08a2f557fe Merge remote-tracking branch 'parent/main' into upstream-20231214 2023-12-14 09:21:21 +09:00
Michael Stanclift
afc8550b63
Change preview card deletes to be done using batch method (#28183) 2023-12-11 07:49:10 +00:00
KMY
94c2396a34 Merge remote-tracking branch 'parent/main' into upstream-20231204 2023-12-04 12:04:52 +09:00
Matt Jankowski
bb0efe16e6
Remove default_scope from MediaAttachment class (#28043) 2023-11-30 13:30:35 +00:00
KMY(雪あすか)
f8280ca5d9
Add: #95 リストへの新着投稿通知 (#192)
* Add: テーブル定義、内部処理

* Add: 通知の定期削除処理、自動削除、テスト

* Add: Web画面の表示、設定

* Fix test
2023-10-31 08:59:31 +09:00
jsgoldstein
30c191aaa0
Add new public status index (#26344)
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-08-24 16:40:04 +02:00
Claire
144a406d33
Clean up unused application records (#24871) 2023-07-21 13:13:16 +02:00
Eugen Rochko
bb4756c823
Change files to be deleted in batches instead of one-by-one (#23302) 2023-06-26 14:17:41 +02:00
Claire
6b0942d107
Change AccessTokensVacuum to also delete expired tokens (#24868) 2023-05-11 04:40:03 +02:00
Claire
32a030dd74
Rewrite import feature (#21054) 2023-05-02 12:08:48 +02:00
Nick Schonning
0592937264
Apply Rubocop Rails/WhereNot (#23448)
* Apply Rubocop Rails/WhereNot

* Update spec for where.not
2023-02-08 10:39:57 +01:00
Kaspar V
ae62e5fa53
Fix/remove calling private method with send in model (#22951)
* fix(status): remove send usage for private unlink_from_conversations

- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service

* fix(feeds_vacuum): replace find_in_batches with in_batches

because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
2023-01-11 21:57:24 +01:00
Kaspar V
47f0d7021e
refactor(vacuum statuses): reduce amount of db queries and load for each query - improve performance (#21487)
* refactor(statuses_vacuum): remove dead code - unused

Method is not called inside class and private.
Clean up dead code.

* refactor(statuses_vacuum): make retention_period present test explicit

This private method only hides functionality.
It is best practice to be as explicit as possible.

* refactor(statuses_vacuum): improve query performance

- fix statuses_scope having sub-select for Account.remote scope by
  `joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
  because it is inexplicit, bad practice and even slower than normal
  `.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
  active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
  because in_batches delivers a full blown active record query result,
  in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
  reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
  the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
  another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send

* refactor(status): simplify unlink_from_conversations

- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?

* refactor(status): searchable_by way less sub selects

These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.

This is how it was:

```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
  Status Load (1.6ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (1.5ms)  SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```

This is how it is with this change:

```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
  Status Load (1.7ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (0.7ms)  SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```
2022-11-27 20:41:18 +01:00
Eugen Rochko
bf0ab3e0fa
Fix vacuum scheduler missing lock, locks never expiring (#19458)
Remove vacuuming of orphaned preview cards
2022-10-26 12:10:48 +02:00
Eugen Rochko
5c9abdeff1
Add retention policy for cached content and media (#19232) 2022-09-27 03:08:19 +02:00