Merge remote-tracking branch 'upstream/develop' into fix-prameter-name-of-accounts-update-credentials

2019-10-01 01:39:22 +09:00 · 2019-10-01 01:39:22 +09:00 · a0f101ee80
commit a0f101ee80
parent 8ca4f145a5 7c9b023a91
432 changed files with 20793 additions and 14802 deletions
--- a/.gitignore
+++ b/.gitignore
@ -38,7 +38,12 @@ erl_crash.dump

 # Prevent committing docs files
 /priv/static/doc/*
+docs/generated_config.md

 # Code test coverage
 /cover
 /Elixir.*.coverdata
+
+.idea
+pleroma.iml
+
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@ -151,6 +151,7 @@ amd64:
  only: &release-only
  - master@pleroma/pleroma
  - develop@pleroma/pleroma
+  - /^maint/.*$/@pleroma/pleroma
  artifacts: &release-artifacts
    name: "pleroma-$CI_COMMIT_REF_NAME-$CI_COMMIT_SHORT_SHA-$CI_JOB_NAME"
    paths:
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -4,33 +4,62 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

 ## [Unreleased]
+### Added
+- Refreshing poll results for remote polls
+- Admin API: Add ability to require password reset
+- Mastodon API: Account entities now include `follow_requests_count` (planned Mastodon 3.x addition)
+- Pleroma API: `GET /api/v1/pleroma/accounts/:id/scrobbles` to get a list of recently scrobbled items
+- Pleroma API: `POST /api/v1/pleroma/scrobble` to scrobble a media item
+
+### Changed
+- **Breaking:** Elixir >=1.8 is now required (was >= 1.7)
+- **Breaking:** Admin API: Return link alongside with token on password reset
+- Replaced [pleroma_job_queue](https://git.pleroma.social/pleroma/pleroma_job_queue) and `Pleroma.Web.Federator.RetryQueue` with [Oban](https://github.com/sorentwo/oban) (see [`docs/config.md`](docs/config.md) on migrating customized worker / retry settings)
+- Introduced [quantum](https://github.com/quantum-elixir/quantum-core) job scheduler
+- Admin API: Return `total` when querying for reports
+- Mastodon API: Return `pleroma.direct_conversation_id` when creating a direct message (`POST /api/v1/statuses`)
+- Admin API: Return link alongside with token on password reset
+
+### Fixed
+- Mastodon API: Fix private and direct statuses not being filtered out from the public timeline for an authenticated user (`GET /api/v1/timelines/public`)
+
+## [1.1.0] - 2019-??-??
 ### Security
- OStatus: eliminate the possibility of a protocol downgrade attack.
- OStatus: prevent following locked accounts, bypassing the approval process.
+- Mastodon API: respect post privacy in `/api/v1/statuses/:id/{favourited,reblogged}_by`
+
+### Removed
+- **Breaking:** GNU Social API with Qvitter extensions support
+- Emoji: Remove longfox emojis.
+- Remove `Reply-To` header from report emails for admins.

 ### Changed
 - **Breaking:** Configuration: A setting to explicitly disable the mailer was added, defaulting to true, if you are using a mailer add `config :pleroma, Pleroma.Emails.Mailer, enabled: true` to your config
 - **Breaking:** Configuration: `/media/` is now removed when `base_url` is configured, append `/media/` to your `base_url` config to keep the old behaviour if desired
+- **Breaking:** `/api/pleroma/notifications/read` is moved to `/api/v1/pleroma/notifications/read` and now supports `max_id` and responds with Mastodon API entities.
+- **Breaking:** `/api/pleroma/admin/users/invite_token` now uses `POST`, changed accepted params and returns full invite in json instead of only token string.
+- Configuration: added `config/description.exs`, from which `docs/config.md` is generated
 - Configuration: OpenGraph and TwitterCard providers enabled by default
 - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
+- Mastodon API: `pleroma.thread_muted` key in the Status entity
 - Federation: Return 403 errors when trying to request pages from a user's follower/following collections if they have `hide_followers`/`hide_follows` set
 - NodeInfo: Return `skipThreadContainment` in `metadata` for the `skip_thread_containment` option
 - NodeInfo: Return `mailerEnabled` in `metadata`
 - Mastodon API: Unsubscribe followers when they unfollow a user
+- Mastodon API: `pleroma.thread_muted` key in the Status entity
 - AdminAPI: Add "godmode" while fetching user statuses (i.e. admin can see private statuses)
 - Improve digest email template
+– Pagination: (optional) return `total` alongside with `items` when paginating
+- Add `rel="ugc"` to all links in statuses, to prevent SEO spam

 ### Fixed
- Not being able to pin unlisted posts
- Objects being re-embedded to activities after being updated (e.g faved/reposted). Running 'mix pleroma.database prune_objects' again is advised.
+- Following from Osada
 - Favorites timeline doing database-intensive queries
 - Metadata rendering errors resulting in the entire page being inaccessible
 - `federation_incoming_replies_max_depth` option being ignored in certain cases
- Federation/MediaProxy not working with instances that have wrong certificate order
 - Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`)
+- Mastodon API: Misskey's endless polls being unable to render
 - Mastodon API: Embedded relationships not being properly rendered in the Account entity of Status entity
- Mastodon API: follower/following counters not being nullified, when `hide_follows`/`hide_followers` is set
- Mastodon API: `muted` in the Status entity, using author's account to determine if the tread was muted
+- Mastodon API: Notifications endpoint crashing if one notification failed to render
 - Mastodon API: Add `account_id`, `type`, `offset`, and `limit` to search API (`/api/v1/search` and `/api/v2/search`)
 - Mastodon API, streaming: Fix filtering of notifications based on blocks/mutes/thread mutes
 - ActivityPub C2S: follower/following collection pages being inaccessible even when authentifucated if `hide_followers`/ `hide_follows` was set
@ -38,29 +67,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Rich Media: Parser failing when no TTL can be found by image TTL setters
 - Rich Media: The crawled URL is now spliced into the rich media data.
 - ActivityPub S2S: sharedInbox usage has been mostly aligned with the rules in the AP specification.
- ActivityPub S2S: remote user deletions now work the same as local user deletions.
- ActivityPub S2S: POST requests are now signed with `(request-target)` pseudo-header.
- Not being able to access the Mastodon FE login page on private instances
- Invalid SemVer version generation, when the current branch does not have commits ahead of tag/checked out on a tag
 - Pleroma.Upload base_url was not automatically whitelisted by MediaProxy. Now your custom CDN or file hosting will be accessed directly as expected.
 - Report email not being sent to admins when the reporter is a remote user
- MRF: ensure that subdomain_match calls are case-insensitive
 - Reverse Proxy limiting `max_body_length` was incorrectly defined and only checked `Content-Length` headers which may not be sufficient in some circumstances
- MRF: fix use of unserializable keyword lists in describe() implementations
 - ActivityPub: Deactivated user deletion
+- ActivityPub: Fix `/users/:nickname/inbox` crashing without an authenticated user
+- MRF: fix ability to follow a relay when AntiFollowbotPolicy was enabled

 ### Added
+- Expiring/ephemeral activites. All activities can have expires_at value set, which controls when they should be deleted automatically.
+- Mastodon API: in post_status, the expires_in parameter lets you set the number of seconds until an activity expires. It must be at least one hour.
+- Mastodon API: all status JSON responses contain a `pleroma.expires_at` item which states when an activity will expire. The value is only shown to the user who created the activity. To everyone else it's empty.
+- Configuration: `ActivityExpiration.enabled` controls whether expired activites will get deleted at the appropriate time. Enabled by default.
 - Conversations: Add Pleroma-specific conversation endpoints and status posting extensions. Run the `bump_all_conversations` task again to create the necessary data.
- **Breaking:** MRF describe API, which adds support for exposing configuration information about MRF policies to NodeInfo.
-  Custom modules will need to be updated by adding, at the very least, `def describe, do: {:ok, %{}}` to the MRF policy modules.
 - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
 - MRF: Support for excluding specific domains from Transparency.
 - MRF: Support for filtering posts based on who they mention (`Pleroma.Web.ActivityPub.MRF.MentionPolicy`)
- MRF: Support for filtering posts based on ActivityStreams vocabulary (`Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`)
- MRF (Simple Policy): Support for wildcard domains.
- Support for wildcard domains in user domain blocks setting.
- Configuration: `quarantined_instances` support wildcard domains.
- Configuration: `federation_incoming_replies_max_depth` option
 - Mastodon API: Support for the [`tagged` filter](https://github.com/tootsuite/mastodon/pull/9755) in [`GET /api/v1/accounts/:id/statuses`](https://docs.joinmastodon.org/api/rest/accounts/#get-api-v1-accounts-id-statuses)
 - Mastodon API, streaming: Add support for passing the token in the `Sec-WebSocket-Protocol` header
 - Mastodon API, extension: Ability to reset avatar, profile banner, and background
@ -72,6 +94,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: added `/auth/password` endpoint for password reset with rate limit.
 - Mastodon API: /api/v1/accounts/:id/statuses now supports nicknames or user id
 - Mastodon API: Improve support for the user profile custom fields
+- Mastodon API: follower/following counters are nullified when `hide_follows`/`hide_followers` and `hide_follows_count`/`hide_followers_count` are set
 - Admin API: Return users' tags when querying reports
 - Admin API: Return avatar and display name when querying users
 - Admin API: Allow querying user by ID
@ -87,9 +110,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - ActivityPub: Optional signing of ActivityPub object fetches.
 - Admin API: Endpoint for fetching latest user's statuses
 - Pleroma API: Add `/api/v1/pleroma/accounts/confirmation_resend?email=<email>` for resending account confirmation.
- Relays: Added a task to list relay subscriptions.
- Mix Tasks: `mix pleroma.database fix_likes_collections`
- Federation: Remove `likes` from objects.
+- Pleroma API: Email change endpoint.
+- Admin API: Added moderation log
+- Support for `X-Forwarded-For` and similar HTTP headers which used by reverse proxies to pass a real user IP address to the backend. Must not be enabled unless your instance is behind at least one reverse proxy (such as Nginx, Apache HTTPD or Varnish Cache).
+- Web response cache (currently, enabled for ActivityPub)
+- Mastodon API: Added an endpoint to get multiple statuses by IDs (`GET /api/v1/statuses/?ids[]=1&ids[]=2`)
+- ActivityPub: Add ActivityPub actor's `discoverable` parameter.
+- Admin API: Added moderation log filters (user/start date/end date/search/pagination)

 ### Changed
 - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
@ -97,10 +124,67 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - RichMedia: parsers and their order are configured in `rich_media` config.
 - RichMedia: add the rich media ttl based on image expiration time.

+## [1.0.7] - 2019-09-26
+### Fixed
+- Broken federation on Erlang 22 (previous versions of hackney http client were using an option that got deprecated)
+### Changed
+- ActivityPub: The first page in inboxes/outboxes is no longer embedded.
+
+## [1.0.6] - 2019-08-14
+### Fixed
+- MRF: fix use of unserializable keyword lists in describe() implementations
+- ActivityPub S2S: POST requests are now signed with `(request-target)` pseudo-header.
+
+## [1.0.5] - 2019-08-13
+### Fixed
+- Mastodon API: follower/following counters not being nullified, when `hide_follows`/`hide_followers` is set
+- Mastodon API: `muted` in the Status entity, using author's account to determine if the thread was muted
+- Mastodon API: return the actual profile URL in the Account entity's `url` property when appropriate
+- Templates: properly style anchor tags
+- Objects being re-embedded to activities after being updated (e.g faved/reposted). Running 'mix pleroma.database prune_objects' again is advised.
+- Not being able to access the Mastodon FE login page on private instances
+- MRF: ensure that subdomain_match calls are case-insensitive
+- Fix internal server error when using the healthcheck API.
+
+### Added
+- **Breaking:** MRF describe API, which adds support for exposing configuration information about MRF policies to NodeInfo.
+  Custom modules will need to be updated by adding, at the very least, `def describe, do: {:ok, %{}}` to the MRF policy modules.
+- Relays: Added a task to list relay subscriptions.
+- MRF: Support for filtering posts based on ActivityStreams vocabulary (`Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`)
+- MRF (Simple Policy): Support for wildcard domains.
+- Support for wildcard domains in user domain blocks setting.
+- Configuration: `quarantined_instances` support wildcard domains.
+- Mix Tasks: `mix pleroma.database fix_likes_collections`
+- Configuration: `federation_incoming_replies_max_depth` option
+
 ### Removed
- Emoji: Remove longfox emojis.
- Remove `Reply-To` header from report emails for admins.
- ActivityPub: The `accept_blocks` configuration setting.
+- Federation: Remove `likes` from objects.
+- **Breaking:** ActivityPub: The `accept_blocks` configuration setting.
+
+## [1.0.4] - 2019-08-01
+### Fixed
+- Invalid SemVer version generation, when the current branch does not have commits ahead of tag/checked out on a tag
+
+## [1.0.3] - 2019-07-31
+### Security
+- OStatus: eliminate the possibility of a protocol downgrade attack.
+- OStatus: prevent following locked accounts, bypassing the approval process.
+- TwitterAPI: use CommonAPI to handle remote follows instead of OStatus.
+
+## [1.0.2] - 2019-07-28
+### Fixed
+- Not being able to pin unlisted posts
+- Mastodon API: represent poll IDs as strings
+- MediaProxy: fix matching filenames
+- MediaProxy: fix filename encoding
+- Migrations: fix a sporadic migration failure
+- Metadata rendering errors resulting in the entire page being inaccessible
+- Federation/MediaProxy not working with instances that have wrong certificate order
+- ActivityPub S2S: remote user deletions now work the same as local user deletions.
+
+### Changed
+- Configuration: OpenGraph and TwitterCard providers enabled by default
+- Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text

 ## [1.0.1] - 2019-07-14
 ### Security
@ -191,6 +275,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Configuration: Added `extra_cookie_attrs` for setting non-standard cookie attributes. Defaults to ["SameSite=Lax"] so that remote follows work.
 - Timelines: Messages involving people you have blocked will be excluded from the timeline in all cases instead of just repeats.
 - Admin API: Move the user related API to `api/pleroma/admin/users`
+- Admin API: `POST /api/pleroma/admin/users` will take list of users
 - Pleroma API: Support for emoji tags in `/api/pleroma/emoji` resulting in a breaking API change
 - Mastodon API: Support for `exclude_types`, `limit` and `min_id` in `/api/v1/notifications`
 - Mastodon API: Add `languages` and `registrations` to `/api/v1/instance`
--- a/4
+++ b/4
@ -1,4 +1,4 @@
-FROM rinpatch/elixir:1.9.0-rc.0-alpine as build
+FROM elixir:1.9-alpine as build

 COPY . .

@ -12,7 +12,7 @@ RUN apk add git gcc g++ musl-dev make &&\
 	mkdir release &&\
 	mix release --path release

-FROM alpine:latest
+FROM alpine:3.9

 ARG HOME=/opt/pleroma
 ARG DATA=/var/lib/pleroma
--- a/README.md
+++ b/README.md
@ -8,7 +8,7 @@ Pleroma is a microblogging server software that can federate (= exchange message

 Pleroma is written in Elixir, high-performance and can run on small devices like a Raspberry Pi.

-For clients it supports both the [GNU Social API with Qvitter extensions](https://twitter-api.readthedocs.io/en/latest/index.html) and the [Mastodon client API](https://docs.joinmastodon.org/api/guidelines/).
+For clients it supports the [Mastodon client API](https://docs.joinmastodon.org/api/guidelines/) with Pleroma extensions (see "Pleroma's APIs and Mastodon API extensions" section on <https://docs-develop.pleroma.social>).

 - [Client Applications for Pleroma](https://docs-develop.pleroma.social/clients.html)

--- a/config/config.exs
+++ b/config/config.exs
@ -51,6 +51,24 @@ config :pleroma, Pleroma.Repo,
  telemetry_event: [Pleroma.Repo.Instrumenter],
  migration_lock: nil

+scheduled_jobs =
+  with digest_config <- Application.get_env(:pleroma, :email_notifications)[:digest],
+       true <- digest_config[:active] do
+    [{digest_config[:schedule], {Pleroma.Daemons.DigestEmailDaemon, :perform, []}}]
+  else
+    _ -> []
+  end
+
+scheduled_jobs =
+  scheduled_jobs ++
+    [{"0 */6 * * * *", {Pleroma.Web.Websub, :refresh_subscriptions, []}}]
+
+config :pleroma, Pleroma.Scheduler,
+  global: true,
+  overlap: true,
+  timezone: :utc,
+  jobs: scheduled_jobs
+
 config :pleroma, Pleroma.Captcha,
  enabled: false,
  seconds_valid: 60,
@ -91,6 +109,7 @@ config :pleroma, Pleroma.Uploaders.Local, uploads: "uploads"

 config :pleroma, Pleroma.Uploaders.S3,
  bucket: nil,
+  streaming_enabled: true,
  public_endpoint: "https://s3.amazonaws.com"

 config :pleroma, Pleroma.Uploaders.MDII,
@ -104,7 +123,8 @@ config :pleroma, :emoji,
    # Put groups that have higher priority than defaults here. Example in `docs/config/custom_emoji.md`
    Custom: ["/emoji/*.png", "/emoji/**/*.png"]
  ],
-  default_manifest: "https://git.pleroma.social/pleroma/emoji-index/raw/master/index.json"
+  default_manifest: "https://git.pleroma.social/pleroma/emoji-index/raw/master/index.json",
+  shared_pack_cache_seconds_per_file: 60

 config :pleroma, :uri_schemes,
  valid_schemes: [
@ -258,7 +278,7 @@ config :pleroma, :instance,
  max_account_fields: 10,
  max_remote_account_fields: 20,
  account_field_name_length: 512,
-  account_field_value_length: 512,
+  account_field_value_length: 2048,
  external_user_synchronization: true

 config :pleroma, :markup,
@ -313,6 +333,10 @@ config :pleroma, :activitypub,
  follow_handshake_timeout: 500,
  sign_object_fetches: true

+config :pleroma, :streamer,
+  workers: 3,
+  overflow_workers: 2
+
 config :pleroma, :user, deny_follow_blocked: true

 config :pleroma, :mrf_normalize_markup, scrub_policy: Pleroma.HTML.Scrubber.Default
@ -373,6 +397,8 @@ config :pleroma, :chat, enabled: true

 config :phoenix, :format_encoders, json: Jason

+config :phoenix, :json_library, Jason
+
 config :pleroma, :gopher,
  enabled: false,
  ip: {0, 0, 0, 0},
@ -449,20 +475,26 @@ config :pleroma, Pleroma.User,
    "web"
  ]

-config :pleroma, Pleroma.Web.Federator.RetryQueue,
-  enabled: false,
-  max_jobs: 20,
-  initial_timeout: 30,
-  max_retries: 5
+config :pleroma, Oban,
+  repo: Pleroma.Repo,
+  verbose: false,
+  prune: {:maxlen, 1500},
+  queues: [
+    activity_expiration: 10,
+    federator_incoming: 50,
+    federator_outgoing: 50,
+    web_push: 50,
+    mailer: 10,
+    transmogrifier: 20,
+    scheduled_activities: 10,
+    background: 5
+  ]

-config :pleroma_job_queue, :queues,
-  federator_incoming: 50,
-  federator_outgoing: 50,
-  web_push: 50,
-  mailer: 10,
-  transmogrifier: 20,
-  scheduled_activities: 10,
-  background: 5
+config :pleroma, :workers,
+  retries: [
+    federator_incoming: 5,
+    federator_outgoing: 5
+  ]

 config :pleroma, :fetch_initial_posts,
  enabled: false,
@ -477,7 +509,7 @@ config :auto_linker,
    class: false,
    strip_prefix: false,
    new_window: false,
-    rel: false
+    rel: "ugc"
  ]

 config :pleroma, :ldap,
@ -555,16 +587,15 @@ config :pleroma, :env, Mix.env()
 config :http_signatures,
  adapter: Pleroma.Signature

-config :pleroma, :rate_limit,
-  search: [{1000, 10}, {1000, 30}],
-  app_account_creation: {1_800_000, 25},
-  relations_actions: {10_000, 10},
-  relation_id_action: {60_000, 2},
-  statuses_actions: {10_000, 15},
-  status_id_action: {60_000, 3},
-  password_reset: {1_800_000, 5},
-  account_confirmation_resend: {8_640_000, 5},
-  ap_routes: {60_000, 15}
+config :pleroma, :rate_limit, nil
+
+config :pleroma, Pleroma.ActivityExpiration, enabled: true
+
+config :pleroma, Pleroma.Plugs.RemoteIp, enabled: false
+
+config :pleroma, :web_cache_ttl,
+  activity_pub: nil,
+  activity_pub_question: 30_000

 # Import environment specific config. This must remain at the bottom
 # of this file so it overrides the configuration defined above.
--- a/config/description.exs
+++ b/config/description.exs
--- a/config/docker.exs
+++ b/config/docker.exs
@ -10,7 +10,7 @@ config :pleroma, :instance,
  notify_email: System.get_env("NOTIFY_EMAIL"),
  limit: 5000,
  registrations_open: false,
-  dynamic_configuration: true
+  healthcheck: true

 config :pleroma, Pleroma.Repo,
  adapter: Ecto.Adapters.Postgres,
--- a/config/test.exs
+++ b/config/test.exs
@ -30,7 +30,8 @@ config :pleroma, :instance,
  notify_email: "noreply@example.com",
  skip_thread_containment: false,
  federating: false,
-  external_user_synchronization: false
+  external_user_synchronization: false,
+  static_dir: "test/instance_static/"

 config :pleroma, :activitypub, sign_object_fetches: false

@ -61,7 +62,11 @@ config :web_push_encryption, :vapid_details,

 config :web_push_encryption, :http_client, Pleroma.Web.WebPushHttpClientMock

-config :pleroma_job_queue, disabled: true
+config :pleroma, Oban,
+  queues: false,
+  prune: :disabled
+
+config :pleroma, Pleroma.Scheduler, jobs: []

 config :pleroma, Pleroma.ScheduledActivity,
  daily_user_limit: 2,
@ -86,11 +91,10 @@ config :joken, default_signer: "yU8uHKq+yyAkZ11Hx//jcdacWc8yQ1bxAAGrplzB0Zwwjkp3

 config :pleroma, Pleroma.ReverseProxy.Client, Pleroma.ReverseProxy.ClientMock

-try do
+if File.exists?("./config/test.secret.exs") do
  import_config "test.secret.exs"
-rescue
-  _ ->
-    IO.puts(
-      "You may want to create test.secret.exs to declare custom database connection parameters."
-    )
+else
+  IO.puts(
+    "You may want to create test.secret.exs to declare custom database connection parameters."
+  )
 end
--- a/docs/api/admin_api.md
+++ b/docs/api/admin_api.md
@ -60,9 +60,13 @@ Authentication is required and the user must be an admin.

 - Method: `POST`
 - Params:
-  - `nickname`
-  - `email`
-  - `password`
+  `users`: [
+    {
+      `nickname`,
+      `email`,
+      `password`
+    }
+  ]
 - Response: User’s nickname

 ## `/api/pleroma/admin/users/follow`
@ -220,15 +224,25 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret

 ## `/api/pleroma/admin/users/invite_token`

-### Get an account registration invite token
+### Create an account registration invite token

- Methods: `GET`
+- Methods: `POST`
 - Params:
-  - *optional* `invite` => [
-    - *optional* `max_use` (integer)
-    - *optional* `expires_at` (date string e.g. "2019-04-07")
-  ]
- Response: invite token (base64 string)
+  - *optional* `max_use` (integer)
+  - *optional* `expires_at` (date string e.g. "2019-04-07")
+- Response:
+
+```json
+{
+  "id": integer,
+  "token": string,
+  "used": boolean,
+  "expires_at": date,
+  "uses": integer,
+  "max_use": integer,
+  "invite_type": string (possible values: `one_time`, `reusable`, `date_limited`, `reusable_date_limited`)
+}
+```

 ## `/api/pleroma/admin/users/invites`

@ -294,16 +308,32 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret

 - Methods: `GET`
 - Params: none
- Response: password reset token (base64 string)
+- Response:
+
+```json
+{
+  "token": "base64 reset token",
+  "link": "https://pleroma.social/api/pleroma/password_reset/url-encoded-base64-token"
+}
+```
+
+
+## `/api/pleroma/admin/users/:nickname/force_password_reset`
+
+### Force passord reset for a user with a given nickname
+
+- Methods: `PATCH`
+- Params: none
+- Response: none (code `204`)

 ## `/api/pleroma/admin/reports`
 ### Get a list of reports
 - Method `GET`
 - Params:
-  - `state`: optional, the state of reports. Valid values are `open`, `closed` and `resolved`
-  - `limit`: optional, the number of records to retrieve
-  - `since_id`: optional, returns results that are more recent than the specified id
-  - `max_id`: optional, returns results that are older than the specified id
+  - *optional* `state`: **string** the state of reports. Valid values are `open`, `closed` and `resolved`
+  - *optional* `limit`: **integer** the number of records to retrieve
+  - *optional* `page`: **integer** page number
+  - *optional* `page_size`: **integer** number of log entries per page (default is `50`)
 - Response:
  - On failure: 403 Forbidden error `{"error": "error_msg"}` when requested by anonymous or non-admin
  - On success: JSON, returns a list of reports, where:
@ -313,6 +343,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret

 ```json
 {
+  "total" : 1,
  "reports": [
    {
      "account": {
@ -680,6 +711,7 @@ Compile time settings (need instance reboot):
    }
  ]
 }
+```

 - Response:

@ -694,3 +726,38 @@ Compile time settings (need instance reboot):
  ]
 }
 ```
+
+## `/api/pleroma/admin/moderation_log`
+### Get moderation log
+- Method `GET`
+- Params:
+  - *optional* `page`: **integer** page number
+  - *optional* `page_size`: **integer** number of log entries per page (default is `50`)
+  - *optional* `start_date`: **datetime (ISO 8601)** filter logs by creation date, start from `start_date`. Accepts datetime in ISO 8601 format (YYYY-MM-DDThh:mm:ss), e.g. `2005-08-09T18:31:42`
+  - *optional* `end_date`: **datetime (ISO 8601)** filter logs by creation date, end by from `end_date`. Accepts datetime in ISO 8601 format (YYYY-MM-DDThh:mm:ss), e.g. 2005-08-09T18:31:42
+  - *optional* `user_id`: **integer** filter logs by actor's id
+  - *optional* `search`: **string** search logs by the log message
+- Response:
+
+```json
+[
+  {
+    "data": {
+      "actor": {
+        "id": 1,
+        "nickname": "lain"
+      },
+      "action": "relay_follow"
+    },
+    "time": 1502812026, // timestamp
+    "message": "[2017-08-15 15:47:06] @nick0 followed relay: https://example.org/relay" // log message
+  }
+]
+```
+
+## `POST /api/pleroma/admin/reload_emoji`
+### Reload the instance's custom emoji
+* Method `POST`
+* Authentication: required
+* Params: None
+* Response: JSON, "ok" and 200 status
--- a/docs/api/differences_in_mastoapi_responses.md
+++ b/docs/api/differences_in_mastoapi_responses.md
@ -21,10 +21,13 @@ Adding the parameter `with_muted=true` to the timeline queries will also return
 Has these additional fields under the `pleroma` object:

 - `local`: true if the post was made on the local instance
- `conversation_id`: the ID of the conversation the status is associated with (if any)
+- `conversation_id`: the ID of the AP context the status is associated with (if any)
+- `direct_conversation_id`: the ID of the Mastodon direct message conversation the status is associated with (if any)
 - `in_reply_to_account_acct`: the `acct` property of User entity for replied user (if any)
 - `content`: a map consisting of alternate representations of the `content` property with the key being it's mimetype. Currently the only alternate representation supported is `text/plain`
 - `spoiler_text`: a map consisting of alternate representations of the `spoiler_text` property with the key being it's mimetype. Currently the only alternate representation supported is `text/plain`
+- `expires_at`: a datetime (iso8601) that states when the post will expire (be deleted automatically), or empty if the post won't expire
+- `thread_muted`: true if the thread the post belongs to is muted

 ## Attachments

@ -48,6 +51,8 @@ Has these additional fields under the `pleroma` object:
 - `confirmation_pending`: boolean, true if a new user account is waiting on email confirmation to be activated
 - `hide_followers`: boolean, true when the user has follower hiding enabled
 - `hide_follows`: boolean, true when the user has follow hiding enabled
+- `hide_followers_count`: boolean, true when the user has follower stat hiding enabled
+- `hide_follows_count`: boolean, true when the user has follow stat hiding enabled
 - `settings_store`: A generic map of settings for frontends. Opaque to the backend. Only returned in `verify_credentials` and `update_credentials`
 - `chat_token`: The token needed for Pleroma chat. Only returned in `verify_credentials`
 - `deactivated`: boolean, true when the user is deactivated
@ -86,8 +91,23 @@ Additional parameters can be added to the JSON body/Form data:
 - `content_type`: string, contain the MIME type of the status, it is transformed into HTML by the backend. You can get the list of the supported MIME types with the nodeinfo endpoint.
 - `to`: A list of nicknames (like `lain@soykaf.club` or `lain` on the local server) that will be used to determine who is going to be addressed by this post. Using this will disable the implicit addressing by mentioned names in the `status` body, only the people in the `to` list will be addressed. The normal rules for for post visibility are not affected by this and will still apply.
 - `visibility`: string, besides standard MastoAPI values (`direct`, `private`, `unlisted` or `public`) it can be used to address a List by setting it to `list:LIST_ID`.
+- `expires_in`: The number of seconds the posted activity should expire in. When a posted activity expires it will be deleted from the server, and a delete request for it will be federated. This needs to be longer than an hour.
 - `in_reply_to_conversation_id`: Will reply to a given conversation, addressing only the people who are part of the recipient set of that conversation. Sets the visibility to `direct`.

+## GET `/api/v1/statuses`
+
+An endpoint to get multiple statuses by IDs.
+
+Required parameters:
+
+- `ids`: array of activity ids
+
+Usage example: `GET /api/v1/statuses/?ids[]=1&ids[]=2`.
+
+Returns: array of Status.
+
+The maximum number of statuses is limited to 100 per request.
+
 ## PATCH `/api/v1/update_credentials`

 Additional parameters can be added to the JSON body/Form data:
@ -95,6 +115,8 @@ Additional parameters can be added to the JSON body/Form data:
 - `no_rich_text` - if true, html tags are stripped from all statuses requested from the API
 - `hide_followers` - if true, user's followers will be hidden
 - `hide_follows` - if true, user's follows will be hidden
+- `hide_followers_count` - if true, user's follower count will be hidden
+- `hide_follows_count` - if true, user's follow count will be hidden
 - `hide_favorites` - if true, user's favorites timeline will be hidden
 - `show_role` - if true, user's role (e.g admin, moderator) will be exposed to anyone in the API
 - `default_scope` - the scope returned under `privacy` key in Source subentity
--- a/docs/api/pleroma_api.md
+++ b/docs/api/pleroma_api.md
@ -126,13 +126,14 @@ Request parameters can be passed via [query strings](https://en.wikipedia.org/wi
 ## `/api/pleroma/admin/`…
 See [Admin-API](Admin-API.md)

-## `/api/pleroma/notifications/read`
-### Mark a single notification as read
+## `/api/v1/pleroma/notifications/read`
+### Mark notifications as read
 * Method `POST`
 * Authentication: required
-* Params:
-    * `id`: notification's id
-* Response: JSON. Returns `{"status": "success"}` if the reading was successful, otherwise returns `{"error": "error_msg"}`
+* Params (mutually exclusive):
+    * `id`: a single notification id to read
+    * `max_id`: read all notifications up to this id
+* Response: Notification entity/Array of Notification entities that were read. In case of `max_id`, only the first 80 read notifications will be returned.

 ## `/api/v1/pleroma/accounts/:id/subscribe`
 ### Subscribe to receive notifications for all statuses posted by a user
@ -251,7 +252,7 @@ See [Admin-API](Admin-API.md)
 * Params:
    * `email`: email of that needs to be verified
 * Authentication: not required
-* Response: 204 No Content 
+* Response: 204 No Content

 ## `/api/v1/pleroma/mascot`
 ### Gets user mascot image
@ -320,11 +321,21 @@ See [Admin-API](Admin-API.md)
 }
 ```

+## `/api/pleroma/change_email`
+### Change account email
+* Method `POST`
+* Authentication: required
+* Params:
+    * `password`: user's password
+    * `email`: new email
+* Response: JSON. Returns `{"status": "success"}` if the change was successful, `{"error": "[error message]"}` otherwise
+* Note: Currently, Mastodon has no API for changing email. If they add it in future it might be incompatible with Pleroma.
+
 # Pleroma Conversations

 Pleroma Conversations have the same general structure that Mastodon Conversations have. The behavior differs in the following ways when using these endpoints:

-1. Pleroma Conversations never add or remove recipients, unless explicitly changed by the user. 
+1. Pleroma Conversations never add or remove recipients, unless explicitly changed by the user.
 2. Pleroma Conversations statuses can be requested by Conversation id.
 3. Pleroma Conversations can be replied to.

@ -354,3 +365,109 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
 * Params:
    * `recipients`: A list of ids of users that should receive posts to this conversation. This will replace the current list of recipients, so submit the full list. The owner of owner of the conversation will always be part of the set of recipients, though.
 * Response: JSON, statuses (200 - healthy, 503 unhealthy)
+
+## `GET /api/pleroma/emoji/packs`
+### Lists the custom emoji packs on the server
+* Method `GET`
+* Authentication: not required
+* Params: None
+* Response: JSON, "ok" and 200 status and the JSON hashmap of "pack name" to "pack contents"
+
+## `PUT /api/pleroma/emoji/packs/:name`
+### Creates an empty custom emoji pack
+* Method `PUT`
+* Authentication: required
+* Params: None
+* Response: JSON, "ok" and 200 status or 409 if the pack with that name already exists
+
+## `DELETE /api/pleroma/emoji/packs/:name`
+### Delete a custom emoji pack
+* Method `DELETE`
+* Authentication: required
+* Params: None
+* Response: JSON, "ok" and 200 status or 500 if there was an error deleting the pack
+
+## `POST /api/pleroma/emoji/packs/:name/update_file`
+### Update a file in a custom emoji pack
+* Method `POST`
+* Authentication: required
+* Params: 
+    * if the `action` is `add`, adds an emoji named `shortcode` to the pack `pack_name`,
+      that means that the emoji file needs to be uploaded with the request
+      (thus requiring it to be a multipart request) and be named `file`.
+      There can also be an optional `filename` that will be the new emoji file name
+      (if it's not there, the name will be taken from the uploaded file).
+    * if the `action` is `update`, changes emoji shortcode
+      (from `shortcode` to `new_shortcode` or moves the file (from the current filename to `new_filename`)
+    * if the `action` is `remove`, removes the emoji named `shortcode` and it's associated file
+* Response: JSON, updated "files" section of the pack and 200 status, 409 if the trying to use a shortcode
+  that is already taken, 400 if there was an error with the shortcode, filename or file (additional info
+  in the "error" part of the response JSON)
+
+## `POST /api/pleroma/emoji/packs/:name/update_metadata`
+### Updates (replaces) pack metadata
+* Method `POST`
+* Authentication: required
+* Params: 
+  * `new_data`: new metadata to replace the old one
+* Response: JSON, updated "metadata" section of the pack and 200 status or 400 if there was a
+  problem with the new metadata (the error is specified in the "error" part of the response JSON)
+
+## `POST /api/pleroma/emoji/packs/download_from`
+### Requests the instance to download the pack from another instance
+* Method `POST`
+* Authentication: required
+* Params: 
+  * `instance_address`: the address of the instance to download from
+  * `pack_name`: the pack to download from that instance
+* Response: JSON, "ok" and 200 status if the pack was downloaded, or 500 if there were
+  errors downloading the pack
+
+## `POST /api/pleroma/emoji/packs/list_from`
+### Requests the instance to list the packs from another instance
+* Method `POST`
+* Authentication: required
+* Params:
+  * `instance_address`: the address of the instance to download from
+* Response: JSON with the pack list, same as if the request was made to that instance's
+  list endpoint directly + 200 status
+
+## `GET /api/pleroma/emoji/packs/:name/download_shared`
+### Requests a local pack from the instance
+* Method `GET`
+* Authentication: not required
+* Params: None
+* Response: the archive of the pack with a 200 status code, 403 if the pack is not set as shared,
+  404 if the pack does not exist
+
+## `GET /api/v1/pleroma/accounts/:id/scrobbles`
+### Requests a list of current and recent Listen activities for an account
+* Method `GET`
+* Authentication: not required
+* Params: None
+* Response: An array of media metadata entities.
+* Example response:
+```json
+[
+   {
+       "account": {...},
+       "id": "1234",
+       "title": "Some Title",
+       "artist": "Some Artist",
+       "album": "Some Album",
+       "length": 180000,
+       "created_at": "2019-09-28T12:40:45.000Z"
+   }
+]
+```
+
+## `POST /api/v1/pleroma/scrobble`
+### Creates a new Listen activity for an account
+* Method `POST`
+* Authentication: required
+* Params:
+  * `title`: the title of the media playing
+  * `album`: the album of the media playing [optional]
+  * `artist`: the artist of the media playing [optional]
+  * `length`: the length of the media playing [optional]
+* Response: the newly created media metadata entity representing the Listen activity
--- a/docs/clients.md
+++ b/docs/clients.md
@ -39,7 +39,7 @@ Feel free to contact us to be added to this list!

 ### Nekonium
 - Homepage: [F-Droid Repository](https://repo.gdgd.jp.net/), [Google Play](https://play.google.com/store/apps/details?id=com.apps.nekonium), [Amazon](https://www.amazon.co.jp/dp/B076FXPRBC/)
- Source: <https://git.gdgd.jp.net/lin/nekonium/>
+- Source: <https://gogs.gdgd.jp.net/lin/nekonium>
 - Contact: [@lin@pleroma.gdgd.jp.net](https://pleroma.gdgd.jp.net/users/lin)
 - Platforms: Android
 - Features: Streaming Ready
@ -67,7 +67,7 @@ Feel free to contact us to be added to this list!
 ## Alternative Web Interfaces
 ### Brutaldon
 - Homepage: <https://jfm.carcosa.net/projects/software/brutaldon/>
- Source Code: <https://github.com/jfmcbrayer/brutaldon>
+- Source Code: <https://git.carcosa.net/jmcbray/brutaldon>
 - Contact: [@gcupc@glitch.social](https://glitch.social/users/gcupc)
 - Features: No Streaming

--- a/docs/config.md
+++ b/docs/config.md
@ -8,7 +8,7 @@ If you run Pleroma with ``MIX_ENV=prod`` the file is ``prod.secret.exs``, otherw
 * `filters`: List of `Pleroma.Upload.Filter` to use.
 * `link_name`: When enabled Pleroma will add a `name` parameter to the url of the upload, for example `https://instance.tld/media/corndog.png?name=corndog.png`. This is needed to provide the correct filename in Content-Disposition headers when using filters like `Pleroma.Upload.Filter.Dedupe`
 * `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host.
-* `proxy_remote`: If you\'re using a remote uploader, Pleroma will proxy media requests instead of redirecting to it.
+* `proxy_remote`: If you're using a remote uploader, Pleroma will proxy media requests instead of redirecting to it.
 * `proxy_opts`: Proxy options, see `Pleroma.ReverseProxy` documentation.

 Note: `strip_exif` has been replaced by `Pleroma.Upload.Filter.Mogrify`.
@ -23,6 +23,7 @@ Note: `strip_exif` has been replaced by `Pleroma.Upload.Filter.Mogrify`.
 * `truncated_namespace`: If you use S3 compatible service such as Digital Ocean Spaces or CDN, set folder name or "" etc.
 For example, when using CDN to S3 virtual host format, set "".
 At this time, write CNAME to CDN in public_endpoint.
+* `streaming_enabled`: Enable streaming uploads, when enabled the file will be sent to the server in chunks as it's being read. This may be unsupported by some providers, try disabling this if you have upload problems.

 ## Pleroma.Upload.Filter.Mogrify

@ -135,7 +136,7 @@ config :pleroma, Pleroma.Emails.Mailer,
 * `max_account_fields`: The maximum number of custom fields in the user profile (default: `10`)
 * `max_remote_account_fields`: The maximum number of custom fields in the remote user profile (default: `20`)
 * `account_field_name_length`: An account field name maximum length (default: `512`)
-* `account_field_value_length`: An account field value maximum length (default: `512`)
+* `account_field_value_length`: An account field value maximum length (default: `2048`)
 * `external_user_synchronization`: Enabling following/followers counters synchronization for external users.


@ -400,35 +401,71 @@ You can then do
 curl "http://localhost:4000/api/pleroma/admin/invite_token?admin_token=somerandomtoken"
 ```

-## :pleroma_job_queue
+## Oban

-[Pleroma Job Queue](https://git.pleroma.social/pleroma/pleroma_job_queue) configuration: a list of queues with maximum concurrent jobs.
+[Oban](https://github.com/sorentwo/oban) asynchronous job processor configuration.
+
+Configuration options described in [Oban readme](https://github.com/sorentwo/oban#usage):
+* `repo` - app's Ecto repo (`Pleroma.Repo`)
+* `verbose` - logs verbosity
+* `prune` - non-retryable jobs [pruning settings](https://github.com/sorentwo/oban#pruning) (`:disabled` / `{:maxlen, value}` / `{:maxage, value}`)
+* `queues` - job queues (see below)

 Pleroma has the following queues:

+* `activity_expiration` - Activity expiration
 * `federator_outgoing` - Outgoing federation
 * `federator_incoming` - Incoming federation
-* `mailer` - Email sender, see [`Pleroma.Emails.Mailer`](#pleroma-emails-mailer)
+* `mailer` - Email sender, see [`Pleroma.Emails.Mailer`](#pleromaemailsmailer)
 * `transmogrifier` - Transmogrifier
 * `web_push` - Web push notifications
-* `scheduled_activities` - Scheduled activities, see [`Pleroma.ScheduledActivities`](#pleromascheduledactivity)
+* `scheduled_activities` - Scheduled activities, see [`Pleroma.ScheduledActivity`](#pleromascheduledactivity)

 Example:

 ```elixir
-config :pleroma_job_queue, :queues,
-  federator_incoming: 50,
-  federator_outgoing: 50
+config :pleroma, Oban,
+  repo: Pleroma.Repo,
+  verbose: false,
+  prune: {:maxlen, 1500},
+  queues: [
+    federator_incoming: 50,
+    federator_outgoing: 50
+  ]
 ```

-This config contains two queues: `federator_incoming` and `federator_outgoing`. Both have the `max_jobs` set to `50`.
+This config contains two queues: `federator_incoming` and `federator_outgoing`. Both have the number of max concurrent jobs set to `50`.

-## Pleroma.Web.Federator.RetryQueue
+### Migrating `pleroma_job_queue` settings

-* `enabled`: If set to `true`, failed federation jobs will be retried
-* `max_jobs`: The maximum amount of parallel federation jobs running at the same time.
-* `initial_timeout`: The initial timeout in seconds
-* `max_retries`: The maximum number of times a federation job is retried
+`config :pleroma_job_queue, :queues` is replaced by `config :pleroma, Oban, :queues` and uses the same format (keys are queues' names, values are max concurrent jobs numbers).
+
+### Note on running with PostgreSQL in silent mode
+
+If you are running PostgreSQL in [`silent_mode`](https://postgresqlco.nf/en/doc/param/silent_mode?version=9.1), it's advised to set [`log_destination`](https://postgresqlco.nf/en/doc/param/log_destination?version=9.1) to `syslog`, 
+otherwise `postmaster.log` file may grow because of "you don't own a lock of type ShareLock" warnings (see https://github.com/sorentwo/oban/issues/52). 
+
+## :workers
+
+Includes custom worker options not interpretable directly by `Oban`.
+
+* `retries` — keyword lists where keys are `Oban` queues (see above) and values are numbers of max attempts for failed jobs.
+
+Example:
+
+```elixir
+config :pleroma, :workers,
+  retries: [
+    federator_incoming: 5,
+    federator_outgoing: 5
+  ]
+```
+
+### Migrating `Pleroma.Web.Federator.RetryQueue` settings
+
+* `max_retries` is replaced with `config :pleroma, :workers, retries: [federator_outgoing: 5]`
+* `enabled: false` corresponds to `config :pleroma, :workers, retries: [federator_outgoing: 1]`
+* deprecated options: `max_jobs`, `initial_timeout`

 ## Pleroma.Web.Metadata
 * `providers`: a list of metadata providers to enable. Providers available:
@ -485,16 +522,38 @@ config :auto_linker,
    class: false,
    strip_prefix: false,
    new_window: false,
-    rel: false
+    rel: "ugc"
  ]
 ```

+## Pleroma.Scheduler
+
+Configuration for [Quantum](https://github.com/quantum-elixir/quantum-core) jobs scheduler.
+
+See [Quantum readme](https://github.com/quantum-elixir/quantum-core#usage) for the list of supported options. 
+
+Example:
+
+```elixir
+config :pleroma, Pleroma.Scheduler,
+  global: true,
+  overlap: true,
+  timezone: :utc,
+  jobs: [{"0 */6 * * * *", {Pleroma.Web.Websub, :refresh_subscriptions, []}}]
+```
+
+The above example defines a single job which invokes `Pleroma.Web.Websub.refresh_subscriptions()` every 6 hours ("0 */6 * * * *", [crontab format](https://en.wikipedia.org/wiki/Cron)).
+
 ## Pleroma.ScheduledActivity

 * `daily_user_limit`: the number of scheduled activities a user is allowed to create in a single day (Default: `25`)
 * `total_user_limit`: the number of scheduled activities a user is allowed to create in total (Default: `300`)
 * `enabled`: whether scheduled activities are sent to the job queue to be executed

+## Pleroma.ActivityExpiration
+
+# `enabled`: whether expired activities will be sent to the job queue to be deleted
+
 ## Pleroma.Web.Auth.Authenticator

 * `Pleroma.Web.Auth.PleromaAuthenticator`: default database authenticator
@ -649,6 +708,8 @@ Configure OAuth 2 provider capabilities:
 * `pack_extensions`: A list of file extensions for emojis, when no emoji.txt for a pack is present. Example `[".png", ".gif"]`
 * `groups`: Emojis are ordered in groups (tags). This is an array of key-value pairs where the key is the groupname and the value the location or array of locations. `*` can be used as a wildcard. Example `[Custom: ["/emoji/*.png", "/emoji/custom/*.png"]]`
 * `default_manifest`: Location of the JSON-manifest. This manifest contains information about the emoji-packs you can download. Currently only one manifest can be added (no arrays).
+* `shared_pack_cache_seconds_per_file`: When an emoji pack is shared, the archive is created and cached in
+  memory for this amount of seconds multiplied by the number of files.

 ## Database options

@ -667,6 +728,10 @@ This will probably take a long time.

 ## :rate_limit

+This is an advanced feature and disabled by default.
+
+If your instance is behind a reverse proxy you must enable and configure [`Pleroma.Plugs.RemoteIp`](#pleroma-plugs-remoteip).
+
 A keyword list of rate limiters where a key is a limiter name and value is the limiter configuration. The basic configuration is a tuple where:

 * The first element: `scale` (Integer). The time scale in milliseconds.
@ -684,3 +749,25 @@ Supported rate limiters:
 * `:relation_id_action` for actions on relation with a specific user (follow, unfollow)
 * `:statuses_actions` for create / delete / fav / unfav / reblog / unreblog actions on any statuses
 * `:status_id_action` for fav / unfav or reblog / unreblog actions on the same status by the same user
+
+## :web_cache_ttl
+
+The expiration time for the web responses cache. Values should be in milliseconds or `nil` to disable expiration.
+
+Available caches:
+
+* `:activity_pub` - activity pub routes (except question activities). Defaults to `nil` (no expiration).
+* `:activity_pub_question` - activity pub routes (question activities). Defaults to `30_000` (30 seconds).
+
+## Pleroma.Plugs.RemoteIp
+
+**If your instance is not behind at least one reverse proxy, you should not enable this plug.**
+
+`Pleroma.Plugs.RemoteIp` is a shim to call [`RemoteIp`](https://git.pleroma.social/pleroma/remote_ip) but with runtime configuration.
+
+Available options:
+
+* `enabled` - Enable/disable the plug. Defaults to `false`.
+* `headers` - A list of strings naming the `req_headers` to use when deriving the `remote_ip`. Order does not matter. Defaults to `~w[forwarded x-forwarded-for x-client-ip x-real-ip]`.
+* `proxies` - A list of strings in [CIDR](https://en.wikipedia.org/wiki/CIDR) notation specifying the IPs of known proxies. Defaults to `[]`.
+* `reserved` - Defaults to [localhost](https://en.wikipedia.org/wiki/Localhost) and [private network](https://en.wikipedia.org/wiki/Private_network).
--- a/docs/installation/alpine_linux_en.md
+++ b/docs/installation/alpine_linux_en.md
@ -1,7 +1,9 @@
 # Installing on Alpine Linux
 ## Installation

-This guide is a step-by-step installation guide for Alpine Linux. It also assumes that you have administrative rights, either as root or a user with [sudo permissions](https://www.linode.com/docs/tools-reference/custom-kernels-distros/install-alpine-linux-on-your-linode/#configuration). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su -l <username> -s $SHELL -c 'command'` instead.
+This guide is a step-by-step installation guide for Alpine Linux. The instructions were verified against Alpine v3.10 standard image. You might miss additional dependencies if you use `netboot` instead.
+
+It assumes that you have administrative rights, either as root or a user with [sudo permissions](https://www.linode.com/docs/tools-reference/custom-kernels-distros/install-alpine-linux-on-your-linode/#configuration). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su -l <username> -s $SHELL -c 'command'` instead.

 ### Required packages

@ -20,12 +22,13 @@ This guide is a step-by-step installation guide for Alpine Linux. It also assume

 ### Prepare the system

-* First make sure to have the community repository enabled:
+* The community repository must be enabled in `/etc/apk/repositories`. Depending on which version and mirror you use this looks like `http://alpine.42.fr/v3.10/community`. If you autogenerated the mirror during installation:

 ```shell
-echo "https://nl.alpinelinux.org/alpine/latest-stable/community" | sudo tee -a /etc/apk/repository
+awk 'NR==2' /etc/apk/repositories | sed 's/main/community/' | tee -a /etc/apk/repositories
 ```

+
 * Then update the system, if not already done:

 ```shell
@ -77,7 +80,8 @@ sudo rc-update add postgresql
 * Add a new system user for the Pleroma service:

 ```shell
-sudo adduser -S -s /bin/false -h /opt/pleroma -H pleroma
+sudo addgroup pleroma
+sudo adduser -S -s /bin/false -h /opt/pleroma -H -G pleroma pleroma
 ```

 **Note**: To execute a single command as the Pleroma system user, use `sudo -Hu pleroma command`. You can also switch to a shell by using `sudo -Hu pleroma $SHELL`. If you don’t have and want `sudo` on your system, you can use `su` as root user (UID 0) for a single command by using `su -l pleroma -s $SHELL -c 'command'` and `su -l pleroma -s $SHELL` for starting a shell.
@ -164,7 +168,26 @@ If that doesn’t work, make sure, that nginx is not already running. If it stil
 sudo cp /opt/pleroma/installation/pleroma.nginx /etc/nginx/conf.d/pleroma.conf
 ```

-* Before starting nginx edit the configuration and change it to your needs (e.g. change servername, change cert paths)
+* Before starting nginx edit the configuration and change it to your needs. You must change change `server_name` and the paths to the certificates. You can use `nano` (install with `apk add nano` if missing).
+
+```
+server {
+    server_name    your.domain;
+    listen         80;
+    ...
+}
+
+server {
+    server_name your.domain;
+    listen 443 ssl http2;
+    ...
+    ssl_trusted_certificate   /etc/letsencrypt/live/your.domain/chain.pem;
+    ssl_certificate           /etc/letsencrypt/live/your.domain/fullchain.pem;
+    ssl_certificate_key       /etc/letsencrypt/live/your.domain/privkey.pem;
+    ...
+}
+```
+
 * Enable and start nginx:

 ```shell
--- a/docs/installation/debian_based_jp.md
+++ b/docs/installation/debian_based_jp.md
@ -5,180 +5,179 @@

 ## インストール

-このガイドはDebian Stretchを仮定しています。Ubuntu 16.04でも可能です。
+このガイドはDebian Stretchを利用することを想定しています。Ubuntu 16.04や18.04でもおそらく動作します。また、ユーザはrootもしくはsudoにより管理者権限を持っていることを前提とします。もし、以下の操作をrootユーザで行う場合は、 `sudo` を無視してください。ただし、`sudo -Hu pleroma` のようにユーザを指定している場合には `su <username> -s $SHELL -c 'command'` を代わりに使ってください。

 ### 必要なソフトウェア

- PostgreSQL 9.6+ (postgresql-contrib-9.6 または他のバージョンの PSQL をインストールしてください)
- Elixir 1.5 以上 ([Debianのリポジトリからインストールしないこと！！！ ここからインストールすること！](https://elixir-lang.org/install.html#unix-and-unix-like))。または [asdf](https://github.com/asdf-vm/asdf) を pleroma ユーザーでインストール。
- erlang-dev
+- PostgreSQL 9.6以上 (Ubuntu16.04では9.5しか提供されていないので，[](https://www.postgresql.org/download/linux/ubuntu/)こちらから新しいバージョンを入手してください)
+- postgresql-contrib 9.6以上 (同上)
+- Elixir 1.5 以上 ([Debianのリポジトリからインストールしないこと！！！ ここからインストールすること！](https://elixir-lang.org/install.html#unix-and-unix-like)。または [asdf](https://github.com/asdf-vm/asdf) をpleromaユーザーでインストールしてください)
+  - erlang-dev
 - erlang-tools
 - erlang-parsetools
+- erlang-eldap (LDAP認証を有効化するときのみ必要)
 - erlang-ssh
- erlang-xmerl (Jessieではバックポートからインストールすること！)
+- erlang-xmerl
 - git
 - build-essential
- openssh
- openssl
- nginx prefered (Apacheも動くかもしれませんが、誰もテストしていません！)
- certbot (または何らかのACME Let's encryptクライアント)
+
+#### このガイドで利用している追加パッケージ
+
+- nginx (おすすめです。他のリバースプロキシを使う場合は、参考となる設定をこのリポジトリから探してください)
+- certbot (または何らかのLet's Encrypt向けACMEクライアント)

 ### システムを準備する

 * まずシステムをアップデートしてください。
 ```
-apt update && apt dist-upgrade
+sudo apt update
+sudo apt full-upgrade
 ```

-* 複数のツールとpostgresqlをインストールします。あとで必要になるので。
+* 上記に挙げたパッケージをインストールしておきます。
 ```
-apt install git build-essential openssl ssh sudo postgresql-9.6 postgresql-contrib-9.6
+sudo apt install git build-essential postgresql postgresql-contrib
 ```
-(postgresqlのバージョンは、あなたのディストロにあわせて変えてください。または、バージョン番号がいらないかもしれません。)
+

 ### ElixirとErlangをインストールします

 * Erlangのリポジトリをダウンロードおよびインストールします。
 ```
-wget -P /tmp/ https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb && sudo dpkg -i /tmp/erlang-solutions_1.0_all.deb
+wget -P /tmp/ https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb
+sudo dpkg -i /tmp/erlang-solutions_1.0_all.deb
 ```

 * ElixirとErlangをインストールします、
 ```
-apt update && apt install elixir erlang-dev erlang-parsetools erlang-xmerl erlang-tools erlang-ssh
+sudo apt update
+sudo apt install elixir erlang-dev erlang-parsetools erlang-xmerl erlang-tools erlang-ssh
 ```

 ### Pleroma BE (バックエンド) をインストールします

-*  新しいユーザーを作ります。
-```
-adduser pleroma
-``` 
-(Give it any password you want, make it STRONG)
+*  Pleroma用に新しいユーザーを作ります。

-*  新しいユーザーをsudoグループに入れます。
 ```
-usermod -aG sudo pleroma
+sudo useradd -r -s /bin/false -m -d /var/lib/pleroma -U pleroma
 ```

-*  新しいユーザーに変身し、ホームディレクトリに移動します。
-```
-su pleroma
-cd ~
-```
+**注意**: Pleromaユーザとして単発のコマンドを実行したい場合はは、`sudo -Hu pleroma command` を使ってください。シェルを使いたい場合は `sudo -Hu pleroma $SHELL`です。もし `sudo` を使わない場合は、rootユーザで `su -l pleroma -s $SHELL -c 'command'` とすることでコマンドを、`su -l pleroma -s $SHELL` とすることでシェルを開始できます。

 *  Gitリポジトリをクローンします。
 ```
-git clone -b master https://git.pleroma.social/pleroma/pleroma
+sudo mkdir -p /opt/pleroma
+sudo chown -R pleroma:pleroma /opt/pleroma
+sudo -Hu pleroma git clone -b master https://git.pleroma.social/pleroma/pleroma /opt/pleroma
 ```

 *  新しいディレクトリに移動します。
 ```
-cd pleroma/
+cd /opt/pleroma
 ```

 * Pleromaが依存するパッケージをインストールします。Hexをインストールしてもよいか聞かれたら、yesを入力してください。
 ```
-mix deps.get
+sudo -Hu pleroma mix deps.get
 ```

 * コンフィギュレーションを生成します。
 ```
-mix pleroma.instance gen
+sudo -Hu pleroma mix pleroma.instance gen
 ```
    * rebar3をインストールしてもよいか聞かれたら、yesを入力してください。
-    * この処理には時間がかかります。私もよく分かりませんが、何らかのコンパイルが行われているようです。
-    * あなたのインスタンスについて、いくつかの質問があります。その回答は `config/generated_config.exs` というコンフィギュレーションファイルに保存されます。
+    * このときにpleromaの一部がコンパイルされるため、この処理には時間がかかります。
+    * あなたのインスタンスについて、いくつかの質問されます。この質問により `config/generated_config.exs` という設定ファイルが生成されます。

-**注意**: メディアプロクシを有効にすると回答して、なおかつ、キャッシュのURLは空欄のままにしている場合は、`generated_config.exs` を編集して、`base_url` で始まる行をコメントアウトまたは削除してください。そして、上にある行の `true` の後にあるコンマを消してください。

 * コンフィギュレーションを確認して、もし問題なければ、ファイル名を変更してください。
 ```
 mv config/{generated_config.exs,prod.secret.exs}
 ```

-* これまでのコマンドで、すでに `config/setup_db.psql` というファイルが作られています。このファイルをもとに、データベースを作成します。
+* 先程のコマンドで、すでに `config/setup_db.psql` というファイルが作られています。このファイルをもとに、データベースを作成します。
 ```
-sudo su postgres -c 'psql -f config/setup_db.psql'
+sudo -Hu pleroma mix pleroma.instance gen
 ```

-* そして、データベースのミグレーションを実行します。
+* そして、データベースのマイグレーションを実行します。
 ```
-MIX_ENV=prod mix ecto.migrate
+sudo -Hu pleroma MIX_ENV=prod mix ecto.migrate
 ```

-* Pleromaを起動できるようになりました。
+* これでPleromaを起動できるようになりました。
 ```
-MIX_ENV=prod mix phx.server
+sudo -Hu pleroma MIX_ENV=prod mix phx.server
 ```

-### インストールを終わらせる
+### インストールの最終段階

-あなたの新しいインスタンスを世界に向けて公開するには、nginxまたは何らかのウェブサーバー (プロクシ) を使用する必要があります。また、Pleroma のためにシステムサービスファイルを作成する必要があります。
+あなたの新しいインスタンスを世界に向けて公開するには、nginx等のWebサーバやプロキシサーバをPleromaの前段に使用する必要があります。また、Pleroma のためにシステムサービスファイルを作成する必要があります。

 #### Nginx

 * まだインストールしていないなら、nginxをインストールします。
 ```
-apt install nginx
+sudo apt install nginx
 ```

 * SSLをセットアップします。他の方法でもよいですが、ここではcertbotを説明します。
 certbotを使うならば、まずそれをインストールします。
 ```
-apt install certbot
+sudo apt install certbot
 ```
 そしてセットアップします。
 ```
-mkdir -p /var/lib/letsencrypt/.well-known
-% certbot certonly --email your@emailaddress --webroot -w /var/lib/letsencrypt/ -d yourdomain
+sudo mkdir -p /var/lib/letsencrypt/
+sudo certbot certonly --email <your@emailaddress> -d <yourdomain> --standalone
 ```
-もしうまくいかないときは、先にnginxを設定してください。ssl "on" を "off" に変えてから再試行してください。
+もしうまくいかないときは、nginxが正しく動いていない可能性があります。先にnginxを設定してください。ssl "on" を "off" に変えてから再試行してください。

 ---

-* nginxコンフィギュレーションの例をnginxフォルダーにコピーします。
+* nginxの設定ファイルサンプルをnginxフォルダーにコピーします。
 ```
-cp /home/pleroma/pleroma/installation/pleroma.nginx /etc/nginx/sites-enabled/pleroma.nginx
+sudo cp /opt/pleroma/installation/pleroma.nginx /etc/nginx/sites-available/pleroma.nginx
+sudo ln -s /etc/nginx/sites-available/pleroma.nginx /etc/nginx/sites-enabled/pleroma.nginx
 ```

-* nginxを起動する前に、コンフィギュレーションを編集してください。例えば、サーバー名、証明書のパスなどを変更する必要があります。
+* nginxを起動する前に、設定ファイルを編集してください。例えば、サーバー名、証明書のパスなどを変更する必要があります。
 * nginxを再起動します。
 ```
-systemctl reload nginx.service
+sudo systemctl enable --now nginx.service
 ```

+もし証明書を更新する必要が出てきた場合には、nginxの関連するlocationブロックのコメントアウトを外し、以下のコマンドを動かします。
+
+```
+sudo certbot certonly --email <your@emailaddress> -d <yourdomain> --webroot -w /var/lib/letsencrypt/
+```
+
+#### 他のWebサーバやプロキシ
+これに関してはサンプルが `/opt/pleroma/installation/` にあるので、探してみてください。
+
 #### Systemd サービス

-* サービスファイルの例をコピーします。
+* サービスファイルのサンプルをコピーします。
 ```
-cp /home/pleroma/pleroma/installation/pleroma.service /usr/lib/systemd/system/pleroma.service
+sudo cp /opt/pleroma/installation/pleroma.service /etc/systemd/system/pleroma.service
 ```

-* サービスファイルを変更します。すべてのパスが正しいことを確認してください。また、`[Service]` セクションに以下の行があることを確認してください。
+* サービスファイルを変更します。すべてのパスが正しいことを確認してください
+* サービスを有効化し `pleroma.service` を開始してください
 ```
-Environment="MIX_ENV=prod"
+sudo systemctl enable --now pleroma.service
 ```

-* `pleroma.service` を enable および start してください。
+#### 初期ユーザの作成
+
+新たにインスタンスを作成したら、以下のコマンドにより管理者権限を持った初期ユーザを作成できます。
+
 ```
-systemctl enable --now pleroma.service
+sudo -Hu pleroma MIX_ENV=prod mix pleroma.user new <username> <your@emailaddress> --admin
 ```

-#### モデレーターを作る
-
-新たにユーザーを作ったら、モデレーター権限を与えたいかもしれません。以下のタスクで可能です。
-```
-mix set_moderator username [true|false]
-```
-
-モデレーターはすべてのポストを消すことができます。将来的には他のことも可能になるかもしれません。
-
-#### メディアプロクシを有効にする
-
-`generate_config` でメディアプロクシを有効にしているなら、すでにメディアプロクシが動作しています。あとから設定を変更したいなら、[How to activate mediaproxy](How-to-activate-mediaproxy) を見てください。
-
-#### コンフィギュレーションとカスタマイズ
+#### その他の設定とカスタマイズ

 * [Backup your instance](backup.html)
 * [Configuration tips](general-tips-for-customizing-pleroma-fe.html)
--- a/installation/pleroma-mongooseim.cfg
+++ b/installation/pleroma-mongooseim.cfg
@ -215,7 +215,9 @@
      ]}
  ]},

-  { 5222, ejabberd_c2s, [
+  %% If you want dual stack, you have to clone this entire config stanza
+  %% and change the bind to "::"
+  { {5222, "0.0.0.0"}, ejabberd_c2s, [

 			%%
 			%% If TLS is compiled in and you installed a SSL
@ -246,7 +248,9 @@
  %%			{max_stanza_size, 65536}
  %%		       ]},

-  { 5269, ejabberd_s2s_in, [
+  %% If you want dual stack, you have to clone this entire config stanza
+  %% and change the bind to "::"
+  { {5269, "0.0.0.0"}, ejabberd_s2s_in, [
 			   {shaper, s2s_shaper},
 			   {max_stanza_size, 131072},
 			   {protocol_options, ["no_sslv3"]}
--- a/installation/pleroma.nginx
+++ b/installation/pleroma.nginx
@ -70,27 +70,28 @@ server {
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

-	# this is explicitly IPv4 since Pleroma.Web.Endpoint binds on IPv4 only
-	# and `localhost.` resolves to [::0] on some systems: see issue #930
+        # this is explicitly IPv4 since Pleroma.Web.Endpoint binds on IPv4 only
+        # and `localhost.` resolves to [::0] on some systems: see issue #930
        proxy_pass http://127.0.0.1:4000;

        client_max_body_size 16m;
    }

    location ~ ^/(media|proxy) {
-        proxy_cache pleroma_media_cache;
+        proxy_cache        pleroma_media_cache;
        slice              1m;
        proxy_cache_key    $host$uri$is_args$args$slice_range;
        proxy_set_header   Range $slice_range;
        proxy_http_version 1.1;
        proxy_cache_valid  200 206 301 304 1h;
-        proxy_cache_lock on;
+        proxy_cache_lock   on;
        proxy_ignore_client_abort on;
-        proxy_buffering on;
+        proxy_buffering    on;
        chunked_transfer_encoding on;
        proxy_ignore_headers Cache-Control;
-        proxy_hide_header Cache-Control;
-        proxy_pass http://localhost:4000;
+        proxy_hide_header  Cache-Control;
+        proxy_pass         http://127.0.0.1:4000;
    }
 }
--- a/lib/mix/pleroma.ex
+++ b/lib/mix/pleroma.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Mix.Pleroma do
--- a/lib/mix/tasks/pleroma/benchmark.ex
+++ b/lib/mix/tasks/pleroma/benchmark.ex
@ -27,7 +27,7 @@ defmodule Mix.Tasks.Pleroma.Benchmark do
    })
  end

-  def run(["render_timeline", nickname]) do
+  def run(["render_timeline", nickname | _] = args) do
    start_pleroma()
    user = Pleroma.User.get_by_nickname(nickname)

@ -37,33 +37,37 @@ defmodule Mix.Tasks.Pleroma.Benchmark do
      |> Map.put("blocking_user", user)
      |> Map.put("muting_user", user)
      |> Map.put("user", user)
-      |> Map.put("limit", 80)
+      |> Map.put("limit", 4096)
      |> Pleroma.Web.ActivityPub.ActivityPub.fetch_public_activities()
      |> Enum.reverse()

    inputs = %{
-      "One activity" => Enum.take_random(activities, 1),
-      "Ten activities" => Enum.take_random(activities, 10),
-      "Twenty activities" => Enum.take_random(activities, 20),
-      "Forty activities" => Enum.take_random(activities, 40),
-      "Eighty activities" => Enum.take_random(activities, 80)
+      "1 activity" => Enum.take_random(activities, 1),
+      "10 activities" => Enum.take_random(activities, 10),
+      "20 activities" => Enum.take_random(activities, 20),
+      "40 activities" => Enum.take_random(activities, 40),
+      "80 activities" => Enum.take_random(activities, 80)
    }

+    inputs =
+      if Enum.at(args, 2) == "extended" do
+        Map.merge(inputs, %{
+          "200 activities" => Enum.take_random(activities, 200),
+          "500 activities" => Enum.take_random(activities, 500),
+          "2000 activities" => Enum.take_random(activities, 2000),
+          "4096 activities" => Enum.take_random(activities, 4096)
+        })
+      else
+        inputs
+      end
+
    Benchee.run(
      %{
-        "Parallel rendering" => fn activities ->
-          Pleroma.Web.MastodonAPI.StatusView.render("index.json", %{
-            activities: activities,
-            for: user,
-            as: :activity
-          })
-        end,
        "Standart rendering" => fn activities ->
          Pleroma.Web.MastodonAPI.StatusView.render("index.json", %{
            activities: activities,
            for: user,
-            as: :activity,
-            parallel: false
+            as: :activity
          })
        end
      },
--- a/lib/mix/tasks/pleroma/database.ex
+++ b/lib/mix/tasks/pleroma/database.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Mix.Tasks.Pleroma.Database do
--- a/lib/mix/tasks/pleroma/docs.ex
+++ b/lib/mix/tasks/pleroma/docs.ex
@ -0,0 +1,42 @@
+defmodule Mix.Tasks.Pleroma.Docs do
+  use Mix.Task
+  import Mix.Pleroma
+
+  @shortdoc "Generates docs from descriptions.exs"
+  @moduledoc """
+  Generates docs from `descriptions.exs`.
+
+  Supports two formats: `markdown` and `json`.
+
+  ## Generate Markdown docs
+
+  `mix pleroma.docs`
+
+  ## Generate JSON docs
+
+  `mix pleroma.docs json`
+  """
+
+  def run(["json"]) do
+    do_run(Pleroma.Docs.JSON)
+  end
+
+  def run(_) do
+    do_run(Pleroma.Docs.Markdown)
+  end
+
+  defp do_run(implementation) do
+    start_pleroma()
+
+    with {descriptions, _paths} <- Mix.Config.eval!("config/description.exs"),
+         {:ok, file_path} <-
+           Pleroma.Docs.Generator.process(
+             implementation,
+             descriptions[:pleroma][:config_description]
+           ) do
+      type = if implementation == Pleroma.Docs.Markdown, do: "Markdown", else: "JSON"
+
+      Mix.shell().info([:green, "#{type} docs successfully generated to #{file_path}."])
+    end
+  end
+end
--- a/lib/mix/tasks/pleroma/ecto/ecto.ex
+++ b/lib/mix/tasks/pleroma/ecto/ecto.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-onl

 defmodule Mix.Tasks.Pleroma.Ecto do
--- a/lib/mix/tasks/pleroma/ecto/migrate.ex
+++ b/lib/mix/tasks/pleroma/ecto/migrate.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-onl

 defmodule Mix.Tasks.Pleroma.Ecto.Migrate do
--- a/lib/mix/tasks/pleroma/ecto/rollback.ex
+++ b/lib/mix/tasks/pleroma/ecto/rollback.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-onl

 defmodule Mix.Tasks.Pleroma.Ecto.Rollback do
--- a/lib/mix/tasks/pleroma/emoji.ex
+++ b/lib/mix/tasks/pleroma/emoji.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Mix.Tasks.Pleroma.Emoji do
@ -235,7 +235,7 @@ defmodule Mix.Tasks.Pleroma.Emoji do
        cwd: tmp_pack_dir
      )

-    emoji_map = Pleroma.Emoji.make_shortcode_to_file_map(tmp_pack_dir, exts)
+    emoji_map = Pleroma.Emoji.Loader.make_shortcode_to_file_map(tmp_pack_dir, exts)

    File.write!(files_name, Jason.encode!(emoji_map, pretty: true))

--- a/lib/mix/tasks/pleroma/instance.ex
+++ b/lib/mix/tasks/pleroma/instance.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Mix.Tasks.Pleroma.Instance do
--- a/lib/mix/tasks/pleroma/relay.ex
+++ b/lib/mix/tasks/pleroma/relay.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Mix.Tasks.Pleroma.Relay do
--- a/lib/mix/tasks/pleroma/uploads.ex
+++ b/lib/mix/tasks/pleroma/uploads.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Mix.Tasks.Pleroma.Uploads do
--- a/lib/mix/tasks/pleroma/user.ex
+++ b/lib/mix/tasks/pleroma/user.ex
@ -1,10 +1,9 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Mix.Tasks.Pleroma.User do
  use Mix.Task
-  import Ecto.Changeset
  import Mix.Pleroma
  alias Pleroma.User
  alias Pleroma.UserInviteToken
@ -228,9 +227,9 @@ defmodule Mix.Tasks.Pleroma.User do
      shell_info("Deactivating #{user.nickname}")
      User.deactivate(user)

-      {:ok, friends} = User.get_friends(user)
-
-      Enum.each(friends, fn friend ->
+      user
+      |> User.get_friends()
+      |> Enum.each(fn friend ->
        user = User.get_cached_by_id(user.id)

        shell_info("Unsubscribing #{friend.nickname} from #{user.nickname}")
@ -405,7 +404,7 @@ defmodule Mix.Tasks.Pleroma.User do
    start_pleroma()

    with %User{local: true} = user <- User.get_cached_by_nickname(nickname) do
-      {:ok, _} = User.delete_user_activities(user)
+      User.delete_user_activities(user)
      shell_info("User #{nickname} statuses deleted.")
    else
      _ ->
@ -443,39 +442,21 @@ defmodule Mix.Tasks.Pleroma.User do
  end

  defp set_moderator(user, value) do
-    info_cng = User.Info.admin_api_update(user.info, %{is_moderator: value})
-
-    user_cng =
-      Ecto.Changeset.change(user)
-      |> put_embed(:info, info_cng)
-
-    {:ok, user} = User.update_and_set_cache(user_cng)
+    {:ok, user} = User.update_info(user, &User.Info.admin_api_update(&1, %{is_moderator: value}))

    shell_info("Moderator status of #{user.nickname}: #{user.info.is_moderator}")
    user
  end

  defp set_admin(user, value) do
-    info_cng = User.Info.admin_api_update(user.info, %{is_admin: value})
-
-    user_cng =
-      Ecto.Changeset.change(user)
-      |> put_embed(:info, info_cng)
-
-    {:ok, user} = User.update_and_set_cache(user_cng)
+    {:ok, user} = User.update_info(user, &User.Info.admin_api_update(&1, %{is_admin: value}))

    shell_info("Admin status of #{user.nickname}: #{user.info.is_admin}")
    user
  end

  defp set_locked(user, value) do
-    info_cng = User.Info.user_upgrade(user.info, %{locked: value})
-
-    user_cng =
-      Ecto.Changeset.change(user)
-      |> put_embed(:info, info_cng)
-
-    {:ok, user} = User.update_and_set_cache(user_cng)
+    {:ok, user} = User.update_info(user, &User.Info.user_upgrade(&1, %{locked: value}))

    shell_info("Locked status of #{user.nickname}: #{user.info.locked}")
    user
--- a/lib/pleroma/activity.ex
+++ b/lib/pleroma/activity.ex
@ -6,6 +6,8 @@ defmodule Pleroma.Activity do
  use Ecto.Schema

  alias Pleroma.Activity
+  alias Pleroma.Activity.Queries
+  alias Pleroma.ActivityExpiration
  alias Pleroma.Bookmark
  alias Pleroma.Notification
  alias Pleroma.Object
@ -19,7 +21,7 @@ defmodule Pleroma.Activity do
  @type t :: %__MODULE__{}
  @type actor :: String.t()

-  @primary_key {:id, Pleroma.FlakeId, autogenerate: true}
+  @primary_key {:id, FlakeId.Ecto.CompatType, autogenerate: true}

  # https://github.com/tootsuite/mastodon/blob/master/app/models/notification.rb#L19
  @mastodon_notification_types %{
@ -59,11 +61,13 @@ defmodule Pleroma.Activity do
    # typical case.
    has_one(:object, Object, on_delete: :nothing, foreign_key: :id)

+    has_one(:expiration, ActivityExpiration, on_delete: :delete_all)
+
    timestamps()
  end

-  def with_joined_object(query) do
-    join(query, :inner, [activity], o in Object,
+  def with_joined_object(query, join_type \\ :inner) do
+    join(query, join_type, [activity], o in Object,
      on:
        fragment(
          "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')",
@ -75,10 +79,10 @@ defmodule Pleroma.Activity do
    )
  end

-  def with_preloaded_object(query) do
+  def with_preloaded_object(query, join_type \\ :inner) do
    query
    |> has_named_binding?(:object)
-    |> if(do: query, else: with_joined_object(query))
+    |> if(do: query, else: with_joined_object(query, join_type))
    |> preload([activity, object: object], object: object)
  end

@ -104,12 +108,9 @@ defmodule Pleroma.Activity do
  def with_set_thread_muted_field(query, _), do: query

  def get_by_ap_id(ap_id) do
-    Repo.one(
-      from(
-        activity in Activity,
-        where: fragment("(?)->>'id' = ?", activity.data, ^to_string(ap_id))
-      )
-    )
+    ap_id
+    |> Queries.by_ap_id()
+    |> Repo.one()
  end

  def get_bookmark(%Activity{} = activity, %User{} = user) do
@ -130,91 +131,55 @@ defmodule Pleroma.Activity do
  end

  def get_by_ap_id_with_object(ap_id) do
-    Repo.one(
-      from(
-        activity in Activity,
-        where: fragment("(?)->>'id' = ?", activity.data, ^to_string(ap_id)),
-        left_join: o in Object,
-        on:
-          fragment(
-            "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')",
-            o.data,
-            activity.data,
-            activity.data
-          ),
-        preload: [object: o]
-      )
-    )
+    ap_id
+    |> Queries.by_ap_id()
+    |> with_preloaded_object(:left)
+    |> Repo.one()
  end

+  @spec get_by_id(String.t()) :: Activity.t() | nil
  def get_by_id(id) do
-    Activity
-    |> where([a], a.id == ^id)
-    |> restrict_deactivated_users()
-    |> Repo.one()
+    case FlakeId.flake_id?(id) do
+      true ->
+        Activity
+        |> where([a], a.id == ^id)
+        |> restrict_deactivated_users()
+        |> Repo.one()
+
+      _ ->
+        nil
+    end
  end

  def get_by_id_with_object(id) do
-    from(activity in Activity,
-      where: activity.id == ^id,
-      inner_join: o in Object,
-      on:
-        fragment(
-          "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')",
-          o.data,
-          activity.data,
-          activity.data
-        ),
-      preload: [object: o]
-    )
+    Activity
+    |> where(id: ^id)
+    |> with_preloaded_object()
    |> Repo.one()
  end

-  def by_object_ap_id(ap_id) do
-    from(
-      activity in Activity,
-      where:
-        fragment(
-          "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-          activity.data,
-          activity.data,
-          ^to_string(ap_id)
-        )
-    )
+  def all_by_ids_with_object(ids) do
+    Activity
+    |> where([a], a.id in ^ids)
+    |> with_preloaded_object()
+    |> Repo.all()
  end

-  def create_by_object_ap_id(ap_ids) when is_list(ap_ids) do
-    from(
-      activity in Activity,
-      where:
-        fragment(
-          "coalesce((?)->'object'->>'id', (?)->>'object') = ANY(?)",
-          activity.data,
-          activity.data,
-          ^ap_ids
-        ),
-      where: fragment("(?)->>'type' = 'Create'", activity.data)
-    )
+  @doc """
+  Accepts `ap_id` or list of `ap_id`.
+  Returns a query.
+  """
+  @spec create_by_object_ap_id(String.t() | [String.t()]) :: Ecto.Queryable.t()
+  def create_by_object_ap_id(ap_id) do
+    ap_id
+    |> Queries.by_object_id()
+    |> Queries.by_type("Create")
  end

-  def create_by_object_ap_id(ap_id) when is_binary(ap_id) do
-    from(
-      activity in Activity,
-      where:
-        fragment(
-          "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-          activity.data,
-          activity.data,
-          ^to_string(ap_id)
-        ),
-      where: fragment("(?)->>'type' = 'Create'", activity.data)
-    )
-  end
-
-  def create_by_object_ap_id(_), do: nil
-
  def get_all_create_by_object_ap_id(ap_id) do
-    Repo.all(create_by_object_ap_id(ap_id))
+    ap_id
+    |> create_by_object_ap_id()
+    |> Repo.all()
  end

  def get_create_by_object_ap_id(ap_id) when is_binary(ap_id) do
@ -225,54 +190,17 @@ defmodule Pleroma.Activity do

  def get_create_by_object_ap_id(_), do: nil

-  def create_by_object_ap_id_with_object(ap_ids) when is_list(ap_ids) do
-    from(
-      activity in Activity,
-      where:
-        fragment(
-          "coalesce((?)->'object'->>'id', (?)->>'object') = ANY(?)",
-          activity.data,
-          activity.data,
-          ^ap_ids
-        ),
-      where: fragment("(?)->>'type' = 'Create'", activity.data),
-      inner_join: o in Object,
-      on:
-        fragment(
-          "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')",
-          o.data,
-          activity.data,
-          activity.data
-        ),
-      preload: [object: o]
-    )
+  @doc """
+  Accepts `ap_id` or list of `ap_id`.
+  Returns a query.
+  """
+  @spec create_by_object_ap_id_with_object(String.t() | [String.t()]) :: Ecto.Queryable.t()
+  def create_by_object_ap_id_with_object(ap_id) do
+    ap_id
+    |> create_by_object_ap_id()
+    |> with_preloaded_object()
  end

-  def create_by_object_ap_id_with_object(ap_id) when is_binary(ap_id) do
-    from(
-      activity in Activity,
-      where:
-        fragment(
-          "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-          activity.data,
-          activity.data,
-          ^to_string(ap_id)
-        ),
-      where: fragment("(?)->>'type' = 'Create'", activity.data),
-      inner_join: o in Object,
-      on:
-        fragment(
-          "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')",
-          o.data,
-          activity.data,
-          activity.data
-        ),
-      preload: [object: o]
-    )
-  end
-
-  def create_by_object_ap_id_with_object(_), do: nil
-
  def get_create_by_object_ap_id_with_object(ap_id) when is_binary(ap_id) do
    ap_id
    |> create_by_object_ap_id_with_object()
@ -296,7 +224,8 @@ defmodule Pleroma.Activity do
  def normalize(_), do: nil

  def delete_by_ap_id(id) when is_binary(id) do
-    by_object_ap_id(id)
+    id
+    |> Queries.by_object_id()
    |> select([u], u)
    |> Repo.delete_all()
    |> elem(1)
@ -305,10 +234,19 @@ defmodule Pleroma.Activity do
      %{data: %{"type" => "Create", "object" => %{"id" => ap_id}}} -> ap_id == id
      _ -> nil
    end)
+    |> purge_web_resp_cache()
  end

  def delete_by_ap_id(_), do: nil

+  defp purge_web_resp_cache(%Activity{} = activity) do
+    %{path: path} = URI.parse(activity.data["id"])
+    Cachex.del(:web_resp_cache, path)
+    activity
+  end
+
+  defp purge_web_resp_cache(nil), do: nil
+
  for {ap_type, type} <- @mastodon_notification_types do
    def mastodon_notification_type(%Activity{data: %{"type" => unquote(ap_type)}}),
      do: unquote(type)
@ -331,40 +269,19 @@ defmodule Pleroma.Activity do
  end

  def follow_requests_for_actor(%Pleroma.User{ap_id: ap_id}) do
-    from(
-      a in Activity,
-      where:
-        fragment(
-          "? ->> 'type' = 'Follow'",
-          a.data
-        ),
-      where:
-        fragment(
-          "? ->> 'state' = 'pending'",
-          a.data
-        ),
-      where:
-        fragment(
-          "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-          a.data,
-          a.data,
-          ^ap_id
-        )
-    )
-  end
-
-  @spec query_by_actor(actor()) :: Ecto.Query.t()
-  def query_by_actor(actor) do
-    from(a in Activity, where: a.actor == ^actor)
+    ap_id
+    |> Queries.by_object_id()
+    |> Queries.by_type("Follow")
+    |> where([a], fragment("? ->> 'state' = 'pending'", a.data))
  end

  def restrict_deactivated_users(query) do
+    deactivated_users =
+      from(u in User.Query.build(deactivated: true), select: u.ap_id)
+      |> Repo.all()
+
    from(activity in query,
-      where:
-        fragment(
-          "? not in (SELECT ap_id FROM users WHERE info->'deactivated' @> 'true')",
-          activity.actor
-        )
+      where: activity.actor not in ^deactivated_users
    )
  end

--- a/lib/pleroma/activity/ir/topics.ex
+++ b/lib/pleroma/activity/ir/topics.ex
@ -0,0 +1,63 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Activity.Ir.Topics do
+  alias Pleroma.Object
+  alias Pleroma.Web.ActivityPub.Visibility
+
+  def get_activity_topics(activity) do
+    activity
+    |> Object.normalize()
+    |> generate_topics(activity)
+    |> List.flatten()
+  end
+
+  defp generate_topics(%{data: %{"type" => "Answer"}}, _) do
+    []
+  end
+
+  defp generate_topics(object, activity) do
+    ["user", "list"] ++ visibility_tags(object, activity)
+  end
+
+  defp visibility_tags(object, activity) do
+    case Visibility.get_visibility(activity) do
+      "public" ->
+        if activity.local do
+          ["public", "public:local"]
+        else
+          ["public"]
+        end
+        |> item_creation_tags(object, activity)
+
+      "direct" ->
+        ["direct"]
+
+      _ ->
+        []
+    end
+  end
+
+  defp item_creation_tags(tags, %{data: %{"type" => "Create"}} = object, activity) do
+    tags ++ hashtags_to_topics(object) ++ attachment_topics(object, activity)
+  end
+
+  defp item_creation_tags(tags, _, _) do
+    tags
+  end
+
+  defp hashtags_to_topics(%{data: %{"tag" => tags}}) do
+    tags
+    |> Enum.filter(&is_bitstring(&1))
+    |> Enum.map(fn tag -> "hashtag:" <> tag end)
+  end
+
+  defp hashtags_to_topics(_), do: []
+
+  defp attachment_topics(%{data: %{"attachment" => []}}, _act), do: []
+
+  defp attachment_topics(_object, %{local: true}), do: ["public:media", "public:local:media"]
+
+  defp attachment_topics(_object, _act), do: ["public:media"]
+end
--- a/lib/pleroma/activity/queries.ex
+++ b/lib/pleroma/activity/queries.ex
@ -0,0 +1,67 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Activity.Queries do
+  @moduledoc """
+  Contains queries for Activity.
+  """
+
+  import Ecto.Query, only: [from: 2]
+
+  @type query :: Ecto.Queryable.t() | Activity.t()
+
+  alias Pleroma.Activity
+
+  @spec by_ap_id(query, String.t()) :: query
+  def by_ap_id(query \\ Activity, ap_id) do
+    from(
+      activity in query,
+      where: fragment("(?)->>'id' = ?", activity.data, ^to_string(ap_id))
+    )
+  end
+
+  @spec by_actor(query, String.t()) :: query
+  def by_actor(query \\ Activity, actor) do
+    from(
+      activity in query,
+      where: fragment("(?)->>'actor' = ?", activity.data, ^actor)
+    )
+  end
+
+  @spec by_object_id(query, String.t() | [String.t()]) :: query
+  def by_object_id(query \\ Activity, object_id)
+
+  def by_object_id(query, object_ids) when is_list(object_ids) do
+    from(
+      activity in query,
+      where:
+        fragment(
+          "coalesce((?)->'object'->>'id', (?)->>'object') = ANY(?)",
+          activity.data,
+          activity.data,
+          ^object_ids
+        )
+    )
+  end
+
+  def by_object_id(query, object_id) when is_binary(object_id) do
+    from(activity in query,
+      where:
+        fragment(
+          "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
+          activity.data,
+          activity.data,
+          ^object_id
+        )
+    )
+  end
+
+  @spec by_type(query, String.t()) :: query
+  def by_type(query \\ Activity, activity_type) do
+    from(
+      activity in query,
+      where: fragment("(?)->>'type' = ?", activity.data, ^activity_type)
+    )
+  end
+end
--- a/lib/pleroma/activity_expiration.ex
+++ b/lib/pleroma/activity_expiration.ex
@ -0,0 +1,67 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.ActivityExpiration do
+  use Ecto.Schema
+
+  alias Pleroma.Activity
+  alias Pleroma.ActivityExpiration
+  alias Pleroma.Repo
+
+  import Ecto.Changeset
+  import Ecto.Query
+
+  @type t :: %__MODULE__{}
+  @min_activity_lifetime :timer.hours(1)
+
+  schema "activity_expirations" do
+    belongs_to(:activity, Activity, type: FlakeId.Ecto.CompatType)
+    field(:scheduled_at, :naive_datetime)
+  end
+
+  def changeset(%ActivityExpiration{} = expiration, attrs) do
+    expiration
+    |> cast(attrs, [:scheduled_at])
+    |> validate_required([:scheduled_at])
+    |> validate_scheduled_at()
+  end
+
+  def get_by_activity_id(activity_id) do
+    ActivityExpiration
+    |> where([exp], exp.activity_id == ^activity_id)
+    |> Repo.one()
+  end
+
+  def create(%Activity{} = activity, scheduled_at) do
+    %ActivityExpiration{activity_id: activity.id}
+    |> changeset(%{scheduled_at: scheduled_at})
+    |> Repo.insert()
+  end
+
+  def due_expirations(offset \\ 0) do
+    naive_datetime =
+      NaiveDateTime.utc_now()
+      |> NaiveDateTime.add(offset, :millisecond)
+
+    ActivityExpiration
+    |> where([exp], exp.scheduled_at < ^naive_datetime)
+    |> Repo.all()
+  end
+
+  def validate_scheduled_at(changeset) do
+    validate_change(changeset, :scheduled_at, fn _, scheduled_at ->
+      if not expires_late_enough?(scheduled_at) do
+        [scheduled_at: "an ephemeral activity must live for at least one hour"]
+      else
+        []
+      end
+    end)
+  end
+
+  def expires_late_enough?(scheduled_at) do
+    now = NaiveDateTime.utc_now()
+    diff = NaiveDateTime.diff(scheduled_at, now, :millisecond)
+    diff >= @min_activity_lifetime
+  end
+end
--- a/lib/pleroma/application.ex
+++ b/lib/pleroma/application.ex
@ -31,33 +31,20 @@ defmodule Pleroma.Application do
    children =
      [
        Pleroma.Repo,
+        Pleroma.Scheduler,
        Pleroma.Config.TransferTask,
        Pleroma.Emoji,
        Pleroma.Captcha,
-        Pleroma.FlakeId,
-        Pleroma.ScheduledActivityWorker
+        Pleroma.Daemons.ScheduledActivityDaemon,
+        Pleroma.Daemons.ActivityExpirationDaemon
      ] ++
        cachex_children() ++
        hackney_pool_children() ++
        [
-          Pleroma.Web.Federator.RetryQueue,
          Pleroma.Stats,
-          %{
-            id: :web_push_init,
-            start: {Task, :start_link, [&Pleroma.Web.Push.init/0]},
-            restart: :temporary
-          },
-          %{
-            id: :federator_init,
-            start: {Task, :start_link, [&Pleroma.Web.Federator.init/0]},
-            restart: :temporary
-          },
-          %{
-            id: :internal_fetch_init,
-            start: {Task, :start_link, [&Pleroma.Web.ActivityPub.InternalFetchActor.init/0]},
-            restart: :temporary
-          }
+          {Oban, Pleroma.Config.get(Oban)}
        ] ++
+        task_children(@env) ++
        oauth_cleanup_child(oauth_cleanup_enabled?()) ++
        streamer_child(@env) ++
        chat_child(@env, chat_enabled?()) ++
@ -69,9 +56,7 @@ defmodule Pleroma.Application do
    # See http://elixir-lang.org/docs/stable/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: Pleroma.Supervisor]
-    result = Supervisor.start_link(children, opts)
-    :ok = after_supervisor_start()
-    result
+    Supervisor.start_link(children, opts)
  end

  defp setup_instrumenters do
@ -115,10 +100,15 @@ defmodule Pleroma.Application do
      build_cachex("object", default_ttl: 25_000, ttl_interval: 1000, limit: 2500),
      build_cachex("rich_media", default_ttl: :timer.minutes(120), limit: 5000),
      build_cachex("scrubber", limit: 2500),
-      build_cachex("idempotency", expiration: idempotency_expiration(), limit: 2500)
+      build_cachex("idempotency", expiration: idempotency_expiration(), limit: 2500),
+      build_cachex("web_resp", limit: 2500),
+      build_cachex("emoji_packs", expiration: emoji_packs_expiration(), limit: 10)
    ]
  end

+  defp emoji_packs_expiration,
+    do: expiration(default: :timer.seconds(5 * 60), interval: :timer.seconds(60))
+
  defp idempotency_expiration,
    do: expiration(default: :timer.seconds(6 * 60 * 60), interval: :timer.seconds(60))

@ -140,7 +130,7 @@ defmodule Pleroma.Application do
  defp streamer_child(:test), do: []

  defp streamer_child(_) do
-    [Pleroma.Web.Streamer]
+    [Pleroma.Web.Streamer.supervisor()]
  end

  defp oauth_cleanup_child(true),
@ -163,16 +153,38 @@ defmodule Pleroma.Application do
    end
  end

-  defp after_supervisor_start do
-    with digest_config <- Application.get_env(:pleroma, :email_notifications)[:digest],
-         true <- digest_config[:active] do
-      PleromaJobQueue.schedule(
-        digest_config[:schedule],
-        :digest_emails,
-        Pleroma.DigestEmailWorker
-      )
-    end
+  defp task_children(:test) do
+    [
+      %{
+        id: :web_push_init,
+        start: {Task, :start_link, [&Pleroma.Web.Push.init/0]},
+        restart: :temporary
+      },
+      %{
+        id: :federator_init,
+        start: {Task, :start_link, [&Pleroma.Web.Federator.init/0]},
+        restart: :temporary
+      }
+    ]
+  end

-    :ok
+  defp task_children(_) do
+    [
+      %{
+        id: :web_push_init,
+        start: {Task, :start_link, [&Pleroma.Web.Push.init/0]},
+        restart: :temporary
+      },
+      %{
+        id: :federator_init,
+        start: {Task, :start_link, [&Pleroma.Web.Federator.init/0]},
+        restart: :temporary
+      },
+      %{
+        id: :internal_fetch_init,
+        start: {Task, :start_link, [&Pleroma.Web.ActivityPub.InternalFetchActor.init/0]},
+        restart: :temporary
+      }
+    ]
  end
 end
--- a/lib/pleroma/bbs/handler.ex
+++ b/lib/pleroma/bbs/handler.ex
@ -42,7 +42,7 @@ defmodule Pleroma.BBS.Handler do
  end

  def puts_activity(activity) do
-    status = Pleroma.Web.MastodonAPI.StatusView.render("status.json", %{activity: activity})
+    status = Pleroma.Web.MastodonAPI.StatusView.render("show.json", %{activity: activity})
    IO.puts("-- #{status.id} by #{status.account.display_name} (#{status.account.acct})")
    IO.puts(HtmlSanitizeEx.strip_tags(status.content))
    IO.puts("")
--- a/lib/pleroma/bookmark.ex
+++ b/lib/pleroma/bookmark.ex
@ -10,20 +10,20 @@ defmodule Pleroma.Bookmark do

  alias Pleroma.Activity
  alias Pleroma.Bookmark
-  alias Pleroma.FlakeId
  alias Pleroma.Repo
  alias Pleroma.User

  @type t :: %__MODULE__{}

  schema "bookmarks" do
-    belongs_to(:user, User, type: FlakeId)
-    belongs_to(:activity, Activity, type: FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
+    belongs_to(:activity, Activity, type: FlakeId.Ecto.CompatType)

    timestamps()
  end

-  @spec create(FlakeId.t(), FlakeId.t()) :: {:ok, Bookmark.t()} | {:error, Changeset.t()}
+  @spec create(FlakeId.Ecto.CompatType.t(), FlakeId.Ecto.CompatType.t()) ::
+          {:ok, Bookmark.t()} | {:error, Changeset.t()}
  def create(user_id, activity_id) do
    attrs = %{
      user_id: user_id,
@ -37,7 +37,7 @@ defmodule Pleroma.Bookmark do
    |> Repo.insert()
  end

-  @spec for_user_query(FlakeId.t()) :: Ecto.Query.t()
+  @spec for_user_query(FlakeId.Ecto.CompatType.t()) :: Ecto.Query.t()
  def for_user_query(user_id) do
    Bookmark
    |> where(user_id: ^user_id)
@ -52,7 +52,8 @@ defmodule Pleroma.Bookmark do
    |> Repo.one()
  end

-  @spec destroy(FlakeId.t(), FlakeId.t()) :: {:ok, Bookmark.t()} | {:error, Changeset.t()}
+  @spec destroy(FlakeId.Ecto.CompatType.t(), FlakeId.Ecto.CompatType.t()) ::
+          {:ok, Bookmark.t()} | {:error, Changeset.t()}
  def destroy(user_id, activity_id) do
    from(b in Bookmark,
      where: b.user_id == ^user_id,
--- a/lib/pleroma/constants.ex
+++ b/lib/pleroma/constants.ex
@ -6,4 +6,16 @@ defmodule Pleroma.Constants do
  use Const

  const(as_public, do: "https://www.w3.org/ns/activitystreams#Public")
+
+  const(object_internal_fields,
+    do: [
+      "likes",
+      "like_count",
+      "announcements",
+      "announcement_count",
+      "emoji",
+      "context_id",
+      "deleted_activity_id"
+    ]
+  )
 end
--- a/lib/pleroma/conversation/participation.ex
+++ b/lib/pleroma/conversation/participation.ex
@ -13,10 +13,10 @@ defmodule Pleroma.Conversation.Participation do
  import Ecto.Query

  schema "conversation_participations" do
-    belongs_to(:user, User, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    belongs_to(:conversation, Conversation)
    field(:read, :boolean, default: false)
-    field(:last_activity_id, Pleroma.FlakeId, virtual: true)
+    field(:last_activity_id, FlakeId.Ecto.CompatType, virtual: true)

    has_many(:recipient_ships, RecipientShip)
    has_many(:recipients, through: [:recipient_ships, :user])
--- a/lib/pleroma/conversation/participation_recipient_ship.ex
+++ b/lib/pleroma/conversation/participation_recipient_ship.ex
@ -12,7 +12,7 @@ defmodule Pleroma.Conversation.Participation.RecipientShip do
  import Ecto.Changeset

  schema "conversation_participation_recipient_ships" do
-    belongs_to(:user, User, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    belongs_to(:participation, Participation)
  end

--- a/lib/pleroma/daemons/activity_expiration_daemon.ex
+++ b/lib/pleroma/daemons/activity_expiration_daemon.ex
@ -0,0 +1,66 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Daemons.ActivityExpirationDaemon do
+  alias Pleroma.Activity
+  alias Pleroma.ActivityExpiration
+  alias Pleroma.Config
+  alias Pleroma.Repo
+  alias Pleroma.User
+  alias Pleroma.Web.CommonAPI
+
+  require Logger
+  use GenServer
+  import Ecto.Query
+
+  @schedule_interval :timer.minutes(1)
+
+  def start_link(_) do
+    GenServer.start_link(__MODULE__, nil)
+  end
+
+  @impl true
+  def init(_) do
+    if Config.get([ActivityExpiration, :enabled]) do
+      schedule_next()
+      {:ok, nil}
+    else
+      :ignore
+    end
+  end
+
+  def perform(:execute, expiration_id) do
+    try do
+      expiration =
+        ActivityExpiration
+        |> where([e], e.id == ^expiration_id)
+        |> Repo.one!()
+
+      activity = Activity.get_by_id_with_object(expiration.activity_id)
+      user = User.get_by_ap_id(activity.object.data["actor"])
+      CommonAPI.delete(activity.id, user)
+    rescue
+      error ->
+        Logger.error("#{__MODULE__} Couldn't delete expired activity: #{inspect(error)}")
+    end
+  end
+
+  @impl true
+  def handle_info(:perform, state) do
+    ActivityExpiration.due_expirations(@schedule_interval)
+    |> Enum.each(fn expiration ->
+      Pleroma.Workers.ActivityExpirationWorker.enqueue(
+        "activity_expiration",
+        %{"activity_expiration_id" => expiration.id}
+      )
+    end)
+
+    schedule_next()
+    {:noreply, state}
+  end
+
+  defp schedule_next do
+    Process.send_after(self(), :perform, @schedule_interval)
+  end
+end
--- a/lib/pleroma/daemons/digest_email_daemon.ex
+++ b/lib/pleroma/daemons/digest_email_daemon.ex
@ -2,10 +2,11 @@
 # Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

-defmodule Pleroma.DigestEmailWorker do
-  import Ecto.Query
+defmodule Pleroma.Daemons.DigestEmailDaemon do
+  alias Pleroma.Repo
+  alias Pleroma.Workers.DigestEmailsWorker

-  @queue_name :digest_emails
+  import Ecto.Query

  def perform do
    config = Pleroma.Config.get([:email_notifications, :digest])
@ -20,8 +21,10 @@ defmodule Pleroma.DigestEmailWorker do
      where: u.last_digest_emailed_at < datetime_add(^now, ^negative_interval, "day"),
      select: u
    )
-    |> Pleroma.Repo.all()
-    |> Enum.each(&PleromaJobQueue.enqueue(@queue_name, __MODULE__, [&1]))
+    |> Repo.all()
+    |> Enum.each(fn user ->
+      DigestEmailsWorker.enqueue("digest_email", %{"user_id" => user.id})
+    end)
  end

  @doc """
--- a/lib/pleroma/daemons/scheduled_activity_daemon.ex
+++ b/lib/pleroma/daemons/scheduled_activity_daemon.ex
@ -2,7 +2,7 @@
 # Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

-defmodule Pleroma.ScheduledActivityWorker do
+defmodule Pleroma.Daemons.ScheduledActivityDaemon do
  @moduledoc """
  Sends scheduled activities to the job queue.
  """
@ -11,6 +11,7 @@ defmodule Pleroma.ScheduledActivityWorker do
  alias Pleroma.ScheduledActivity
  alias Pleroma.User
  alias Pleroma.Web.CommonAPI
+
  use GenServer
  require Logger

@ -45,7 +46,10 @@ defmodule Pleroma.ScheduledActivityWorker do
  def handle_info(:perform, state) do
    ScheduledActivity.due_activities(@schedule_interval)
    |> Enum.each(fn scheduled_activity ->
-      PleromaJobQueue.enqueue(:scheduled_activities, __MODULE__, [:execute, scheduled_activity.id])
+      Pleroma.Workers.ScheduledActivityWorker.enqueue(
+        "execute",
+        %{"activity_id" => scheduled_activity.id}
+      )
    end)

    schedule_next()
--- a/lib/pleroma/delivery.ex
+++ b/lib/pleroma/delivery.ex
@ -0,0 +1,50 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Delivery do
+  use Ecto.Schema
+
+  alias Pleroma.Delivery
+  alias Pleroma.Object
+  alias Pleroma.Repo
+  alias Pleroma.User
+  alias Pleroma.User
+
+  import Ecto.Changeset
+  import Ecto.Query
+
+  schema "deliveries" do
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
+    belongs_to(:object, Object)
+  end
+
+  def changeset(delivery, params \\ %{}) do
+    delivery
+    |> cast(params, [:user_id, :object_id])
+    |> validate_required([:user_id, :object_id])
+    |> foreign_key_constraint(:object_id)
+    |> foreign_key_constraint(:user_id)
+    |> unique_constraint(:user_id, name: :deliveries_user_id_object_id_index)
+  end
+
+  def create(object_id, user_id) do
+    %Delivery{}
+    |> changeset(%{user_id: user_id, object_id: object_id})
+    |> Repo.insert(on_conflict: :nothing)
+  end
+
+  def get(object_id, user_id) do
+    from(d in Delivery, where: d.user_id == ^user_id and d.object_id == ^object_id)
+    |> Repo.one()
+  end
+
+  # A hack because user delete activities have a fake id for whatever reason
+  # TODO: Get rid of this
+  def delete_all_by_object_id("pleroma:fake_object_id"), do: {0, []}
+
+  def delete_all_by_object_id(object_id) do
+    from(d in Delivery, where: d.object_id == ^object_id)
+    |> Repo.delete_all()
+  end
+end
--- a/lib/pleroma/docs/generator.ex
+++ b/lib/pleroma/docs/generator.ex
@ -0,0 +1,73 @@
+defmodule Pleroma.Docs.Generator do
+  @callback process(keyword()) :: {:ok, String.t()}
+
+  @spec process(module(), keyword()) :: {:ok, String.t()}
+  def process(implementation, descriptions) do
+    implementation.process(descriptions)
+  end
+
+  @spec uploaders_list() :: [module()]
+  def uploaders_list do
+    {:ok, modules} = :application.get_key(:pleroma, :modules)
+
+    Enum.filter(modules, fn module ->
+      name_as_list = Module.split(module)
+
+      List.starts_with?(name_as_list, ["Pleroma", "Uploaders"]) and
+        List.last(name_as_list) != "Uploader"
+    end)
+  end
+
+  @spec filters_list() :: [module()]
+  def filters_list do
+    {:ok, modules} = :application.get_key(:pleroma, :modules)
+
+    Enum.filter(modules, fn module ->
+      name_as_list = Module.split(module)
+
+      List.starts_with?(name_as_list, ["Pleroma", "Upload", "Filter"])
+    end)
+  end
+
+  @spec mrf_list() :: [module()]
+  def mrf_list do
+    {:ok, modules} = :application.get_key(:pleroma, :modules)
+
+    Enum.filter(modules, fn module ->
+      name_as_list = Module.split(module)
+
+      List.starts_with?(name_as_list, ["Pleroma", "Web", "ActivityPub", "MRF"]) and
+        length(name_as_list) > 4
+    end)
+  end
+
+  @spec richmedia_parsers() :: [module()]
+  def richmedia_parsers do
+    {:ok, modules} = :application.get_key(:pleroma, :modules)
+
+    Enum.filter(modules, fn module ->
+      name_as_list = Module.split(module)
+
+      List.starts_with?(name_as_list, ["Pleroma", "Web", "RichMedia", "Parsers"]) and
+        length(name_as_list) == 5
+    end)
+  end
+end
+
+defimpl Jason.Encoder, for: Tuple do
+  def encode(tuple, opts) do
+    Jason.Encode.list(Tuple.to_list(tuple), opts)
+  end
+end
+
+defimpl Jason.Encoder, for: [Regex, Function] do
+  def encode(term, opts) do
+    Jason.Encode.string(inspect(term), opts)
+  end
+end
+
+defimpl String.Chars, for: Regex do
+  def to_string(term) do
+    inspect(term)
+  end
+end
--- a/lib/pleroma/docs/json.ex
+++ b/lib/pleroma/docs/json.ex
@ -0,0 +1,20 @@
+defmodule Pleroma.Docs.JSON do
+  @behaviour Pleroma.Docs.Generator
+
+  @spec process(keyword()) :: {:ok, String.t()}
+  def process(descriptions) do
+    config_path = "docs/generate_config.json"
+
+    with {:ok, file} <- File.open(config_path, [:write]),
+         json <- generate_json(descriptions),
+         :ok <- IO.write(file, json),
+         :ok <- File.close(file) do
+      {:ok, config_path}
+    end
+  end
+
+  @spec generate_json([keyword()]) :: String.t()
+  def generate_json(descriptions) do
+    Jason.encode!(descriptions)
+  end
+end
--- a/lib/pleroma/docs/markdown.ex
+++ b/lib/pleroma/docs/markdown.ex
@ -0,0 +1,88 @@
+defmodule Pleroma.Docs.Markdown do
+  @behaviour Pleroma.Docs.Generator
+
+  @spec process(keyword()) :: {:ok, String.t()}
+  def process(descriptions) do
+    config_path = "docs/generated_config.md"
+    {:ok, file} = File.open(config_path, [:utf8, :write])
+    IO.write(file, "# Generated configuration\n")
+    IO.write(file, "Date of generation: #{Date.utc_today()}\n\n")
+
+    IO.write(
+      file,
+      "This file describe the configuration, it is recommended to edit the relevant `*.secret.exs` file instead of the others founds in the ``config`` directory.\n\n" <>
+        "If you run Pleroma with ``MIX_ENV=prod`` the file is ``prod.secret.exs``, otherwise it is ``dev.secret.exs``.\n\n"
+    )
+
+    for group <- descriptions do
+      if is_nil(group[:key]) do
+        IO.write(file, "## #{inspect(group[:group])}\n")
+      else
+        IO.write(file, "## #{inspect(group[:key])}\n")
+      end
+
+      IO.write(file, "#{group[:description]}\n")
+
+      for child <- group[:children] || [] do
+        print_child_header(file, child)
+
+        print_suggestions(file, child[:suggestions])
+
+        if child[:children] do
+          for subchild <- child[:children] do
+            print_child_header(file, subchild)
+
+            print_suggestions(file, subchild[:suggestions])
+          end
+        end
+      end
+
+      IO.write(file, "\n")
+    end
+
+    :ok = File.close(file)
+    {:ok, config_path}
+  end
+
+  defp print_child_header(file, %{key: key, type: type, description: description} = _child) do
+    IO.write(
+      file,
+      "- `#{inspect(key)}` (`#{inspect(type)}`): #{description}  \n"
+    )
+  end
+
+  defp print_child_header(file, %{key: key, type: type} = _child) do
+    IO.write(file, "- `#{inspect(key)}` (`#{inspect(type)}`)  \n")
+  end
+
+  defp print_suggestion(file, suggestion) when is_list(suggestion) do
+    IO.write(file, "  `#{inspect(suggestion)}`\n")
+  end
+
+  defp print_suggestion(file, suggestion) when is_function(suggestion) do
+    IO.write(file, "  `#{inspect(suggestion.())}`\n")
+  end
+
+  defp print_suggestion(file, suggestion, as_list \\ false) do
+    list_mark = if as_list, do: "- ", else: ""
+    IO.write(file, "  #{list_mark}`#{inspect(suggestion)}`\n")
+  end
+
+  defp print_suggestions(_file, nil), do: nil
+
+  defp print_suggestions(_file, ""), do: nil
+
+  defp print_suggestions(file, suggestions) do
+    if length(suggestions) > 1 do
+      IO.write(file, "Suggestions:\n")
+
+      for suggestion <- suggestions do
+        print_suggestion(file, suggestion, true)
+      end
+    else
+      IO.write(file, "  Suggestion: ")
+
+      print_suggestion(file, List.first(suggestions))
+    end
+  end
+end
--- a/lib/pleroma/emails/mailer.ex
+++ b/lib/pleroma/emails/mailer.ex
@ -9,6 +9,7 @@ defmodule Pleroma.Emails.Mailer do
  The module contains functions to delivery email using Swoosh.Mailer.
  """

+  alias Pleroma.Workers.MailerWorker
  alias Swoosh.DeliveryError

  @otp_app :pleroma
@ -19,7 +20,12 @@ defmodule Pleroma.Emails.Mailer do

  @doc "add email to queue"
  def deliver_async(email, config \\ []) do
-    PleromaJobQueue.enqueue(:mailer, __MODULE__, [:deliver_async, email, config])
+    encoded_email =
+      email
+      |> :erlang.term_to_binary()
+      |> Base.encode64()
+
+    MailerWorker.enqueue("email", %{"encoded_email" => encoded_email, "config" => config})
  end

  @doc "callback to perform send email from queue"
--- a/lib/pleroma/emoji.ex
+++ b/lib/pleroma/emoji.ex
@ -4,24 +4,37 @@

 defmodule Pleroma.Emoji do
  @moduledoc """
-  The emojis are loaded from:
-
-    * emoji packs in INSTANCE-DIR/emoji
-    * the files: `config/emoji.txt` and `config/custom_emoji.txt`
-    * glob paths, nested folder is used as tag name for grouping e.g. priv/static/emoji/custom/nested_folder
-
-  This GenServer stores in an ETS table the list of the loaded emojis, and also allows to reload the list at runtime.
+  This GenServer stores in an ETS table the list of the loaded emojis,
+  and also allows to reload the list at runtime.
  """
  use GenServer

+  alias Pleroma.Emoji.Loader
+
  require Logger

-  @type pattern :: Regex.t() | module() | String.t()
-  @type patterns :: pattern() | [pattern()]
-  @type group_patterns :: keyword(patterns())
-
  @ets __MODULE__.Ets
-  @ets_options [:ordered_set, :protected, :named_table, {:read_concurrency, true}]
+  @ets_options [
+    :ordered_set,
+    :protected,
+    :named_table,
+    {:read_concurrency, true}
+  ]
+
+  defstruct [:code, :file, :tags, :safe_code, :safe_file]
+
+  @doc "Build emoji struct"
+  def build({code, file, tags}) do
+    %__MODULE__{
+      code: code,
+      file: file,
+      tags: tags,
+      safe_code: Pleroma.HTML.strip_tags(code),
+      safe_file: Pleroma.HTML.strip_tags(file)
+    }
+  end
+
+  def build({code, file}), do: build({code, file, []})

  @doc false
  def start_link(_) do
@ -44,11 +57,14 @@ defmodule Pleroma.Emoji do
  end

  @doc "Returns all the emojos!!"
-  @spec get_all() :: [{String.t(), String.t()}, ...]
+  @spec get_all() :: list({String.t(), String.t(), String.t()})
  def get_all do
    :ets.tab2list(@ets)
  end

+  @doc "Clear out old emojis"
+  def clear_all, do: :ets.delete_all_objects(@ets)
+
  @doc false
  def init(_) do
    @ets = :ets.new(@ets, @ets_options)
@ -58,13 +74,13 @@ defmodule Pleroma.Emoji do

  @doc false
  def handle_cast(:reload, state) do
-    load()
+    update_emojis(Loader.load())
    {:noreply, state}
  end

  @doc false
  def handle_call(:reload, _from, state) do
-    load()
+    update_emojis(Loader.load())
    {:reply, :ok, state}
  end

@ -75,189 +91,11 @@ defmodule Pleroma.Emoji do

  @doc false
  def code_change(_old_vsn, state, _extra) do
-    load()
+    update_emojis(Loader.load())
    {:ok, state}
  end

-  defp load do
-    emoji_dir_path =
-      Path.join(
-        Pleroma.Config.get!([:instance, :static_dir]),
-        "emoji"
-      )
-
-    emoji_groups = Pleroma.Config.get([:emoji, :groups])
-
-    case File.ls(emoji_dir_path) do
-      {:error, :enoent} ->
-        # The custom emoji directory doesn't exist,
-        # don't do anything
-        nil
-
-      {:error, e} ->
-        # There was some other error
-        Logger.error("Could not access the custom emoji directory #{emoji_dir_path}: #{e}")
-
-      {:ok, results} ->
-        grouped =
-          Enum.group_by(results, fn file -> File.dir?(Path.join(emoji_dir_path, file)) end)
-
-        packs = grouped[true] || []
-        files = grouped[false] || []
-
-        # Print the packs we've found
-        Logger.info("Found emoji packs: #{Enum.join(packs, ", ")}")
-
-        if not Enum.empty?(files) do
-          Logger.warn(
-            "Found files in the emoji folder. These will be ignored, please move them to a subdirectory\nFound files: #{
-              Enum.join(files, ", ")
-            }"
-          )
-        end
-
-        emojis =
-          Enum.flat_map(
-            packs,
-            fn pack -> load_pack(Path.join(emoji_dir_path, pack), emoji_groups) end
-          )
-
-        true = :ets.insert(@ets, emojis)
-    end
-
-    # Compat thing for old custom emoji handling & default emoji,
-    # it should run even if there are no emoji packs
-    shortcode_globs = Pleroma.Config.get([:emoji, :shortcode_globs], [])
-
-    emojis =
-      (load_from_file("config/emoji.txt", emoji_groups) ++
-         load_from_file("config/custom_emoji.txt", emoji_groups) ++
-         load_from_globs(shortcode_globs, emoji_groups))
-      |> Enum.reject(fn value -> value == nil end)
-
-    true = :ets.insert(@ets, emojis)
-
-    :ok
-  end
-
-  defp load_pack(pack_dir, emoji_groups) do
-    pack_name = Path.basename(pack_dir)
-
-    emoji_txt = Path.join(pack_dir, "emoji.txt")
-
-    if File.exists?(emoji_txt) do
-      load_from_file(emoji_txt, emoji_groups)
-    else
-      extensions = Pleroma.Config.get([:emoji, :pack_extensions])
-
-      Logger.info(
-        "No emoji.txt found for pack \"#{pack_name}\", assuming all #{Enum.join(extensions, ", ")} files are emoji"
-      )
-
-      make_shortcode_to_file_map(pack_dir, extensions)
-      |> Enum.map(fn {shortcode, rel_file} ->
-        filename = Path.join("/emoji/#{pack_name}", rel_file)
-
-        {shortcode, filename, [to_string(match_extra(emoji_groups, filename))]}
-      end)
-    end
-  end
-
-  def make_shortcode_to_file_map(pack_dir, exts) do
-    find_all_emoji(pack_dir, exts)
-    |> Enum.map(&Path.relative_to(&1, pack_dir))
-    |> Enum.map(fn f -> {f |> Path.basename() |> Path.rootname(), f} end)
-    |> Enum.into(%{})
-  end
-
-  def find_all_emoji(dir, exts) do
-    Enum.reduce(
-      File.ls!(dir),
-      [],
-      fn f, acc ->
-        filepath = Path.join(dir, f)
-
-        if File.dir?(filepath) do
-          acc ++ find_all_emoji(filepath, exts)
-        else
-          acc ++ [filepath]
-        end
-      end
-    )
-    |> Enum.filter(fn f -> Path.extname(f) in exts end)
-  end
-
-  defp load_from_file(file, emoji_groups) do
-    if File.exists?(file) do
-      load_from_file_stream(File.stream!(file), emoji_groups)
-    else
-      []
-    end
-  end
-
-  defp load_from_file_stream(stream, emoji_groups) do
-    stream
-    |> Stream.map(&String.trim/1)
-    |> Stream.map(fn line ->
-      case String.split(line, ~r/,\s*/) do
-        [name, file] ->
-          {name, file, [to_string(match_extra(emoji_groups, file))]}
-
-        [name, file | tags] ->
-          {name, file, tags}
-
-        _ ->
-          nil
-      end
-    end)
-    |> Enum.to_list()
-  end
-
-  defp load_from_globs(globs, emoji_groups) do
-    static_path = Path.join(:code.priv_dir(:pleroma), "static")
-
-    paths =
-      Enum.map(globs, fn glob ->
-        Path.join(static_path, glob)
-        |> Path.wildcard()
-      end)
-      |> Enum.concat()
-
-    Enum.map(paths, fn path ->
-      tag = match_extra(emoji_groups, Path.join("/", Path.relative_to(path, static_path)))
-      shortcode = Path.basename(path, Path.extname(path))
-      external_path = Path.join("/", Path.relative_to(path, static_path))
-      {shortcode, external_path, [to_string(tag)]}
-    end)
-  end
-
-  @doc """
-  Finds a matching group for the given emoji filename
-  """
-  @spec match_extra(group_patterns(), String.t()) :: atom() | nil
-  def match_extra(group_patterns, filename) do
-    match_group_patterns(group_patterns, fn pattern ->
-      case pattern do
-        %Regex{} = regex -> Regex.match?(regex, filename)
-        string when is_binary(string) -> filename == string
-      end
-    end)
-  end
-
-  defp match_group_patterns(group_patterns, matcher) do
-    Enum.find_value(group_patterns, fn {group, patterns} ->
-      patterns =
-        patterns
-        |> List.wrap()
-        |> Enum.map(fn pattern ->
-          if String.contains?(pattern, "*") do
-            ~r(#{String.replace(pattern, "*", ".*")})
-          else
-            pattern
-          end
-        end)
-
-      Enum.any?(patterns, matcher) && group
-    end)
+  defp update_emojis(emojis) do
+    :ets.insert(@ets, emojis)
  end
 end
--- a/lib/pleroma/emoji/formatter.ex
+++ b/lib/pleroma/emoji/formatter.ex
@ -0,0 +1,59 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Emoji.Formatter do
+  alias Pleroma.Emoji
+  alias Pleroma.HTML
+  alias Pleroma.Web.MediaProxy
+
+  def emojify(text) do
+    emojify(text, Emoji.get_all())
+  end
+
+  def emojify(text, nil), do: text
+
+  def emojify(text, emoji, strip \\ false) do
+    Enum.reduce(emoji, text, fn
+      {_, %Emoji{safe_code: emoji, safe_file: file}}, text ->
+        String.replace(text, ":#{emoji}:", prepare_emoji_html(emoji, file, strip))
+
+      {unsafe_emoji, unsafe_file}, text ->
+        emoji = HTML.strip_tags(unsafe_emoji)
+        file = HTML.strip_tags(unsafe_file)
+        String.replace(text, ":#{emoji}:", prepare_emoji_html(emoji, file, strip))
+    end)
+    |> HTML.filter_tags()
+  end
+
+  defp prepare_emoji_html(_emoji, _file, true), do: ""
+
+  defp prepare_emoji_html(emoji, file, _strip) do
+    "<img class='emoji' alt='#{emoji}' title='#{emoji}' src='#{MediaProxy.url(file)}' />"
+  end
+
+  def demojify(text) do
+    emojify(text, Emoji.get_all(), true)
+  end
+
+  def demojify(text, nil), do: text
+
+  @doc "Outputs a list of the emoji-shortcodes in a text"
+  def get_emoji(text) when is_binary(text) do
+    Enum.filter(Emoji.get_all(), fn {emoji, %Emoji{}} ->
+      String.contains?(text, ":#{emoji}:")
+    end)
+  end
+
+  def get_emoji(_), do: []
+
+  @doc "Outputs a list of the emoji-Maps in a text"
+  def get_emoji_map(text) when is_binary(text) do
+    get_emoji(text)
+    |> Enum.reduce(%{}, fn {name, %Emoji{file: file}}, acc ->
+      Map.put(acc, name, "#{Pleroma.Web.Endpoint.static_url()}#{file}")
+    end)
+  end
+
+  def get_emoji_map(_), do: []
+end
--- a/lib/pleroma/emoji/loader.ex
+++ b/lib/pleroma/emoji/loader.ex
@ -0,0 +1,224 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Emoji.Loader do
+  @moduledoc """
+  The Loader emoji from:
+
+    * emoji packs in INSTANCE-DIR/emoji
+    * the files: `config/emoji.txt` and `config/custom_emoji.txt`
+    * glob paths, nested folder is used as tag name for grouping e.g. priv/static/emoji/custom/nested_folder
+  """
+  alias Pleroma.Config
+  alias Pleroma.Emoji
+
+  require Logger
+
+  @type pattern :: Regex.t() | module() | String.t()
+  @type patterns :: pattern() | [pattern()]
+  @type group_patterns :: keyword(patterns())
+  @type emoji :: {String.t(), Emoji.t()}
+
+  @doc """
+  Loads emojis from files/packs.
+
+  returns list emojis in format:
+  `{"000", "/emoji/freespeechextremist.com/000.png", ["Custom"]}`
+  """
+  @spec load() :: list(emoji)
+  def load do
+    emoji_dir_path = Path.join(Config.get!([:instance, :static_dir]), "emoji")
+
+    emoji_groups = Config.get([:emoji, :groups])
+
+    emojis =
+      case File.ls(emoji_dir_path) do
+        {:error, :enoent} ->
+          # The custom emoji directory doesn't exist,
+          # don't do anything
+          []
+
+        {:error, e} ->
+          # There was some other error
+          Logger.error("Could not access the custom emoji directory #{emoji_dir_path}: #{e}")
+          []
+
+        {:ok, results} ->
+          grouped =
+            Enum.group_by(results, fn file ->
+              File.dir?(Path.join(emoji_dir_path, file))
+            end)
+
+          packs = grouped[true] || []
+          files = grouped[false] || []
+
+          # Print the packs we've found
+          Logger.info("Found emoji packs: #{Enum.join(packs, ", ")}")
+
+          if not Enum.empty?(files) do
+            Logger.warn(
+              "Found files in the emoji folder. These will be ignored, please move them to a subdirectory\nFound files: #{
+                Enum.join(files, ", ")
+              }"
+            )
+          end
+
+          emojis =
+            Enum.flat_map(packs, fn pack ->
+              load_pack(Path.join(emoji_dir_path, pack), emoji_groups)
+            end)
+
+          Emoji.clear_all()
+          emojis
+      end
+
+    # Compat thing for old custom emoji handling & default emoji,
+    # it should run even if there are no emoji packs
+    shortcode_globs = Config.get([:emoji, :shortcode_globs], [])
+
+    emojis_txt =
+      (load_from_file("config/emoji.txt", emoji_groups) ++
+         load_from_file("config/custom_emoji.txt", emoji_groups) ++
+         load_from_globs(shortcode_globs, emoji_groups))
+      |> Enum.reject(fn value -> value == nil end)
+
+    Enum.map(emojis ++ emojis_txt, &prepare_emoji/1)
+  end
+
+  defp prepare_emoji({code, _, _} = emoji), do: {code, Emoji.build(emoji)}
+
+  defp load_pack(pack_dir, emoji_groups) do
+    pack_name = Path.basename(pack_dir)
+
+    pack_file = Path.join(pack_dir, "pack.json")
+
+    if File.exists?(pack_file) do
+      contents = Jason.decode!(File.read!(pack_file))
+
+      contents["files"]
+      |> Enum.map(fn {name, rel_file} ->
+        filename = Path.join("/emoji/#{pack_name}", rel_file)
+        {name, filename, ["pack:#{pack_name}"]}
+      end)
+    else
+      # Load from emoji.txt / all files
+      emoji_txt = Path.join(pack_dir, "emoji.txt")
+
+      if File.exists?(emoji_txt) do
+        load_from_file(emoji_txt, emoji_groups)
+      else
+        extensions = Pleroma.Config.get([:emoji, :pack_extensions])
+
+        Logger.info(
+          "No emoji.txt found for pack \"#{pack_name}\", assuming all #{
+            Enum.join(extensions, ", ")
+          } files are emoji"
+        )
+
+        make_shortcode_to_file_map(pack_dir, extensions)
+        |> Enum.map(fn {shortcode, rel_file} ->
+          filename = Path.join("/emoji/#{pack_name}", rel_file)
+
+          {shortcode, filename, [to_string(match_extra(emoji_groups, filename))]}
+        end)
+      end
+    end
+  end
+
+  def make_shortcode_to_file_map(pack_dir, exts) do
+    find_all_emoji(pack_dir, exts)
+    |> Enum.map(&Path.relative_to(&1, pack_dir))
+    |> Enum.map(fn f -> {f |> Path.basename() |> Path.rootname(), f} end)
+    |> Enum.into(%{})
+  end
+
+  def find_all_emoji(dir, exts) do
+    dir
+    |> File.ls!()
+    |> Enum.flat_map(fn f ->
+      filepath = Path.join(dir, f)
+
+      if File.dir?(filepath) do
+        find_all_emoji(filepath, exts)
+      else
+        [filepath]
+      end
+    end)
+    |> Enum.filter(fn f -> Path.extname(f) in exts end)
+  end
+
+  defp load_from_file(file, emoji_groups) do
+    if File.exists?(file) do
+      load_from_file_stream(File.stream!(file), emoji_groups)
+    else
+      []
+    end
+  end
+
+  defp load_from_file_stream(stream, emoji_groups) do
+    stream
+    |> Stream.map(&String.trim/1)
+    |> Stream.map(fn line ->
+      case String.split(line, ~r/,\s*/) do
+        [name, file] ->
+          {name, file, [to_string(match_extra(emoji_groups, file))]}
+
+        [name, file | tags] ->
+          {name, file, tags}
+
+        _ ->
+          nil
+      end
+    end)
+    |> Enum.to_list()
+  end
+
+  defp load_from_globs(globs, emoji_groups) do
+    static_path = Path.join(:code.priv_dir(:pleroma), "static")
+
+    paths =
+      Enum.map(globs, fn glob ->
+        Path.join(static_path, glob)
+        |> Path.wildcard()
+      end)
+      |> Enum.concat()
+
+    Enum.map(paths, fn path ->
+      tag = match_extra(emoji_groups, Path.join("/", Path.relative_to(path, static_path)))
+      shortcode = Path.basename(path, Path.extname(path))
+      external_path = Path.join("/", Path.relative_to(path, static_path))
+      {shortcode, external_path, [to_string(tag)]}
+    end)
+  end
+
+  @doc """
+  Finds a matching group for the given emoji filename
+  """
+  @spec match_extra(group_patterns(), String.t()) :: atom() | nil
+  def match_extra(group_patterns, filename) do
+    match_group_patterns(group_patterns, fn pattern ->
+      case pattern do
+        %Regex{} = regex -> Regex.match?(regex, filename)
+        string when is_binary(string) -> filename == string
+      end
+    end)
+  end
+
+  defp match_group_patterns(group_patterns, matcher) do
+    Enum.find_value(group_patterns, fn {group, patterns} ->
+      patterns =
+        patterns
+        |> List.wrap()
+        |> Enum.map(fn pattern ->
+          if String.contains?(pattern, "*") do
+            ~r(#{String.replace(pattern, "*", ".*")})
+          else
+            pattern
+          end
+        end)
+
+      Enum.any?(patterns, matcher) && group
+    end)
+  end
+end
--- a/lib/pleroma/filter.ex
+++ b/lib/pleroma/filter.ex
@ -12,7 +12,7 @@ defmodule Pleroma.Filter do
  alias Pleroma.User

  schema "filters" do
-    belongs_to(:user, User, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    field(:filter_id, :integer)
    field(:hide, :boolean, default: false)
    field(:whole_word, :boolean, default: true)
--- a/lib/pleroma/flake_id.ex
+++ b/lib/pleroma/flake_id.ex
@ -1,182 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.FlakeId do
-  @moduledoc """
-  Flake is a decentralized, k-ordered id generation service.
-
-  Adapted from:
-
-  * [flaky](https://github.com/nirvana/flaky), released under the terms of the Truly Free License,
-  * [Flake](https://github.com/boundary/flake), Copyright 2012, Boundary, Apache License, Version 2.0
-  """
-
-  @type t :: binary
-
-  @behaviour Ecto.Type
-  use GenServer
-  require Logger
-  alias __MODULE__
-  import Kernel, except: [to_string: 1]
-
-  defstruct node: nil, time: 0, sq: 0
-
-  @doc "Converts a binary Flake to a String"
-  def to_string(<<0::integer-size(64), id::integer-size(64)>>) do
-    Kernel.to_string(id)
-  end
-
-  def to_string(<<_::integer-size(64), _::integer-size(48), _::integer-size(16)>> = flake) do
-    encode_base62(flake)
-  end
-
-  def to_string(s), do: s
-
-  def from_string(int) when is_integer(int) do
-    from_string(Kernel.to_string(int))
-  end
-
-  for i <- [-1, 0] do
-    def from_string(unquote(i)), do: <<0::integer-size(128)>>
-    def from_string(unquote(Kernel.to_string(i))), do: <<0::integer-size(128)>>
-  end
-
-  def from_string(<<_::integer-size(128)>> = flake), do: flake
-
-  def from_string(string) when is_binary(string) and byte_size(string) < 18 do
-    case Integer.parse(string) do
-      {id, ""} -> <<0::integer-size(64), id::integer-size(64)>>
-      _ -> nil
-    end
-  end
-
-  def from_string(string) do
-    string |> decode_base62 |> from_integer
-  end
-
-  def to_integer(<<integer::integer-size(128)>>), do: integer
-
-  def from_integer(integer) do
-    <<_time::integer-size(64), _node::integer-size(48), _seq::integer-size(16)>> =
-      <<integer::integer-size(128)>>
-  end
-
-  @doc "Generates a Flake"
-  @spec get :: binary
-  def get, do: to_string(:gen_server.call(:flake, :get))
-
-  # checks that ID is is valid FlakeID
-  #
-  @spec is_flake_id?(String.t()) :: boolean
-  def is_flake_id?(id), do: is_flake_id?(String.to_charlist(id), true)
-  defp is_flake_id?([c | cs], true) when c >= ?0 and c <= ?9, do: is_flake_id?(cs, true)
-  defp is_flake_id?([c | cs], true) when c >= ?A and c <= ?Z, do: is_flake_id?(cs, true)
-  defp is_flake_id?([c | cs], true) when c >= ?a and c <= ?z, do: is_flake_id?(cs, true)
-  defp is_flake_id?([], true), do: true
-  defp is_flake_id?(_, _), do: false
-
-  # -- Ecto.Type API
-  @impl Ecto.Type
-  def type, do: :uuid
-
-  @impl Ecto.Type
-  def cast(value) do
-    {:ok, FlakeId.to_string(value)}
-  end
-
-  @impl Ecto.Type
-  def load(value) do
-    {:ok, FlakeId.to_string(value)}
-  end
-
-  @impl Ecto.Type
-  def dump(value) do
-    {:ok, FlakeId.from_string(value)}
-  end
-
-  def autogenerate, do: get()
-
-  # -- GenServer API
-  def start_link(_) do
-    :gen_server.start_link({:local, :flake}, __MODULE__, [], [])
-  end
-
-  @impl GenServer
-  def init([]) do
-    {:ok, %FlakeId{node: worker_id(), time: time()}}
-  end
-
-  @impl GenServer
-  def handle_call(:get, _from, state) do
-    {flake, new_state} = get(time(), state)
-    {:reply, flake, new_state}
-  end
-
-  # Matches when the calling time is the same as the state time. Incr. sq
-  defp get(time, %FlakeId{time: time, node: node, sq: seq}) do
-    new_state = %FlakeId{time: time, node: node, sq: seq + 1}
-    {gen_flake(new_state), new_state}
-  end
-
-  # Matches when the times are different, reset sq
-  defp get(newtime, %FlakeId{time: time, node: node}) when newtime > time do
-    new_state = %FlakeId{time: newtime, node: node, sq: 0}
-    {gen_flake(new_state), new_state}
-  end
-
-  # Error when clock is running backwards
-  defp get(newtime, %FlakeId{time: time}) when newtime < time do
-    {:error, :clock_running_backwards}
-  end
-
-  defp gen_flake(%FlakeId{time: time, node: node, sq: seq}) do
-    <<time::integer-size(64), node::integer-size(48), seq::integer-size(16)>>
-  end
-
-  defp nthchar_base62(n) when n <= 9, do: ?0 + n
-  defp nthchar_base62(n) when n <= 35, do: ?A + n - 10
-  defp nthchar_base62(n), do: ?a + n - 36
-
-  defp encode_base62(<<integer::integer-size(128)>>) do
-    integer
-    |> encode_base62([])
-    |> List.to_string()
-  end
-
-  defp encode_base62(int, acc) when int < 0, do: encode_base62(-int, acc)
-  defp encode_base62(int, []) when int == 0, do: '0'
-  defp encode_base62(int, acc) when int == 0, do: acc
-
-  defp encode_base62(int, acc) do
-    r = rem(int, 62)
-    id = div(int, 62)
-    acc = [nthchar_base62(r) | acc]
-    encode_base62(id, acc)
-  end
-
-  defp decode_base62(s) do
-    decode_base62(String.to_charlist(s), 0)
-  end
-
-  defp decode_base62([c | cs], acc) when c >= ?0 and c <= ?9,
-    do: decode_base62(cs, 62 * acc + (c - ?0))
-
-  defp decode_base62([c | cs], acc) when c >= ?A and c <= ?Z,
-    do: decode_base62(cs, 62 * acc + (c - ?A + 10))
-
-  defp decode_base62([c | cs], acc) when c >= ?a and c <= ?z,
-    do: decode_base62(cs, 62 * acc + (c - ?a + 36))
-
-  defp decode_base62([], acc), do: acc
-
-  defp time do
-    {mega_seconds, seconds, micro_seconds} = :erlang.timestamp()
-    1_000_000_000 * mega_seconds + seconds * 1000 + :erlang.trunc(micro_seconds / 1000)
-  end
-
-  defp worker_id do
-    <<worker::integer-size(48)>> = :crypto.strong_rand_bytes(6)
-    worker
-  end
-end
--- a/lib/pleroma/formatter.ex
+++ b/lib/pleroma/formatter.ex
@ -3,10 +3,8 @@
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Pleroma.Formatter do
-  alias Pleroma.Emoji
  alias Pleroma.HTML
  alias Pleroma.User
-  alias Pleroma.Web.MediaProxy

  @safe_mention_regex ~r/^(\s*(?<mentions>(@.+?\s+){1,})+)(?<rest>.*)/s
  @link_regex ~r"((?:http(s)?:\/\/)?[\w.-]+(?:\.[\w\.-]+)+[\w\-\._~%:/?#[\]@!\$&'\(\)\*\+,;=.]+)|[0-9a-z+\-\.]+:[0-9a-z$-_.+!*'(),]+"ui
@ -36,9 +34,9 @@ defmodule Pleroma.Formatter do
        nickname_text = get_nickname_text(nickname, opts)

        link =
-          "<span class='h-card'><a data-user='#{id}' class='u-url mention' href='#{ap_id}'>@<span>#{
+          ~s(<span class="h-card"><a data-user="#{id}" class="u-url mention" href="#{ap_id}" rel="ugc">@<span>#{
            nickname_text
-          }</span></a></span>"
+          }</span></a></span>)

        {link, %{acc | mentions: MapSet.put(acc.mentions, {"@" <> nickname, user})}}

@ -50,7 +48,7 @@ defmodule Pleroma.Formatter do
  def hashtag_handler("#" <> tag = tag_text, _buffer, _opts, acc) do
    tag = String.downcase(tag)
    url = "#{Pleroma.Web.base_url()}/tag/#{tag}"
-    link = "<a class='hashtag' data-tag='#{tag}' href='#{url}' rel='tag'>#{tag_text}</a>"
+    link = ~s(<a class="hashtag" data-tag="#{tag}" href="#{url}" rel="tag ugc">#{tag_text}</a>)

    {link, %{acc | tags: MapSet.put(acc.tags, {tag_text, tag})}}
  end
@ -100,51 +98,6 @@ defmodule Pleroma.Formatter do
    end
  end

-  def emojify(text) do
-    emojify(text, Emoji.get_all())
-  end
-
-  def emojify(text, nil), do: text
-
-  def emojify(text, emoji, strip \\ false) do
-    Enum.reduce(emoji, text, fn emoji_data, text ->
-      emoji = HTML.strip_tags(elem(emoji_data, 0))
-      file = HTML.strip_tags(elem(emoji_data, 1))
-
-      html =
-        if not strip do
-          "<img class='emoji' alt='#{emoji}' title='#{emoji}' src='#{MediaProxy.url(file)}' />"
-        else
-          ""
-        end
-
-      String.replace(text, ":#{emoji}:", html) |> HTML.filter_tags()
-    end)
-  end
-
-  def demojify(text) do
-    emojify(text, Emoji.get_all(), true)
-  end
-
-  def demojify(text, nil), do: text
-
-  @doc "Outputs a list of the emoji-shortcodes in a text"
-  def get_emoji(text) when is_binary(text) do
-    Enum.filter(Emoji.get_all(), fn {emoji, _, _} -> String.contains?(text, ":#{emoji}:") end)
-  end
-
-  def get_emoji(_), do: []
-
-  @doc "Outputs a list of the emoji-Maps in a text"
-  def get_emoji_map(text) when is_binary(text) do
-    get_emoji(text)
-    |> Enum.reduce(%{}, fn {name, file, _group}, acc ->
-      Map.put(acc, name, "#{Pleroma.Web.Endpoint.static_url()}#{file}")
-    end)
-  end
-
-  def get_emoji_map(_), do: []
-
  def html_escape({text, mentions, hashtags}, type) do
    {html_escape(text, type), mentions, hashtags}
  end
--- a/lib/pleroma/healthcheck.ex
+++ b/lib/pleroma/healthcheck.ex
@ -9,6 +9,7 @@ defmodule Pleroma.Healthcheck do
  alias Pleroma.Healthcheck
  alias Pleroma.Repo

+  @derive Jason.Encoder
  defstruct pool_size: 0,
            active: 0,
            idle: 0,
--- a/lib/pleroma/html.ex
+++ b/lib/pleroma/html.ex
@ -184,7 +184,8 @@ defmodule Pleroma.HTML.Scrubber.Default do
    "tag",
    "nofollow",
    "noopener",
-    "noreferrer"
+    "noreferrer",
+    "ugc"
  ])

  Meta.allow_tag_with_these_attributes("a", ["name", "title"])
@ -304,7 +305,8 @@ defmodule Pleroma.HTML.Scrubber.LinksOnly do
    "nofollow",
    "noopener",
    "noreferrer",
-    "me"
+    "me",
+    "ugc"
  ])

  Meta.allow_tag_with_these_attributes("a", ["name", "title"])
--- a/lib/pleroma/instances/instance.ex
+++ b/lib/pleroma/instances/instance.ex
@ -90,7 +90,7 @@ defmodule Pleroma.Instances.Instance do
  def set_unreachable(url_or_host, unreachable_since \\ nil)

  def set_unreachable(url_or_host, unreachable_since) when is_binary(url_or_host) do
-    unreachable_since = unreachable_since || DateTime.utc_now()
+    unreachable_since = parse_datetime(unreachable_since) || NaiveDateTime.utc_now()
    host = host(url_or_host)
    existing_record = Repo.get_by(Instance, %{host: host})

@ -114,4 +114,10 @@ defmodule Pleroma.Instances.Instance do
  end

  def set_unreachable(_, _), do: {:error, nil}
+
+  defp parse_datetime(datetime) when is_binary(datetime) do
+    NaiveDateTime.from_iso8601(datetime)
+  end
+
+  defp parse_datetime(datetime), do: datetime
 end
--- a/lib/pleroma/list.ex
+++ b/lib/pleroma/list.ex
@ -13,7 +13,7 @@ defmodule Pleroma.List do
  alias Pleroma.User

  schema "lists" do
-    belongs_to(:user, User, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    field(:title, :string)
    field(:following, {:array, :string}, default: [])
    field(:ap_id, :string)
@ -84,22 +84,11 @@ defmodule Pleroma.List do
  end

  # Get lists to which the account belongs.
-  def get_lists_account_belongs(%User{} = owner, account_id) do
-    user = User.get_cached_by_id(account_id)
-
-    query =
-      from(
-        l in Pleroma.List,
-        where:
-          l.user_id == ^owner.id and
-            fragment(
-              "? = ANY(?)",
-              ^user.follower_address,
-              l.following
-            )
-      )
-
-    Repo.all(query)
+  def get_lists_account_belongs(%User{} = owner, user) do
+    Pleroma.List
+    |> where([l], l.user_id == ^owner.id)
+    |> where([l], fragment("? = ANY(?)", ^user.follower_address, l.following))
+    |> Repo.all()
  end

  def rename(%Pleroma.List{} = list, title) do
@ -109,15 +98,19 @@ defmodule Pleroma.List do
  end

  def create(title, %User{} = creator) do
-    list = %Pleroma.List{user_id: creator.id, title: title}
+    changeset = title_changeset(%Pleroma.List{user_id: creator.id}, %{title: title})

-    Repo.transaction(fn ->
-      list = Repo.insert!(list)
+    if changeset.valid? do
+      Repo.transaction(fn ->
+        list = Repo.insert!(changeset)

-      list
-      |> change(ap_id: "#{creator.ap_id}/lists/#{list.id}")
-      |> Repo.update!()
-    end)
+        list
+        |> change(ap_id: "#{creator.ap_id}/lists/#{list.id}")
+        |> Repo.update!()
+      end)
+    else
+      {:error, changeset}
+    end
  end

  def follow(%Pleroma.List{following: following} = list, %User{} = followed) do
--- a/lib/pleroma/moderation_log.ex
+++ b/lib/pleroma/moderation_log.ex
@ -0,0 +1,554 @@
+defmodule Pleroma.ModerationLog do
+  use Ecto.Schema
+
+  alias Pleroma.Activity
+  alias Pleroma.ModerationLog
+  alias Pleroma.Repo
+  alias Pleroma.User
+
+  import Ecto.Query
+
+  schema "moderation_log" do
+    field(:data, :map)
+
+    timestamps()
+  end
+
+  def get_all(params) do
+    base_query =
+      get_all_query()
+      |> maybe_filter_by_date(params)
+      |> maybe_filter_by_user(params)
+      |> maybe_filter_by_search(params)
+
+    query_with_pagination = base_query |> paginate_query(params)
+
+    %{
+      items: Repo.all(query_with_pagination),
+      count: Repo.aggregate(base_query, :count, :id)
+    }
+  end
+
+  defp maybe_filter_by_date(query, %{start_date: nil, end_date: nil}), do: query
+
+  defp maybe_filter_by_date(query, %{start_date: start_date, end_date: nil}) do
+    from(q in query,
+      where: q.inserted_at >= ^parse_datetime(start_date)
+    )
+  end
+
+  defp maybe_filter_by_date(query, %{start_date: nil, end_date: end_date}) do
+    from(q in query,
+      where: q.inserted_at <= ^parse_datetime(end_date)
+    )
+  end
+
+  defp maybe_filter_by_date(query, %{start_date: start_date, end_date: end_date}) do
+    from(q in query,
+      where: q.inserted_at >= ^parse_datetime(start_date),
+      where: q.inserted_at <= ^parse_datetime(end_date)
+    )
+  end
+
+  defp maybe_filter_by_user(query, %{user_id: nil}), do: query
+
+  defp maybe_filter_by_user(query, %{user_id: user_id}) do
+    from(q in query,
+      where: fragment("(?)->'actor'->>'id' = ?", q.data, ^user_id)
+    )
+  end
+
+  defp maybe_filter_by_search(query, %{search: search}) when is_nil(search) or search == "",
+    do: query
+
+  defp maybe_filter_by_search(query, %{search: search}) do
+    from(q in query,
+      where: fragment("(?)->>'message' ILIKE ?", q.data, ^"%#{search}%")
+    )
+  end
+
+  defp paginate_query(query, %{page: page, page_size: page_size}) do
+    from(q in query,
+      limit: ^page_size,
+      offset: ^((page - 1) * page_size)
+    )
+  end
+
+  defp get_all_query do
+    from(q in __MODULE__,
+      order_by: [desc: q.inserted_at]
+    )
+  end
+
+  defp parse_datetime(datetime) do
+    {:ok, parsed_datetime, _} = DateTime.from_iso8601(datetime)
+
+    parsed_datetime
+  end
+
+  @spec insert_log(%{actor: User, subject: User, action: String.t(), permission: String.t()}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        subject: %User{} = subject,
+        action: action,
+        permission: permission
+      }) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "subject" => user_to_map(subject),
+        "action" => action,
+        "permission" => permission,
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{actor: User, subject: User, action: String.t()}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        action: "report_update",
+        subject: %Activity{data: %{"type" => "Flag"}} = subject
+      }) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => "report_update",
+        "subject" => report_to_map(subject),
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{actor: User, subject: Activity, action: String.t(), text: String.t()}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        action: "report_response",
+        subject: %Activity{} = subject,
+        text: text
+      }) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => "report_response",
+        "subject" => report_to_map(subject),
+        "text" => text,
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{
+          actor: User,
+          subject: Activity,
+          action: String.t(),
+          sensitive: String.t(),
+          visibility: String.t()
+        }) :: {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        action: "status_update",
+        subject: %Activity{} = subject,
+        sensitive: sensitive,
+        visibility: visibility
+      }) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => "status_update",
+        "subject" => status_to_map(subject),
+        "sensitive" => sensitive,
+        "visibility" => visibility,
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{actor: User, action: String.t(), subject_id: String.t()}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        action: "status_delete",
+        subject_id: subject_id
+      }) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => "status_delete",
+        "subject_id" => subject_id,
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{actor: User, subject: User, action: String.t()}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{actor: %User{} = actor, subject: subject, action: action}) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => action,
+        "subject" => user_to_map(subject),
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{actor: User, subjects: [User], action: String.t()}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{actor: %User{} = actor, subjects: subjects, action: action}) do
+    subjects = Enum.map(subjects, &user_to_map/1)
+
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => action,
+        "subjects" => subjects,
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{actor: User, action: String.t(), followed: User, follower: User}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        followed: %User{} = followed,
+        follower: %User{} = follower,
+        action: "follow"
+      }) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => "follow",
+        "followed" => user_to_map(followed),
+        "follower" => user_to_map(follower),
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{actor: User, action: String.t(), followed: User, follower: User}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        followed: %User{} = followed,
+        follower: %User{} = follower,
+        action: "unfollow"
+      }) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => "unfollow",
+        "followed" => user_to_map(followed),
+        "follower" => user_to_map(follower),
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{
+          actor: User,
+          action: String.t(),
+          nicknames: [String.t()],
+          tags: [String.t()]
+        }) :: {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        nicknames: nicknames,
+        tags: tags,
+        action: action
+      }) do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "nicknames" => nicknames,
+        "tags" => tags,
+        "action" => action,
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log(%{actor: User, action: String.t(), target: String.t()}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{
+        actor: %User{} = actor,
+        action: action,
+        target: target
+      })
+      when action in ["relay_follow", "relay_unfollow"] do
+    %ModerationLog{
+      data: %{
+        "actor" => user_to_map(actor),
+        "action" => action,
+        "target" => target,
+        "message" => ""
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
+  @spec insert_log_entry_with_message(ModerationLog) :: {:ok, ModerationLog} | {:error, any}
+
+  defp insert_log_entry_with_message(entry) do
+    entry.data["message"]
+    |> put_in(get_log_entry_message(entry))
+    |> Repo.insert()
+  end
+
+  defp user_to_map(%User{} = user) do
+    user
+    |> Map.from_struct()
+    |> Map.take([:id, :nickname])
+    |> Map.new(fn {k, v} -> {Atom.to_string(k), v} end)
+    |> Map.put("type", "user")
+  end
+
+  defp report_to_map(%Activity{} = report) do
+    %{
+      "type" => "report",
+      "id" => report.id,
+      "state" => report.data["state"]
+    }
+  end
+
+  defp status_to_map(%Activity{} = status) do
+    %{
+      "type" => "status",
+      "id" => status.id
+    }
+  end
+
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => action,
+          "followed" => %{"nickname" => followed_nickname},
+          "follower" => %{"nickname" => follower_nickname}
+        }
+      }) do
+    "@#{actor_nickname} made @#{follower_nickname} #{action} @#{followed_nickname}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "delete",
+          "subject" => %{"nickname" => subject_nickname, "type" => "user"}
+        }
+      }) do
+    "@#{actor_nickname} deleted user @#{subject_nickname}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "create",
+          "subjects" => subjects
+        }
+      }) do
+    nicknames =
+      subjects
+      |> Enum.map(&"@#{&1["nickname"]}")
+      |> Enum.join(", ")
+
+    "@#{actor_nickname} created users: #{nicknames}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "activate",
+          "subject" => %{"nickname" => subject_nickname, "type" => "user"}
+        }
+      }) do
+    "@#{actor_nickname} activated user @#{subject_nickname}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "deactivate",
+          "subject" => %{"nickname" => subject_nickname, "type" => "user"}
+        }
+      }) do
+    "@#{actor_nickname} deactivated user @#{subject_nickname}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "nicknames" => nicknames,
+          "tags" => tags,
+          "action" => "tag"
+        }
+      }) do
+    nicknames_string =
+      nicknames
+      |> Enum.map(&"@#{&1}")
+      |> Enum.join(", ")
+
+    tags_string = tags |> Enum.join(", ")
+
+    "@#{actor_nickname} added tags: #{tags_string} to users: #{nicknames_string}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "nicknames" => nicknames,
+          "tags" => tags,
+          "action" => "untag"
+        }
+      }) do
+    nicknames_string =
+      nicknames
+      |> Enum.map(&"@#{&1}")
+      |> Enum.join(", ")
+
+    tags_string = tags |> Enum.join(", ")
+
+    "@#{actor_nickname} removed tags: #{tags_string} from users: #{nicknames_string}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "grant",
+          "subject" => %{"nickname" => subject_nickname},
+          "permission" => permission
+        }
+      }) do
+    "@#{actor_nickname} made @#{subject_nickname} #{permission}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "revoke",
+          "subject" => %{"nickname" => subject_nickname},
+          "permission" => permission
+        }
+      }) do
+    "@#{actor_nickname} revoked #{permission} role from @#{subject_nickname}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "relay_follow",
+          "target" => target
+        }
+      }) do
+    "@#{actor_nickname} followed relay: #{target}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "relay_unfollow",
+          "target" => target
+        }
+      }) do
+    "@#{actor_nickname} unfollowed relay: #{target}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "report_update",
+          "subject" => %{"id" => subject_id, "state" => state, "type" => "report"}
+        }
+      }) do
+    "@#{actor_nickname} updated report ##{subject_id} with '#{state}' state"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "report_response",
+          "subject" => %{"id" => subject_id, "type" => "report"},
+          "text" => text
+        }
+      }) do
+    "@#{actor_nickname} responded with '#{text}' to report ##{subject_id}"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "status_update",
+          "subject" => %{"id" => subject_id, "type" => "status"},
+          "sensitive" => nil,
+          "visibility" => visibility
+        }
+      }) do
+    "@#{actor_nickname} updated status ##{subject_id}, set visibility: '#{visibility}'"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "status_update",
+          "subject" => %{"id" => subject_id, "type" => "status"},
+          "sensitive" => sensitive,
+          "visibility" => nil
+        }
+      }) do
+    "@#{actor_nickname} updated status ##{subject_id}, set sensitive: '#{sensitive}'"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "status_update",
+          "subject" => %{"id" => subject_id, "type" => "status"},
+          "sensitive" => sensitive,
+          "visibility" => visibility
+        }
+      }) do
+    "@#{actor_nickname} updated status ##{subject_id}, set sensitive: '#{sensitive}', visibility: '#{
+      visibility
+    }'"
+  end
+
+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "status_delete",
+          "subject_id" => subject_id
+        }
+      }) do
+    "@#{actor_nickname} deleted status ##{subject_id}"
+  end
+end
--- a/lib/pleroma/notification.ex
+++ b/lib/pleroma/notification.ex
@ -22,8 +22,8 @@ defmodule Pleroma.Notification do

  schema "notifications" do
    field(:seen, :boolean, default: false)
-    belongs_to(:user, User, type: Pleroma.FlakeId)
-    belongs_to(:activity, Activity, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
+    belongs_to(:activity, Activity, type: FlakeId.Ecto.CompatType)

    timestamps()
  end
@ -102,15 +102,33 @@ defmodule Pleroma.Notification do
        n in Notification,
        where: n.user_id == ^user_id,
        where: n.id <= ^id,
+        where: n.seen == false,
        update: [
          set: [
            seen: true,
            updated_at: ^NaiveDateTime.utc_now()
          ]
-        ]
+        ],
+        # Ideally we would preload object and activities here
+        # but Ecto does not support preloads in update_all
+        select: n.id
      )

-    Repo.update_all(query, [])
+    {_, notification_ids} = Repo.update_all(query, [])
+
+    Notification
+    |> where([n], n.id in ^notification_ids)
+    |> join(:inner, [n], activity in assoc(n, :activity))
+    |> join(:left, [n, a], object in Object,
+      on:
+        fragment(
+          "(?->>'id') = COALESCE((? -> 'object'::text) ->> 'id'::text)",
+          object.data,
+          a.data
+        )
+    )
+    |> preload([n, a, o], activity: {a, object: o})
+    |> Repo.all()
  end

  def read_one(%User{} = user, notification_id) do
@ -192,8 +210,10 @@ defmodule Pleroma.Notification do
    unless skip?(activity, user) do
      notification = %Notification{user_id: user.id, activity: activity}
      {:ok, notification} = Repo.insert(notification)
-      Streamer.stream("user", notification)
-      Streamer.stream("user:notification", notification)
+
+      ["user", "user:notification"]
+      |> Streamer.stream(notification)
+
      Push.send(notification)
      notification
    end
--- a/lib/pleroma/object.ex
+++ b/lib/pleroma/object.ex
@ -38,6 +38,24 @@ defmodule Pleroma.Object do
  def get_by_id(nil), do: nil
  def get_by_id(id), do: Repo.get(Object, id)

+  def get_by_id_and_maybe_refetch(id, opts \\ []) do
+    %{updated_at: updated_at} = object = get_by_id(id)
+
+    if opts[:interval] &&
+         NaiveDateTime.diff(NaiveDateTime.utc_now(), updated_at) > opts[:interval] do
+      case Fetcher.refetch_object(object) do
+        {:ok, %Object{} = object} ->
+          object
+
+        e ->
+          Logger.error("Couldn't refresh #{object.data["id"]}:\n#{inspect(e)}")
+          object
+      end
+    else
+      object
+    end
+  end
+
  def get_by_ap_id(nil), do: nil

  def get_by_ap_id(ap_id) do
@ -130,14 +148,16 @@ defmodule Pleroma.Object do
  def delete(%Object{data: %{"id" => id}} = object) do
    with {:ok, _obj} = swap_object_with_tombstone(object),
         deleted_activity = Activity.delete_by_ap_id(id),
-         {:ok, true} <- Cachex.del(:object_cache, "object:#{id}") do
+         {:ok, true} <- Cachex.del(:object_cache, "object:#{id}"),
+         {:ok, _} <- Cachex.del(:web_resp_cache, URI.parse(id).path) do
      {:ok, object, deleted_activity}
    end
  end

  def prune(%Object{data: %{"id" => id}} = object) do
    with {:ok, object} <- Repo.delete(object),
-         {:ok, true} <- Cachex.del(:object_cache, "object:#{id}") do
+         {:ok, true} <- Cachex.del(:object_cache, "object:#{id}"),
+         {:ok, _} <- Cachex.del(:web_resp_cache, URI.parse(id).path) do
      {:ok, object}
    end
  end
@ -150,8 +170,6 @@ defmodule Pleroma.Object do
  def update_and_set_cache(changeset) do
    with {:ok, object} <- Repo.update(changeset) do
      set_cache(object)
-    else
-      e -> e
    end
  end

@ -230,4 +248,11 @@ defmodule Pleroma.Object do
      _ -> :noop
    end
  end
+
+  @doc "Updates data field of an object"
+  def update_data(%Object{data: data} = object, attrs \\ %{}) do
+    object
+    |> Object.change(%{data: Map.merge(data || %{}, attrs)})
+    |> Repo.update()
+  end
 end
--- a/lib/pleroma/object/fetcher.ex
+++ b/lib/pleroma/object/fetcher.ex
@ -6,18 +6,40 @@ defmodule Pleroma.Object.Fetcher do
  alias Pleroma.HTTP
  alias Pleroma.Object
  alias Pleroma.Object.Containment
+  alias Pleroma.Repo
  alias Pleroma.Signature
  alias Pleroma.Web.ActivityPub.InternalFetchActor
  alias Pleroma.Web.ActivityPub.Transmogrifier
  alias Pleroma.Web.OStatus

  require Logger
+  require Pleroma.Constants

-  defp reinject_object(data) do
+  defp touch_changeset(changeset) do
+    updated_at =
+      NaiveDateTime.utc_now()
+      |> NaiveDateTime.truncate(:second)
+
+    Ecto.Changeset.put_change(changeset, :updated_at, updated_at)
+  end
+
+  defp maybe_reinject_internal_fields(data, %{data: %{} = old_data}) do
+    internal_fields = Map.take(old_data, Pleroma.Constants.object_internal_fields())
+
+    Map.merge(data, internal_fields)
+  end
+
+  defp maybe_reinject_internal_fields(data, _), do: data
+
+  @spec reinject_object(struct(), map()) :: {:ok, Object.t()} | {:error, any()}
+  defp reinject_object(struct, data) do
    Logger.debug("Reinjecting object #{data["id"]}")

    with data <- Transmogrifier.fix_object(data),
-         {:ok, object} <- Object.create(data) do
+         data <- maybe_reinject_internal_fields(data, struct),
+         changeset <- Object.change(struct, %{data: data}),
+         changeset <- touch_changeset(changeset),
+         {:ok, object} <- Repo.insert_or_update(changeset) do
      {:ok, object}
    else
      e ->
@ -26,55 +48,68 @@ defmodule Pleroma.Object.Fetcher do
    end
  end

+  def refetch_object(%Object{data: %{"id" => id}} = object) do
+    with {:local, false} <- {:local, String.starts_with?(id, Pleroma.Web.base_url() <> "/")},
+         {:ok, data} <- fetch_and_contain_remote_object_from_id(id),
+         {:ok, object} <- reinject_object(object, data) do
+      {:ok, object}
+    else
+      {:local, true} -> object
+      e -> {:error, e}
+    end
+  end
+
  # TODO:
  # This will create a Create activity, which we need internally at the moment.
  def fetch_object_from_id(id, options \\ []) do
-    if object = Object.get_cached_by_ap_id(id) do
+    with {:fetch_object, nil} <- {:fetch_object, Object.get_cached_by_ap_id(id)},
+         {:fetch, {:ok, data}} <- {:fetch, fetch_and_contain_remote_object_from_id(id)},
+         {:normalize, nil} <- {:normalize, Object.normalize(data, false)},
+         params <- prepare_activity_params(data),
+         {:containment, :ok} <- {:containment, Containment.contain_origin(id, params)},
+         {:ok, activity} <- Transmogrifier.handle_incoming(params, options),
+         {:object, _data, %Object{} = object} <-
+           {:object, data, Object.normalize(activity, false)} do
      {:ok, object}
    else
-      Logger.info("Fetching #{id} via AP")
+      {:containment, _} ->
+        {:error, "Object containment failed."}

-      with {:fetch, {:ok, data}} <- {:fetch, fetch_and_contain_remote_object_from_id(id)},
-           {:normalize, nil} <- {:normalize, Object.normalize(data, false)},
-           params <- %{
-             "type" => "Create",
-             "to" => data["to"],
-             "cc" => data["cc"],
-             # Should we seriously keep this attributedTo thing?
-             "actor" => data["actor"] || data["attributedTo"],
-             "object" => data
-           },
-           {:containment, :ok} <- {:containment, Containment.contain_origin(id, params)},
-           {:ok, activity} <- Transmogrifier.handle_incoming(params, options),
-           {:object, _data, %Object{} = object} <-
-             {:object, data, Object.normalize(activity, false)} do
+      {:error, {:reject, nil}} ->
+        {:reject, nil}
+
+      {:object, data, nil} ->
+        reinject_object(%Object{}, data)
+
+      {:normalize, object = %Object{}} ->
        {:ok, object}
-      else
-        {:containment, _} ->
-          {:error, "Object containment failed."}

-        {:error, {:reject, nil}} ->
-          {:reject, nil}
+      {:fetch_object, %Object{} = object} ->
+        {:ok, object}

-        {:object, data, nil} ->
-          reinject_object(data)
+      _e ->
+        # Only fallback when receiving a fetch/normalization error with ActivityPub
+        Logger.info("Couldn't get object via AP, trying out OStatus fetching...")

-        {:normalize, object = %Object{}} ->
-          {:ok, object}
-
-        _e ->
-          # Only fallback when receiving a fetch/normalization error with ActivityPub
-          Logger.info("Couldn't get object via AP, trying out OStatus fetching...")
-
-          # FIXME: OStatus Object Containment?
-          case OStatus.fetch_activity_from_url(id) do
-            {:ok, [activity | _]} -> {:ok, Object.normalize(activity, false)}
-            e -> e
-          end
-      end
+        # FIXME: OStatus Object Containment?
+        case OStatus.fetch_activity_from_url(id) do
+          {:ok, [activity | _]} -> {:ok, Object.normalize(activity, false)}
+          e -> e
+        end
    end
  end

+  defp prepare_activity_params(data) do
+    %{
+      "type" => "Create",
+      "to" => data["to"],
+      "cc" => data["cc"],
+      # Should we seriously keep this attributedTo thing?
+      "actor" => data["actor"] || data["attributedTo"],
+      "object" => data
+    }
+  end
+
  def fetch_object_from_id!(id, options \\ []) do
    with {:ok, object} <- fetch_object_from_id(id, options) do
      object
@ -117,9 +152,7 @@ defmodule Pleroma.Object.Fetcher do
  def fetch_and_contain_remote_object_from_id(id) when is_binary(id) do
    Logger.info("Fetching object #{id} via AP")

-    date =
-      NaiveDateTime.utc_now()
-      |> Timex.format!("{WDshort}, {0D} {Mshort} {YYYY} {h24}:{m}:{s} GMT")
+    date = Pleroma.Signature.signed_date()

    headers =
      [{:Accept, "application/activity+json"}]
--- a/lib/pleroma/pagination.ex
+++ b/lib/pleroma/pagination.ex
@ -16,6 +16,15 @@ defmodule Pleroma.Pagination do

  def fetch_paginated(query, params, type \\ :keyset)

+  def fetch_paginated(query, %{"total" => true} = params, :keyset) do
+    total = Repo.aggregate(query, :count, :id)
+
+    %{
+      total: total,
+      items: fetch_paginated(query, Map.drop(params, ["total"]), :keyset)
+    }
+  end
+
  def fetch_paginated(query, params, :keyset) do
    options = cast_params(params)

@ -25,6 +34,15 @@ defmodule Pleroma.Pagination do
    |> enforce_order(options)
  end

+  def fetch_paginated(query, %{"total" => true} = params, :offset) do
+    total = Repo.aggregate(query, :count, :id)
+
+    %{
+      total: total,
+      items: fetch_paginated(query, Map.drop(params, ["total"]), :offset)
+    }
+  end
+
  def fetch_paginated(query, params, :offset) do
    options = cast_params(params)

@ -46,6 +64,7 @@ defmodule Pleroma.Pagination do

  def paginate(query, options, :offset) do
    query
+    |> restrict(:order, options)
    |> restrict(:offset, options)
    |> restrict(:limit, options)
  end
--- a/lib/pleroma/password_reset_token.ex
+++ b/lib/pleroma/password_reset_token.ex
@ -12,7 +12,7 @@ defmodule Pleroma.PasswordResetToken do
  alias Pleroma.User

  schema "password_reset_tokens" do
-    belongs_to(:user, User, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    field(:token, :string)
    field(:used, :boolean, default: false)

--- a/lib/pleroma/plugs/cache.ex
+++ b/lib/pleroma/plugs/cache.ex
@ -0,0 +1,136 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.Cache do
+  @moduledoc """
+  Caches successful GET responses.
+
+  To enable the cache add the plug to a router pipeline or controller:
+
+      plug(Pleroma.Plugs.Cache)
+
+  ## Configuration
+
+  To configure the plug you need to pass settings as the second argument to the `plug/2` macro:
+
+      plug(Pleroma.Plugs.Cache, [ttl: nil, query_params: true])
+
+  Available options:
+
+  - `ttl`:  An expiration time (time-to-live). This value should be in milliseconds or `nil` to disable expiration. Defaults to `nil`.
+  - `query_params`: Take URL query string into account (`true`), ignore it (`false`) or limit to specific params only (list). Defaults to `true`.
+  - `tracking_fun`: A function that is called on successfull responses, no matter if the request is cached or not. It should accept a conn as the first argument and the value assigned to `tracking_fun_data` as the second.
+
+  Additionally, you can overwrite the TTL inside a controller action by assigning `cache_ttl` to the connection struct:
+
+      def index(conn, _params) do
+        ttl = 60_000 # one minute
+
+        conn
+        |> assign(:cache_ttl, ttl)
+        |> render("index.html")
+      end
+
+  """
+
+  import Phoenix.Controller, only: [current_path: 1, json: 2]
+  import Plug.Conn
+
+  @behaviour Plug
+
+  @defaults %{ttl: nil, query_params: true}
+
+  @impl true
+  def init([]), do: @defaults
+
+  def init(opts) do
+    opts = Map.new(opts)
+    Map.merge(@defaults, opts)
+  end
+
+  @impl true
+  def call(%{method: "GET"} = conn, opts) do
+    key = cache_key(conn, opts)
+
+    case Cachex.get(:web_resp_cache, key) do
+      {:ok, nil} ->
+        cache_resp(conn, opts)
+
+      {:ok, {content_type, body, tracking_fun_data}} ->
+        conn = opts.tracking_fun.(conn, tracking_fun_data)
+
+        send_cached(conn, {content_type, body})
+
+      {:ok, record} ->
+        send_cached(conn, record)
+
+      {atom, message} when atom in [:ignore, :error] ->
+        render_error(conn, message)
+    end
+  end
+
+  def call(conn, _), do: conn
+
+  # full path including query params
+  defp cache_key(conn, %{query_params: true}), do: current_path(conn)
+
+  # request path without query params
+  defp cache_key(conn, %{query_params: false}), do: conn.request_path
+
+  # request path with specific query params
+  defp cache_key(conn, %{query_params: query_params}) when is_list(query_params) do
+    query_string =
+      conn.params
+      |> Map.take(query_params)
+      |> URI.encode_query()
+
+    conn.request_path <> "?" <> query_string
+  end
+
+  defp cache_resp(conn, opts) do
+    register_before_send(conn, fn
+      %{status: 200, resp_body: body} = conn ->
+        ttl = Map.get(conn.assigns, :cache_ttl, opts.ttl)
+        key = cache_key(conn, opts)
+        content_type = content_type(conn)
+
+        conn =
+          unless opts[:tracking_fun] do
+            Cachex.put(:web_resp_cache, key, {content_type, body}, ttl: ttl)
+            conn
+          else
+            tracking_fun_data = Map.get(conn.assigns, :tracking_fun_data, nil)
+            Cachex.put(:web_resp_cache, key, {content_type, body, tracking_fun_data}, ttl: ttl)
+
+            opts.tracking_fun.(conn, tracking_fun_data)
+          end
+
+        put_resp_header(conn, "x-cache", "MISS from Pleroma")
+
+      conn ->
+        conn
+    end)
+  end
+
+  defp content_type(conn) do
+    conn
+    |> Plug.Conn.get_resp_header("content-type")
+    |> hd()
+  end
+
+  defp send_cached(conn, {content_type, body}) do
+    conn
+    |> put_resp_content_type(content_type, nil)
+    |> put_resp_header("x-cache", "HIT from Pleroma")
+    |> send_resp(:ok, body)
+    |> halt()
+  end
+
+  defp render_error(conn, message) do
+    conn
+    |> put_status(:internal_server_error)
+    |> json(%{error: message})
+    |> halt()
+  end
+end
--- a/lib/pleroma/plugs/http_signature.ex
+++ b/lib/pleroma/plugs/http_signature.ex
@ -15,7 +15,8 @@ defmodule Pleroma.Web.Plugs.HTTPSignaturePlug do
  end

  def call(conn, _opts) do
-    [signature | _] = get_req_header(conn, "signature")
+    headers = get_req_header(conn, "signature")
+    signature = Enum.at(headers, 0)

    if signature do
      # set (request-target) header to the appropriate value
--- a/lib/pleroma/plugs/remote_ip.ex
+++ b/lib/pleroma/plugs/remote_ip.ex
@ -0,0 +1,54 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.RemoteIp do
+  @moduledoc """
+  This is a shim to call [`RemoteIp`](https://git.pleroma.social/pleroma/remote_ip) but with runtime configuration.
+  """
+
+  @behaviour Plug
+
+  @headers ~w[
+    forwarded
+    x-forwarded-for
+    x-client-ip
+    x-real-ip
+  ]
+
+  # https://en.wikipedia.org/wiki/Localhost
+  # https://en.wikipedia.org/wiki/Private_network
+  @reserved ~w[
+    127.0.0.0/8
+    ::1/128
+    fc00::/7
+    10.0.0.0/8
+    172.16.0.0/12
+    192.168.0.0/16
+  ]
+
+  def init(_), do: nil
+
+  def call(conn, _) do
+    config = Pleroma.Config.get(__MODULE__, [])
+
+    if Keyword.get(config, :enabled, false) do
+      RemoteIp.call(conn, remote_ip_opts(config))
+    else
+      conn
+    end
+  end
+
+  defp remote_ip_opts(config) do
+    headers = config |> Keyword.get(:headers, @headers) |> MapSet.new()
+    reserved = Keyword.get(config, :reserved, @reserved)
+
+    proxies =
+      config
+      |> Keyword.get(:proxies, [])
+      |> Enum.concat(reserved)
+      |> Enum.map(&InetCidr.parse/1)
+
+    {headers, proxies}
+  end
+end
--- a/lib/pleroma/plugs/trailing_format_plug.ex
+++ b/lib/pleroma/plugs/trailing_format_plug.ex
@ -0,0 +1,41 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.TrailingFormatPlug do
+  @moduledoc "Calls TrailingFormatPlug for specific paths. Ideally we would just do this in the router, but TrailingFormatPlug needs to be called before Plug.Parsers."
+
+  @behaviour Plug
+  @paths [
+    "/api/statusnet",
+    "/api/statuses",
+    "/api/qvitter",
+    "/api/search",
+    "/api/account",
+    "/api/friends",
+    "/api/mutes",
+    "/api/media",
+    "/api/favorites",
+    "/api/blocks",
+    "/api/friendships",
+    "/api/users",
+    "/users",
+    "/nodeinfo",
+    "/api/help",
+    "/api/externalprofile",
+    "/notice",
+    "/api/pleroma/emoji"
+  ]
+
+  def init(opts) do
+    TrailingFormatPlug.init(opts)
+  end
+
+  for path <- @paths do
+    def call(%{request_path: unquote(path) <> _} = conn, opts) do
+      TrailingFormatPlug.call(conn, opts)
+    end
+  end
+
+  def call(conn, _opts), do: conn
+end
--- a/lib/pleroma/registration.ex
+++ b/lib/pleroma/registration.ex
@ -11,10 +11,10 @@ defmodule Pleroma.Registration do
  alias Pleroma.Repo
  alias Pleroma.User

-  @primary_key {:id, Pleroma.FlakeId, autogenerate: true}
+  @primary_key {:id, FlakeId.Ecto.CompatType, autogenerate: true}

  schema "registrations" do
-    belongs_to(:user, User, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    field(:provider, :string)
    field(:uid, :string)
    field(:info, :map, default: %{})
--- a/lib/pleroma/scheduled_activity.ex
+++ b/lib/pleroma/scheduled_activity.ex
@ -17,7 +17,7 @@ defmodule Pleroma.ScheduledActivity do
  @min_offset :timer.minutes(5)

  schema "scheduled_activities" do
-    belongs_to(:user, User, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    field(:scheduled_at, :naive_datetime)
    field(:params, :map)

--- a/lib/pleroma/scheduler.ex
+++ b/lib/pleroma/scheduler.ex
@ -0,0 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Scheduler do
+  use Quantum.Scheduler, otp_app: :pleroma
+end
--- a/lib/pleroma/signature.ex
+++ b/lib/pleroma/signature.ex
@ -53,4 +53,10 @@ defmodule Pleroma.Signature do
      HTTPSignatures.sign(private_key, user.ap_id <> "#main-key", headers)
    end
  end
+
+  def signed_date, do: signed_date(NaiveDateTime.utc_now())
+
+  def signed_date(%NaiveDateTime{} = date) do
+    Timex.format!(date, "{WDshort}, {0D} {Mshort} {YYYY} {h24}:{m}:{s} GMT")
+  end
 end
--- a/lib/pleroma/thread_mute.ex
+++ b/lib/pleroma/thread_mute.ex
@ -12,7 +12,7 @@ defmodule Pleroma.ThreadMute do
  require Ecto.Query

  schema "thread_mutes" do
-    belongs_to(:user, User, type: Pleroma.FlakeId)
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    field(:context, :string)
  end

@ -24,7 +24,7 @@ defmodule Pleroma.ThreadMute do
  end

  def query(user_id, context) do
-    user_id = Pleroma.FlakeId.from_string(user_id)
+    {:ok, user_id} = FlakeId.Ecto.CompatType.dump(user_id)

    ThreadMute
    |> Ecto.Query.where(user_id: ^user_id)
--- a/lib/pleroma/uploaders/s3.ex
+++ b/lib/pleroma/uploaders/s3.ex
@ -38,16 +38,26 @@ defmodule Pleroma.Uploaders.S3 do
  def put_file(%Pleroma.Upload{} = upload) do
    config = Config.get([__MODULE__])
    bucket = Keyword.get(config, :bucket)
+    streaming = Keyword.get(config, :streaming_enabled)

    s3_name = strict_encode(upload.path)

    op =
-      upload.tempfile
-      |> ExAws.S3.Upload.stream_file()
-      |> ExAws.S3.upload(bucket, s3_name, [
-        {:acl, :public_read},
-        {:content_type, upload.content_type}
-      ])
+      if streaming do
+        upload.tempfile
+        |> ExAws.S3.Upload.stream_file()
+        |> ExAws.S3.upload(bucket, s3_name, [
+          {:acl, :public_read},
+          {:content_type, upload.content_type}
+        ])
+      else
+        {:ok, file_data} = File.read(upload.tempfile)
+
+        ExAws.S3.put_object(bucket, s3_name, file_data, [
+          {:acl, :public_read},
+          {:content_type, upload.content_type}
+        ])
+      end

    case ExAws.request(op) do
      {:ok, _} ->
--- a/lib/pleroma/user.ex
+++ b/lib/pleroma/user.ex
@ -11,6 +11,7 @@ defmodule Pleroma.User do
  alias Comeonin.Pbkdf2
  alias Ecto.Multi
  alias Pleroma.Activity
+  alias Pleroma.Delivery
  alias Pleroma.Keys
  alias Pleroma.Notification
  alias Pleroma.Object
@ -27,12 +28,13 @@ defmodule Pleroma.User do
  alias Pleroma.Web.OStatus
  alias Pleroma.Web.RelMe
  alias Pleroma.Web.Websub
+  alias Pleroma.Workers.BackgroundWorker

  require Logger

  @type t :: %__MODULE__{}

-  @primary_key {:id, Pleroma.FlakeId, autogenerate: true}
+  @primary_key {:id, FlakeId.Ecto.CompatType, autogenerate: true}

  # credo:disable-for-next-line Credo.Check.Readability.MaxLineLength
  @email_regex ~r/^[a-zA-Z0-9.!#$%&'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/
@ -61,6 +63,7 @@ defmodule Pleroma.User do
    field(:last_digest_emailed_at, :naive_datetime)
    has_many(:notifications, Notification)
    has_many(:registrations, Registration)
+    has_many(:deliveries, Delivery)
    embeds_one(:info, User.Info)

    timestamps()
@ -103,9 +106,7 @@ defmodule Pleroma.User do
  def profile_url(%User{ap_id: ap_id}), do: ap_id
  def profile_url(_), do: nil

-  def ap_id(%User{nickname: nickname}) do
-    "#{Web.base_url()}/users/#{nickname}"
-  end
+  def ap_id(%User{nickname: nickname}), do: "#{Web.base_url()}/users/#{nickname}"

  def ap_followers(%User{follower_address: fa}) when is_binary(fa), do: fa
  def ap_followers(%User{} = user), do: "#{ap_id(user)}/followers"
@ -116,12 +117,9 @@ defmodule Pleroma.User do

  def user_info(%User{} = user, args \\ %{}) do
    following_count =
-      if args[:following_count],
-        do: args[:following_count],
-        else: user.info.following_count || following_count(user)
+      Map.get(args, :following_count, user.info.following_count || following_count(user))

-    follower_count =
-      if args[:follower_count], do: args[:follower_count], else: user.info.follower_count
+    follower_count = Map.get(args, :follower_count, user.info.follower_count)

    %{
      note_count: user.info.note_count,
@ -134,12 +132,11 @@ defmodule Pleroma.User do
  end

  def follow_state(%User{} = user, %User{} = target) do
-    follow_activity = Utils.fetch_latest_follow(user, target)
-
-    if follow_activity,
-      do: follow_activity.data["state"],
+    case Utils.fetch_latest_follow(user, target) do
+      %{data: %{"state" => state}} -> state
      # Ideally this would be nil, but then Cachex does not commit the value
-      else: false
+      _ -> false
+    end
  end

  def get_cached_follow_state(user, target) do
@ -147,12 +144,9 @@ defmodule Pleroma.User do
    Cachex.fetch!(:user_cache, key, fn _ -> {:commit, follow_state(user, target)} end)
  end

+  @spec set_follow_state_cache(String.t(), String.t(), String.t()) :: {:ok | :error, boolean()}
  def set_follow_state_cache(user_ap_id, target_ap_id, state) do
-    Cachex.put(
-      :user_cache,
-      "follow_state:#{user_ap_id}|#{target_ap_id}",
-      state
-    )
+    Cachex.put(:user_cache, "follow_state:#{user_ap_id}|#{target_ap_id}", state)
  end

  def set_info_cache(user, args) do
@ -174,39 +168,44 @@ defmodule Pleroma.User do
    |> Repo.aggregate(:count, :id)
  end

+  defp truncate_if_exists(params, key, max_length) do
+    if Map.has_key?(params, key) and is_binary(params[key]) do
+      {value, _chopped} = String.split_at(params[key], max_length)
+      Map.put(params, key, value)
+    else
+      params
+    end
+  end
+
  def remote_user_creation(params) do
    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)

-    params = Map.put(params, :info, params[:info] || %{})
-    info_cng = User.Info.remote_user_creation(%User.Info{}, params[:info])
+    params =
+      params
+      |> Map.put(:info, params[:info] || %{})
+      |> truncate_if_exists(:name, name_limit)
+      |> truncate_if_exists(:bio, bio_limit)

-    changes =
-      %User{}
+    changeset =
+      %User{local: false}
      |> cast(params, [:bio, :name, :ap_id, :nickname, :avatar])
      |> validate_required([:name, :ap_id])
      |> unique_constraint(:nickname)
      |> validate_format(:nickname, @email_regex)
      |> validate_length(:bio, max: bio_limit)
      |> validate_length(:name, max: name_limit)
-      |> put_change(:local, false)
-      |> put_embed(:info, info_cng)
+      |> change_info(&User.Info.remote_user_creation(&1, params[:info]))

-    if changes.valid? do
-      case info_cng.changes[:source_data] do
-        %{"followers" => followers, "following" => following} ->
-          changes
-          |> put_change(:follower_address, followers)
-          |> put_change(:following_address, following)
+    case params[:info][:source_data] do
+      %{"followers" => followers, "following" => following} ->
+        changeset
+        |> put_change(:follower_address, followers)
+        |> put_change(:following_address, following)

-        _ ->
-          followers = User.ap_followers(%User{nickname: changes.changes[:nickname]})
-
-          changes
-          |> put_change(:follower_address, followers)
-      end
-    else
-      changes
+      _ ->
+        followers = ap_followers(%User{nickname: get_field(changeset, :nickname)})
+        put_change(changeset, :follower_address, followers)
    end
  end

@ -227,7 +226,6 @@ defmodule Pleroma.User do
    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)

    params = Map.put(params, :last_refreshed_at, NaiveDateTime.utc_now())
-    info_cng = User.Info.user_upgrade(struct.info, params[:info], remote?)

    struct
    |> cast(params, [
@ -242,7 +240,7 @@ defmodule Pleroma.User do
    |> validate_format(:nickname, local_nickname_regex())
    |> validate_length(:bio, max: bio_limit)
    |> validate_length(:name, max: name_limit)
-    |> put_embed(:info, info_cng)
+    |> change_info(&User.Info.user_upgrade(&1, params[:info], remote?))
  end

  def password_update_changeset(struct, params) do
@ -251,6 +249,7 @@ defmodule Pleroma.User do
    |> validate_required([:password, :password_confirmation])
    |> validate_confirmation(:password)
    |> put_password_hash
+    |> put_embed(:info, User.Info.set_password_reset_pending(struct.info, false))
  end

  @spec reset_password(User.t(), map) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
@ -267,6 +266,20 @@ defmodule Pleroma.User do
    end
  end

+  def force_password_reset_async(user) do
+    BackgroundWorker.enqueue("force_password_reset", %{"user_id" => user.id})
+  end
+
+  @spec force_password_reset(User.t()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  def force_password_reset(user) do
+    info_cng = User.Info.set_password_reset_pending(user.info, true)
+
+    user
+    |> change()
+    |> put_embed(:info, info_cng)
+    |> update_and_set_cache()
+  end
+
  def register_changeset(struct, params \\ %{}, opts \\ []) do
    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
@ -278,43 +291,39 @@ defmodule Pleroma.User do
        opts[:need_confirmation]
      end

-    info_change =
-      User.Info.confirmation_changeset(%User.Info{}, need_confirmation: need_confirmation?)
+    struct
+    |> cast(params, [:bio, :email, :name, :nickname, :password, :password_confirmation])
+    |> validate_required([:name, :nickname, :password, :password_confirmation])
+    |> validate_confirmation(:password)
+    |> unique_constraint(:email)
+    |> unique_constraint(:nickname)
+    |> validate_exclusion(:nickname, Pleroma.Config.get([User, :restricted_nicknames]))
+    |> validate_format(:nickname, local_nickname_regex())
+    |> validate_format(:email, @email_regex)
+    |> validate_length(:bio, max: bio_limit)
+    |> validate_length(:name, min: 1, max: name_limit)
+    |> change_info(&User.Info.confirmation_changeset(&1, need_confirmation: need_confirmation?))
+    |> maybe_validate_required_email(opts[:external])
+    |> put_password_hash
+    |> put_ap_id()
+    |> unique_constraint(:ap_id)
+    |> put_following_and_follower_address()
+  end

-    changeset =
-      struct
-      |> cast(params, [:bio, :email, :name, :nickname, :password, :password_confirmation])
-      |> validate_required([:name, :nickname, :password, :password_confirmation])
-      |> validate_confirmation(:password)
-      |> unique_constraint(:email)
-      |> unique_constraint(:nickname)
-      |> validate_exclusion(:nickname, Pleroma.Config.get([User, :restricted_nicknames]))
-      |> validate_format(:nickname, local_nickname_regex())
-      |> validate_format(:email, @email_regex)
-      |> validate_length(:bio, max: bio_limit)
-      |> validate_length(:name, min: 1, max: name_limit)
-      |> put_change(:info, info_change)
+  def maybe_validate_required_email(changeset, true), do: changeset
+  def maybe_validate_required_email(changeset, _), do: validate_required(changeset, [:email])

-    changeset =
-      if opts[:external] do
-        changeset
-      else
-        validate_required(changeset, [:email])
-      end
+  defp put_ap_id(changeset) do
+    ap_id = ap_id(%User{nickname: get_field(changeset, :nickname)})
+    put_change(changeset, :ap_id, ap_id)
+  end

-    if changeset.valid? do
-      ap_id = User.ap_id(%User{nickname: changeset.changes[:nickname]})
-      followers = User.ap_followers(%User{nickname: changeset.changes[:nickname]})
+  defp put_following_and_follower_address(changeset) do
+    followers = ap_followers(%User{nickname: get_field(changeset, :nickname)})

-      changeset
-      |> put_password_hash
-      |> put_change(:ap_id, ap_id)
-      |> unique_constraint(:ap_id)
-      |> put_change(:following, [followers])
-      |> put_change(:follower_address, followers)
-    else
-      changeset
-    end
+    changeset
+    |> put_change(:following, [followers])
+    |> put_change(:follower_address, followers)
  end

  defp autofollow_users(user) do
@ -329,8 +338,13 @@ defmodule Pleroma.User do

  @doc "Inserts provided changeset, performs post-registration actions (confirmation email sending etc.)"
  def register(%Ecto.Changeset{} = changeset) do
-    with {:ok, user} <- Repo.insert(changeset),
-         {:ok, user} <- autofollow_users(user),
+    with {:ok, user} <- Repo.insert(changeset) do
+      post_register_action(user)
+    end
+  end
+
+  def post_register_action(%User{} = user) do
+    with {:ok, user} <- autofollow_users(user),
         {:ok, user} <- set_cache(user),
         {:ok, _} <- User.WelcomeMessage.post_welcome_message_to_user(user),
         {:ok, _} <- try_send_confirmation_email(user) do
@ -371,7 +385,7 @@ defmodule Pleroma.User do
  end

  def maybe_direct_follow(%User{} = follower, %User{} = followed) do
-    if not User.ap_enabled?(followed) do
+    if not ap_enabled?(followed) do
      follow(follower, followed)
    else
      {:ok, follower}
@ -404,9 +418,7 @@ defmodule Pleroma.User do

    {1, [follower]} = Repo.update_all(q, [])

-    Enum.each(followeds, fn followed ->
-      update_follower_count(followed)
-    end)
+    Enum.each(followeds, &update_follower_count/1)

    set_cache(follower)
  end
@ -493,6 +505,11 @@ defmodule Pleroma.User do
    |> Repo.all()
  end

+  def get_all_by_ids(ids) do
+    from(u in __MODULE__, where: u.id in ^ids)
+    |> Repo.all()
+  end
+
  # This is mostly an SPC migration fix. This guesses the user nickname by taking the last part
  # of the ap_id and the domain and tries to get that user
  def get_by_guessed_nickname(ap_id) do
@ -516,8 +533,6 @@ defmodule Pleroma.User do
  def update_and_set_cache(changeset) do
    with {:ok, user} <- Repo.update(changeset, stale_error_field: :id) do
      set_cache(user)
-    else
-      e -> e
    end
  end

@ -554,17 +569,29 @@ defmodule Pleroma.User do
    key = "nickname:#{nickname}"

    Cachex.fetch!(:user_cache, key, fn ->
-      user_result = get_or_fetch_by_nickname(nickname)
-
-      case user_result do
+      case get_or_fetch_by_nickname(nickname) do
        {:ok, user} -> {:commit, user}
        {:error, _error} -> {:ignore, nil}
      end
    end)
  end

-  def get_cached_by_nickname_or_id(nickname_or_id) do
-    get_cached_by_id(nickname_or_id) || get_cached_by_nickname(nickname_or_id)
+  def get_cached_by_nickname_or_id(nickname_or_id, opts \\ []) do
+    restrict_to_local = Pleroma.Config.get([:instance, :limit_to_local_content])
+
+    cond do
+      is_integer(nickname_or_id) or FlakeId.flake_id?(nickname_or_id) ->
+        get_cached_by_id(nickname_or_id) || get_cached_by_nickname(nickname_or_id)
+
+      restrict_to_local == false ->
+        get_cached_by_nickname(nickname_or_id)
+
+      restrict_to_local == :unauthenticated and match?(%User{}, opts[:for]) ->
+        get_cached_by_nickname(nickname_or_id)
+
+      true ->
+        nil
+    end
  end

  def get_by_nickname(nickname) do
@ -582,13 +609,11 @@ defmodule Pleroma.User do

  def get_cached_user_info(user) do
    key = "user_info:#{user.id}"
-    Cachex.fetch!(:user_cache, key, fn _ -> user_info(user) end)
+    Cachex.fetch!(:user_cache, key, fn -> user_info(user) end)
  end

  def fetch_by_nickname(nickname) do
-    ap_try = ActivityPub.make_user_from_nickname(nickname)
-
-    case ap_try do
+    case ActivityPub.make_user_from_nickname(nickname) do
      {:ok, user} -> {:ok, user}
      _ -> OStatus.make_user(nickname)
    end
@ -613,8 +638,9 @@ defmodule Pleroma.User do
  end

  @doc "Fetch some posts when the user has just been federated with"
-  def fetch_initial_posts(user),
-    do: PleromaJobQueue.enqueue(:background, __MODULE__, [:fetch_initial_posts, user])
+  def fetch_initial_posts(user) do
+    BackgroundWorker.enqueue("fetch_initial_posts", %{"user_id" => user.id})
+  end

  @spec get_followers_query(User.t(), pos_integer() | nil) :: Ecto.Query.t()
  def get_followers_query(%User{} = user, nil) do
@ -622,7 +648,8 @@ defmodule Pleroma.User do
  end

  def get_followers_query(user, page) do
-    from(u in get_followers_query(user, nil))
+    user
+    |> get_followers_query(nil)
    |> User.Query.paginate(page, 20)
  end

@ -631,25 +658,24 @@ defmodule Pleroma.User do

  @spec get_followers(User.t(), pos_integer()) :: {:ok, list(User.t())}
  def get_followers(user, page \\ nil) do
-    q = get_followers_query(user, page)
-
-    {:ok, Repo.all(q)}
+    user
+    |> get_followers_query(page)
+    |> Repo.all()
  end

  @spec get_external_followers(User.t(), pos_integer()) :: {:ok, list(User.t())}
  def get_external_followers(user, page \\ nil) do
-    q =
-      user
-      |> get_followers_query(page)
-      |> User.Query.build(%{external: true})
-
-    {:ok, Repo.all(q)}
+    user
+    |> get_followers_query(page)
+    |> User.Query.build(%{external: true})
+    |> Repo.all()
  end

  def get_followers_ids(user, page \\ nil) do
-    q = get_followers_query(user, page)
-
-    Repo.all(from(u in q, select: u.id))
+    user
+    |> get_followers_query(page)
+    |> select([u], u.id)
+    |> Repo.all()
  end

  @spec get_friends_query(User.t(), pos_integer() | nil) :: Ecto.Query.t()
@ -658,7 +684,8 @@ defmodule Pleroma.User do
  end

  def get_friends_query(user, page) do
-    from(u in get_friends_query(user, nil))
+    user
+    |> get_friends_query(nil)
    |> User.Query.paginate(page, 20)
  end

@ -666,28 +693,27 @@ defmodule Pleroma.User do
  def get_friends_query(user), do: get_friends_query(user, nil)

  def get_friends(user, page \\ nil) do
-    q = get_friends_query(user, page)
-
-    {:ok, Repo.all(q)}
+    user
+    |> get_friends_query(page)
+    |> Repo.all()
  end

  def get_friends_ids(user, page \\ nil) do
-    q = get_friends_query(user, page)
-
-    Repo.all(from(u in q, select: u.id))
+    user
+    |> get_friends_query(page)
+    |> select([u], u.id)
+    |> Repo.all()
  end

  @spec get_follow_requests(User.t()) :: {:ok, [User.t()]}
  def get_follow_requests(%User{} = user) do
-    users =
-      Activity.follow_requests_for_actor(user)
-      |> join(:inner, [a], u in User, on: a.actor == u.ap_id)
-      |> where([a, u], not fragment("? @> ?", u.following, ^[user.follower_address]))
-      |> group_by([a, u], u.id)
-      |> select([a, u], u)
-      |> Repo.all()
-
-    {:ok, users}
+    user
+    |> Activity.follow_requests_for_actor()
+    |> join(:inner, [a], u in User, on: a.actor == u.ap_id)
+    |> where([a, u], not fragment("? @> ?", u.following, ^[user.follower_address]))
+    |> group_by([a, u], u.id)
+    |> select([a, u], u)
+    |> Repo.all()
  end

  def increase_note_count(%User{} = user) do
@ -733,20 +759,27 @@ defmodule Pleroma.User do
  end

  def update_note_count(%User{} = user) do
-    note_count_query =
+    note_count =
      from(
        a in Object,
        where: fragment("?->>'actor' = ? and ?->>'type' = 'Note'", a.data, ^user.ap_id, a.data),
        select: count(a.id)
      )
+      |> Repo.one()

-    note_count = Repo.one(note_count_query)
+    update_info(user, &User.Info.set_note_count(&1, note_count))
+  end

-    info_cng = User.Info.set_note_count(user.info, note_count)
+  def update_mascot(user, url) do
+    info_changeset =
+      User.Info.mascot_update(
+        user.info,
+        url
+      )

    user
    |> change()
-    |> put_embed(:info, info_cng)
+    |> put_embed(:info, info_changeset)
    |> update_and_set_cache()
  end

@ -764,17 +797,7 @@ defmodule Pleroma.User do

  def fetch_follow_information(user) do
    with {:ok, info} <- ActivityPub.fetch_follow_information_for_user(user) do
-      info_cng = User.Info.follow_information_update(user.info, info)
-
-      changeset =
-        user
-        |> change()
-        |> put_embed(:info, info_cng)
-
-      update_and_set_cache(changeset)
-    else
-      {:error, _} = e -> e
-      e -> {:error, e}
+      update_info(user, &User.Info.follow_information_update(&1, info))
    end
  end

@ -848,62 +871,28 @@ defmodule Pleroma.User do

  @spec mute(User.t(), User.t(), boolean()) :: {:ok, User.t()} | {:error, String.t()}
  def mute(muter, %User{ap_id: ap_id}, notifications? \\ true) do
-    info = muter.info
-
-    info_cng =
-      User.Info.add_to_mutes(info, ap_id)
-      |> User.Info.add_to_muted_notifications(info, ap_id, notifications?)
-
-    cng =
-      change(muter)
-      |> put_embed(:info, info_cng)
-
-    update_and_set_cache(cng)
+    update_info(muter, &User.Info.add_to_mutes(&1, ap_id, notifications?))
  end

  def unmute(muter, %{ap_id: ap_id}) do
-    info = muter.info
-
-    info_cng =
-      User.Info.remove_from_mutes(info, ap_id)
-      |> User.Info.remove_from_muted_notifications(info, ap_id)
-
-    cng =
-      change(muter)
-      |> put_embed(:info, info_cng)
-
-    update_and_set_cache(cng)
+    update_info(muter, &User.Info.remove_from_mutes(&1, ap_id))
  end

  def subscribe(subscriber, %{ap_id: ap_id}) do
-    deny_follow_blocked = Pleroma.Config.get([:user, :deny_follow_blocked])
-
    with %User{} = subscribed <- get_cached_by_ap_id(ap_id) do
-      blocked = blocks?(subscribed, subscriber) and deny_follow_blocked
+      deny_follow_blocked = Pleroma.Config.get([:user, :deny_follow_blocked])

-      if blocked do
+      if blocks?(subscribed, subscriber) and deny_follow_blocked do
        {:error, "Could not subscribe: #{subscribed.nickname} is blocking you"}
      else
-        info_cng =
-          subscribed.info
-          |> User.Info.add_to_subscribers(subscriber.ap_id)
-
-        change(subscribed)
-        |> put_embed(:info, info_cng)
-        |> update_and_set_cache()
+        update_info(subscribed, &User.Info.add_to_subscribers(&1, subscriber.ap_id))
      end
    end
  end

  def unsubscribe(unsubscriber, %{ap_id: ap_id}) do
    with %User{} = user <- get_cached_by_ap_id(ap_id) do
-      info_cng =
-        user.info
-        |> User.Info.remove_from_subscribers(unsubscriber.ap_id)
-
-      change(user)
-      |> put_embed(:info, info_cng)
-      |> update_and_set_cache()
+      update_info(user, &User.Info.remove_from_subscribers(&1, unsubscriber.ap_id))
    end
  end

@ -932,21 +921,11 @@ defmodule Pleroma.User do
        blocker
      end

-    if following?(blocked, blocker) do
-      unfollow(blocked, blocker)
-    end
+    if following?(blocked, blocker), do: unfollow(blocked, blocker)

    {:ok, blocker} = update_follower_count(blocker)

-    info_cng =
-      blocker.info
-      |> User.Info.add_to_block(ap_id)
-
-    cng =
-      change(blocker)
-      |> put_embed(:info, info_cng)
-
-    update_and_set_cache(cng)
+    update_info(blocker, &User.Info.add_to_block(&1, ap_id))
  end

  # helper to handle the block given only an actor's AP id
@ -955,15 +934,7 @@ defmodule Pleroma.User do
  end

  def unblock(blocker, %{ap_id: ap_id}) do
-    info_cng =
-      blocker.info
-      |> User.Info.remove_from_block(ap_id)
-
-    cng =
-      change(blocker)
-      |> put_embed(:info, info_cng)
-
-    update_and_set_cache(cng)
+    update_info(blocker, &User.Info.remove_from_block(&1, ap_id))
  end

  def mutes?(nil, _), do: false
@ -1020,79 +991,53 @@ defmodule Pleroma.User do
  end

  def block_domain(user, domain) do
-    info_cng =
-      user.info
-      |> User.Info.add_to_domain_block(domain)
-
-    cng =
-      change(user)
-      |> put_embed(:info, info_cng)
-
-    update_and_set_cache(cng)
+    update_info(user, &User.Info.add_to_domain_block(&1, domain))
  end

  def unblock_domain(user, domain) do
-    info_cng =
-      user.info
-      |> User.Info.remove_from_domain_block(domain)
-
-    cng =
-      change(user)
-      |> put_embed(:info, info_cng)
-
-    update_and_set_cache(cng)
+    update_info(user, &User.Info.remove_from_domain_block(&1, domain))
  end

  def deactivate_async(user, status \\ true) do
-    PleromaJobQueue.enqueue(:background, __MODULE__, [:deactivate_async, user, status])
+    BackgroundWorker.enqueue("deactivate_user", %{"user_id" => user.id, "status" => status})
  end

  def deactivate(%User{} = user, status \\ true) do
-    info_cng = User.Info.set_activation_status(user.info, status)
-
-    with {:ok, friends} <- User.get_friends(user),
-         {:ok, followers} <- User.get_followers(user),
-         {:ok, user} <-
-           user
-           |> change()
-           |> put_embed(:info, info_cng)
-           |> update_and_set_cache() do
-      Enum.each(followers, &invalidate_cache(&1))
-      Enum.each(friends, &update_follower_count(&1))
+    with {:ok, user} <- update_info(user, &User.Info.set_activation_status(&1, status)) do
+      Enum.each(get_followers(user), &invalidate_cache/1)
+      Enum.each(get_friends(user), &update_follower_count/1)

      {:ok, user}
    end
  end

  def update_notification_settings(%User{} = user, settings \\ %{}) do
-    info_changeset = User.Info.update_notification_settings(user.info, settings)
-
-    change(user)
-    |> put_embed(:info, info_changeset)
-    |> update_and_set_cache()
+    update_info(user, &User.Info.update_notification_settings(&1, settings))
  end

-  @spec delete(User.t()) :: :ok
-  def delete(%User{} = user),
-    do: PleromaJobQueue.enqueue(:background, __MODULE__, [:delete, user])
+  def delete(%User{} = user) do
+    BackgroundWorker.enqueue("delete_user", %{"user_id" => user.id})
+  end
+
+  def perform(:force_password_reset, user), do: force_password_reset(user)

  @spec perform(atom(), User.t()) :: {:ok, User.t()}
  def perform(:delete, %User{} = user) do
    {:ok, _user} = ActivityPub.delete(user)

    # Remove all relationships
-    {:ok, followers} = User.get_followers(user)
-
-    Enum.each(followers, fn follower ->
+    user
+    |> get_followers()
+    |> Enum.each(fn follower ->
      ActivityPub.unfollow(follower, user)
-      User.unfollow(follower, user)
+      unfollow(follower, user)
    end)

-    {:ok, friends} = User.get_friends(user)
-
-    Enum.each(friends, fn followed ->
+    user
+    |> get_friends()
+    |> Enum.each(fn followed ->
      ActivityPub.unfollow(user, followed)
-      User.unfollow(user, followed)
+      unfollow(user, followed)
    end)

    delete_user_activities(user)
@ -1104,13 +1049,11 @@ defmodule Pleroma.User do
  def perform(:fetch_initial_posts, %User{} = user) do
    pages = Pleroma.Config.get!([:fetch_initial_posts, :pages])

-    Enum.each(
-      # Insert all the posts in reverse order, so they're in the right order on the timeline
-      Enum.reverse(Utils.fetch_ordered_collection(user.info.source_data["outbox"], pages)),
-      &Pleroma.Web.Federator.incoming_ap_doc/1
-    )
-
-    {:ok, user}
+    # Insert all the posts in reverse order, so they're in the right order on the timeline
+    user.info.source_data["outbox"]
+    |> Utils.fetch_ordered_collection(pages)
+    |> Enum.reverse()
+    |> Enum.each(&Pleroma.Web.Federator.incoming_ap_doc/1)
  end

  def perform(:deactivate_async, user, status), do: deactivate(user, status)
@ -1181,32 +1124,27 @@ defmodule Pleroma.User do
    Repo.all(query)
  end

-  def blocks_import(%User{} = blocker, blocked_identifiers) when is_list(blocked_identifiers),
-    do:
-      PleromaJobQueue.enqueue(:background, __MODULE__, [
-        :blocks_import,
-        blocker,
-        blocked_identifiers
-      ])
+  def blocks_import(%User{} = blocker, blocked_identifiers) when is_list(blocked_identifiers) do
+    BackgroundWorker.enqueue("blocks_import", %{
+      "blocker_id" => blocker.id,
+      "blocked_identifiers" => blocked_identifiers
+    })
+  end

-  def follow_import(%User{} = follower, followed_identifiers) when is_list(followed_identifiers),
-    do:
-      PleromaJobQueue.enqueue(:background, __MODULE__, [
-        :follow_import,
-        follower,
-        followed_identifiers
-      ])
+  def follow_import(%User{} = follower, followed_identifiers)
+      when is_list(followed_identifiers) do
+    BackgroundWorker.enqueue("follow_import", %{
+      "follower_id" => follower.id,
+      "followed_identifiers" => followed_identifiers
+    })
+  end

-  def delete_user_activities(%User{ap_id: ap_id} = user) do
+  def delete_user_activities(%User{ap_id: ap_id}) do
    ap_id
-    |> Activity.query_by_actor()
+    |> Activity.Queries.by_actor()
    |> RepoStreamer.chunk_stream(50)
-    |> Stream.each(fn activities ->
-      Enum.each(activities, &delete_activity(&1))
-    end)
+    |> Stream.each(fn activities -> Enum.each(activities, &delete_activity/1) end)
    |> Stream.run()
-
-    {:ok, user}
  end

  defp delete_activity(%{data: %{"type" => "Create"}} = activity) do
@ -1216,17 +1154,19 @@ defmodule Pleroma.User do
  end

  defp delete_activity(%{data: %{"type" => "Like"}} = activity) do
-    user = get_cached_by_ap_id(activity.actor)
    object = Object.normalize(activity)

-    ActivityPub.unlike(user, object)
+    activity.actor
+    |> get_cached_by_ap_id()
+    |> ActivityPub.unlike(object)
  end

  defp delete_activity(%{data: %{"type" => "Announce"}} = activity) do
-    user = get_cached_by_ap_id(activity.actor)
    object = Object.normalize(activity)

-    ActivityPub.unannounce(user, object)
+    activity.actor
+    |> get_cached_by_ap_id()
+    |> ActivityPub.unannounce(object)
  end

  defp delete_activity(_activity), do: "Doing nothing"
@ -1238,9 +1178,7 @@ defmodule Pleroma.User do
  def html_filter_policy(_), do: Pleroma.Config.get([:markup, :scrub_policy])

  def fetch_by_ap_id(ap_id) do
-    ap_try = ActivityPub.make_user_from_ap_id(ap_id)
-
-    case ap_try do
+    case ActivityPub.make_user_from_ap_id(ap_id) do
      {:ok, user} ->
        {:ok, user}

@ -1255,7 +1193,7 @@ defmodule Pleroma.User do
  def get_or_fetch_by_ap_id(ap_id) do
    user = get_cached_by_ap_id(ap_id)

-    if !is_nil(user) and !User.needs_update?(user) do
+    if !is_nil(user) and !needs_update?(user) do
      {:ok, user}
    else
      # Whether to fetch initial posts for the user (if it's a new user & the fetching is enabled)
@ -1275,19 +1213,20 @@ defmodule Pleroma.User do

  @doc "Creates an internal service actor by URI if missing.  Optionally takes nickname for addressing."
  def get_or_create_service_actor_by_ap_id(uri, nickname \\ nil) do
-    if user = get_cached_by_ap_id(uri) do
+    with %User{} = user <- get_cached_by_ap_id(uri) do
      user
    else
-      changes =
-        %User{info: %User.Info{}}
-        |> cast(%{}, [:ap_id, :nickname, :local])
-        |> put_change(:ap_id, uri)
-        |> put_change(:nickname, nickname)
-        |> put_change(:local, true)
-        |> put_change(:follower_address, uri <> "/followers")
+      _ ->
+        {:ok, user} =
+          %User{info: %User.Info{}}
+          |> cast(%{}, [:ap_id, :nickname, :local])
+          |> put_change(:ap_id, uri)
+          |> put_change(:nickname, nickname)
+          |> put_change(:local, true)
+          |> put_change(:follower_address, uri <> "/followers")
+          |> Repo.insert()

-      {:ok, user} = Repo.insert(changes)
-      user
+        user
    end
  end

@ -1344,23 +1283,21 @@ defmodule Pleroma.User do
  # this is because we have synchronous follow APIs and need to simulate them
  # with an async handshake
  def wait_and_refresh(_, %User{local: true} = a, %User{local: true} = b) do
-    with %User{} = a <- User.get_cached_by_id(a.id),
-         %User{} = b <- User.get_cached_by_id(b.id) do
+    with %User{} = a <- get_cached_by_id(a.id),
+         %User{} = b <- get_cached_by_id(b.id) do
      {:ok, a, b}
    else
-      _e ->
-        :error
+      nil -> :error
    end
  end

  def wait_and_refresh(timeout, %User{} = a, %User{} = b) do
    with :ok <- :timer.sleep(timeout),
-         %User{} = a <- User.get_cached_by_id(a.id),
-         %User{} = b <- User.get_cached_by_id(b.id) do
+         %User{} = a <- get_cached_by_id(a.id),
+         %User{} = b <- get_cached_by_id(b.id) do
      {:ok, a, b}
    else
-      _e ->
-        :error
+      nil -> :error
    end
  end

@ -1422,7 +1359,7 @@ defmodule Pleroma.User do
  defp normalize_tags(tags) do
    [tags]
    |> List.flatten()
-    |> Enum.map(&String.downcase(&1))
+    |> Enum.map(&String.downcase/1)
  end

  defp local_nickname_regex do
@ -1515,11 +1452,7 @@ defmodule Pleroma.User do
  @spec switch_email_notifications(t(), String.t(), boolean()) ::
          {:ok, t()} | {:error, Ecto.Changeset.t()}
  def switch_email_notifications(user, type, status) do
-    info = Pleroma.User.Info.update_email_notifications(user.info, %{type => status})
-
-    change(user)
-    |> put_embed(:info, info)
-    |> update_and_set_cache()
+    update_info(user, &User.Info.update_email_notifications(&1, %{type => status}))
  end

  @doc """
@ -1541,13 +1474,8 @@ defmodule Pleroma.User do
  def toggle_confirmation(%User{} = user) do
    need_confirmation? = !user.info.confirmation_pending

-    info_changeset =
-      User.Info.confirmation_changeset(user.info, need_confirmation: need_confirmation?)
-
    user
-    |> change()
-    |> put_embed(:info, info_changeset)
-    |> update_and_set_cache()
+    |> update_info(&User.Info.confirmation_changeset(&1, need_confirmation: need_confirmation?))
  end

  def get_mascot(%{info: %{mascot: %{} = mascot}}) when not is_nil(mascot) do
@ -1570,16 +1498,11 @@ defmodule Pleroma.User do
    }
  end

-  def ensure_keys_present(%User{info: info} = user) do
-    if info.keys do
-      {:ok, user}
-    else
-      {:ok, pem} = Keys.generate_rsa_pem()
+  def ensure_keys_present(%{info: %{keys: keys}} = user) when not is_nil(keys), do: {:ok, user}

-      user
-      |> Ecto.Changeset.change()
-      |> Ecto.Changeset.put_embed(:info, User.Info.set_keys(info, pem))
-      |> update_and_set_cache()
+  def ensure_keys_present(%User{} = user) do
+    with {:ok, pem} <- Keys.generate_rsa_pem() do
+      update_info(user, &User.Info.set_keys(&1, pem))
    end
  end

@ -1604,4 +1527,47 @@ defmodule Pleroma.User do
  def is_internal_user?(%User{nickname: nil}), do: true
  def is_internal_user?(%User{local: true, nickname: "internal." <> _}), do: true
  def is_internal_user?(_), do: false
+
+  # A hack because user delete activities have a fake id for whatever reason
+  # TODO: Get rid of this
+  def get_delivered_users_by_object_id("pleroma:fake_object_id"), do: []
+
+  def get_delivered_users_by_object_id(object_id) do
+    from(u in User,
+      inner_join: delivery in assoc(u, :deliveries),
+      where: delivery.object_id == ^object_id
+    )
+    |> Repo.all()
+  end
+
+  def change_email(user, email) do
+    user
+    |> cast(%{email: email}, [:email])
+    |> validate_required([:email])
+    |> unique_constraint(:email)
+    |> validate_format(:email, @email_regex)
+    |> update_and_set_cache()
+  end
+
+  @doc """
+  Changes `user.info` and returns the user changeset.
+
+  `fun` is called with the `user.info`.
+  """
+  def change_info(user, fun) do
+    changeset = change(user)
+    info = get_field(changeset, :info) || %User.Info{}
+    put_embed(changeset, :info, fun.(info))
+  end
+
+  @doc """
+  Updates `user.info` and sets cache.
+
+  `fun` is called with the `user.info`.
+  """
+  def update_info(user, fun) do
+    user
+    |> change_info(fun)
+    |> update_and_set_cache()
+  end
 end
--- a/lib/pleroma/user/info.ex
+++ b/lib/pleroma/user/info.ex
@ -20,6 +20,7 @@ defmodule Pleroma.User.Info do
    field(:following_count, :integer, default: nil)
    field(:locked, :boolean, default: false)
    field(:confirmation_pending, :boolean, default: false)
+    field(:password_reset_pending, :boolean, default: false)
    field(:confirmation_token, :string, default: nil)
    field(:default_scope, :string, default: "public")
    field(:blocks, {:array, :string}, default: [])
@ -41,6 +42,8 @@ defmodule Pleroma.User.Info do
    field(:topic, :string, default: nil)
    field(:hub, :string, default: nil)
    field(:salmon, :string, default: nil)
+    field(:hide_followers_count, :boolean, default: false)
+    field(:hide_follows_count, :boolean, default: false)
    field(:hide_followers, :boolean, default: false)
    field(:hide_follows, :boolean, default: false)
    field(:hide_favorites, :boolean, default: true)
@ -49,8 +52,9 @@ defmodule Pleroma.User.Info do
    field(:mascot, :map, default: nil)
    field(:emoji, {:array, :map}, default: [])
    field(:pleroma_settings_store, :map, default: %{})
-    field(:fields, {:array, :map}, default: [])
+    field(:fields, {:array, :map}, default: nil)
    field(:raw_fields, {:array, :map}, default: [])
+    field(:discoverable, :boolean, default: false)

    field(:notification_settings, :map,
      default: %{
@ -80,6 +84,14 @@ defmodule Pleroma.User.Info do
    |> validate_required([:deactivated])
  end

+  def set_password_reset_pending(info, pending) do
+    params = %{password_reset_pending: pending}
+
+    info
+    |> cast(params, [:password_reset_pending])
+    |> validate_required([:password_reset_pending])
+  end
+
  def update_notification_settings(info, settings) do
    settings =
      settings
@ -176,16 +188,11 @@ defmodule Pleroma.User.Info do
    |> validate_required([:subscribers])
  end

-  @spec add_to_mutes(Info.t(), String.t()) :: Changeset.t()
-  def add_to_mutes(info, muted) do
-    set_mutes(info, Enum.uniq([muted | info.mutes]))
-  end
-
-  @spec add_to_muted_notifications(Changeset.t(), Info.t(), String.t(), boolean()) ::
-          Changeset.t()
-  def add_to_muted_notifications(changeset, info, muted, notifications?) do
-    set_notification_mutes(
-      changeset,
+  @spec add_to_mutes(Info.t(), String.t(), boolean()) :: Changeset.t()
+  def add_to_mutes(info, muted, notifications?) do
+    info
+    |> set_mutes(Enum.uniq([muted | info.mutes]))
+    |> set_notification_mutes(
      Enum.uniq([muted | info.muted_notifications]),
      notifications?
    )
@ -193,12 +200,9 @@ defmodule Pleroma.User.Info do

  @spec remove_from_mutes(Info.t(), String.t()) :: Changeset.t()
  def remove_from_mutes(info, muted) do
-    set_mutes(info, List.delete(info.mutes, muted))
-  end
-
-  @spec remove_from_muted_notifications(Changeset.t(), Info.t(), String.t()) :: Changeset.t()
-  def remove_from_muted_notifications(changeset, info, muted) do
-    set_notification_mutes(changeset, List.delete(info.muted_notifications, muted), true)
+    info
+    |> set_mutes(List.delete(info.mutes, muted))
+    |> set_notification_mutes(List.delete(info.muted_notifications, muted), true)
  end

  def add_to_block(info, blocked) do
@ -242,6 +246,13 @@ defmodule Pleroma.User.Info do
  end

  def remote_user_creation(info, params) do
+    params =
+      if Map.has_key?(params, :fields) do
+        Map.put(params, :fields, Enum.map(params[:fields], &truncate_field/1))
+      else
+        params
+      end
+
    info
    |> cast(params, [
      :ap_enabled,
@ -255,9 +266,12 @@ defmodule Pleroma.User.Info do
      :salmon,
      :hide_followers,
      :hide_follows,
+      :hide_followers_count,
+      :hide_follows_count,
      :follower_count,
      :fields,
-      :following_count
+      :following_count,
+      :discoverable
    ])
    |> validate_fields(true)
  end
@ -274,7 +288,10 @@ defmodule Pleroma.User.Info do
      :following_count,
      :hide_follows,
      :fields,
-      :hide_followers
+      :hide_followers,
+      :discoverable,
+      :hide_followers_count,
+      :hide_follows_count
    ])
    |> validate_fields(remote?)
  end
@ -288,13 +305,16 @@ defmodule Pleroma.User.Info do
      :banner,
      :hide_follows,
      :hide_followers,
+      :hide_followers_count,
+      :hide_follows_count,
      :hide_favorites,
      :background,
      :show_role,
      :skip_thread_containment,
      :fields,
      :raw_fields,
-      :pleroma_settings_store
+      :pleroma_settings_store,
+      :discoverable
    ])
    |> validate_fields()
  end
@ -318,14 +338,22 @@ defmodule Pleroma.User.Info do
    name_limit = Pleroma.Config.get([:instance, :account_field_name_length], 255)
    value_limit = Pleroma.Config.get([:instance, :account_field_value_length], 255)

-    is_binary(name) &&
-      is_binary(value) &&
-      String.length(name) <= name_limit &&
+    is_binary(name) && is_binary(value) && String.length(name) <= name_limit &&
      String.length(value) <= value_limit
  end

  defp valid_field?(_), do: false

+  defp truncate_field(%{"name" => name, "value" => value}) do
+    {name, _chopped} =
+      String.split_at(name, Pleroma.Config.get([:instance, :account_field_name_length], 255))
+
+    {value, _chopped} =
+      String.split_at(value, Pleroma.Config.get([:instance, :account_field_value_length], 255))
+
+    %{"name" => name, "value" => value}
+  end
+
  @spec confirmation_changeset(Info.t(), keyword()) :: Changeset.t()
  def confirmation_changeset(info, opts) do
    need_confirmation? = Keyword.get(opts, :need_confirmation)
@ -422,7 +450,7 @@ defmodule Pleroma.User.Info do

  # ``fields`` is an array of mastodon profile field, containing ``{"name": "…", "value": "…"}``.
  # For example: [{"name": "Pronoun", "value": "she/her"}, …]
-  def fields(%{fields: [], source_data: %{"attachment" => attachment}}) do
+  def fields(%{fields: nil, source_data: %{"attachment" => attachment}}) do
    limit = Pleroma.Config.get([:instance, :max_remote_account_fields], 0)

    attachment
@ -431,6 +459,8 @@ defmodule Pleroma.User.Info do
    |> Enum.take(limit)
  end

+  def fields(%{fields: nil}), do: []
+
  def fields(%{fields: fields}), do: fields

  def follow_information_update(info, params) do
@ -439,7 +469,9 @@ defmodule Pleroma.User.Info do
      :hide_followers,
      :hide_follows,
      :follower_count,
-      :following_count
+      :following_count,
+      :hide_followers_count,
+      :hide_follows_count
    ])
  end
 end
--- a/lib/pleroma/user/query.ex
+++ b/lib/pleroma/user/query.ex
@ -1,5 +1,5 @@
 # Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Pleroma.User.Query do
--- a/lib/pleroma/web/activity_pub/activity_pub.ex
+++ b/lib/pleroma/web/activity_pub/activity_pub.ex
@ -4,6 +4,7 @@

 defmodule Pleroma.Web.ActivityPub.ActivityPub do
  alias Pleroma.Activity
+  alias Pleroma.Activity.Ir.Topics
  alias Pleroma.Config
  alias Pleroma.Conversation
  alias Pleroma.Notification
@ -16,7 +17,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
  alias Pleroma.User
  alias Pleroma.Web.ActivityPub.MRF
  alias Pleroma.Web.ActivityPub.Transmogrifier
+  alias Pleroma.Web.Streamer
  alias Pleroma.Web.WebFinger
+  alias Pleroma.Workers.BackgroundWorker

  import Ecto.Query
  import Pleroma.Web.ActivityPub.Utils
@ -139,13 +142,13 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do

      # Splice in the child object if we have one.
      activity =
-        if !is_nil(object) do
+        if not is_nil(object) do
          Map.put(activity, :object, object)
        else
          activity
        end

-      PleromaJobQueue.enqueue(:background, Pleroma.Web.RichMedia.Helpers, [:fetch, activity])
+      BackgroundWorker.enqueue("fetch_data_for_activity", %{"activity_id" => activity.id})

      Notification.create_notifications(activity)

@ -186,9 +189,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
      participations
      |> Repo.preload(:user)

-    Enum.each(participations, fn participation ->
-      Pleroma.Web.Streamer.stream("participation", participation)
-    end)
+    Streamer.stream("participation", participations)
  end

  def stream_out_participations(%Object{data: %{"context" => context}}, user) do
@ -207,41 +208,15 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do

  def stream_out_participations(_, _), do: :noop

-  def stream_out(activity) do
-    if activity.data["type"] in ["Create", "Announce", "Delete"] do
-      object = Object.normalize(activity)
-      # Do not stream out poll replies
-      unless object.data["type"] == "Answer" do
-        Pleroma.Web.Streamer.stream("user", activity)
-        Pleroma.Web.Streamer.stream("list", activity)
+  def stream_out(%Activity{data: %{"type" => data_type}} = activity)
+      when data_type in ["Create", "Announce", "Delete"] do
+    activity
+    |> Topics.get_activity_topics()
+    |> Streamer.stream(activity)
+  end

-        if get_visibility(activity) == "public" do
-          Pleroma.Web.Streamer.stream("public", activity)
-
-          if activity.local do
-            Pleroma.Web.Streamer.stream("public:local", activity)
-          end
-
-          if activity.data["type"] in ["Create"] do
-            object.data
-            |> Map.get("tag", [])
-            |> Enum.filter(fn tag -> is_bitstring(tag) end)
-            |> Enum.each(fn tag -> Pleroma.Web.Streamer.stream("hashtag:" <> tag, activity) end)
-
-            if object.data["attachment"] != [] do
-              Pleroma.Web.Streamer.stream("public:media", activity)
-
-              if activity.local do
-                Pleroma.Web.Streamer.stream("public:local:media", activity)
-              end
-            end
-          end
-        else
-          if get_visibility(activity) == "direct",
-            do: Pleroma.Web.Streamer.stream("direct", activity)
-        end
-      end
-    end
+  def stream_out(_activity) do
+    :noop
  end

  def create(%{to: to, actor: actor, context: context, object: object} = params, fake \\ false) do
@ -273,6 +248,26 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
    end
  end

+  def listen(%{to: to, actor: actor, context: context, object: object} = params) do
+    additional = params[:additional] || %{}
+    # only accept false as false value
+    local = !(params[:local] == false)
+    published = params[:published]
+
+    with listen_data <-
+           make_listen_data(
+             %{to: to, actor: actor, published: published, context: context, object: object},
+             additional
+           ),
+         {:ok, activity} <- insert(listen_data, local),
+         :ok <- maybe_federate(activity) do
+      {:ok, activity}
+    else
+      {:error, message} ->
+        {:error, message}
+    end
+  end
+
  def accept(%{to: to, actor: actor, object: object} = params) do
    # only accept false as false value
    local = !(params[:local] == false)
@ -331,12 +326,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
    end
  end

-  def unlike(
-        %User{} = actor,
-        %Object{} = object,
-        activity_id \\ nil,
-        local \\ true
-      ) do
+  def unlike(%User{} = actor, %Object{} = object, activity_id \\ nil, local \\ true) do
    with %Activity{} = like_activity <- get_existing_like(actor.ap_id, object),
         unlike_data <- make_unlike_data(actor, like_activity, activity_id),
         {:ok, unlike_activity} <- insert(unlike_data, local),
@ -440,6 +430,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
    end
  end

+  @spec block(User.t(), User.t(), String.t() | nil, boolean) :: {:ok, Activity.t() | nil}
  def block(blocker, blocked, activity_id \\ nil, local \\ true) do
    outgoing_blocks = Config.get([:activitypub, :outgoing_blocks])
    unfollow_blocked = Config.get([:activitypub, :unfollow_blocked])
@ -468,10 +459,11 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
    end
  end

+  @spec flag(map()) :: {:ok, Activity.t()} | any
  def flag(
        %{
          actor: actor,
-          context: context,
+          context: _context,
          account: account,
          statuses: statuses,
          content: content
@ -483,14 +475,6 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do

    additional = params[:additional] || %{}

-    params = %{
-      actor: actor,
-      context: context,
-      account: account,
-      statuses: statuses,
-      content: content
-    }
-
    additional =
      if forward do
        Map.merge(additional, %{"to" => [], "cc" => [account.ap_id]})
@ -546,7 +530,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
  end

  @spec fetch_latest_activity_id_for_context(String.t(), keyword() | map()) ::
-          Pleroma.FlakeId.t() | nil
+          FlakeId.Ecto.CompatType.t() | nil
  def fetch_latest_activity_id_for_context(context, opts \\ %{}) do
    context
    |> fetch_activities_for_context_query(Map.merge(%{"skip_preload" => true}, opts))
@ -555,12 +539,13 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
    |> Repo.one()
  end

-  def fetch_public_activities(opts \\ %{}) do
-    q = fetch_activities_query([Pleroma.Constants.as_public()], opts)
+  def fetch_public_activities(opts \\ %{}, pagination \\ :keyset) do
+    opts = Map.drop(opts, ["user"])

-    q
+    [Pleroma.Constants.as_public()]
+    |> fetch_activities_query(opts)
    |> restrict_unlisted()
-    |> Pagination.fetch_paginated(opts)
+    |> Pagination.fetch_paginated(opts, pagination)
    |> Enum.reverse()
  end

@ -623,6 +608,23 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do

  defp restrict_thread_visibility(query, _, _), do: query

+  def fetch_user_abstract_activities(user, reading_user, params \\ %{}) do
+    params =
+      params
+      |> Map.put("user", reading_user)
+      |> Map.put("actor_id", user.ap_id)
+      |> Map.put("whole_db", true)
+
+    recipients =
+      user_activities_recipients(%{
+        "godmode" => params["godmode"],
+        "reading_user" => reading_user
+      })
+
+    fetch_activities(recipients, params)
+    |> Enum.reverse()
+  end
+
  def fetch_user_activities(user, reading_user, params \\ %{}) do
    params =
      params
@ -801,7 +803,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
      )

    unless opts["skip_preload"] do
-      from([thread_mute: tm] in query, where: is_nil(tm))
+      from([thread_mute: tm] in query, where: is_nil(tm.user_id))
    else
      query
    end
@ -869,7 +871,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do

  defp restrict_muted_reblogs(query, _), do: query

-  defp exclude_poll_votes(query, %{"include_poll_votes" => "true"}), do: query
+  defp exclude_poll_votes(query, %{"include_poll_votes" => true}), do: query

  defp exclude_poll_votes(query, _) do
    if has_named_binding?(query, :object) do
@ -953,11 +955,11 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
    |> exclude_poll_votes(opts)
  end

-  def fetch_activities(recipients, opts \\ %{}) do
+  def fetch_activities(recipients, opts \\ %{}, pagination \\ :keyset) do
    list_memberships = Pleroma.List.memberships(opts["user"])

    fetch_activities_query(recipients ++ list_memberships, opts)
-    |> Pagination.fetch_paginated(opts)
+    |> Pagination.fetch_paginated(opts, pagination)
    |> Enum.reverse()
    |> maybe_update_cc(list_memberships, opts["user"])
  end
@ -988,10 +990,15 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
    )
  end

-  def fetch_activities_bounded(recipients, recipients_with_public, opts \\ %{}) do
+  def fetch_activities_bounded(
+        recipients,
+        recipients_with_public,
+        opts \\ %{},
+        pagination \\ :keyset
+      ) do
    fetch_activities_query([], opts)
    |> fetch_activities_bounded_query(recipients, recipients_with_public)
-    |> Pagination.fetch_paginated(opts)
+    |> Pagination.fetch_paginated(opts, pagination)
    |> Enum.reverse()
  end

@ -1031,6 +1038,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do

    locked = data["manuallyApprovesFollowers"] || false
    data = Transmogrifier.maybe_fix_user_object(data)
+    discoverable = data["discoverable"] || false

    user_data = %{
      ap_id: data["id"],
@ -1039,7 +1047,8 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
        source_data: data,
        banner: banner,
        fields: fields,
-        locked: locked
+        locked: locked,
+        discoverable: discoverable
      },
      avatar: avatar,
      name: data["name"],
--- a/lib/pleroma/web/activity_pub/activity_pub_controller.ex
+++ b/lib/pleroma/web/activity_pub/activity_pub_controller.ex
@ -6,6 +6,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
  use Pleroma.Web, :controller

  alias Pleroma.Activity
+  alias Pleroma.Delivery
  alias Pleroma.Object
  alias Pleroma.Object.Fetcher
  alias Pleroma.User
@ -23,6 +24,12 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do

  action_fallback(:errors)

+  plug(
+    Pleroma.Plugs.Cache,
+    [query_params: false, tracking_fun: &__MODULE__.track_object_fetch/2]
+    when action in [:activity, :object]
+  )
+
  plug(Pleroma.Web.FederatingPlug when action in [:inbox, :relay])
  plug(:set_requester_reachable when action in [:inbox])
  plug(:relay_active? when action in [:relay])
@ -41,8 +48,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
    with %User{} = user <- User.get_cached_by_nickname(nickname),
         {:ok, user} <- User.ensure_keys_present(user) do
      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("user.json", %{user: user}))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("user.json", %{user: user})
    else
      nil -> {:error, :not_found}
    end
@ -53,14 +61,27 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
         %Object{} = object <- Object.get_cached_by_ap_id(ap_id),
         {_, true} <- {:public?, Visibility.is_public?(object)} do
      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(ObjectView.render("object.json", %{object: object}))
+      |> assign(:tracking_fun_data, object.id)
+      |> set_cache_ttl_for(object)
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(ObjectView)
+      |> render("object.json", object: object)
    else
      {:public?, false} ->
        {:error, :not_found}
    end
  end

+  def track_object_fetch(conn, nil), do: conn
+
+  def track_object_fetch(conn, object_id) do
+    with %{assigns: %{user: %User{id: user_id}}} <- conn do
+      Delivery.create(object_id, user_id)
+    end
+
+    conn
+  end
+
  def object_likes(conn, %{"uuid" => uuid, "page" => page}) do
    with ap_id <- o_status_url(conn, :object, uuid),
         %Object{} = object <- Object.get_cached_by_ap_id(ap_id),
@ -69,8 +90,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
      {page, _} = Integer.parse(page)

      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(ObjectView.render("likes.json", ap_id, likes, page))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(ObjectView)
+      |> render("likes.json", %{ap_id: ap_id, likes: likes, page: page})
    else
      {:public?, false} ->
        {:error, :not_found}
@ -83,8 +105,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
         {_, true} <- {:public?, Visibility.is_public?(object)},
         likes <- Utils.get_object_likes(object) do
      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(ObjectView.render("likes.json", ap_id, likes))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(ObjectView)
+      |> render("likes.json", %{ap_id: ap_id, likes: likes})
    else
      {:public?, false} ->
        {:error, :not_found}
@ -96,14 +119,52 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
         %Activity{} = activity <- Activity.normalize(ap_id),
         {_, true} <- {:public?, Visibility.is_public?(activity)} do
      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(ObjectView.render("object.json", %{object: activity}))
+      |> maybe_set_tracking_data(activity)
+      |> set_cache_ttl_for(activity)
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(ObjectView)
+      |> render("object.json", object: activity)
    else
-      {:public?, false} ->
-        {:error, :not_found}
+      {:public?, false} -> {:error, :not_found}
+      nil -> {:error, :not_found}
    end
  end

+  defp maybe_set_tracking_data(conn, %Activity{data: %{"type" => "Create"}} = activity) do
+    object_id = Object.normalize(activity).id
+    assign(conn, :tracking_fun_data, object_id)
+  end
+
+  defp maybe_set_tracking_data(conn, _activity), do: conn
+
+  defp set_cache_ttl_for(conn, %Activity{object: object}) do
+    set_cache_ttl_for(conn, object)
+  end
+
+  defp set_cache_ttl_for(conn, entity) do
+    ttl =
+      case entity do
+        %Object{data: %{"type" => "Question"}} ->
+          Pleroma.Config.get([:web_cache_ttl, :activity_pub_question])
+
+        %Object{} ->
+          Pleroma.Config.get([:web_cache_ttl, :activity_pub])
+
+        _ ->
+          nil
+      end
+
+    assign(conn, :cache_ttl, ttl)
+  end
+
+  # GET /relay/following
+  def following(%{assigns: %{relay: true}} = conn, _params) do
+    conn
+    |> put_resp_content_type("application/activity+json")
+    |> put_view(UserView)
+    |> render("following.json", %{user: Relay.get_actor()})
+  end
+
  def following(%{assigns: %{user: for_user}} = conn, %{"nickname" => nickname, "page" => page}) do
    with %User{} = user <- User.get_cached_by_nickname(nickname),
         {user, for_user} <- ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user),
@ -112,12 +173,13 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
      {page, _} = Integer.parse(page)

      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("following.json", %{user: user, page: page, for: for_user}))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("following.json", %{user: user, page: page, for: for_user})
    else
      {:show_follows, _} ->
        conn
-        |> put_resp_header("content-type", "application/activity+json")
+        |> put_resp_content_type("application/activity+json")
        |> send_resp(403, "")
    end
  end
@ -126,11 +188,20 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
    with %User{} = user <- User.get_cached_by_nickname(nickname),
         {user, for_user} <- ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user) do
      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("following.json", %{user: user, for: for_user}))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("following.json", %{user: user, for: for_user})
    end
  end

+  # GET /relay/followers
+  def followers(%{assigns: %{relay: true}} = conn, _params) do
+    conn
+    |> put_resp_content_type("application/activity+json")
+    |> put_view(UserView)
+    |> render("followers.json", %{user: Relay.get_actor()})
+  end
+
  def followers(%{assigns: %{user: for_user}} = conn, %{"nickname" => nickname, "page" => page}) do
    with %User{} = user <- User.get_cached_by_nickname(nickname),
         {user, for_user} <- ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user),
@ -139,12 +210,13 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
      {page, _} = Integer.parse(page)

      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("followers.json", %{user: user, page: page, for: for_user}))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("followers.json", %{user: user, page: page, for: for_user})
    else
      {:show_followers, _} ->
        conn
-        |> put_resp_header("content-type", "application/activity+json")
+        |> put_resp_content_type("application/activity+json")
        |> send_resp(403, "")
    end
  end
@ -153,17 +225,49 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
    with %User{} = user <- User.get_cached_by_nickname(nickname),
         {user, for_user} <- ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user) do
      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("followers.json", %{user: user, for: for_user}))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("followers.json", %{user: user, for: for_user})
    end
  end

-  def outbox(conn, %{"nickname" => nickname} = params) do
+  def outbox(conn, %{"nickname" => nickname, "page" => page?} = params)
+      when page? in [true, "true"] do
+    with %User{} = user <- User.get_cached_by_nickname(nickname),
+         {:ok, user} <- User.ensure_keys_present(user) do
+      activities =
+        if params["max_id"] do
+          ActivityPub.fetch_user_activities(user, nil, %{
+            "max_id" => params["max_id"],
+            # This is a hack because postgres generates inefficient queries when filtering by
+            # 'Answer', poll votes will be hidden by the visibility filter in this case anyway
+            "include_poll_votes" => true,
+            "limit" => 10
+          })
+        else
+          ActivityPub.fetch_user_activities(user, nil, %{
+            "limit" => 10,
+            "include_poll_votes" => true
+          })
+        end
+
+      conn
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("activity_collection_page.json", %{
+        activities: activities,
+        iri: "#{user.ap_id}/outbox"
+      })
+    end
+  end
+
+  def outbox(conn, %{"nickname" => nickname}) do
    with %User{} = user <- User.get_cached_by_nickname(nickname),
         {:ok, user} <- User.ensure_keys_present(user) do
      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("outbox.json", %{user: user, max_id: params["max_id"]}))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("activity_collection.json", %{iri: "#{user.ap_id}/outbox"})
    end
  end

@ -210,8 +314,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
  defp represent_service_actor(%User{} = user, conn) do
    with {:ok, user} <- User.ensure_keys_present(user) do
      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("user.json", %{user: user}))
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("user.json", %{user: user})
    else
      nil -> {:error, :not_found}
    end
@ -229,32 +334,73 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
    |> represent_service_actor(conn)
  end

+  @doc "Returns the authenticated user's ActivityPub User object or a 404 Not Found if non-authenticated"
  def whoami(%{assigns: %{user: %User{} = user}} = conn, _params) do
    conn
-    |> put_resp_header("content-type", "application/activity+json")
-    |> json(UserView.render("user.json", %{user: user}))
+    |> put_resp_content_type("application/activity+json")
+    |> put_view(UserView)
+    |> render("user.json", %{user: user})
  end

  def whoami(_conn, _params), do: {:error, :not_found}

-  def read_inbox(%{assigns: %{user: user}} = conn, %{"nickname" => nickname} = params) do
-    if nickname == user.nickname do
-      conn
-      |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("inbox.json", %{user: user, max_id: params["max_id"]}))
-    else
-      err =
-        dgettext("errors", "can't read inbox of %{nickname} as %{as_nickname}",
-          nickname: nickname,
-          as_nickname: user.nickname
-        )
+  def read_inbox(
+        %{assigns: %{user: %{nickname: nickname} = user}} = conn,
+        %{"nickname" => nickname, "page" => page?} = params
+      )
+      when page? in [true, "true"] do
+    activities =
+      if params["max_id"] do
+        ActivityPub.fetch_activities([user.ap_id | user.following], %{
+          "max_id" => params["max_id"],
+          "limit" => 10
+        })
+      else
+        ActivityPub.fetch_activities([user.ap_id | user.following], %{"limit" => 10})
+      end

+    conn
+    |> put_resp_content_type("application/activity+json")
+    |> put_view(UserView)
+    |> render("activity_collection_page.json", %{
+      activities: activities,
+      iri: "#{user.ap_id}/inbox"
+    })
+  end
+
+  def read_inbox(%{assigns: %{user: %{nickname: nickname} = user}} = conn, %{
+        "nickname" => nickname
+      }) do
+    with {:ok, user} <- User.ensure_keys_present(user) do
      conn
-      |> put_status(:forbidden)
-      |> json(err)
+      |> put_resp_content_type("application/activity+json")
+      |> put_view(UserView)
+      |> render("activity_collection.json", %{iri: "#{user.ap_id}/inbox"})
    end
  end

+  def read_inbox(%{assigns: %{user: nil}} = conn, %{"nickname" => nickname}) do
+    err = dgettext("errors", "can't read inbox of %{nickname}", nickname: nickname)
+
+    conn
+    |> put_status(:forbidden)
+    |> json(err)
+  end
+
+  def read_inbox(%{assigns: %{user: %{nickname: as_nickname}}} = conn, %{
+        "nickname" => nickname
+      }) do
+    err =
+      dgettext("errors", "can't read inbox of %{nickname} as %{as_nickname}",
+        nickname: nickname,
+        as_nickname: as_nickname
+      )
+
+    conn
+    |> put_status(:forbidden)
+    |> json(err)
+  end
+
  def handle_user_activity(user, %{"type" => "Create"} = params) do
    object =
      params["object"]
@ -295,42 +441,42 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
  end

  def update_outbox(
-        %{assigns: %{user: user}} = conn,
+        %{assigns: %{user: %User{nickname: nickname} = user}} = conn,
        %{"nickname" => nickname} = params
      ) do
-    if nickname == user.nickname do
-      actor = user.ap_id()
+    actor = user.ap_id()

-      params =
-        params
-        |> Map.drop(["id"])
-        |> Map.put("actor", actor)
-        |> Transmogrifier.fix_addressing()
-
-      with {:ok, %Activity{} = activity} <- handle_user_activity(user, params) do
-        conn
-        |> put_status(:created)
-        |> put_resp_header("location", activity.data["id"])
-        |> json(activity.data)
-      else
-        {:error, message} ->
-          conn
-          |> put_status(:bad_request)
-          |> json(message)
-      end
-    else
-      err =
-        dgettext("errors", "can't update outbox of %{nickname} as %{as_nickname}",
-          nickname: nickname,
-          as_nickname: user.nickname
-        )
+    params =
+      params
+      |> Map.drop(["id"])
+      |> Map.put("actor", actor)
+      |> Transmogrifier.fix_addressing()

+    with {:ok, %Activity{} = activity} <- handle_user_activity(user, params) do
      conn
-      |> put_status(:forbidden)
-      |> json(err)
+      |> put_status(:created)
+      |> put_resp_header("location", activity.data["id"])
+      |> json(activity.data)
+    else
+      {:error, message} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(message)
    end
  end

+  def update_outbox(%{assigns: %{user: user}} = conn, %{"nickname" => nickname} = _) do
+    err =
+      dgettext("errors", "can't update outbox of %{nickname} as %{as_nickname}",
+        nickname: nickname,
+        as_nickname: user.nickname
+      )
+
+    conn
+    |> put_status(:forbidden)
+    |> json(err)
+  end
+
  def errors(conn, {:error, :not_found}) do
    conn
    |> put_status(:not_found)
@ -364,4 +510,31 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do

    {new_user, for_user}
  end
+
+  # TODO: Add support for "object" field
+  @doc """
+  Endpoint based on <https://www.w3.org/wiki/SocialCG/ActivityPub/MediaUpload>
+
+  Parameters:
+  - (required) `file`: data of the media
+  - (optionnal) `description`: description of the media, intended for accessibility
+
+  Response:
+  - HTTP Code: 201 Created
+  - HTTP Body: ActivityPub object to be inserted into another's `attachment` field
+  """
+  def upload_media(%{assigns: %{user: user}} = conn, %{"file" => file} = data) do
+    with {:ok, object} <-
+           ActivityPub.upload(
+             file,
+             actor: User.ap_id(user),
+             description: Map.get(data, "description")
+           ) do
+      Logger.debug(inspect(object))
+
+      conn
+      |> put_status(:created)
+      |> json(object.data)
+    end
+  end
 end
--- a/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex
@ -25,11 +25,15 @@ defmodule Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicy do
  defp score_displayname(_), do: 0.0

  defp determine_if_followbot(%User{nickname: nickname, name: displayname}) do
-    # nickname will always be a binary string because it's generated by Pleroma.
+    # nickname will be a binary string except when following a relay
    nick_score =
-      nickname
-      |> String.downcase()
-      |> score_nickname()
+      if is_binary(nickname) do
+        nickname
+        |> String.downcase()
+        |> score_nickname()
+      else
+        0.0
+      end

    # displayname will either be a binary string or nil, if a displayname isn't set.
    name_score =
--- a/lib/pleroma/web/activity_pub/mrf/mediaproxy_warming_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/mediaproxy_warming_policy.ex
@ -8,6 +8,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy do

  alias Pleroma.HTTP
  alias Pleroma.Web.MediaProxy
+  alias Pleroma.Workers.BackgroundWorker

  require Logger

@ -30,7 +31,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy do
        url
        |> Enum.each(fn
          %{"href" => href} ->
-            PleromaJobQueue.enqueue(:background, __MODULE__, [:prefetch, href])
+            BackgroundWorker.enqueue("media_proxy_prefetch", %{"url" => href})

          x ->
            Logger.debug("Unhandled attachment URL object #{inspect(x)}")
@ -46,7 +47,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy do
        %{"type" => "Create", "object" => %{"attachment" => attachments} = _object} = message
      )
      when is_list(attachments) and length(attachments) > 0 do
-    PleromaJobQueue.enqueue(:background, __MODULE__, [:preload, message])
+    BackgroundWorker.enqueue("media_proxy_preload", %{"message" => message})

    {:ok, message}
  end
--- a/lib/pleroma/web/activity_pub/publisher.ex
+++ b/lib/pleroma/web/activity_pub/publisher.ex
@ -5,8 +5,10 @@
 defmodule Pleroma.Web.ActivityPub.Publisher do
  alias Pleroma.Activity
  alias Pleroma.Config
+  alias Pleroma.Delivery
  alias Pleroma.HTTP
  alias Pleroma.Instances
+  alias Pleroma.Object
  alias Pleroma.User
  alias Pleroma.Web.ActivityPub.Relay
  alias Pleroma.Web.ActivityPub.Transmogrifier
@ -50,9 +52,7 @@ defmodule Pleroma.Web.ActivityPub.Publisher do

    digest = "SHA-256=" <> (:crypto.hash(:sha256, json) |> Base.encode64())

-    date =
-      NaiveDateTime.utc_now()
-      |> Timex.format!("{WDshort}, {0D} {Mshort} {YYYY} {h24}:{m}:{s} GMT")
+    date = Pleroma.Signature.signed_date()

    signature =
      Pleroma.Signature.sign(actor, %{
@ -86,6 +86,15 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
    end
  end

+  def publish_one(%{actor_id: actor_id} = params) do
+    actor = User.get_cached_by_id(actor_id)
+
+    params
+    |> Map.delete(:actor_id)
+    |> Map.put(:actor, actor)
+    |> publish_one()
+  end
+
  defp should_federate?(inbox, public) do
    if public do
      true
@ -102,14 +111,25 @@ defmodule Pleroma.Web.ActivityPub.Publisher do

  @spec recipients(User.t(), Activity.t()) :: list(User.t()) | []
  defp recipients(actor, activity) do
-    {:ok, followers} =
+    followers =
      if actor.follower_address in activity.recipients do
        User.get_external_followers(actor)
      else
-        {:ok, []}
+        []
      end

-    Pleroma.Web.Salmon.remote_users(actor, activity) ++ followers
+    fetchers =
+      with %Activity{data: %{"type" => "Delete"}} <- activity,
+           %Object{id: object_id} <- Object.normalize(activity),
+           fetchers <- User.get_delivered_users_by_object_id(object_id),
+           _ <- Delivery.delete_all_by_object_id(object_id) do
+        fetchers
+      else
+        _ ->
+          []
+      end
+
+    Pleroma.Web.Salmon.remote_users(actor, activity) ++ followers ++ fetchers
  end

  defp get_cc_ap_ids(ap_id, recipients) do
@ -161,7 +181,8 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
  Publishes an activity with BCC to all relevant peers.
  """

-  def publish(actor, %{data: %{"bcc" => bcc}} = activity) when is_list(bcc) and bcc != [] do
+  def publish(%User{} = actor, %{data: %{"bcc" => bcc}} = activity)
+      when is_list(bcc) and bcc != [] do
    public = is_public?(activity)
    {:ok, data} = Transmogrifier.prepare_outgoing(activity.data)

@ -188,7 +209,7 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
      Pleroma.Web.Federator.Publisher.enqueue_one(__MODULE__, %{
        inbox: inbox,
        json: json,
-        actor: actor,
+        actor_id: actor.id,
        id: activity.data["id"],
        unreachable_since: unreachable_since
      })
@ -223,7 +244,7 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
        %{
          inbox: inbox,
          json: json,
-          actor: actor,
+          actor_id: actor.id,
          id: activity.data["id"],
          unreachable_since: unreachable_since
        }
--- a/lib/pleroma/web/activity_pub/relay.ex
+++ b/lib/pleroma/web/activity_pub/relay.ex
@ -22,13 +22,7 @@ defmodule Pleroma.Web.ActivityPub.Relay do
      Logger.info("relay: followed instance: #{target_instance}; id=#{activity.data["id"]}")
      {:ok, activity}
    else
-      {:error, _} = error ->
-        Logger.error("error: #{inspect(error)}")
-        error
-
-      e ->
-        Logger.error("error: #{inspect(e)}")
-        {:error, e}
+      error -> format_error(error)
    end
  end

@ -37,16 +31,11 @@ defmodule Pleroma.Web.ActivityPub.Relay do
    with %User{} = local_user <- get_actor(),
         {:ok, %User{} = target_user} <- User.get_or_fetch_by_ap_id(target_instance),
         {:ok, activity} <- ActivityPub.unfollow(local_user, target_user) do
+      User.unfollow(local_user, target_user)
      Logger.info("relay: unfollowed instance: #{target_instance}: id=#{activity.data["id"]}")
      {:ok, activity}
    else
-      {:error, _} = error ->
-        Logger.error("error: #{inspect(error)}")
-        error
-
-      e ->
-        Logger.error("error: #{inspect(e)}")
-        {:error, e}
+      error -> format_error(error)
    end
  end

@ -56,11 +45,16 @@ defmodule Pleroma.Web.ActivityPub.Relay do
         %Object{} = object <- Object.normalize(activity) do
      ActivityPub.announce(user, object, nil, true, false)
    else
-      e ->
-        Logger.error("error: #{inspect(e)}")
-        {:error, inspect(e)}
+      error -> format_error(error)
    end
  end

  def publish(_), do: {:error, "Not implemented"}
+
+  defp format_error({:error, error}), do: format_error(error)
+
+  defp format_error(error) do
+    Logger.error("error: #{inspect(error)}")
+    {:error, error}
+  end
 end
--- a/lib/pleroma/web/activity_pub/transmogrifier.ex
+++ b/lib/pleroma/web/activity_pub/transmogrifier.ex
@ -15,6 +15,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
  alias Pleroma.Web.ActivityPub.Utils
  alias Pleroma.Web.ActivityPub.Visibility
  alias Pleroma.Web.Federator
+  alias Pleroma.Workers.TransmogrifierWorker

  import Ecto.Query

@ -41,8 +42,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
  end

  def fix_summary(%{"summary" => nil} = object) do
-    object
-    |> Map.put("summary", "")
+    Map.put(object, "summary", "")
  end

  def fix_summary(%{"summary" => _} = object) do
@ -50,10 +50,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    object
  end

-  def fix_summary(object) do
-    object
-    |> Map.put("summary", "")
-  end
+  def fix_summary(object), do: Map.put(object, "summary", "")

  def fix_addressing_list(map, field) do
    cond do
@ -73,13 +70,9 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
        explicit_mentions,
        follower_collection
      ) do
-    explicit_to =
-      to
-      |> Enum.filter(fn x -> x in explicit_mentions end)
+    explicit_to = Enum.filter(to, fn x -> x in explicit_mentions end)

-    explicit_cc =
-      to
-      |> Enum.filter(fn x -> x not in explicit_mentions end)
+    explicit_cc = Enum.filter(to, fn x -> x not in explicit_mentions end)

    final_cc =
      (cc ++ explicit_cc)
@ -97,13 +90,19 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
  def fix_explicit_addressing(%{"directMessage" => true} = object), do: object

  def fix_explicit_addressing(object) do
-    explicit_mentions =
+    explicit_mentions = Utils.determine_explicit_mentions(object)
+
+    %User{follower_address: follower_collection} =
      object
-      |> Utils.determine_explicit_mentions()
+      |> Containment.get_actor()
+      |> User.get_cached_by_ap_id()

-    follower_collection = User.get_cached_by_ap_id(Containment.get_actor(object)).follower_address
-
-    explicit_mentions = explicit_mentions ++ [Pleroma.Constants.as_public(), follower_collection]
+    explicit_mentions =
+      explicit_mentions ++
+        [
+          Pleroma.Constants.as_public(),
+          follower_collection
+        ]

    fix_explicit_addressing(object, explicit_mentions, follower_collection)
  end
@ -147,50 +146,27 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
  end

  def fix_actor(%{"attributedTo" => actor} = object) do
-    object
-    |> Map.put("actor", Containment.get_actor(%{"actor" => actor}))
+    Map.put(object, "actor", Containment.get_actor(%{"actor" => actor}))
  end

  def fix_in_reply_to(object, options \\ [])

  def fix_in_reply_to(%{"inReplyTo" => in_reply_to} = object, options)
      when not is_nil(in_reply_to) do
-    in_reply_to_id =
-      cond do
-        is_bitstring(in_reply_to) ->
-          in_reply_to
-
-        is_map(in_reply_to) && is_bitstring(in_reply_to["id"]) ->
-          in_reply_to["id"]
-
-        is_list(in_reply_to) && is_bitstring(Enum.at(in_reply_to, 0)) ->
-          Enum.at(in_reply_to, 0)
-
-        # Maybe I should output an error too?
-        true ->
-          ""
-      end
-
+    in_reply_to_id = prepare_in_reply_to(in_reply_to)
    object = Map.put(object, "inReplyToAtomUri", in_reply_to_id)

    if Federator.allowed_incoming_reply_depth?(options[:depth]) do
-      case get_obj_helper(in_reply_to_id, options) do
-        {:ok, replied_object} ->
-          with %Activity{} = _activity <-
-                 Activity.get_create_by_object_ap_id(replied_object.data["id"]) do
-            object
-            |> Map.put("inReplyTo", replied_object.data["id"])
-            |> Map.put("inReplyToAtomUri", object["inReplyToAtomUri"] || in_reply_to_id)
-            |> Map.put("conversation", replied_object.data["context"] || object["conversation"])
-            |> Map.put("context", replied_object.data["context"] || object["conversation"])
-          else
-            e ->
-              Logger.error("Couldn't fetch \"#{inspect(in_reply_to_id)}\", error: #{inspect(e)}")
-              object
-          end
-
+      with {:ok, replied_object} <- get_obj_helper(in_reply_to_id, options),
+           %Activity{} = _ <- Activity.get_create_by_object_ap_id(replied_object.data["id"]) do
+        object
+        |> Map.put("inReplyTo", replied_object.data["id"])
+        |> Map.put("inReplyToAtomUri", object["inReplyToAtomUri"] || in_reply_to_id)
+        |> Map.put("conversation", replied_object.data["context"] || object["conversation"])
+        |> Map.put("context", replied_object.data["context"] || object["conversation"])
+      else
        e ->
-          Logger.error("Couldn't fetch \"#{inspect(in_reply_to_id)}\", error: #{inspect(e)}")
+          Logger.error("Couldn't fetch #{inspect(in_reply_to_id)}, error: #{inspect(e)}")
          object
      end
    else
@ -200,6 +176,22 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do

  def fix_in_reply_to(object, _options), do: object

+  defp prepare_in_reply_to(in_reply_to) do
+    cond do
+      is_bitstring(in_reply_to) ->
+        in_reply_to
+
+      is_map(in_reply_to) && is_bitstring(in_reply_to["id"]) ->
+        in_reply_to["id"]
+
+      is_list(in_reply_to) && is_bitstring(Enum.at(in_reply_to, 0)) ->
+        Enum.at(in_reply_to, 0)
+
+      true ->
+        ""
+    end
+  end
+
  def fix_context(object) do
    context = object["context"] || object["conversation"] || Utils.generate_context_id()

@ -210,11 +202,9 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do

  def fix_attachments(%{"attachment" => attachment} = object) when is_list(attachment) do
    attachments =
-      attachment
-      |> Enum.map(fn data ->
+      Enum.map(attachment, fn data ->
        media_type = data["mediaType"] || data["mimeType"]
        href = data["url"] || data["href"]
-
        url = [%{"type" => "Link", "mediaType" => media_type, "href" => href}]

        data
@ -222,30 +212,25 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
        |> Map.put("url", url)
      end)

-    object
-    |> Map.put("attachment", attachments)
+    Map.put(object, "attachment", attachments)
  end

  def fix_attachments(%{"attachment" => attachment} = object) when is_map(attachment) do
-    Map.put(object, "attachment", [attachment])
+    object
+    |> Map.put("attachment", [attachment])
    |> fix_attachments()
  end

  def fix_attachments(object), do: object

  def fix_url(%{"url" => url} = object) when is_map(url) do
-    object
-    |> Map.put("url", url["href"])
+    Map.put(object, "url", url["href"])
  end

  def fix_url(%{"type" => "Video", "url" => url} = object) when is_list(url) do
    first_element = Enum.at(url, 0)

-    link_element =
-      url
-      |> Enum.filter(fn x -> is_map(x) end)
-      |> Enum.filter(fn x -> x["mimeType"] == "text/html" end)
-      |> Enum.at(0)
+    link_element = Enum.find(url, fn x -> is_map(x) and x["mimeType"] == "text/html" end)

    object
    |> Map.put("attachment", [first_element])
@ -263,36 +248,32 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
        true -> ""
      end

-    object
-    |> Map.put("url", url_string)
+    Map.put(object, "url", url_string)
  end

  def fix_url(object), do: object

  def fix_emoji(%{"tag" => tags} = object) when is_list(tags) do
-    emoji = tags |> Enum.filter(fn data -> data["type"] == "Emoji" and data["icon"] end)
-
    emoji =
-      emoji
+      tags
+      |> Enum.filter(fn data -> data["type"] == "Emoji" and data["icon"] end)
      |> Enum.reduce(%{}, fn data, mapping ->
        name = String.trim(data["name"], ":")

-        mapping |> Map.put(name, data["icon"]["url"])
+        Map.put(mapping, name, data["icon"]["url"])
      end)

    # we merge mastodon and pleroma emoji into a single mapping, to allow for both wire formats
    emoji = Map.merge(object["emoji"] || %{}, emoji)

-    object
-    |> Map.put("emoji", emoji)
+    Map.put(object, "emoji", emoji)
  end

  def fix_emoji(%{"tag" => %{"type" => "Emoji"} = tag} = object) do
    name = String.trim(tag["name"], ":")
    emoji = %{name => tag["icon"]["url"]}

-    object
-    |> Map.put("emoji", emoji)
+    Map.put(object, "emoji", emoji)
  end

  def fix_emoji(object), do: object
@ -303,17 +284,13 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
      |> Enum.filter(fn data -> data["type"] == "Hashtag" and data["name"] end)
      |> Enum.map(fn data -> String.slice(data["name"], 1..-1) end)

-    combined = tag ++ tags
-
-    object
-    |> Map.put("tag", combined)
+    Map.put(object, "tag", tag ++ tags)
  end

  def fix_tag(%{"tag" => %{"type" => "Hashtag", "name" => hashtag} = tag} = object) do
    combined = [tag, String.slice(hashtag, 1..-1)]

-    object
-    |> Map.put("tag", combined)
+    Map.put(object, "tag", combined)
  end

  def fix_tag(%{"tag" => %{} = tag} = object), do: Map.put(object, "tag", [tag])
@ -325,8 +302,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    content_groups = Map.to_list(content_map)
    {_, content} = Enum.at(content_groups, 0)

-    object
-    |> Map.put("content", content)
+    Map.put(object, "content", content)
  end

  def fix_content_map(object), do: object
@ -335,16 +311,11 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do

  def fix_type(%{"inReplyTo" => reply_id, "name" => _} = object, options)
      when is_binary(reply_id) do
-    reply =
-      with true <- Federator.allowed_incoming_reply_depth?(options[:depth]),
-           {:ok, object} <- get_obj_helper(reply_id, options) do
-        object
-      end
-
-    if reply && reply.data["type"] == "Question" do
+    with true <- Federator.allowed_incoming_reply_depth?(options[:depth]),
+         {:ok, %{data: %{"type" => "Question"} = _} = _} <- get_obj_helper(reply_id, options) do
      Map.put(object, "type", "Answer")
    else
-      object
+      _ -> object
    end
  end

@ -376,6 +347,17 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    end
  end

+  # Reduce the object list to find the reported user.
+  defp get_reported(objects) do
+    Enum.reduce_while(objects, nil, fn ap_id, _ ->
+      with %User{} = user <- User.get_cached_by_ap_id(ap_id) do
+        {:halt, user}
+      else
+        _ -> {:cont, nil}
+      end
+    end)
+  end
+
  def handle_incoming(data, options \\ [])

  # Flag objects are placed ahead of the ID check because Mastodon 2.8 and earlier send them
@ -384,31 +366,19 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    with context <- data["context"] || Utils.generate_context_id(),
         content <- data["content"] || "",
         %User{} = actor <- User.get_cached_by_ap_id(actor),
-
         # Reduce the object list to find the reported user.
-         %User{} = account <-
-           Enum.reduce_while(objects, nil, fn ap_id, _ ->
-             with %User{} = user <- User.get_cached_by_ap_id(ap_id) do
-               {:halt, user}
-             else
-               _ -> {:cont, nil}
-             end
-           end),
-
+         %User{} = account <- get_reported(objects),
         # Remove the reported user from the object list.
         statuses <- Enum.filter(objects, fn ap_id -> ap_id != account.ap_id end) do
-      params = %{
+      %{
        actor: actor,
        context: context,
        account: account,
        statuses: statuses,
        content: content,
-        additional: %{
-          "cc" => [account.ap_id]
-        }
+        additional: %{"cc" => [account.ap_id]}
      }
-
-      ActivityPub.flag(params)
+      |> ActivityPub.flag()
    end
  end

@ -460,12 +430,44 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    end
  end

+  def handle_incoming(
+        %{"type" => "Listen", "object" => %{"type" => "Audio"} = object} = data,
+        options
+      ) do
+    actor = Containment.get_actor(data)
+
+    data =
+      Map.put(data, "actor", actor)
+      |> fix_addressing
+
+    with {:ok, %User{} = user} <- User.get_or_fetch_by_ap_id(data["actor"]) do
+      options = Keyword.put(options, :depth, (options[:depth] || 0) + 1)
+      object = fix_object(object, options)
+
+      params = %{
+        to: data["to"],
+        object: object,
+        actor: user,
+        context: nil,
+        local: false,
+        published: data["published"],
+        additional: Map.take(data, ["cc", "id"])
+      }
+
+      ActivityPub.listen(params)
+    else
+      _e -> :error
+    end
+  end
+
  def handle_incoming(
        %{"type" => "Follow", "object" => followed, "actor" => follower, "id" => id} = data,
        _options
      ) do
-    with %User{local: true} = followed <- User.get_cached_by_ap_id(followed),
-         {:ok, %User{} = follower} <- User.get_or_fetch_by_ap_id(follower),
+    with %User{local: true} = followed <-
+           User.get_cached_by_ap_id(Containment.get_actor(%{"actor" => followed})),
+         {:ok, %User{} = follower} <-
+           User.get_or_fetch_by_ap_id(Containment.get_actor(%{"actor" => follower})),
         {:ok, activity} <- ActivityPub.follow(follower, followed, id, false) do
      with deny_follow_blocked <- Pleroma.Config.get([:user, :deny_follow_blocked]),
           {_, false} <- {:user_blocked, User.blocks?(followed, follower) && deny_follow_blocked},
@ -753,8 +755,12 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do

  def handle_incoming(_, _), do: :error

+  @spec get_obj_helper(String.t(), Keyword.t()) :: {:ok, Object.t()} | nil
  def get_obj_helper(id, options \\ []) do
-    if object = Object.normalize(id, true, options), do: {:ok, object}, else: nil
+    case Object.normalize(id, true, options) do
+      %Object{} = object -> {:ok, object}
+      _ -> nil
+    end
  end

  def set_reply_to_uri(%{"inReplyTo" => in_reply_to} = object) when is_binary(in_reply_to) do
@ -789,7 +795,8 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
  #  internal -> Mastodon
  #  """

-  def prepare_outgoing(%{"type" => "Create", "object" => object_id} = data) do
+  def prepare_outgoing(%{"type" => activity_type, "object" => object_id} = data)
+      when activity_type in ["Create", "Listen"] do
    object =
      object_id
      |> Object.normalize()
@ -853,27 +860,24 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    {:ok, data}
  end

-  def maybe_fix_object_url(data) do
-    if is_binary(data["object"]) and not String.starts_with?(data["object"], "http") do
-      case get_obj_helper(data["object"]) do
-        {:ok, relative_object} ->
-          if relative_object.data["external_url"] do
-            _data =
-              data
-              |> Map.put("object", relative_object.data["external_url"])
-          else
-            data
-          end
-
-        e ->
-          Logger.error("Couldn't fetch #{data["object"]} #{inspect(e)}")
-          data
-      end
+  def maybe_fix_object_url(%{"object" => object} = data) when is_binary(object) do
+    with false <- String.starts_with?(object, "http"),
+         {:fetch, {:ok, relative_object}} <- {:fetch, get_obj_helper(object)},
+         %{data: %{"external_url" => external_url}} when not is_nil(external_url) <-
+           relative_object do
+      Map.put(data, "object", external_url)
    else
-      data
+      {:fetch, e} ->
+        Logger.error("Couldn't fetch #{object} #{inspect(e)}")
+        data
+
+      _ ->
+        data
    end
  end

+  def maybe_fix_object_url(data), do: data
+
  def add_hashtags(object) do
    tags =
      (object["tag"] || [])
@ -891,53 +895,49 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
          tag
      end)

-    object
-    |> Map.put("tag", tags)
+    Map.put(object, "tag", tags)
  end

  def add_mention_tags(object) do
    mentions =
      object
      |> Utils.get_notified_from_object()
-      |> Enum.map(fn user ->
-        %{"type" => "Mention", "href" => user.ap_id, "name" => "@#{user.nickname}"}
-      end)
+      |> Enum.map(&build_mention_tag/1)

    tags = object["tag"] || []

-    object
-    |> Map.put("tag", tags ++ mentions)
+    Map.put(object, "tag", tags ++ mentions)
  end

-  def add_emoji_tags(%User{info: %{"emoji" => _emoji} = user_info} = object) do
-    user_info = add_emoji_tags(user_info)
+  defp build_mention_tag(%{ap_id: ap_id, nickname: nickname} = _) do
+    %{"type" => "Mention", "href" => ap_id, "name" => "@#{nickname}"}
+  end

-    object
-    |> Map.put(:info, user_info)
+  def take_emoji_tags(%User{info: %{emoji: emoji} = _user_info} = _user) do
+    emoji
+    |> Enum.flat_map(&Map.to_list/1)
+    |> Enum.map(&build_emoji_tag/1)
  end

  # TODO: we should probably send mtime instead of unix epoch time for updated
  def add_emoji_tags(%{"emoji" => emoji} = object) do
    tags = object["tag"] || []

-    out =
-      emoji
-      |> Enum.map(fn {name, url} ->
-        %{
-          "icon" => %{"url" => url, "type" => "Image"},
-          "name" => ":" <> name <> ":",
-          "type" => "Emoji",
-          "updated" => "1970-01-01T00:00:00Z",
-          "id" => url
-        }
-      end)
+    out = Enum.map(emoji, &build_emoji_tag/1)

-    object
-    |> Map.put("tag", tags ++ out)
+    Map.put(object, "tag", tags ++ out)
  end

-  def add_emoji_tags(object) do
-    object
+  def add_emoji_tags(object), do: object
+
+  defp build_emoji_tag({name, url}) do
+    %{
+      "icon" => %{"url" => url, "type" => "Image"},
+      "name" => ":" <> name <> ":",
+      "type" => "Emoji",
+      "updated" => "1970-01-01T00:00:00Z",
+      "id" => url
+    }
  end

  def set_conversation(object) do
@ -957,9 +957,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do

  def add_attributed_to(object) do
    attributed_to = object["attributedTo"] || object["actor"]
-
-    object
-    |> Map.put("attributedTo", attributed_to)
+    Map.put(object, "attributedTo", attributed_to)
  end

  def prepare_attachments(object) do
@ -970,30 +968,18 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
        %{"url" => href, "mediaType" => media_type, "name" => data["name"], "type" => "Document"}
      end)

-    object
-    |> Map.put("attachment", attachments)
+    Map.put(object, "attachment", attachments)
  end

  defp strip_internal_fields(object) do
    object
-    |> Map.drop([
-      "likes",
-      "like_count",
-      "announcements",
-      "announcement_count",
-      "emoji",
-      "context_id",
-      "deleted_activity_id"
-    ])
+    |> Map.drop(Pleroma.Constants.object_internal_fields())
  end

  defp strip_internal_tags(%{"tag" => tags} = object) do
-    tags =
-      tags
-      |> Enum.filter(fn x -> is_map(x) end)
+    tags = Enum.filter(tags, fn x -> is_map(x) end)

-    object
-    |> Map.put("tag", tags)
+    Map.put(object, "tag", tags)
  end

  defp strip_internal_tags(object), do: object
@ -1047,9 +1033,9 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    with %User{local: false} = user <- User.get_cached_by_ap_id(ap_id),
         {:ok, data} <- ActivityPub.fetch_and_prepare_user_from_ap_id(ap_id),
         already_ap <- User.ap_enabled?(user),
-         {:ok, user} <- user |> User.upgrade_changeset(data) |> User.update_and_set_cache() do
-      unless already_ap do
-        PleromaJobQueue.enqueue(:transmogrifier, __MODULE__, [:user_upgrade, user])
+         {:ok, user} <- upgrade_user(user, data) do
+      if not already_ap do
+        TransmogrifierWorker.enqueue("user_upgrade", %{"user_id" => user.id})
      end

      {:ok, user}
@ -1059,6 +1045,12 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    end
  end

+  defp upgrade_user(user, data) do
+    user
+    |> User.upgrade_changeset(data, true)
+    |> User.update_and_set_cache()
+  end
+
  def maybe_retire_websub(ap_id) do
    # some sanity checks
    if is_binary(ap_id) && String.length(ap_id) > 8 do
@ -1072,16 +1064,11 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
    end
  end

-  def maybe_fix_user_url(data) do
-    if is_map(data["url"]) do
-      Map.put(data, "url", data["url"]["href"])
-    else
-      data
-    end
+  def maybe_fix_user_url(%{"url" => url} = data) when is_map(url) do
+    Map.put(data, "url", url["href"])
  end

-  def maybe_fix_user_object(data) do
-    data
-    |> maybe_fix_user_url
-  end
+  def maybe_fix_user_url(data), do: data
+
+  def maybe_fix_user_object(data), do: maybe_fix_user_url(data)
 end
--- a/lib/pleroma/web/activity_pub/utils.ex
+++ b/lib/pleroma/web/activity_pub/utils.ex
@ -20,7 +20,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  require Logger
  require Pleroma.Constants

-  @supported_object_types ["Article", "Note", "Video", "Page", "Question", "Answer"]
+  @supported_object_types ["Article", "Note", "Video", "Page", "Question", "Answer", "Audio"]
  @supported_report_states ~w(open closed resolved)
  @valid_visibilities ~w(public unlisted private direct)

@ -33,50 +33,40 @@ defmodule Pleroma.Web.ActivityPub.Utils do
    Map.put(params, "actor", get_ap_id(params["actor"]))
  end

-  def determine_explicit_mentions(%{"tag" => tag} = _object) when is_list(tag) do
-    tag
-    |> Enum.filter(fn x -> is_map(x) end)
-    |> Enum.filter(fn x -> x["type"] == "Mention" end)
-    |> Enum.map(fn x -> x["href"] end)
+  @spec determine_explicit_mentions(map()) :: map()
+  def determine_explicit_mentions(%{"tag" => tag} = _) when is_list(tag) do
+    Enum.flat_map(tag, fn
+      %{"type" => "Mention", "href" => href} -> [href]
+      _ -> []
+    end)
  end

  def determine_explicit_mentions(%{"tag" => tag} = object) when is_map(tag) do
-    Map.put(object, "tag", [tag])
+    object
+    |> Map.put("tag", [tag])
    |> determine_explicit_mentions()
  end

  def determine_explicit_mentions(_), do: []

+  @spec recipient_in_collection(any(), any()) :: boolean()
  defp recipient_in_collection(ap_id, coll) when is_binary(coll), do: ap_id == coll
  defp recipient_in_collection(ap_id, coll) when is_list(coll), do: ap_id in coll
  defp recipient_in_collection(_, _), do: false

+  @spec recipient_in_message(User.t(), User.t(), map()) :: boolean()
  def recipient_in_message(%User{ap_id: ap_id} = recipient, %User{} = actor, params) do
+    addresses = [params["to"], params["cc"], params["bto"], params["bcc"]]
+
    cond do
-      recipient_in_collection(ap_id, params["to"]) ->
-        true
-
-      recipient_in_collection(ap_id, params["cc"]) ->
-        true
-
-      recipient_in_collection(ap_id, params["bto"]) ->
-        true
-
-      recipient_in_collection(ap_id, params["bcc"]) ->
-        true
-
+      Enum.any?(addresses, &recipient_in_collection(ap_id, &1)) -> true
      # if the message is unaddressed at all, then assume it is directly addressed
      # to the recipient
-      !params["to"] && !params["cc"] && !params["bto"] && !params["bcc"] ->
-        true
-
+      Enum.all?(addresses, &is_nil(&1)) -> true
      # if the message is sent from somebody the user is following, then assume it
      # is addressed to the recipient
-      User.following?(recipient, actor) ->
-        true
-
-      true ->
-        false
+      User.following?(recipient, actor) -> true
+      true -> false
    end
  end

@ -85,15 +75,13 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  defp extract_list(_), do: []

  def maybe_splice_recipient(ap_id, params) do
-    need_splice =
+    need_splice? =
      !recipient_in_collection(ap_id, params["to"]) &&
        !recipient_in_collection(ap_id, params["cc"])

-    cc_list = extract_list(params["cc"])
-
-    if need_splice do
-      params
-      |> Map.put("cc", [ap_id | cc_list])
+    if need_splice? do
+      cc_list = extract_list(params["cc"])
+      Map.put(params, "cc", [ap_id | cc_list])
    else
      params
    end
@ -139,7 +127,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
      "object" => object
    }

-    Notification.get_notified_from_activity(%Activity{data: fake_create_activity}, false)
+    get_notified_from_object(fake_create_activity)
  end

  def get_notified_from_object(object) do
@ -166,16 +154,10 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  @doc """
  Enqueues an activity for federation if it's local
  """
+  @spec maybe_federate(any()) :: :ok
  def maybe_federate(%Activity{local: true} = activity) do
    if Pleroma.Config.get!([:instance, :federating]) do
-      priority =
-        case activity.data["type"] do
-          "Delete" -> 10
-          "Create" -> 1
-          _ -> 5
-        end
-
-      Pleroma.Web.Federator.publish(activity, priority)
+      Pleroma.Web.Federator.publish(activity)
    end

    :ok
@ -187,53 +169,58 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  Adds an id and a published data if they aren't there,
  also adds it to an included object
  """
-  def lazy_put_activity_defaults(map, fake \\ false) do
-    map =
-      unless fake do
-        %{data: %{"id" => context}, id: context_id} = create_context(map["context"])
+  @spec lazy_put_activity_defaults(map(), boolean) :: map()
+  def lazy_put_activity_defaults(map, fake? \\ false)

-        map
-        |> Map.put_new_lazy("id", &generate_activity_id/0)
-        |> Map.put_new_lazy("published", &make_date/0)
-        |> Map.put_new("context", context)
-        |> Map.put_new("context_id", context_id)
-      else
-        map
-        |> Map.put_new("id", "pleroma:fakeid")
-        |> Map.put_new_lazy("published", &make_date/0)
-        |> Map.put_new("context", "pleroma:fakecontext")
-        |> Map.put_new("context_id", -1)
-      end
+  def lazy_put_activity_defaults(map, true) do
+    map
+    |> Map.put_new("id", "pleroma:fakeid")
+    |> Map.put_new_lazy("published", &make_date/0)
+    |> Map.put_new("context", "pleroma:fakecontext")
+    |> Map.put_new("context_id", -1)
+    |> lazy_put_object_defaults(true)
+  end

-    if is_map(map["object"]) do
-      object = lazy_put_object_defaults(map["object"], map, fake)
-      %{map | "object" => object}
-    else
+  def lazy_put_activity_defaults(map, _fake?) do
+    %{data: %{"id" => context}, id: context_id} = create_context(map["context"])
+
+    map
+    |> Map.put_new_lazy("id", &generate_activity_id/0)
+    |> Map.put_new_lazy("published", &make_date/0)
+    |> Map.put_new("context", context)
+    |> Map.put_new("context_id", context_id)
+    |> lazy_put_object_defaults(false)
+  end
+
+  # Adds an id and published date if they aren't there.
+  #
+  @spec lazy_put_object_defaults(map(), boolean()) :: map()
+  defp lazy_put_object_defaults(%{"object" => map} = activity, true)
+       when is_map(map) do
+    object =
      map
-    end
+      |> Map.put_new("id", "pleroma:fake_object_id")
+      |> Map.put_new_lazy("published", &make_date/0)
+      |> Map.put_new("context", activity["context"])
+      |> Map.put_new("context_id", activity["context_id"])
+      |> Map.put_new("fake", true)
+
+    %{activity | "object" => object}
  end

-  @doc """
-  Adds an id and published date if they aren't there.
-  """
-  def lazy_put_object_defaults(map, activity \\ %{}, fake)
+  defp lazy_put_object_defaults(%{"object" => map} = activity, _)
+       when is_map(map) do
+    object =
+      map
+      |> Map.put_new_lazy("id", &generate_object_id/0)
+      |> Map.put_new_lazy("published", &make_date/0)
+      |> Map.put_new("context", activity["context"])
+      |> Map.put_new("context_id", activity["context_id"])

-  def lazy_put_object_defaults(map, activity, true = _fake) do
-    map
-    |> Map.put_new_lazy("published", &make_date/0)
-    |> Map.put_new("id", "pleroma:fake_object_id")
-    |> Map.put_new("context", activity["context"])
-    |> Map.put_new("fake", true)
-    |> Map.put_new("context_id", activity["context_id"])
+    %{activity | "object" => object}
  end

-  def lazy_put_object_defaults(map, activity, _fake) do
-    map
-    |> Map.put_new_lazy("id", &generate_object_id/0)
-    |> Map.put_new_lazy("published", &make_date/0)
-    |> Map.put_new("context", activity["context"])
-    |> Map.put_new("context_id", activity["context_id"])
-  end
+  defp lazy_put_object_defaults(activity, _), do: activity

  @doc """
  Inserts a full object if it is contained in an activity.
@ -241,9 +228,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  def insert_full_object(%{"object" => %{"type" => type} = object_data} = map)
      when is_map(object_data) and type in @supported_object_types do
    with {:ok, object} <- Object.create(object_data) do
-      map =
-        map
-        |> Map.put("object", object.data["id"])
+      map = Map.put(map, "object", object.data["id"])

      {:ok, map, object}
    end
@ -256,46 +241,27 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  @doc """
  Returns an existing like if a user already liked an object
  """
+  @spec get_existing_like(String.t(), map()) :: Activity.t() | nil
  def get_existing_like(actor, %{data: %{"id" => id}}) do
-    query =
-      from(
-        activity in Activity,
-        where: fragment("(?)->>'actor' = ?", activity.data, ^actor),
-        # this is to use the index
-        where:
-          fragment(
-            "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-            activity.data,
-            activity.data,
-            ^id
-          ),
-        where: fragment("(?)->>'type' = 'Like'", activity.data)
-      )
-
-    Repo.one(query)
+    actor
+    |> Activity.Queries.by_actor()
+    |> Activity.Queries.by_object_id(id)
+    |> Activity.Queries.by_type("Like")
+    |> limit(1)
+    |> Repo.one()
  end

  @doc """
  Returns like activities targeting an object
  """
  def get_object_likes(%{data: %{"id" => id}}) do
-    query =
-      from(
-        activity in Activity,
-        # this is to use the index
-        where:
-          fragment(
-            "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-            activity.data,
-            activity.data,
-            ^id
-          ),
-        where: fragment("(?)->>'type' = 'Like'", activity.data)
-      )
-
-    Repo.all(query)
+    id
+    |> Activity.Queries.by_object_id()
+    |> Activity.Queries.by_type("Like")
+    |> Repo.all()
  end

+  @spec make_like_data(User.t(), map(), String.t()) :: map()
  def make_like_data(
        %User{ap_id: ap_id} = actor,
        %{data: %{"actor" => object_actor_id, "id" => id}} = object,
@ -315,7 +281,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
      |> List.delete(actor.ap_id)
      |> List.delete(object_actor.follower_address)

-    data = %{
+    %{
      "type" => "Like",
      "actor" => ap_id,
      "object" => id,
@ -323,38 +289,49 @@ defmodule Pleroma.Web.ActivityPub.Utils do
      "cc" => cc,
      "context" => object.data["context"]
    }
-
-    if activity_id, do: Map.put(data, "id", activity_id), else: data
+    |> maybe_put("id", activity_id)
  end

+  @spec update_element_in_object(String.t(), list(any), Object.t()) ::
+          {:ok, Object.t()} | {:error, Ecto.Changeset.t()}
  def update_element_in_object(property, element, object) do
-    with new_data <-
-           object.data
-           |> Map.put("#{property}_count", length(element))
-           |> Map.put("#{property}s", element),
-         changeset <- Changeset.change(object, data: new_data),
-         {:ok, object} <- Object.update_and_set_cache(changeset) do
-      {:ok, object}
-    end
+    data =
+      Map.merge(
+        object.data,
+        %{"#{property}_count" => length(element), "#{property}s" => element}
+      )
+
+    object
+    |> Changeset.change(data: data)
+    |> Object.update_and_set_cache()
  end

-  def update_likes_in_object(likes, object) do
+  @spec add_like_to_object(Activity.t(), Object.t()) ::
+          {:ok, Object.t()} | {:error, Ecto.Changeset.t()}
+  def add_like_to_object(%Activity{data: %{"actor" => actor}}, object) do
+    [actor | fetch_likes(object)]
+    |> Enum.uniq()
+    |> update_likes_in_object(object)
+  end
+
+  @spec remove_like_from_object(Activity.t(), Object.t()) ::
+          {:ok, Object.t()} | {:error, Ecto.Changeset.t()}
+  def remove_like_from_object(%Activity{data: %{"actor" => actor}}, object) do
+    object
+    |> fetch_likes()
+    |> List.delete(actor)
+    |> update_likes_in_object(object)
+  end
+
+  defp update_likes_in_object(likes, object) do
    update_element_in_object("like", likes, object)
  end

-  def add_like_to_object(%Activity{data: %{"actor" => actor}}, object) do
-    likes = if is_list(object.data["likes"]), do: object.data["likes"], else: []
-
-    with likes <- [actor | likes] |> Enum.uniq() do
-      update_likes_in_object(likes, object)
-    end
-  end
-
-  def remove_like_from_object(%Activity{data: %{"actor" => actor}}, object) do
-    likes = if is_list(object.data["likes"]), do: object.data["likes"], else: []
-
-    with likes <- likes |> List.delete(actor) do
-      update_likes_in_object(likes, object)
+  defp fetch_likes(object) do
+    if is_list(object.data["likes"]) do
+      object.data["likes"]
+    else
+      []
    end
  end

@ -363,36 +340,35 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  @doc """
  Updates a follow activity's state (for locked accounts).
  """
+  @spec update_follow_state_for_all(Activity.t(), String.t()) :: {:ok, Activity} | {:error, any()}
  def update_follow_state_for_all(
        %Activity{data: %{"actor" => actor, "object" => object}} = activity,
        state
      ) do
-    try do
-      Ecto.Adapters.SQL.query!(
-        Repo,
-        "UPDATE activities SET data = jsonb_set(data, '{state}', $1) WHERE data->>'type' = 'Follow' AND data->>'actor' = $2 AND data->>'object' = $3 AND data->>'state' = 'pending'",
-        [state, actor, object]
-      )
+    "Follow"
+    |> Activity.Queries.by_type()
+    |> Activity.Queries.by_actor(actor)
+    |> Activity.Queries.by_object_id(object)
+    |> where(fragment("data->>'state' = 'pending'"))
+    |> update(set: [data: fragment("jsonb_set(data, '{state}', ?)", ^state)])
+    |> Repo.update_all([])

-      User.set_follow_state_cache(actor, object, state)
-      activity = Activity.get_by_id(activity.id)
-      {:ok, activity}
-    rescue
-      e ->
-        {:error, e}
-    end
+    User.set_follow_state_cache(actor, object, state)
+
+    activity = Activity.get_by_id(activity.id)
+
+    {:ok, activity}
  end

  def update_follow_state(
        %Activity{data: %{"actor" => actor, "object" => object}} = activity,
        state
      ) do
-    with new_data <-
-           activity.data
-           |> Map.put("state", state),
-         changeset <- Changeset.change(activity, data: new_data),
-         {:ok, activity} <- Repo.update(changeset),
-         _ <- User.set_follow_state_cache(actor, object, state) do
+    new_data = Map.put(activity.data, "state", state)
+    changeset = Changeset.change(activity, data: new_data)
+
+    with {:ok, activity} <- Repo.update(changeset) do
+      User.set_follow_state_cache(actor, object, state)
      {:ok, activity}
    end
  end
@ -405,7 +381,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
        %User{ap_id: followed_id} = _followed,
        activity_id
      ) do
-    data = %{
+    %{
      "type" => "Follow",
      "actor" => follower_id,
      "to" => [followed_id],
@ -413,35 +389,18 @@ defmodule Pleroma.Web.ActivityPub.Utils do
      "object" => followed_id,
      "state" => "pending"
    }
-
-    data = if activity_id, do: Map.put(data, "id", activity_id), else: data
-
-    data
+    |> maybe_put("id", activity_id)
  end

  def fetch_latest_follow(%User{ap_id: follower_id}, %User{ap_id: followed_id}) do
-    query =
-      from(
-        activity in Activity,
-        where:
-          fragment(
-            "? ->> 'type' = 'Follow'",
-            activity.data
-          ),
-        where: activity.actor == ^follower_id,
-        # this is to use the index
-        where:
-          fragment(
-            "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-            activity.data,
-            activity.data,
-            ^followed_id
-          ),
-        order_by: [fragment("? desc nulls last", activity.id)],
-        limit: 1
-      )
-
-    Repo.one(query)
+    "Follow"
+    |> Activity.Queries.by_type()
+    |> where(actor: ^follower_id)
+    # this is to use the index
+    |> Activity.Queries.by_object_id(followed_id)
+    |> order_by([activity], fragment("? desc nulls last", activity.id))
+    |> limit(1)
+    |> Repo.one()
  end

  #### Announce-related helpers
@ -449,23 +408,14 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  @doc """
  Retruns an existing announce activity if the notice has already been announced
  """
-  def get_existing_announce(actor, %{data: %{"id" => id}}) do
-    query =
-      from(
-        activity in Activity,
-        where: activity.actor == ^actor,
-        # this is to use the index
-        where:
-          fragment(
-            "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-            activity.data,
-            activity.data,
-            ^id
-          ),
-        where: fragment("(?)->>'type' = 'Announce'", activity.data)
-      )
-
-    Repo.one(query)
+  @spec get_existing_announce(String.t(), map()) :: Activity.t() | nil
+  def get_existing_announce(actor, %{data: %{"id" => ap_id}}) do
+    "Announce"
+    |> Activity.Queries.by_type()
+    |> where(actor: ^actor)
+    # this is to use the index
+    |> Activity.Queries.by_object_id(ap_id)
+    |> Repo.one()
  end

  @doc """
@ -478,7 +428,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
        activity_id,
        false
      ) do
-    data = %{
+    %{
      "type" => "Announce",
      "actor" => ap_id,
      "object" => id,
@ -486,8 +436,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
      "cc" => [],
      "context" => object.data["context"]
    }
-
-    if activity_id, do: Map.put(data, "id", activity_id), else: data
+    |> maybe_put("id", activity_id)
  end

  def make_announce_data(
@ -496,7 +445,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
        activity_id,
        true
      ) do
-    data = %{
+    %{
      "type" => "Announce",
      "actor" => ap_id,
      "object" => id,
@ -504,8 +453,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
      "cc" => [Pleroma.Constants.as_public()],
      "context" => object.data["context"]
    }
-
-    if activity_id, do: Map.put(data, "id", activity_id), else: data
+    |> maybe_put("id", activity_id)
  end

  @doc """
@ -516,7 +464,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
        %Activity{data: %{"context" => context}} = activity,
        activity_id
      ) do
-    data = %{
+    %{
      "type" => "Undo",
      "actor" => ap_id,
      "object" => activity.data,
@ -524,8 +472,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
      "cc" => [Pleroma.Constants.as_public()],
      "context" => context
    }
-
-    if activity_id, do: Map.put(data, "id", activity_id), else: data
+    |> maybe_put("id", activity_id)
  end

  def make_unlike_data(
@ -533,7 +480,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
        %Activity{data: %{"context" => context}} = activity,
        activity_id
      ) do
-    data = %{
+    %{
      "type" => "Undo",
      "actor" => ap_id,
      "object" => activity.data,
@ -541,94 +488,81 @@ defmodule Pleroma.Web.ActivityPub.Utils do
      "cc" => [Pleroma.Constants.as_public()],
      "context" => context
    }
-
-    if activity_id, do: Map.put(data, "id", activity_id), else: data
+    |> maybe_put("id", activity_id)
  end

+  @spec add_announce_to_object(Activity.t(), Object.t()) ::
+          {:ok, Object.t()} | {:error, Ecto.Changeset.t()}
  def add_announce_to_object(
-        %Activity{
-          data: %{"actor" => actor, "cc" => [Pleroma.Constants.as_public()]}
-        },
+        %Activity{data: %{"actor" => actor, "cc" => [Pleroma.Constants.as_public()]}},
        object
      ) do
-    announcements =
-      if is_list(object.data["announcements"]), do: object.data["announcements"], else: []
+    announcements = take_announcements(object)

-    with announcements <- [actor | announcements] |> Enum.uniq() do
+    with announcements <- Enum.uniq([actor | announcements]) do
      update_element_in_object("announcement", announcements, object)
    end
  end

  def add_announce_to_object(_, object), do: {:ok, object}

+  @spec remove_announce_from_object(Activity.t(), Object.t()) ::
+          {:ok, Object.t()} | {:error, Ecto.Changeset.t()}
  def remove_announce_from_object(%Activity{data: %{"actor" => actor}}, object) do
-    announcements =
-      if is_list(object.data["announcements"]), do: object.data["announcements"], else: []
-
-    with announcements <- announcements |> List.delete(actor) do
+    with announcements <- List.delete(take_announcements(object), actor) do
      update_element_in_object("announcement", announcements, object)
    end
  end

+  defp take_announcements(%{data: %{"announcements" => announcements}} = _)
+       when is_list(announcements),
+       do: announcements
+
+  defp take_announcements(_), do: []
+
  #### Unfollow-related helpers

  def make_unfollow_data(follower, followed, follow_activity, activity_id) do
-    data = %{
+    %{
      "type" => "Undo",
      "actor" => follower.ap_id,
      "to" => [followed.ap_id],
      "object" => follow_activity.data
    }
-
-    if activity_id, do: Map.put(data, "id", activity_id), else: data
+    |> maybe_put("id", activity_id)
  end

  #### Block-related helpers
+  @spec fetch_latest_block(User.t(), User.t()) :: Activity.t() | nil
  def fetch_latest_block(%User{ap_id: blocker_id}, %User{ap_id: blocked_id}) do
-    query =
-      from(
-        activity in Activity,
-        where:
-          fragment(
-            "? ->> 'type' = 'Block'",
-            activity.data
-          ),
-        where: activity.actor == ^blocker_id,
-        # this is to use the index
-        where:
-          fragment(
-            "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-            activity.data,
-            activity.data,
-            ^blocked_id
-          ),
-        order_by: [fragment("? desc nulls last", activity.id)],
-        limit: 1
-      )
-
-    Repo.one(query)
+    "Block"
+    |> Activity.Queries.by_type()
+    |> where(actor: ^blocker_id)
+    # this is to use the index
+    |> Activity.Queries.by_object_id(blocked_id)
+    |> order_by([activity], fragment("? desc nulls last", activity.id))
+    |> limit(1)
+    |> Repo.one()
  end

  def make_block_data(blocker, blocked, activity_id) do
-    data = %{
+    %{
      "type" => "Block",
      "actor" => blocker.ap_id,
      "to" => [blocked.ap_id],
      "object" => blocked.ap_id
    }
-
-    if activity_id, do: Map.put(data, "id", activity_id), else: data
+    |> maybe_put("id", activity_id)
  end

  def make_unblock_data(blocker, blocked, block_activity, activity_id) do
-    data = %{
+    %{
      "type" => "Undo",
      "actor" => blocker.ap_id,
      "to" => [blocked.ap_id],
      "object" => block_activity.data
    }
-
-    if activity_id, do: Map.put(data, "id", activity_id), else: data
+    |> maybe_put("id", activity_id)
  end

  #### Create-related helpers
@ -647,29 +581,48 @@ defmodule Pleroma.Web.ActivityPub.Utils do
    |> Map.merge(additional)
  end

-  #### Flag-related helpers
-
-  def make_flag_data(params, additional) do
-    status_ap_ids =
-      Enum.map(params.statuses || [], fn
-        %Activity{} = act -> act.data["id"]
-        act when is_map(act) -> act["id"]
-        act when is_binary(act) -> act
-      end)
-
-    object = [params.account.ap_id] ++ status_ap_ids
+  #### Listen-related helpers
+  def make_listen_data(params, additional) do
+    published = params.published || make_date()

    %{
-      "type" => "Flag",
+      "type" => "Listen",
+      "to" => params.to |> Enum.uniq(),
      "actor" => params.actor.ap_id,
-      "content" => params.content,
-      "object" => object,
-      "context" => params.context,
+      "object" => params.object,
+      "published" => published,
+      "context" => params.context
+    }
+    |> Map.merge(additional)
+  end
+
+  #### Flag-related helpers
+  @spec make_flag_data(map(), map()) :: map()
+  def make_flag_data(%{actor: actor, context: context, content: content} = params, additional) do
+    %{
+      "type" => "Flag",
+      "actor" => actor.ap_id,
+      "content" => content,
+      "object" => build_flag_object(params),
+      "context" => context,
      "state" => "open"
    }
    |> Map.merge(additional)
  end

+  def make_flag_data(_, _), do: %{}
+
+  defp build_flag_object(%{account: account, statuses: statuses} = _) do
+    [account.ap_id] ++
+      Enum.map(statuses || [], fn
+        %Activity{} = act -> act.data["id"]
+        act when is_map(act) -> act["id"]
+        act when is_binary(act) -> act
+      end)
+  end
+
+  defp build_flag_object(_), do: []
+
  @doc """
  Fetches the OrderedCollection/OrderedCollectionPage from `from`, limiting the amount of pages fetched after
  the first one to `pages_left` pages.
@ -712,11 +665,11 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  #### Report-related helpers

  def update_report_state(%Activity{} = activity, state) when state in @supported_report_states do
-    with new_data <- Map.put(activity.data, "state", state),
-         changeset <- Changeset.change(activity, data: new_data),
-         {:ok, activity} <- Repo.update(changeset) do
-      {:ok, activity}
-    end
+    new_data = Map.put(activity.data, "state", state)
+
+    activity
+    |> Changeset.change(data: new_data)
+    |> Repo.update()
  end

  def update_report_state(_, _), do: {:error, "Unsupported state"}
@ -783,20 +736,15 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  end

  def get_existing_votes(actor, %{data: %{"id" => id}}) do
-    query =
-      from(
-        [activity, object: object] in Activity.with_preloaded_object(Activity),
-        where: fragment("(?)->>'type' = 'Create'", activity.data),
-        where: fragment("(?)->>'actor' = ?", activity.data, ^actor),
-        where:
-          fragment(
-            "(?)->>'inReplyTo' = ?",
-            object.data,
-            ^to_string(id)
-          ),
-        where: fragment("(?)->>'type' = 'Answer'", object.data)
-      )
-
-    Repo.all(query)
+    actor
+    |> Activity.Queries.by_actor()
+    |> Activity.Queries.by_type("Create")
+    |> Activity.with_preloaded_object()
+    |> where([a, object: o], fragment("(?)->>'inReplyTo' = ?", o.data, ^to_string(id)))
+    |> where([a, object: o], fragment("(?)->>'type' = 'Answer'", o.data))
+    |> Repo.all()
  end
+
+  defp maybe_put(map, _key, nil), do: map
+  defp maybe_put(map, key, value), do: Map.put(map, key, value)
 end
--- a/lib/pleroma/web/activity_pub/views/object_view.ex
+++ b/lib/pleroma/web/activity_pub/views/object_view.ex
@ -15,7 +15,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectView do
    Map.merge(base, additional)
  end

-  def render("object.json", %{object: %Activity{data: %{"type" => "Create"}} = activity}) do
+  def render("object.json", %{object: %Activity{data: %{"type" => activity_type}} = activity})
+      when activity_type in ["Create", "Listen"] do
    base = Pleroma.Web.ActivityPub.Utils.make_json_ld_header()
    object = Object.normalize(activity)

@ -37,12 +38,12 @@ defmodule Pleroma.Web.ActivityPub.ObjectView do
    Map.merge(base, additional)
  end

-  def render("likes.json", ap_id, likes, page) do
+  def render("likes.json", %{ap_id: ap_id, likes: likes, page: page}) do
    collection(likes, "#{ap_id}/likes", page)
    |> Map.merge(Pleroma.Web.ActivityPub.Utils.make_json_ld_header())
  end

-  def render("likes.json", ap_id, likes) do
+  def render("likes.json", %{ap_id: ap_id, likes: likes}) do
    %{
      "id" => "#{ap_id}/likes",
      "type" => "OrderedCollection",
--- a/lib/pleroma/web/activity_pub/views/user_view.ex
+++ b/lib/pleroma/web/activity_pub/views/user_view.ex
@ -8,7 +8,6 @@ defmodule Pleroma.Web.ActivityPub.UserView do
  alias Pleroma.Keys
  alias Pleroma.Repo
  alias Pleroma.User
-  alias Pleroma.Web.ActivityPub.ActivityPub
  alias Pleroma.Web.ActivityPub.Transmogrifier
  alias Pleroma.Web.ActivityPub.Utils
  alias Pleroma.Web.Endpoint
@ -25,7 +24,8 @@ defmodule Pleroma.Web.ActivityPub.UserView do
      "oauthAuthorizationEndpoint" => Helpers.o_auth_url(Endpoint, :authorize),
      "oauthRegistrationEndpoint" => Helpers.mastodon_api_url(Endpoint, :create_app),
      "oauthTokenEndpoint" => Helpers.o_auth_url(Endpoint, :token_exchange),
-      "sharedInbox" => Helpers.activity_pub_url(Endpoint, :inbox)
+      "sharedInbox" => Helpers.activity_pub_url(Endpoint, :inbox),
+      "uploadMedia" => Helpers.activity_pub_url(Endpoint, :upload_media)
    }
  end

@ -75,10 +75,7 @@ defmodule Pleroma.Web.ActivityPub.UserView do

    endpoints = render("endpoints.json", %{user: user})

-    user_tags =
-      user
-      |> Transmogrifier.add_emoji_tags()
-      |> Map.get("tag", [])
+    emoji_tags = Transmogrifier.take_emoji_tags(user)

    fields =
      user.info
@ -110,7 +107,8 @@ defmodule Pleroma.Web.ActivityPub.UserView do
      },
      "endpoints" => endpoints,
      "attachment" => fields,
-      "tag" => (user.info.source_data["tag"] || []) ++ user_tags
+      "tag" => (user.info.source_data["tag"] || []) ++ emoji_tags,
+      "discoverable" => user.info.discoverable
    }
    |> Map.merge(maybe_make_image(&User.avatar_url/2, "icon", user))
    |> Map.merge(maybe_make_image(&User.banner_url/2, "image", user))
@ -118,30 +116,34 @@ defmodule Pleroma.Web.ActivityPub.UserView do
  end

  def render("following.json", %{user: user, page: page} = opts) do
-    showing = (opts[:for] && opts[:for] == user) || !user.info.hide_follows
+    showing_items = (opts[:for] && opts[:for] == user) || !user.info.hide_follows
+    showing_count = showing_items || !user.info.hide_follows_count
+
    query = User.get_friends_query(user)
    query = from(user in query, select: [:ap_id])
    following = Repo.all(query)

    total =
-      if showing do
+      if showing_count do
        length(following)
      else
        0
      end

-    collection(following, "#{user.ap_id}/following", page, showing, total)
+    collection(following, "#{user.ap_id}/following", page, showing_items, total)
    |> Map.merge(Utils.make_json_ld_header())
  end

  def render("following.json", %{user: user} = opts) do
-    showing = (opts[:for] && opts[:for] == user) || !user.info.hide_follows
+    showing_items = (opts[:for] && opts[:for] == user) || !user.info.hide_follows
+    showing_count = showing_items || !user.info.hide_follows_count
+
    query = User.get_friends_query(user)
    query = from(user in query, select: [:ap_id])
    following = Repo.all(query)

    total =
-      if showing do
+      if showing_count do
        length(following)
      else
        0
@ -152,7 +154,7 @@ defmodule Pleroma.Web.ActivityPub.UserView do
      "type" => "OrderedCollection",
      "totalItems" => total,
      "first" =>
-        if showing do
+        if showing_items do
          collection(following, "#{user.ap_id}/following", 1, !user.info.hide_follows)
        else
          "#{user.ap_id}/following?page=1"
@ -162,32 +164,34 @@ defmodule Pleroma.Web.ActivityPub.UserView do
  end

  def render("followers.json", %{user: user, page: page} = opts) do
-    showing = (opts[:for] && opts[:for] == user) || !user.info.hide_followers
+    showing_items = (opts[:for] && opts[:for] == user) || !user.info.hide_followers
+    showing_count = showing_items || !user.info.hide_followers_count

    query = User.get_followers_query(user)
    query = from(user in query, select: [:ap_id])
    followers = Repo.all(query)

    total =
-      if showing do
+      if showing_count do
        length(followers)
      else
        0
      end

-    collection(followers, "#{user.ap_id}/followers", page, showing, total)
+    collection(followers, "#{user.ap_id}/followers", page, showing_items, total)
    |> Map.merge(Utils.make_json_ld_header())
  end

  def render("followers.json", %{user: user} = opts) do
-    showing = (opts[:for] && opts[:for] == user) || !user.info.hide_followers
+    showing_items = (opts[:for] && opts[:for] == user) || !user.info.hide_followers
+    showing_count = showing_items || !user.info.hide_followers_count

    query = User.get_followers_query(user)
    query = from(user in query, select: [:ap_id])
    followers = Repo.all(query)

    total =
-      if showing do
+      if showing_count do
        length(followers)
      else
        0
@ -198,8 +202,8 @@ defmodule Pleroma.Web.ActivityPub.UserView do
      "type" => "OrderedCollection",
      "totalItems" => total,
      "first" =>
-        if showing do
-          collection(followers, "#{user.ap_id}/followers", 1, showing, total)
+        if showing_items do
+          collection(followers, "#{user.ap_id}/followers", 1, showing_items, total)
        else
          "#{user.ap_id}/followers?page=1"
        end
@ -207,25 +211,22 @@ defmodule Pleroma.Web.ActivityPub.UserView do
    |> Map.merge(Utils.make_json_ld_header())
  end

-  def render("outbox.json", %{user: user, max_id: max_qid}) do
-    params = %{
-      "limit" => "10"
+  def render("activity_collection.json", %{iri: iri}) do
+    %{
+      "id" => iri,
+      "type" => "OrderedCollection",
+      "first" => "#{iri}?page=true"
    }
+    |> Map.merge(Utils.make_json_ld_header())
+  end

-    params =
-      if max_qid != nil do
-        Map.put(params, "max_id", max_qid)
-      else
-        params
-      end
-
-    activities = ActivityPub.fetch_user_activities(user, nil, params)
-
+  def render("activity_collection_page.json", %{activities: activities, iri: iri}) do
+    # this is sorted chronologically, so first activity is the newest (max)
    {max_id, min_id, collection} =
      if length(activities) > 0 do
        {
-          Enum.at(Enum.reverse(activities), 0).id,
          Enum.at(activities, 0).id,
+          Enum.at(Enum.reverse(activities), 0).id,
          Enum.map(activities, fn act ->
            {:ok, data} = Transmogrifier.prepare_outgoing(act.data)
            data
@ -239,71 +240,14 @@ defmodule Pleroma.Web.ActivityPub.UserView do
        }
      end

-    iri = "#{user.ap_id}/outbox"
-
-    page = %{
-      "id" => "#{iri}?max_id=#{max_id}",
+    %{
+      "id" => "#{iri}?max_id=#{max_id}&page=true",
      "type" => "OrderedCollectionPage",
      "partOf" => iri,
      "orderedItems" => collection,
-      "next" => "#{iri}?max_id=#{min_id}"
+      "next" => "#{iri}?max_id=#{min_id}&page=true"
    }
-
-    if max_qid == nil do
-      %{
-        "id" => iri,
-        "type" => "OrderedCollection",
-        "first" => page
-      }
-      |> Map.merge(Utils.make_json_ld_header())
-    else
-      page |> Map.merge(Utils.make_json_ld_header())
-    end
-  end
-
-  def render("inbox.json", %{user: user, max_id: max_qid}) do
-    params = %{
-      "limit" => "10"
-    }
-
-    params =
-      if max_qid != nil do
-        Map.put(params, "max_id", max_qid)
-      else
-        params
-      end
-
-    activities = ActivityPub.fetch_activities([user.ap_id | user.following], params)
-
-    min_id = Enum.at(Enum.reverse(activities), 0).id
-    max_id = Enum.at(activities, 0).id
-
-    collection =
-      Enum.map(activities, fn act ->
-        {:ok, data} = Transmogrifier.prepare_outgoing(act.data)
-        data
-      end)
-
-    iri = "#{user.ap_id}/inbox"
-
-    page = %{
-      "id" => "#{iri}?max_id=#{max_id}",
-      "type" => "OrderedCollectionPage",
-      "partOf" => iri,
-      "orderedItems" => collection,
-      "next" => "#{iri}?max_id=#{min_id}"
-    }
-
-    if max_qid == nil do
-      %{
-        "id" => iri,
-        "type" => "OrderedCollection",
-        "first" => page
-      }
-      |> Map.merge(Utils.make_json_ld_header())
-    else
-      page |> Map.merge(Utils.make_json_ld_header())
-    end
+    |> Map.merge(Utils.make_json_ld_header())
  end

  def collection(collection, iri, page, show_items \\ true, total \\ nil) do
--- a/lib/pleroma/web/admin_api/admin_api_controller.ex
+++ b/lib/pleroma/web/admin_api/admin_api_controller.ex
@ -5,6 +5,7 @@
 defmodule Pleroma.Web.AdminAPI.AdminAPIController do
  use Pleroma.Web, :controller
  alias Pleroma.Activity
+  alias Pleroma.ModerationLog
  alias Pleroma.User
  alias Pleroma.UserInviteToken
  alias Pleroma.Web.ActivityPub.ActivityPub
@ -12,10 +13,14 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
  alias Pleroma.Web.AdminAPI.AccountView
  alias Pleroma.Web.AdminAPI.Config
  alias Pleroma.Web.AdminAPI.ConfigView
+  alias Pleroma.Web.AdminAPI.ModerationLogView
+  alias Pleroma.Web.AdminAPI.Report
  alias Pleroma.Web.AdminAPI.ReportView
  alias Pleroma.Web.AdminAPI.Search
  alias Pleroma.Web.CommonAPI
+  alias Pleroma.Web.Endpoint
  alias Pleroma.Web.MastodonAPI.StatusView
+  alias Pleroma.Web.Router

  import Pleroma.Web.ControllerHelper, only: [json_response: 3]

@ -25,58 +30,120 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do

  action_fallback(:errors)

-  def user_delete(conn, %{"nickname" => nickname}) do
-    User.get_cached_by_nickname(nickname)
-    |> User.delete()
+  def user_delete(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
+    user = User.get_cached_by_nickname(nickname)
+    User.delete(user)
+
+    ModerationLog.insert_log(%{
+      actor: admin,
+      subject: user,
+      action: "delete"
+    })

    conn
    |> json(nickname)
  end

-  def user_follow(conn, %{"follower" => follower_nick, "followed" => followed_nick}) do
+  def user_follow(%{assigns: %{user: admin}} = conn, %{
+        "follower" => follower_nick,
+        "followed" => followed_nick
+      }) do
    with %User{} = follower <- User.get_cached_by_nickname(follower_nick),
         %User{} = followed <- User.get_cached_by_nickname(followed_nick) do
      User.follow(follower, followed)
+
+      ModerationLog.insert_log(%{
+        actor: admin,
+        followed: followed,
+        follower: follower,
+        action: "follow"
+      })
    end

    conn
    |> json("ok")
  end

-  def user_unfollow(conn, %{"follower" => follower_nick, "followed" => followed_nick}) do
+  def user_unfollow(%{assigns: %{user: admin}} = conn, %{
+        "follower" => follower_nick,
+        "followed" => followed_nick
+      }) do
    with %User{} = follower <- User.get_cached_by_nickname(follower_nick),
         %User{} = followed <- User.get_cached_by_nickname(followed_nick) do
      User.unfollow(follower, followed)
+
+      ModerationLog.insert_log(%{
+        actor: admin,
+        followed: followed,
+        follower: follower,
+        action: "unfollow"
+      })
    end

    conn
    |> json("ok")
  end

-  def user_create(
-        conn,
-        %{"nickname" => nickname, "email" => email, "password" => password}
-      ) do
-    user_data = %{
-      nickname: nickname,
-      name: nickname,
-      email: email,
-      password: password,
-      password_confirmation: password,
-      bio: "."
-    }
+  def users_create(%{assigns: %{user: admin}} = conn, %{"users" => users}) do
+    changesets =
+      Enum.map(users, fn %{"nickname" => nickname, "email" => email, "password" => password} ->
+        user_data = %{
+          nickname: nickname,
+          name: nickname,
+          email: email,
+          password: password,
+          password_confirmation: password,
+          bio: "."
+        }

-    changeset = User.register_changeset(%User{}, user_data, need_confirmation: false)
-    {:ok, user} = User.register(changeset)
+        User.register_changeset(%User{}, user_data, need_confirmation: false)
+      end)
+      |> Enum.reduce(Ecto.Multi.new(), fn changeset, multi ->
+        Ecto.Multi.insert(multi, Ecto.UUID.generate(), changeset)
+      end)

-    conn
-    |> json(user.nickname)
+    case Pleroma.Repo.transaction(changesets) do
+      {:ok, users} ->
+        res =
+          users
+          |> Map.values()
+          |> Enum.map(fn user ->
+            {:ok, user} = User.post_register_action(user)
+
+            user
+          end)
+          |> Enum.map(&AccountView.render("created.json", %{user: &1}))
+
+        ModerationLog.insert_log(%{
+          actor: admin,
+          subjects: Map.values(users),
+          action: "create"
+        })
+
+        conn
+        |> json(res)
+
+      {:error, id, changeset, _} ->
+        res =
+          Enum.map(changesets.operations, fn
+            {current_id, {:changeset, _current_changeset, _}} when current_id == id ->
+              AccountView.render("create-error.json", %{changeset: changeset})
+
+            {_, {:changeset, current_changeset, _}} ->
+              AccountView.render("create-error.json", %{changeset: current_changeset})
+          end)
+
+        conn
+        |> put_status(:conflict)
+        |> json(res)
+    end
  end

  def user_show(conn, %{"nickname" => nickname}) do
    with %User{} = user <- User.get_cached_by_nickname_or_id(nickname) do
      conn
-      |> json(AccountView.render("show.json", %{user: user}))
+      |> put_view(AccountView)
+      |> render("show.json", %{user: user})
    else
      _ -> {:error, :not_found}
    end
@ -95,29 +162,55 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
        })

      conn
-      |> json(StatusView.render("index.json", %{activities: activities, as: :activity}))
+      |> put_view(StatusView)
+      |> render("index.json", %{activities: activities, as: :activity})
    else
      _ -> {:error, :not_found}
    end
  end

-  def user_toggle_activation(conn, %{"nickname" => nickname}) do
+  def user_toggle_activation(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
    user = User.get_cached_by_nickname(nickname)

    {:ok, updated_user} = User.deactivate(user, !user.info.deactivated)

+    action = if user.info.deactivated, do: "activate", else: "deactivate"
+
+    ModerationLog.insert_log(%{
+      actor: admin,
+      subject: user,
+      action: action
+    })
+
    conn
-    |> json(AccountView.render("show.json", %{user: updated_user}))
+    |> put_view(AccountView)
+    |> render("show.json", %{user: updated_user})
  end

-  def tag_users(conn, %{"nicknames" => nicknames, "tags" => tags}) do
-    with {:ok, _} <- User.tag(nicknames, tags),
-         do: json_response(conn, :no_content, "")
+  def tag_users(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames, "tags" => tags}) do
+    with {:ok, _} <- User.tag(nicknames, tags) do
+      ModerationLog.insert_log(%{
+        actor: admin,
+        nicknames: nicknames,
+        tags: tags,
+        action: "tag"
+      })
+
+      json_response(conn, :no_content, "")
+    end
  end

-  def untag_users(conn, %{"nicknames" => nicknames, "tags" => tags}) do
-    with {:ok, _} <- User.untag(nicknames, tags),
-         do: json_response(conn, :no_content, "")
+  def untag_users(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames, "tags" => tags}) do
+    with {:ok, _} <- User.untag(nicknames, tags) do
+      ModerationLog.insert_log(%{
+        actor: admin,
+        nicknames: nicknames,
+        tags: tags,
+        action: "untag"
+      })
+
+      json_response(conn, :no_content, "")
+    end
  end

  def list_users(conn, params) do
@ -158,22 +251,24 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    |> Enum.into(%{}, &{&1, true})
  end

-  def right_add(conn, %{"permission_group" => permission_group, "nickname" => nickname})
+  def right_add(%{assigns: %{user: admin}} = conn, %{
+        "permission_group" => permission_group,
+        "nickname" => nickname
+      })
      when permission_group in ["moderator", "admin"] do
-    user = User.get_cached_by_nickname(nickname)
+    info = Map.put(%{}, "is_" <> permission_group, true)

-    info =
-      %{}
-      |> Map.put("is_" <> permission_group, true)
+    {:ok, user} =
+      nickname
+      |> User.get_cached_by_nickname()
+      |> User.update_info(&User.Info.admin_api_update(&1, info))

-    info_cng = User.Info.admin_api_update(user.info, info)
-
-    cng =
-      user
-      |> Ecto.Changeset.change()
-      |> Ecto.Changeset.put_embed(:info, info_cng)
-
-    {:ok, _user} = User.update_and_set_cache(cng)
+    ModerationLog.insert_log(%{
+      action: "grant",
+      actor: admin,
+      subject: user,
+      permission: permission_group
+    })

    json(conn, info)
  end
@ -192,48 +287,66 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    })
  end

+  def right_delete(%{assigns: %{user: %{nickname: nickname}}} = conn, %{"nickname" => nickname}) do
+    render_error(conn, :forbidden, "You can't revoke your own admin status.")
+  end
+
  def right_delete(
-        %{assigns: %{user: %User{:nickname => admin_nickname}}} = conn,
+        %{assigns: %{user: admin}} = conn,
        %{
          "permission_group" => permission_group,
          "nickname" => nickname
        }
      )
      when permission_group in ["moderator", "admin"] do
-    if admin_nickname == nickname do
-      render_error(conn, :forbidden, "You can't revoke your own admin status.")
-    else
-      user = User.get_cached_by_nickname(nickname)
+    info = Map.put(%{}, "is_" <> permission_group, false)

-      info =
-        %{}
-        |> Map.put("is_" <> permission_group, false)
+    {:ok, user} =
+      nickname
+      |> User.get_cached_by_nickname()
+      |> User.update_info(&User.Info.admin_api_update(&1, info))

-      info_cng = User.Info.admin_api_update(user.info, info)
+    ModerationLog.insert_log(%{
+      action: "revoke",
+      actor: admin,
+      subject: user,
+      permission: permission_group
+    })

-      cng =
-        Ecto.Changeset.change(user)
-        |> Ecto.Changeset.put_embed(:info, info_cng)
-
-      {:ok, _user} = User.update_and_set_cache(cng)
-
-      json(conn, info)
-    end
+    json(conn, info)
  end

  def right_delete(conn, _) do
    render_error(conn, :not_found, "No such permission_group")
  end

-  def set_activation_status(conn, %{"nickname" => nickname, "status" => status}) do
+  def set_activation_status(%{assigns: %{user: admin}} = conn, %{
+        "nickname" => nickname,
+        "status" => status
+      }) do
    with {:ok, status} <- Ecto.Type.cast(:boolean, status),
         %User{} = user <- User.get_cached_by_nickname(nickname),
-         {:ok, _} <- User.deactivate(user, !status),
-         do: json_response(conn, :no_content, "")
+         {:ok, _} <- User.deactivate(user, !status) do
+      action = if(user.info.deactivated, do: "activate", else: "deactivate")
+
+      ModerationLog.insert_log(%{
+        actor: admin,
+        subject: user,
+        action: action
+      })
+
+      json_response(conn, :no_content, "")
+    end
  end

-  def relay_follow(conn, %{"relay_url" => target}) do
+  def relay_follow(%{assigns: %{user: admin}} = conn, %{"relay_url" => target}) do
    with {:ok, _message} <- Relay.follow(target) do
+      ModerationLog.insert_log(%{
+        action: "relay_follow",
+        actor: admin,
+        target: target
+      })
+
      json(conn, target)
    else
      _ ->
@ -243,8 +356,14 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    end
  end

-  def relay_unfollow(conn, %{"relay_url" => target}) do
+  def relay_unfollow(%{assigns: %{user: admin}} = conn, %{"relay_url" => target}) do
    with {:ok, _message} <- Relay.unfollow(target) do
+      ModerationLog.insert_log(%{
+        action: "relay_unfollow",
+        actor: admin,
+        target: target
+      })
+
      json(conn, target)
    else
      _ ->
@ -272,13 +391,23 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    end
  end

-  @doc "Get a account registeration invite token (base64 string)"
-  def get_invite_token(conn, params) do
-    options = params["invite"] || %{}
-    {:ok, invite} = UserInviteToken.create_invite(options)
+  @doc "Create an account registration invite token"
+  def create_invite_token(conn, params) do
+    opts = %{}

-    conn
-    |> json(invite.token)
+    opts =
+      if params["max_use"],
+        do: Map.put(opts, :max_use, params["max_use"]),
+        else: opts
+
+    opts =
+      if params["expires_at"],
+        do: Map.put(opts, :expires_at, params["expires_at"]),
+        else: opts
+
+    {:ok, invite} = UserInviteToken.create_invite(opts)
+
+    json(conn, AccountView.render("invite.json", %{invite: invite}))
  end

  @doc "Get list of created invites"
@ -286,7 +415,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    invites = UserInviteToken.list_invites()

    conn
-    |> json(AccountView.render("invites.json", %{invites: invites}))
+    |> put_view(AccountView)
+    |> render("invites.json", %{invites: invites})
  end

  @doc "Revokes invite by token"
@ -294,7 +424,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    with {:ok, invite} <- UserInviteToken.find_by_token(token),
         {:ok, updated_invite} = UserInviteToken.update_invite(invite, %{used: true}) do
      conn
-      |> json(AccountView.render("invite.json", %{invite: updated_invite}))
+      |> put_view(AccountView)
+      |> render("invite.json", %{invite: updated_invite})
    else
      nil -> {:error, :not_found}
    end
@ -306,19 +437,33 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    {:ok, token} = Pleroma.PasswordResetToken.create_token(user)

    conn
-    |> json(token.token)
+    |> json(%{
+      token: token.token,
+      link: Router.Helpers.reset_password_url(Endpoint, :reset, token.token)
+    })
+  end
+
+  @doc "Force password reset for a given user"
+  def force_password_reset(conn, %{"nickname" => nickname}) do
+    (%User{local: true} = user) = User.get_cached_by_nickname(nickname)
+
+    User.force_password_reset_async(user)
+
+    json_response(conn, :no_content, "")
  end

  def list_reports(conn, params) do
+    {page, page_size} = page_params(params)
+
    params =
      params
      |> Map.put("type", "Flag")
      |> Map.put("skip_preload", true)
+      |> Map.put("total", true)
+      |> Map.put("limit", page_size)
+      |> Map.put("offset", (page - 1) * page_size)

-    reports =
-      []
-      |> ActivityPub.fetch_activities(params)
-      |> Enum.reverse()
+    reports = ActivityPub.fetch_activities([], params, :offset)

    conn
    |> put_view(ReportView)
@ -329,17 +474,23 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    with %Activity{} = report <- Activity.get_by_id(id) do
      conn
      |> put_view(ReportView)
-      |> render("show.json", %{report: report})
+      |> render("show.json", Report.extract_report_info(report))
    else
      _ -> {:error, :not_found}
    end
  end

-  def report_update_state(conn, %{"id" => id, "state" => state}) do
+  def report_update_state(%{assigns: %{user: admin}} = conn, %{"id" => id, "state" => state}) do
    with {:ok, report} <- CommonAPI.update_report_state(id, state) do
+      ModerationLog.insert_log(%{
+        action: "report_update",
+        actor: admin,
+        subject: report
+      })
+
      conn
      |> put_view(ReportView)
-      |> render("show.json", %{report: report})
+      |> render("show.json", Report.extract_report_info(report))
    end
  end

@ -353,9 +504,16 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do

      {:ok, activity} = CommonAPI.post(user, params)

+      ModerationLog.insert_log(%{
+        action: "report_response",
+        actor: user,
+        subject: activity,
+        text: params["status"]
+      })
+
      conn
      |> put_view(StatusView)
-      |> render("status.json", %{activity: activity})
+      |> render("show.json", %{activity: activity})
    else
      true ->
        {:param_cast, nil}
@ -365,20 +523,54 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    end
  end

-  def status_update(conn, %{"id" => id} = params) do
+  def status_update(%{assigns: %{user: admin}} = conn, %{"id" => id} = params) do
    with {:ok, activity} <- CommonAPI.update_activity_scope(id, params) do
+      {:ok, sensitive} = Ecto.Type.cast(:boolean, params["sensitive"])
+
+      ModerationLog.insert_log(%{
+        action: "status_update",
+        actor: admin,
+        subject: activity,
+        sensitive: sensitive,
+        visibility: params["visibility"]
+      })
+
      conn
      |> put_view(StatusView)
-      |> render("status.json", %{activity: activity})
+      |> render("show.json", %{activity: activity})
    end
  end

  def status_delete(%{assigns: %{user: user}} = conn, %{"id" => id}) do
    with {:ok, %Activity{}} <- CommonAPI.delete(id, user) do
+      ModerationLog.insert_log(%{
+        action: "status_delete",
+        actor: user,
+        subject_id: id
+      })
+
      json(conn, %{})
    end
  end

+  def list_log(conn, params) do
+    {page, page_size} = page_params(params)
+
+    log =
+      ModerationLog.get_all(%{
+        page: page,
+        page_size: page_size,
+        start_date: params["start_date"],
+        end_date: params["end_date"],
+        user_id: params["user_id"],
+        search: params["search"]
+      })
+
+    conn
+    |> put_view(ModerationLogView)
+    |> render("index.json", %{log: log})
+  end
+
  def migrate_to_db(conn, _params) do
    Mix.Tasks.Pleroma.Config.run(["migrate_to_db"])
    json(conn, %{})
@ -424,6 +616,12 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    |> render("index.json", %{configs: updated})
  end

+  def reload_emoji(conn, _params) do
+    Pleroma.Emoji.reload()
+
+    conn |> json("ok")
+  end
+
  def errors(conn, {:error, :not_found}) do
    conn
    |> put_status(:not_found)
--- a/lib/pleroma/web/admin_api/config.ex
+++ b/lib/pleroma/web/admin_api/config.ex
@ -90,6 +90,8 @@ defmodule Pleroma.Web.AdminAPI.Config do
    for v <- entity, into: [], do: do_convert(v)
  end

+  defp do_convert(%Regex{} = entity), do: inspect(entity)
+
  defp do_convert(entity) when is_map(entity) do
    for {k, v} <- entity, into: %{}, do: {do_convert(k), do_convert(v)}
  end
@ -122,7 +124,7 @@ defmodule Pleroma.Web.AdminAPI.Config do

  def transform(entity), do: :erlang.term_to_binary(entity)

-  defp do_transform(%Regex{} = entity) when is_map(entity), do: entity
+  defp do_transform(%Regex{} = entity), do: entity

  defp do_transform(%{"tuple" => [":dispatch", [entity]]}) do
    {dispatch_settings, []} = do_eval(entity)
@ -154,8 +156,15 @@ defmodule Pleroma.Web.AdminAPI.Config do
  defp do_transform(entity), do: entity

  defp do_transform_string("~r/" <> pattern) do
-    pattern = String.trim_trailing(pattern, "/")
-    ~r/#{pattern}/
+    modificator = String.split(pattern, "/") |> List.last()
+    pattern = String.trim_trailing(pattern, "/" <> modificator)
+
+    case modificator do
+      "" -> ~r/#{pattern}/
+      "i" -> ~r/#{pattern}/i
+      "u" -> ~r/#{pattern}/u
+      "s" -> ~r/#{pattern}/s
+    end
  end

  defp do_transform_string(":" <> atom), do: String.to_atom(atom)
--- a/lib/pleroma/web/admin_api/report.ex
+++ b/lib/pleroma/web/admin_api/report.ex
@ -0,0 +1,22 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.Report do
+  alias Pleroma.Activity
+  alias Pleroma.User
+
+  def extract_report_info(
+        %{data: %{"actor" => actor, "object" => [account_ap_id | status_ap_ids]}} = report
+      ) do
+    user = User.get_cached_by_ap_id(actor)
+    account = User.get_cached_by_ap_id(account_ap_id)
+
+    statuses =
+      Enum.map(status_ap_ids, fn ap_id ->
+        Activity.get_by_ap_id_with_object(ap_id)
+      end)
+
+    %{report: report, user: user, account: account, statuses: statuses}
+  end
+end
--- a/lib/pleroma/web/admin_api/views/account_view.ex
+++ b/lib/pleroma/web/admin_api/views/account_view.ex
@ -52,4 +52,50 @@ defmodule Pleroma.Web.AdminAPI.AccountView do
      invites: render_many(invites, AccountView, "invite.json", as: :invite)
    }
  end
+
+  def render("created.json", %{user: user}) do
+    %{
+      type: "success",
+      code: 200,
+      data: %{
+        nickname: user.nickname,
+        email: user.email
+      }
+    }
+  end
+
+  def render("create-error.json", %{changeset: %Ecto.Changeset{changes: changes, errors: errors}}) do
+    %{
+      type: "error",
+      code: 409,
+      error: parse_error(errors),
+      data: %{
+        nickname: Map.get(changes, :nickname),
+        email: Map.get(changes, :email)
+      }
+    }
+  end
+
+  defp parse_error([]), do: ""
+
+  defp parse_error(errors) do
+    ## when nickname is duplicate ap_id constraint error is raised
+    nickname_error = Keyword.get(errors, :nickname) || Keyword.get(errors, :ap_id)
+    email_error = Keyword.get(errors, :email)
+    password_error = Keyword.get(errors, :password)
+
+    cond do
+      nickname_error ->
+        "nickname #{elem(nickname_error, 0)}"
+
+      email_error ->
+        "email #{elem(email_error, 0)}"
+
+      password_error ->
+        "password #{elem(password_error, 0)}"
+
+      true ->
+        ""
+    end
+  end
 end
--- a/lib/pleroma/web/admin_api/views/moderation_log_view.ex
+++ b/lib/pleroma/web/admin_api/views/moderation_log_view.ex
@ -0,0 +1,29 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.ModerationLogView do
+  use Pleroma.Web, :view
+
+  alias Pleroma.ModerationLog
+
+  def render("index.json", %{log: log}) do
+    %{
+      items: render_many(log.items, __MODULE__, "show.json", as: :log_entry),
+      total: log.count
+    }
+  end
+
+  def render("show.json", %{log_entry: log_entry}) do
+    time =
+      log_entry.inserted_at
+      |> DateTime.from_naive!("Etc/UTC")
+      |> DateTime.to_unix()
+
+    %{
+      data: log_entry.data,
+      time: time,
+      message: ModerationLog.get_log_entry_message(log_entry)
+    }
+  end
+end
--- a/lib/pleroma/web/admin_api/views/report_view.ex
+++ b/lib/pleroma/web/admin_api/views/report_view.ex
@ -4,25 +4,26 @@

 defmodule Pleroma.Web.AdminAPI.ReportView do
  use Pleroma.Web, :view
-  alias Pleroma.Activity
  alias Pleroma.HTML
  alias Pleroma.User
+  alias Pleroma.Web.AdminAPI.Report
  alias Pleroma.Web.CommonAPI.Utils
  alias Pleroma.Web.MastodonAPI.StatusView

  def render("index.json", %{reports: reports}) do
    %{
-      reports: render_many(reports, __MODULE__, "show.json", as: :report)
+      reports:
+        reports[:items]
+        |> Enum.map(&Report.extract_report_info(&1))
+        |> Enum.map(&render(__MODULE__, "show.json", &1))
+        |> Enum.reverse(),
+      total: reports[:total]
    }
  end

-  def render("show.json", %{report: report}) do
-    user = User.get_cached_by_ap_id(report.data["actor"])
+  def render("show.json", %{report: report, user: user, account: account, statuses: statuses}) do
    created_at = Utils.to_masto_date(report.data["published"])

-    [account_ap_id | status_ap_ids] = report.data["object"]
-    account = User.get_cached_by_ap_id(account_ap_id)
-
    content =
      unless is_nil(report.data["content"]) do
        HTML.filter_tags(report.data["content"])
@ -30,11 +31,6 @@ defmodule Pleroma.Web.AdminAPI.ReportView do
        nil
      end

-    statuses =
-      Enum.map(status_ap_ids, fn ap_id ->
-        Activity.get_by_ap_id_with_object(ap_id)
-      end)
-
    %{
      id: report.id,
      account: merge_account_views(account),
@ -47,7 +43,7 @@ defmodule Pleroma.Web.AdminAPI.ReportView do
  end

  defp merge_account_views(%User{} = user) do
-    Pleroma.Web.MastodonAPI.AccountView.render("account.json", %{user: user})
+    Pleroma.Web.MastodonAPI.AccountView.render("show.json", %{user: user})
    |> Map.merge(Pleroma.Web.AdminAPI.AccountView.render("show.json", %{user: user}))
  end

--- a/lib/pleroma/web/chat_channel.ex
+++ b/lib/pleroma/web/chat_channel.ex
@ -22,7 +22,7 @@ defmodule Pleroma.Web.ChatChannel do

    if String.length(text) > 0 do
      author = User.get_cached_by_nickname(user_name)
-      author = Pleroma.Web.MastodonAPI.AccountView.render("account.json", user: author)
+      author = Pleroma.Web.MastodonAPI.AccountView.render("show.json", user: author)
      message = ChatChannelState.add_message(%{text: text, author: author})

      broadcast!(socket, "new_msg", message)
--- a/lib/pleroma/web/common_api/activity_draft.ex
+++ b/lib/pleroma/web/common_api/activity_draft.ex
@ -0,0 +1,219 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.CommonAPI.ActivityDraft do
+  alias Pleroma.Activity
+  alias Pleroma.Conversation.Participation
+  alias Pleroma.Web.CommonAPI
+  alias Pleroma.Web.CommonAPI.Utils
+
+  import Pleroma.Web.Gettext
+
+  defstruct valid?: true,
+            errors: [],
+            user: nil,
+            params: %{},
+            status: nil,
+            summary: nil,
+            full_payload: nil,
+            attachments: [],
+            in_reply_to: nil,
+            in_reply_to_conversation: nil,
+            visibility: nil,
+            expires_at: nil,
+            poll: nil,
+            emoji: %{},
+            content_html: nil,
+            mentions: [],
+            tags: [],
+            to: [],
+            cc: [],
+            context: nil,
+            sensitive: false,
+            object: nil,
+            preview?: false,
+            changes: %{}
+
+  def create(user, params) do
+    %__MODULE__{user: user}
+    |> put_params(params)
+    |> status()
+    |> summary()
+    |> with_valid(&attachments/1)
+    |> full_payload()
+    |> expires_at()
+    |> poll()
+    |> with_valid(&in_reply_to/1)
+    |> with_valid(&in_reply_to_conversation/1)
+    |> with_valid(&visibility/1)
+    |> content()
+    |> with_valid(&to_and_cc/1)
+    |> with_valid(&context/1)
+    |> sensitive()
+    |> with_valid(&object/1)
+    |> preview?()
+    |> with_valid(&changes/1)
+    |> validate()
+  end
+
+  defp put_params(draft, params) do
+    params = Map.put_new(params, "in_reply_to_status_id", params["in_reply_to_id"])
+    %__MODULE__{draft | params: params}
+  end
+
+  defp status(%{params: %{"status" => status}} = draft) do
+    %__MODULE__{draft | status: String.trim(status)}
+  end
+
+  defp summary(%{params: params} = draft) do
+    %__MODULE__{draft | summary: Map.get(params, "spoiler_text", "")}
+  end
+
+  defp full_payload(%{status: status, summary: summary} = draft) do
+    full_payload = String.trim(status <> summary)
+
+    case Utils.validate_character_limit(full_payload, draft.attachments) do
+      :ok -> %__MODULE__{draft | full_payload: full_payload}
+      {:error, message} -> add_error(draft, message)
+    end
+  end
+
+  defp attachments(%{params: params} = draft) do
+    attachments = Utils.attachments_from_ids(params)
+    %__MODULE__{draft | attachments: attachments}
+  end
+
+  defp in_reply_to(draft) do
+    case Map.get(draft.params, "in_reply_to_status_id") do
+      "" -> draft
+      nil -> draft
+      id -> %__MODULE__{draft | in_reply_to: Activity.get_by_id(id)}
+    end
+  end
+
+  defp in_reply_to_conversation(draft) do
+    in_reply_to_conversation = Participation.get(draft.params["in_reply_to_conversation_id"])
+    %__MODULE__{draft | in_reply_to_conversation: in_reply_to_conversation}
+  end
+
+  defp visibility(%{params: params} = draft) do
+    case CommonAPI.get_visibility(params, draft.in_reply_to, draft.in_reply_to_conversation) do
+      {visibility, "direct"} when visibility != "direct" ->
+        add_error(draft, dgettext("errors", "The message visibility must be direct"))
+
+      {visibility, _} ->
+        %__MODULE__{draft | visibility: visibility}
+    end
+  end
+
+  defp expires_at(draft) do
+    case CommonAPI.check_expiry_date(draft.params["expires_in"]) do
+      {:ok, expires_at} -> %__MODULE__{draft | expires_at: expires_at}
+      {:error, message} -> add_error(draft, message)
+    end
+  end
+
+  defp poll(draft) do
+    case Utils.make_poll_data(draft.params) do
+      {:ok, {poll, poll_emoji}} ->
+        %__MODULE__{draft | poll: poll, emoji: Map.merge(draft.emoji, poll_emoji)}
+
+      {:error, message} ->
+        add_error(draft, message)
+    end
+  end
+
+  defp content(draft) do
+    {content_html, mentions, tags} =
+      Utils.make_content_html(
+        draft.status,
+        draft.attachments,
+        draft.params,
+        draft.visibility
+      )
+
+    %__MODULE__{draft | content_html: content_html, mentions: mentions, tags: tags}
+  end
+
+  defp to_and_cc(draft) do
+    addressed_users =
+      draft.mentions
+      |> Enum.map(fn {_, mentioned_user} -> mentioned_user.ap_id end)
+      |> Utils.get_addressed_users(draft.params["to"])
+
+    {to, cc} =
+      Utils.get_to_and_cc(
+        draft.user,
+        addressed_users,
+        draft.in_reply_to,
+        draft.visibility,
+        draft.in_reply_to_conversation
+      )
+
+    %__MODULE__{draft | to: to, cc: cc}
+  end
+
+  defp context(draft) do
+    context = Utils.make_context(draft.in_reply_to, draft.in_reply_to_conversation)
+    %__MODULE__{draft | context: context}
+  end
+
+  defp sensitive(draft) do
+    sensitive = draft.params["sensitive"] || Enum.member?(draft.tags, {"#nsfw", "nsfw"})
+    %__MODULE__{draft | sensitive: sensitive}
+  end
+
+  defp object(draft) do
+    emoji = Map.merge(Pleroma.Emoji.Formatter.get_emoji_map(draft.full_payload), draft.emoji)
+
+    object =
+      Utils.make_note_data(
+        draft.user.ap_id,
+        draft.to,
+        draft.context,
+        draft.content_html,
+        draft.attachments,
+        draft.in_reply_to,
+        draft.tags,
+        draft.summary,
+        draft.cc,
+        draft.sensitive,
+        draft.poll
+      )
+      |> Map.put("emoji", emoji)
+
+    %__MODULE__{draft | object: object}
+  end
+
+  defp preview?(draft) do
+    preview? = Pleroma.Web.ControllerHelper.truthy_param?(draft.params["preview"]) || false
+    %__MODULE__{draft | preview?: preview?}
+  end
+
+  defp changes(draft) do
+    direct? = draft.visibility == "direct"
+
+    changes =
+      %{
+        to: draft.to,
+        actor: draft.user,
+        context: draft.context,
+        object: draft.object,
+        additional: %{"cc" => draft.cc, "directMessage" => direct?}
+      }
+      |> Utils.maybe_add_list_data(draft.user, draft.visibility)
+
+    %__MODULE__{draft | changes: changes}
+  end
+
+  defp with_valid(%{valid?: true} = draft, func), do: func.(draft)
+  defp with_valid(draft, _func), do: draft
+
+  defp add_error(draft, message) do
+    %__MODULE__{draft | valid?: false, errors: [message | draft.errors]}
+  end
+
+  defp validate(%{valid?: true} = draft), do: {:ok, draft}
+  defp validate(%{errors: [message | _]}), do: {:error, message}
+end
--- a/lib/pleroma/web/common_api/common_api.ex
+++ b/lib/pleroma/web/common_api/common_api.ex
@ -4,8 +4,8 @@

 defmodule Pleroma.Web.CommonAPI do
  alias Pleroma.Activity
+  alias Pleroma.ActivityExpiration
  alias Pleroma.Conversation.Participation
-  alias Pleroma.Formatter
  alias Pleroma.Object
  alias Pleroma.ThreadMute
  alias Pleroma.User
@ -17,14 +17,11 @@ defmodule Pleroma.Web.CommonAPI do
  import Pleroma.Web.CommonAPI.Utils

  def follow(follower, followed) do
+    timeout = Pleroma.Config.get([:activitypub, :follow_handshake_timeout])
+
    with {:ok, follower} <- User.maybe_direct_follow(follower, followed),
         {:ok, activity} <- ActivityPub.follow(follower, followed),
-         {:ok, follower, followed} <-
-           User.wait_and_refresh(
-             Pleroma.Config.get([:activitypub, :follow_handshake_timeout]),
-             follower,
-             followed
-           ) do
+         {:ok, follower, followed} <- User.wait_and_refresh(timeout, follower, followed) do
      {:ok, follower, followed, activity}
    end
  end
@ -75,8 +72,7 @@ defmodule Pleroma.Web.CommonAPI do
         {:ok, delete} <- ActivityPub.delete(object) do
      {:ok, delete}
    else
-      _ ->
-        {:error, dgettext("errors", "Could not delete")}
+      _ -> {:error, dgettext("errors", "Could not delete")}
    end
  end

@ -86,18 +82,16 @@ defmodule Pleroma.Web.CommonAPI do
         nil <- Utils.get_existing_announce(user.ap_id, object) do
      ActivityPub.announce(user, object)
    else
-      _ ->
-        {:error, dgettext("errors", "Could not repeat")}
+      _ -> {:error, dgettext("errors", "Could not repeat")}
    end
  end

  def unrepeat(id_or_ap_id, user) do
-    with %Activity{} = activity <- get_by_id_or_ap_id(id_or_ap_id),
-         object <- Object.normalize(activity) do
+    with %Activity{} = activity <- get_by_id_or_ap_id(id_or_ap_id) do
+      object = Object.normalize(activity)
      ActivityPub.unannounce(user, object)
    else
-      _ ->
-        {:error, dgettext("errors", "Could not unrepeat")}
+      _ -> {:error, dgettext("errors", "Could not unrepeat")}
    end
  end

@ -107,30 +101,23 @@ defmodule Pleroma.Web.CommonAPI do
         nil <- Utils.get_existing_like(user.ap_id, object) do
      ActivityPub.like(user, object)
    else
-      _ ->
-        {:error, dgettext("errors", "Could not favorite")}
+      _ -> {:error, dgettext("errors", "Could not favorite")}
    end
  end

  def unfavorite(id_or_ap_id, user) do
-    with %Activity{} = activity <- get_by_id_or_ap_id(id_or_ap_id),
-         object <- Object.normalize(activity) do
+    with %Activity{} = activity <- get_by_id_or_ap_id(id_or_ap_id) do
+      object = Object.normalize(activity)
      ActivityPub.unlike(user, object)
    else
-      _ ->
-        {:error, dgettext("errors", "Could not unfavorite")}
+      _ -> {:error, dgettext("errors", "Could not unfavorite")}
    end
  end

-  def vote(user, object, choices) do
-    with "Question" <- object.data["type"],
-         {:author, false} <- {:author, object.data["actor"] == user.ap_id},
-         {:existing_votes, []} <- {:existing_votes, Utils.get_existing_votes(user.ap_id, object)},
-         {options, max_count} <- get_options_and_max_count(object),
-         option_count <- Enum.count(options),
-         {:choice_check, {choices, true}} <-
-           {:choice_check, normalize_and_validate_choice_indices(choices, option_count)},
-         {:count_check, true} <- {:count_check, Enum.count(choices) <= max_count} do
+  def vote(user, %{data: %{"type" => "Question"}} = object, choices) do
+    with :ok <- validate_not_author(object, user),
+         :ok <- validate_existing_votes(user, object),
+         {:ok, options, choices} <- normalize_and_validate_choices(choices, object) do
      answer_activities =
        Enum.map(choices, fn index ->
          answer_data = make_answer_data(user, object, Enum.at(options, index)["name"])
@ -149,32 +136,40 @@ defmodule Pleroma.Web.CommonAPI do

      object = Object.get_cached_by_ap_id(object.data["id"])
      {:ok, answer_activities, object}
-    else
-      {:author, _} -> {:error, dgettext("errors", "Poll's author can't vote")}
-      {:existing_votes, _} -> {:error, dgettext("errors", "Already voted")}
-      {:choice_check, {_, false}} -> {:error, dgettext("errors", "Invalid indices")}
-      {:count_check, false} -> {:error, dgettext("errors", "Too many choices")}
    end
  end

-  defp get_options_and_max_count(object) do
-    if Map.has_key?(object.data, "anyOf") do
-      {object.data["anyOf"], Enum.count(object.data["anyOf"])}
+  defp validate_not_author(%{data: %{"actor" => ap_id}}, %{ap_id: ap_id}),
+    do: {:error, dgettext("errors", "Poll's author can't vote")}
+
+  defp validate_not_author(_, _), do: :ok
+
+  defp validate_existing_votes(%{ap_id: ap_id}, object) do
+    if Utils.get_existing_votes(ap_id, object) == [] do
+      :ok
    else
-      {object.data["oneOf"], 1}
+      {:error, dgettext("errors", "Already voted")}
    end
  end

-  defp normalize_and_validate_choice_indices(choices, count) do
-    Enum.map_reduce(choices, true, fn index, valid ->
-      index = if is_binary(index), do: String.to_integer(index), else: index
-      {index, if(valid, do: index < count, else: valid)}
-    end)
+  defp get_options_and_max_count(%{data: %{"anyOf" => any_of}}), do: {any_of, Enum.count(any_of)}
+  defp get_options_and_max_count(%{data: %{"oneOf" => one_of}}), do: {one_of, 1}
+
+  defp normalize_and_validate_choices(choices, object) do
+    choices = Enum.map(choices, fn i -> if is_binary(i), do: String.to_integer(i), else: i end)
+    {options, max_count} = get_options_and_max_count(object)
+    count = Enum.count(options)
+
+    with {_, true} <- {:valid_choice, Enum.all?(choices, &(&1 < count))},
+         {_, true} <- {:count_check, Enum.count(choices) <= max_count} do
+      {:ok, options, choices}
+    else
+      {:valid_choice, _} -> {:error, dgettext("errors", "Invalid indices")}
+      {:count_check, _} -> {:error, dgettext("errors", "Too many choices")}
+    end
  end

-  def get_visibility(_, _, %Participation{}) do
-    {"direct", "direct"}
-  end
+  def get_visibility(_, _, %Participation{}), do: {"direct", "direct"}

  def get_visibility(%{"visibility" => visibility}, in_reply_to, _)
      when visibility in ~w{public unlisted private direct},
@ -196,94 +191,73 @@ defmodule Pleroma.Web.CommonAPI do

  def get_replied_to_visibility(activity) do
    with %Object{} = object <- Object.normalize(activity) do
-      Pleroma.Web.ActivityPub.Visibility.get_visibility(object)
+      Visibility.get_visibility(object)
    end
  end

-  def post(user, %{"status" => status} = data) do
-    limit = Pleroma.Config.get([:instance, :limit])
+  def check_expiry_date({:ok, nil} = res), do: res

-    with status <- String.trim(status),
-         attachments <- attachments_from_ids(data),
-         in_reply_to <- get_replied_to_activity(data["in_reply_to_status_id"]),
-         in_reply_to_conversation <- Participation.get(data["in_reply_to_conversation_id"]),
-         {visibility, in_reply_to_visibility} <-
-           get_visibility(data, in_reply_to, in_reply_to_conversation),
-         {_, false} <-
-           {:private_to_public, in_reply_to_visibility == "direct" && visibility != "direct"},
-         {content_html, mentions, tags} <-
-           make_content_html(
-             status,
-             attachments,
-             data,
-             visibility
-           ),
-         mentioned_users <- for({_, mentioned_user} <- mentions, do: mentioned_user.ap_id),
-         addressed_users <- get_addressed_users(mentioned_users, data["to"]),
-         {poll, poll_emoji} <- make_poll_data(data),
-         {to, cc} <-
-           get_to_and_cc(user, addressed_users, in_reply_to, visibility, in_reply_to_conversation),
-         context <- make_context(in_reply_to, in_reply_to_conversation),
-         cw <- data["spoiler_text"] || "",
-         sensitive <- data["sensitive"] || Enum.member?(tags, {"#nsfw", "nsfw"}),
-         full_payload <- String.trim(status <> cw),
-         :ok <- validate_character_limit(full_payload, attachments, limit),
-         object <-
-           make_note_data(
-             user.ap_id,
-             to,
-             context,
-             content_html,
-             attachments,
-             in_reply_to,
-             tags,
-             cw,
-             cc,
-             sensitive,
-             poll
-           ),
-         object <-
-           Map.put(
-             object,
-             "emoji",
-             Map.merge(Formatter.get_emoji_map(full_payload), poll_emoji)
-           ) do
-      preview? = Pleroma.Web.ControllerHelper.truthy_param?(data["preview"]) || false
-      direct? = visibility == "direct"
+  def check_expiry_date({:ok, in_seconds}) do
+    expiry = NaiveDateTime.utc_now() |> NaiveDateTime.add(in_seconds)

-      %{
-        to: to,
-        actor: user,
-        context: context,
-        object: object,
-        additional: %{"cc" => cc, "directMessage" => direct?}
-      }
-      |> maybe_add_list_data(user, visibility)
-      |> ActivityPub.create(preview?)
+    if ActivityExpiration.expires_late_enough?(expiry) do
+      {:ok, expiry}
    else
-      {:private_to_public, true} ->
-        {:error, dgettext("errors", "The message visibility must be direct")}
-
-      {:error, _} = e ->
-        e
-
-      e ->
-        {:error, e}
+      {:error, "Expiry date is too soon"}
    end
  end

+  def check_expiry_date(expiry_str) do
+    Ecto.Type.cast(:integer, expiry_str)
+    |> check_expiry_date()
+  end
+
+  def listen(user, %{"title" => _} = data) do
+    with visibility <- data["visibility"] || "public",
+         {to, cc} <- get_to_and_cc(user, [], nil, visibility, nil),
+         listen_data <-
+           Map.take(data, ["album", "artist", "title", "length"])
+           |> Map.put("type", "Audio")
+           |> Map.put("to", to)
+           |> Map.put("cc", cc)
+           |> Map.put("actor", user.ap_id),
+         {:ok, activity} <-
+           ActivityPub.listen(%{
+             actor: user,
+             to: to,
+             object: listen_data,
+             context: Utils.generate_context_id(),
+             additional: %{"cc" => cc}
+           }) do
+      {:ok, activity}
+    end
+  end
+
+  def post(user, %{"status" => _} = data) do
+    with {:ok, draft} <- Pleroma.Web.CommonAPI.ActivityDraft.create(user, data) do
+      draft.changes
+      |> ActivityPub.create(draft.preview?)
+      |> maybe_create_activity_expiration(draft.expires_at)
+    end
+  end
+
+  defp maybe_create_activity_expiration({:ok, activity}, %NaiveDateTime{} = expires_at) do
+    with {:ok, _} <- ActivityExpiration.create(activity, expires_at) do
+      {:ok, activity}
+    end
+  end
+
+  defp maybe_create_activity_expiration(result, _), do: result
+
  # Updates the emojis for a user based on their profile
  def update(user) do
+    emoji = emoji_from_profile(user)
+    source_data = user.info |> Map.get(:source_data, %{}) |> Map.put("tag", emoji)
+
    user =
-      with emoji <- emoji_from_profile(user),
-           source_data <- (user.info.source_data || %{}) |> Map.put("tag", emoji),
-           info_cng <- User.Info.set_source_data(user.info, source_data),
-           change <- Ecto.Changeset.change(user) |> Ecto.Changeset.put_embed(:info, info_cng),
-           {:ok, user} <- User.update_and_set_cache(change) do
-        user
-      else
-        _e ->
-          user
+      case User.update_info(user, &User.Info.set_source_data(&1, source_data)) do
+        {:ok, user} -> user
+        _ -> user
      end

    ActivityPub.update(%{
@ -298,44 +272,25 @@ defmodule Pleroma.Web.CommonAPI do
  def pin(id_or_ap_id, %{ap_id: user_ap_id} = user) do
    with %Activity{
           actor: ^user_ap_id,
-           data: %{
-             "type" => "Create"
-           },
-           object: %Object{
-             data: %{
-               "type" => "Note"
-             }
-           }
+           data: %{"type" => "Create"},
+           object: %Object{data: %{"type" => "Note"}}
         } = activity <- get_by_id_or_ap_id(id_or_ap_id),
         true <- Visibility.is_public?(activity),
-         %{valid?: true} = info_changeset <- User.Info.add_pinnned_activity(user.info, activity),
-         changeset <-
-           Ecto.Changeset.change(user) |> Ecto.Changeset.put_embed(:info, info_changeset),
-         {:ok, _user} <- User.update_and_set_cache(changeset) do
+         {:ok, _user} <- User.update_info(user, &User.Info.add_pinnned_activity(&1, activity)) do
      {:ok, activity}
    else
-      %{errors: [pinned_activities: {err, _}]} ->
-        {:error, err}
-
-      _ ->
-        {:error, dgettext("errors", "Could not pin")}
+      {:error, %{changes: %{info: %{errors: [pinned_activities: {err, _}]}}}} -> {:error, err}
+      _ -> {:error, dgettext("errors", "Could not pin")}
    end
  end

  def unpin(id_or_ap_id, user) do
    with %Activity{} = activity <- get_by_id_or_ap_id(id_or_ap_id),
-         %{valid?: true} = info_changeset <-
-           User.Info.remove_pinnned_activity(user.info, activity),
-         changeset <-
-           Ecto.Changeset.change(user) |> Ecto.Changeset.put_embed(:info, info_changeset),
-         {:ok, _user} <- User.update_and_set_cache(changeset) do
+         {:ok, _user} <- User.update_info(user, &User.Info.remove_pinnned_activity(&1, activity)) do
      {:ok, activity}
    else
-      %{errors: [pinned_activities: {err, _}]} ->
-        {:error, err}
-
-      _ ->
-        {:error, dgettext("errors", "Could not unpin")}
+      %{errors: [pinned_activities: {err, _}]} -> {:error, err}
+      _ -> {:error, dgettext("errors", "Could not unpin")}
    end
  end

@ -355,51 +310,46 @@ defmodule Pleroma.Web.CommonAPI do
  def thread_muted?(%{id: nil} = _user, _activity), do: false

  def thread_muted?(user, activity) do
-    with [] <- ThreadMute.check_muted(user.id, activity.data["context"]) do
-      false
-    else
-      _ -> true
+    ThreadMute.check_muted(user.id, activity.data["context"]) != []
+  end
+
+  def report(user, %{"account_id" => account_id} = data) do
+    with {:ok, account} <- get_reported_account(account_id),
+         {:ok, {content_html, _, _}} <- make_report_content_html(data["comment"]),
+         {:ok, statuses} <- get_report_statuses(account, data) do
+      ActivityPub.flag(%{
+        context: Utils.generate_context_id(),
+        actor: user,
+        account: account,
+        statuses: statuses,
+        content: content_html,
+        forward: data["forward"] || false
+      })
    end
  end

-  def report(user, data) do
-    with {:account_id, %{"account_id" => account_id}} <- {:account_id, data},
-         {:account, %User{} = account} <- {:account, User.get_cached_by_id(account_id)},
-         {:ok, {content_html, _, _}} <- make_report_content_html(data["comment"]),
-         {:ok, statuses} <- get_report_statuses(account, data),
-         {:ok, activity} <-
-           ActivityPub.flag(%{
-             context: Utils.generate_context_id(),
-             actor: user,
-             account: account,
-             statuses: statuses,
-             content: content_html,
-             forward: data["forward"] || false
-           }) do
-      {:ok, activity}
-    else
-      {:error, err} -> {:error, err}
-      {:account_id, %{}} -> {:error, dgettext("errors", "Valid `account_id` required")}
-      {:account, nil} -> {:error, dgettext("errors", "Account not found")}
+  def report(_user, _params), do: {:error, dgettext("errors", "Valid `account_id` required")}
+
+  defp get_reported_account(account_id) do
+    case User.get_cached_by_id(account_id) do
+      %User{} = account -> {:ok, account}
+      _ -> {:error, dgettext("errors", "Account not found")}
    end
  end

  def update_report_state(activity_id, state) do
-    with %Activity{} = activity <- Activity.get_by_id(activity_id),
-         {:ok, activity} <- Utils.update_report_state(activity, state) do
-      {:ok, activity}
+    with %Activity{} = activity <- Activity.get_by_id(activity_id) do
+      Utils.update_report_state(activity, state)
    else
      nil -> {:error, :not_found}
-      {:error, reason} -> {:error, reason}
      _ -> {:error, dgettext("errors", "Could not update state")}
    end
  end

  def update_activity_scope(activity_id, opts \\ %{}) do
    with %Activity{} = activity <- Activity.get_by_id_with_object(activity_id),
-         {:ok, activity} <- toggle_sensitive(activity, opts),
-         {:ok, activity} <- set_visibility(activity, opts) do
-      {:ok, activity}
+         {:ok, activity} <- toggle_sensitive(activity, opts) do
+      set_visibility(activity, opts)
    else
      nil -> {:error, :not_found}
      {:error, reason} -> {:error, reason}
@ -430,23 +380,15 @@ defmodule Pleroma.Web.CommonAPI do

  defp set_visibility(activity, _), do: {:ok, activity}

-  def hide_reblogs(user, muted) do
-    ap_id = muted.ap_id
-
+  def hide_reblogs(user, %{ap_id: ap_id} = _muted) do
    if ap_id not in user.info.muted_reblogs do
-      info_changeset = User.Info.add_reblog_mute(user.info, ap_id)
-      changeset = Ecto.Changeset.change(user) |> Ecto.Changeset.put_embed(:info, info_changeset)
-      User.update_and_set_cache(changeset)
+      User.update_info(user, &User.Info.add_reblog_mute(&1, ap_id))
    end
  end

-  def show_reblogs(user, muted) do
-    ap_id = muted.ap_id
-
+  def show_reblogs(user, %{ap_id: ap_id} = _muted) do
    if ap_id in user.info.muted_reblogs do
-      info_changeset = User.Info.remove_reblog_mute(user.info, ap_id)
-      changeset = Ecto.Changeset.change(user) |> Ecto.Changeset.put_embed(:info, info_changeset)
-      User.update_and_set_cache(changeset)
+      User.update_info(user, &User.Info.remove_reblog_mute(&1, ap_id))
    end
  end
 end
--- a/lib/pleroma/web/common_api/utils.ex
+++ b/lib/pleroma/web/common_api/utils.ex
@ -4,11 +4,13 @@

 defmodule Pleroma.Web.CommonAPI.Utils do
  import Pleroma.Web.Gettext
+  import Pleroma.Web.ControllerHelper, only: [truthy_param?: 1]

  alias Calendar.Strftime
  alias Pleroma.Activity
  alias Pleroma.Config
  alias Pleroma.Conversation.Participation
+  alias Pleroma.Emoji
  alias Pleroma.Formatter
  alias Pleroma.Object
  alias Pleroma.Plugs.AuthenticationPlug
@ -25,7 +27,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
  # This is a hack for twidere.
  def get_by_id_or_ap_id(id) do
    activity =
-      with true <- Pleroma.FlakeId.is_flake_id?(id),
+      with true <- FlakeId.flake_id?(id),
           %Activity{} = activity <- Activity.get_by_id_with_object(id) do
        activity
      else
@ -40,14 +42,6 @@ defmodule Pleroma.Web.CommonAPI.Utils do
      end
  end

-  def get_replied_to_activity(""), do: nil
-
-  def get_replied_to_activity(id) when not is_nil(id) do
-    Activity.get_by_id(id)
-  end
-
-  def get_replied_to_activity(_), do: nil
-
  def attachments_from_ids(%{"media_ids" => ids, "descriptions" => desc} = _) do
    attachments_from_ids_descs(ids, desc)
  end
@ -93,8 +87,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
          Activity.t() | nil,
          String.t(),
          Participation.t() | nil
-        ) ::
-          {list(String.t()), list(String.t())}
+        ) :: {list(String.t()), list(String.t())}

  def get_to_and_cc(_, _, _, _, %Participation{} = participation) do
    participation = Repo.preload(participation, :recipients)
@ -159,70 +152,74 @@ defmodule Pleroma.Web.CommonAPI.Utils do

  def maybe_add_list_data(activity_params, _, _), do: activity_params

+  def make_poll_data(%{"poll" => %{"expires_in" => expires_in}} = data)
+      when is_binary(expires_in) do
+    # In some cases mastofe sends out strings instead of integers
+    data
+    |> put_in(["poll", "expires_in"], String.to_integer(expires_in))
+    |> make_poll_data()
+  end
+
  def make_poll_data(%{"poll" => %{"options" => options, "expires_in" => expires_in}} = data)
      when is_list(options) do
-    %{max_expiration: max_expiration, min_expiration: min_expiration} =
-      limits = Pleroma.Config.get([:instance, :poll_limits])
+    limits = Pleroma.Config.get([:instance, :poll_limits])

-    # XXX: There is probably a cleaner way of doing this
-    try do
-      # In some cases mastofe sends out strings instead of integers
-      expires_in = if is_binary(expires_in), do: String.to_integer(expires_in), else: expires_in
-
-      if Enum.count(options) > limits.max_options do
-        raise ArgumentError, message: "Poll can't contain more than #{limits.max_options} options"
-      end
-
-      {poll, emoji} =
+    with :ok <- validate_poll_expiration(expires_in, limits),
+         :ok <- validate_poll_options_amount(options, limits),
+         :ok <- validate_poll_options_length(options, limits) do
+      {option_notes, emoji} =
        Enum.map_reduce(options, %{}, fn option, emoji ->
-          if String.length(option) > limits.max_option_chars do
-            raise ArgumentError,
-              message:
-                "Poll options cannot be longer than #{limits.max_option_chars} characters each"
-          end
+          note = %{
+            "name" => option,
+            "type" => "Note",
+            "replies" => %{"type" => "Collection", "totalItems" => 0}
+          }

-          {%{
-             "name" => option,
-             "type" => "Note",
-             "replies" => %{"type" => "Collection", "totalItems" => 0}
-           }, Map.merge(emoji, Formatter.get_emoji_map(option))}
+          {note, Map.merge(emoji, Emoji.Formatter.get_emoji_map(option))}
        end)

-      case expires_in do
-        expires_in when expires_in > max_expiration ->
-          raise ArgumentError, message: "Expiration date is too far in the future"
-
-        expires_in when expires_in < min_expiration ->
-          raise ArgumentError, message: "Expiration date is too soon"
-
-        _ ->
-          :noop
-      end
-
      end_time =
        NaiveDateTime.utc_now()
        |> NaiveDateTime.add(expires_in)
        |> NaiveDateTime.to_iso8601()

-      poll =
-        if Pleroma.Web.ControllerHelper.truthy_param?(data["poll"]["multiple"]) do
-          %{"type" => "Question", "anyOf" => poll, "closed" => end_time}
-        else
-          %{"type" => "Question", "oneOf" => poll, "closed" => end_time}
-        end
+      key = if truthy_param?(data["poll"]["multiple"]), do: "anyOf", else: "oneOf"
+      poll = %{"type" => "Question", key => option_notes, "closed" => end_time}

-      {poll, emoji}
-    rescue
-      e in ArgumentError -> e.message
+      {:ok, {poll, emoji}}
    end
  end

  def make_poll_data(%{"poll" => poll}) when is_map(poll) do
-    "Invalid poll"
+    {:error, "Invalid poll"}
  end

  def make_poll_data(_data) do
-    {%{}, %{}}
+    {:ok, {%{}, %{}}}
+  end
+
+  defp validate_poll_options_amount(options, %{max_options: max_options}) do
+    if Enum.count(options) > max_options do
+      {:error, "Poll can't contain more than #{max_options} options"}
+    else
+      :ok
+    end
+  end
+
+  defp validate_poll_options_length(options, %{max_option_chars: max_option_chars}) do
+    if Enum.any?(options, &(String.length(&1) > max_option_chars)) do
+      {:error, "Poll options cannot be longer than #{max_option_chars} characters each"}
+    else
+      :ok
+    end
+  end
+
+  defp validate_poll_expiration(expires_in, %{min_expiration: min, max_expiration: max}) do
+    cond do
+      expires_in > max -> {:error, "Expiration date is too far in the future"}
+      expires_in < min -> {:error, "Expiration date is too soon"}
+      true -> :ok
+    end
  end

  def make_content_html(
@ -234,7 +231,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
    no_attachment_links =
      data
      |> Map.get("no_attachment_links", Config.get([:instance, :no_attachment_links]))
-      |> Kernel.in([true, "true"])
+      |> truthy_param?()

    content_type = get_content_type(data["content_type"])

@ -347,25 +344,25 @@ defmodule Pleroma.Web.CommonAPI.Utils do
        attachments,
        in_reply_to,
        tags,
-        cw \\ nil,
+        summary \\ nil,
        cc \\ [],
        sensitive \\ false,
-        merge \\ %{}
+        extra_params \\ %{}
      ) do
    %{
      "type" => "Note",
      "to" => to,
      "cc" => cc,
      "content" => content_html,
-      "summary" => cw,
-      "sensitive" => !Enum.member?(["false", "False", "0", false], sensitive),
+      "summary" => summary,
+      "sensitive" => truthy_param?(sensitive),
      "context" => context,
      "attachment" => attachments,
      "actor" => actor,
      "tag" => Keyword.values(tags) |> Enum.uniq()
    }
    |> add_in_reply_to(in_reply_to)
-    |> Map.merge(merge)
+    |> Map.merge(extra_params)
  end

  defp add_in_reply_to(object, nil), do: object
@ -434,12 +431,14 @@ defmodule Pleroma.Web.CommonAPI.Utils do
    end
  end

-  def emoji_from_profile(%{info: _info} = user) do
-    (Formatter.get_emoji(user.bio) ++ Formatter.get_emoji(user.name))
-    |> Enum.map(fn {shortcode, url, _} ->
+  def emoji_from_profile(%User{bio: bio, name: name}) do
+    [bio, name]
+    |> Enum.map(&Emoji.Formatter.get_emoji/1)
+    |> Enum.concat()
+    |> Enum.map(fn {shortcode, %Emoji{file: path}} ->
      %{
        "type" => "Emoji",
-        "icon" => %{"type" => "Image", "url" => "#{Endpoint.url()}#{url}"},
+        "icon" => %{"type" => "Image", "url" => "#{Endpoint.url()}#{path}"},
        "name" => ":#{shortcode}:"
      }
    end)
@ -571,15 +570,16 @@ defmodule Pleroma.Web.CommonAPI.Utils do
    }
  end

-  def validate_character_limit(full_payload, attachments, limit) do
+  def validate_character_limit("" = _full_payload, [] = _attachments) do
+    {:error, dgettext("errors", "Cannot post an empty status without attachments")}
+  end
+
+  def validate_character_limit(full_payload, _attachments) do
+    limit = Pleroma.Config.get([:instance, :limit])
    length = String.length(full_payload)

    if length < limit do
-      if length > 0 or Enum.count(attachments) > 0 do
-        :ok
-      else
-        {:error, dgettext("errors", "Cannot post an empty status without attachments")}
-      end
+      :ok
    else
      {:error, dgettext("errors", "The status is over the character limit")}
    end
--- a/lib/pleroma/web/controller_helper.ex
+++ b/lib/pleroma/web/controller_helper.ex
@ -6,7 +6,7 @@ defmodule Pleroma.Web.ControllerHelper do
  use Pleroma.Web, :controller

  # As in MastoAPI, per https://api.rubyonrails.org/classes/ActiveModel/Type/Boolean.html
-  @falsy_param_values [false, 0, "0", "f", "F", "false", "FALSE", "off", "OFF"]
+  @falsy_param_values [false, 0, "0", "f", "F", "false", "False", "FALSE", "off", "OFF"]
  def truthy_param?(blank_value) when blank_value in [nil, ""], do: nil
  def truthy_param?(value), do: value not in @falsy_param_values

@ -34,79 +34,45 @@ defmodule Pleroma.Web.ControllerHelper do

  defp param_to_integer(_, default), do: default

-  def add_link_headers(
-        conn,
-        method,
-        activities,
-        param \\ nil,
-        params \\ %{},
-        func3 \\ nil,
-        func4 \\ nil
-      ) do
-    params =
-      conn.params
-      |> Map.drop(["since_id", "max_id", "min_id"])
-      |> Map.merge(params)
+  def add_link_headers(conn, activities, extra_params \\ %{}) do
+    case List.last(activities) do
+      %{id: max_id} ->
+        params =
+          conn.params
+          |> Map.drop(Map.keys(conn.path_params))
+          |> Map.drop(["since_id", "max_id", "min_id"])
+          |> Map.merge(extra_params)

-    last = List.last(activities)
+        limit =
+          params
+          |> Map.get("limit", "20")
+          |> String.to_integer()

-    func3 = func3 || (&mastodon_api_url/3)
-    func4 = func4 || (&mastodon_api_url/4)
+        min_id =
+          if length(activities) <= limit do
+            activities
+            |> List.first()
+            |> Map.get(:id)
+          else
+            activities
+            |> Enum.at(limit * -1)
+            |> Map.get(:id)
+          end

-    if last do
-      max_id = last.id
+        next_url = current_url(conn, Map.merge(params, %{max_id: max_id}))
+        prev_url = current_url(conn, Map.merge(params, %{min_id: min_id}))

-      limit =
-        params
-        |> Map.get("limit", "20")
-        |> String.to_integer()
+        put_resp_header(conn, "link", "<#{next_url}>; rel=\"next\", <#{prev_url}>; rel=\"prev\"")

-      min_id =
-        if length(activities) <= limit do
-          activities
-          |> List.first()
-          |> Map.get(:id)
-        else
-          activities
-          |> Enum.at(limit * -1)
-          |> Map.get(:id)
-        end
+      _ ->
+        conn
+    end
+  end

-      {next_url, prev_url} =
-        if param do
-          {
-            func4.(
-              Pleroma.Web.Endpoint,
-              method,
-              param,
-              Map.merge(params, %{max_id: max_id})
-            ),
-            func4.(
-              Pleroma.Web.Endpoint,
-              method,
-              param,
-              Map.merge(params, %{min_id: min_id})
-            )
-          }
-        else
-          {
-            func3.(
-              Pleroma.Web.Endpoint,
-              method,
-              Map.merge(params, %{max_id: max_id})
-            ),
-            func3.(
-              Pleroma.Web.Endpoint,
-              method,
-              Map.merge(params, %{min_id: min_id})
-            )
-          }
-        end
-
-      conn
-      |> put_resp_header("link", "<#{next_url}>; rel=\"next\", <#{prev_url}>; rel=\"prev\"")
-    else
-      conn
+  def assign_account_by_id(%{params: %{"id" => id}} = conn, _) do
+    case Pleroma.User.get_cached_by_id(id) do
+      %Pleroma.User{} = account -> assign(conn, :account, account)
+      nil -> Pleroma.Web.MastodonAPI.FallbackController.call(conn, {:error, :not_found}) |> halt()
    end
  end
 end
--- a/lib/pleroma/web/endpoint.ex
+++ b/lib/pleroma/web/endpoint.ex
@ -57,7 +57,7 @@ defmodule Pleroma.Web.Endpoint do
    plug(Phoenix.CodeReloader)
  end

-  plug(TrailingFormatPlug)
+  plug(Pleroma.Plugs.TrailingFormatPlug)
  plug(Plug.RequestId)
  plug(Plug.Logger)

@ -97,10 +97,7 @@ defmodule Pleroma.Web.Endpoint do
    extra: extra
  )

-  # Note: the plug and its configuration is compile-time this can't be upstreamed yet
-  if proxies = Pleroma.Config.get([__MODULE__, :reverse_proxies]) do
-    plug(RemoteIp, proxies: proxies)
-  end
+  plug(Pleroma.Plugs.RemoteIp)

  defmodule Instrumenter do
    use Prometheus.PhoenixInstrumenter
--- a/lib/pleroma/web/federator/federator.ex
+++ b/lib/pleroma/web/federator/federator.ex
@ -10,16 +10,17 @@ defmodule Pleroma.Web.Federator do
  alias Pleroma.Web.ActivityPub.Transmogrifier
  alias Pleroma.Web.ActivityPub.Utils
  alias Pleroma.Web.Federator.Publisher
-  alias Pleroma.Web.Federator.RetryQueue
  alias Pleroma.Web.OStatus
  alias Pleroma.Web.Websub
+  alias Pleroma.Workers.PublisherWorker
+  alias Pleroma.Workers.ReceiverWorker
+  alias Pleroma.Workers.SubscriberWorker

  require Logger

  def init do
-    # 1 minute
-    Process.sleep(1000 * 60)
-    refresh_subscriptions()
+    # To do: consider removing this call in favor of scheduled execution (`quantum`-based)
+    refresh_subscriptions(schedule_in: 60)
  end

  @doc "Addresses [memory leaks on recursive replies fetching](https://git.pleroma.social/pleroma/pleroma/issues/161)"
@ -37,50 +38,38 @@ defmodule Pleroma.Web.Federator do
  # Client API

  def incoming_doc(doc) do
-    PleromaJobQueue.enqueue(:federator_incoming, __MODULE__, [:incoming_doc, doc])
+    ReceiverWorker.enqueue("incoming_doc", %{"body" => doc})
  end

  def incoming_ap_doc(params) do
-    PleromaJobQueue.enqueue(:federator_incoming, __MODULE__, [:incoming_ap_doc, params])
+    ReceiverWorker.enqueue("incoming_ap_doc", %{"params" => params})
  end

-  def publish(activity, priority \\ 1) do
-    PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:publish, activity], priority)
+  def publish(%{id: "pleroma:fakeid"} = activity) do
+    perform(:publish, activity)
+  end
+
+  def publish(activity) do
+    PublisherWorker.enqueue("publish", %{"activity_id" => activity.id})
  end

  def verify_websub(websub) do
-    PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:verify_websub, websub])
+    SubscriberWorker.enqueue("verify_websub", %{"websub_id" => websub.id})
  end

-  def request_subscription(sub) do
-    PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:request_subscription, sub])
+  def request_subscription(websub) do
+    SubscriberWorker.enqueue("request_subscription", %{"websub_id" => websub.id})
  end

-  def refresh_subscriptions do
-    PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:refresh_subscriptions])
+  def refresh_subscriptions(worker_args \\ []) do
+    SubscriberWorker.enqueue("refresh_subscriptions", %{}, worker_args ++ [max_attempts: 1])
  end

  # Job Worker Callbacks

-  def perform(:refresh_subscriptions) do
-    Logger.debug("Federator running refresh subscriptions")
-    Websub.refresh_subscriptions()
-
-    spawn(fn ->
-      # 6 hours
-      Process.sleep(1000 * 60 * 60 * 6)
-      refresh_subscriptions()
-    end)
-  end
-
-  def perform(:request_subscription, websub) do
-    Logger.debug("Refreshing #{websub.topic}")
-
-    with {:ok, websub} <- Websub.request_subscription(websub) do
-      Logger.debug("Successfully refreshed #{websub.topic}")
-    else
-      _e -> Logger.debug("Couldn't refresh #{websub.topic}")
-    end
+  @spec perform(atom(), module(), any()) :: {:ok, any()} | {:error, any()}
+  def perform(:publish_one, module, params) do
+    apply(module, :publish_one, [params])
  end

  def perform(:publish, activity) do
@ -92,14 +81,6 @@ defmodule Pleroma.Web.Federator do
    end
  end

-  def perform(:verify_websub, websub) do
-    Logger.debug(fn ->
-      "Running WebSub verification for #{websub.id} (#{websub.topic}, #{websub.callback})"
-    end)
-
-    Websub.verify(websub)
-  end
-
  def perform(:incoming_doc, doc) do
    Logger.info("Got document, trying to parse")
    OStatus.handle_incoming(doc)
@ -130,22 +111,27 @@ defmodule Pleroma.Web.Federator do
    end
  end

-  def perform(
-        :publish_single_websub,
-        %{xml: _xml, topic: _topic, callback: _callback, secret: _secret} = params
-      ) do
-    case Websub.publish_one(params) do
-      {:ok, _} ->
-        :ok
+  def perform(:request_subscription, websub) do
+    Logger.debug("Refreshing #{websub.topic}")

-      {:error, _} ->
-        RetryQueue.enqueue(params, Websub)
+    with {:ok, websub} <- Websub.request_subscription(websub) do
+      Logger.debug("Successfully refreshed #{websub.topic}")
+    else
+      _e -> Logger.debug("Couldn't refresh #{websub.topic}")
    end
  end

-  def perform(type, _) do
-    Logger.debug(fn -> "Unknown task: #{type}" end)
-    {:error, "Don't know what to do with this"}
+  def perform(:verify_websub, websub) do
+    Logger.debug(fn ->
+      "Running WebSub verification for #{websub.id} (#{websub.topic}, #{websub.callback})"
+    end)
+
+    Websub.verify(websub)
+  end
+
+  def perform(:refresh_subscriptions) do
+    Logger.debug("Federator running refresh subscriptions")
+    Websub.refresh_subscriptions()
  end

  def ap_enabled_actor(id) do
--- a/Show More
+++ b/Show More