MSC2938: Report content to moderators

proposals/2938-report-content-to-moderators.md

@@ -0,0 +1,273 @@
+# MSC2938: Report Content to Moderators
+
+Matrix currently offers [a mechanism](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-report-eventid)
+to report bad content content (e.g. trolling, spam, abuse, etc.) to homeserver admins.
+However, the homeserver admin is generally the wrong person for this as:
+
+- on encrypted channels, they often do not have access to the content;
+- on large servers, they quickly end up bombarded with reports, which makes them unable to take action.
+
+This proposal extends the existing mechanism to let the server route
+the report to room moderators and/or admins instead, who can better
+take action.
+
+## Proposal
+
+### Introduction
+
+Bad content (trolling, spam, abuse, etc.) on good rooms happens. When it happens, the entire
+room suffers from it.
+
+Whenever bad content show up, a room moderator or admin needs to intervene and either explain
+the rules to the bad user or kick/ban them. However, room moderators/admins are often not monitoring
+their rooms actively and are therefore not aware of bad content until someone points it to them.
+
+As of this writing, there is no good mechanism for doing so. There is [a mechanism to
+report bad content](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-report-eventid),
+but this mechanism is delivered to the *homeserver admin*, who often cannot take action against
+such bad content. To make things worse, clients that support this mechanism typically label
+it "Report Content", which misleads users into asssuming that this is the right mechanism
+to get in touch with room moderators/admins. The end result is that 80%+ of the reports that
+reach the homeserver administrators are reports that can only be actioned by room
+moderators/administrators (numbers from matrix.org for November 2020).
+
+
+We propose an extension to the mechanism to request that the report be sent to room moderators/admins
+instead of homeserver admins.
+
+### Proposal
+
+In the following, we call **moderators** for a room `room_id` any user with a powerlevel sufficient
+to both kick and ban room members.
+
+#### Client API
+
+We extend `POST /_matrix/client/r0/rooms/{room_id}/report/{event_id}` with a JSON Body Parameter
+
+| Parameter | Type   | Description |
+| --------- |------- | ----------- |
+| target    | string | One of "server-notice", "homeserver_admins". |
+
+If `target` is `"homeserver_admins"` or unspecified, the behavior is unchanged.
+
+If `target` is `"room_moderators"`, the message is intended to be propagated as a server notice to
+moderators for this room.
+
+#### Server notices
+
+We further extend server notices so that the field `server_notice_type` may also admit a new
+value:
+
+- `m.server_notice.content_report`:
+  A user has reported some content as bad (trolling, spam, abuse, etc.).
+
+Whenever `server_notice_type` is `m.server_notice.content_report`, 
+
+| Parameter | Type   | Description |
+| --------- | ------ | ----------- |
+| body      | string | **Required** A human-readable description of the problem, generated by the server (e.g. "User %s has reported that '%s'"). |
+| msgtype 	| enum 	 | **Required** In this case, `m.server_notice.content_report`. |
+| room_id   | string | **Required** The id of the room being reported. |
+| event_id  | string | **Required** The id of the event being reported. |
+| user_id   | string | **Required** The id of the user reporting the event. |
+| score     | integer | The score to rate this content as where -100 is most offensive and 0 is inoffensive, as given by the reporting user. |
+| reason    | string | The reason given by the reporting user. |
+| nature    | enum   | The type of activity being reported. |
+
+Field `nature` is designed to be filled by the reporter's front-end and to aid moderator-side tools (e.g. bots) to make
+a decision on the event. As of this MSC, `nature` is one of:
+
+- `abuse.spam` (reporting spam);
+- `abuse.moderation` (reporting a behavior that should be moderated, typically trolling).
+
+
+#### Client behavior (sending reports)
+
+Clients may offer UX to let users report content to the room moderators instead of homeserver administrators.
+When this UX is used, a call to `POST /_matrix/client/r0/rooms/{room_id}/report/{event_id}` is sent, e.g.
+
+```jsonc
+{
+  "target": "room_moderators",
+  "score": -100,
+  "reason": "this makes me sad",
+  "nature": "abuse.moderation",
+}
+```
+
+#### Server behavior
+
+Whenever a server receives a `POST /_matrix/client/r0/rooms/{room_id}/report/{event_id}` with `target` specified
+as `"room_moderators"` they should:
+
+1. if `event_id` is not an event in `room_id` or if the user is not a member of `room_id`, return a HTTP NOT FOUND and Matrix error code NOT FOUND;
+2. if `nature` is not a known nature, normalize it to `None`;
+3. if `room_id` has no moderators, return a HTTP NOT FOUND and Matrix error code NOT FOUND;
+4. for all moderators in room `room_id` *on the local homeserver*:
+
+  1. send a server notice
+  ```jsonc
+  {
+    "body": "User has reported content", // Or any other human-readable description
+    "msgtype": "m.server_notice.content_report",
+    "room_id": "{room_id}",   // From path parameter
+    "event_id": "{event_id}", // From path parameter
+    "reporter_id": "{user_id}",   // From authentication token
+    "score": "{score}",     // From JSON body parameter
+    "reason": "{reason}"    // From JSON body parameter
+  }
+  ```
+
+5. for all homeservers other than local with at least one moderator in room `room_id`:
+
+  1. send a EDU
+  ```jsonc
+  {
+    "room_id": "{room_id}", // From path parameter
+    "sender":  "{user_id}", // From authentication token
+    "origin":  "{server_name}", // From server configuration
+    "origin_server_ts": "{now}", // From clock
+    "type": "m.server_notice.content_report",
+    "content": {
+      "body": "User has reported content", // Or any other human-readable description
+      "msgtype": "m.server_notice.content_report",
+      "room_id": "{room_id}",   // From path parameter
+      "event_id": "{event_id}", // From path parameter
+      "reporter_id": "{user_id}",// From authentication token
+      "score": "{score}",       // From JSON body parameter
+      "reason": "{reason}",     // From JSON body parameter
+      "nature": "{nature}",     // From JSON body parameter
+    },
+    "prev_events": [...],     // Most recent events in room `{room_id}`, as per usual EDU mechanisms
+    "depth": ?????,           // FIXME: How should we pick this?
+    "auth_events": [
+      "{userAuthEventId}",    // Event ID for the authentication of user {user_id}
+      "{reportedUserAuthEventId}", // Event ID for the authentication of the user who created event {event_id}
+    ],
+    "hashes": {
+      ...                      // Hashes, as per usual EDU mechanisms
+    },
+    "signatures": {
+      ...                      // Signatures, as per usual EDU mechanism
+    },
+  }
+  ```
+
+1. When a server receives a EDU with the following content:
+  ```jsonc
+  {
+    "body": "{body}",
+    "msgtype": "m.server_notice.content_report",
+    "room_id": "{room_id}",
+    "event_id": "{event_id}",
+    "reporter_id": "{user_id}",
+    "score": "{score}",
+    "reason": "{reason}",
+    "nature": "{nature}",
+  }
+  ```
+
+  1. ignore the message if it is an invalid EDU, as per usual EDU rules; otherwise
+  2. ignore the message if server is not part of `room_id`; otherwise
+  3. ignore the message if `event_id` is not an event in `room_id` or if `reporter_id` is not a member of `room_id`; otherwise
+  4. for all moderators in room `room_id` on server:
+
+    1. send a server notice
+    ```jsonc
+    {
+      "body": "{body}", // Or any other human-readable description.
+      "msgtype": "m.server_notice.content_report",
+      "room_id": "{room_id}",   // From EDU
+      "event_id": "{event_id}", // From EDU
+      "reporter_id": "{user_id}",// From EDU
+      "score": "{score}",       // From EDU
+      "reason": "{reason}",     // From EDU
+      "nature": "{nature}",     // From EDU
+    }
+    ```
+
+#### Client behavior (receiving reports)
+
+General client behavior is unchanged from https://matrix.org/docs/spec/client_server/r0.6.1#id166.
+
+In this specific case, clients may additionally offer a link to event `event_id` in `room_id` or a
+preview to simplify the work of moderators.
+
+
+## Potential issues
+
+This proposal may end up being used for spamming the moderators of rooms. We believe
+that this is no worse than the current implementation which may be used for spamming the administrator
+of the homeserver.
+
+## Alternatives
+
+### Send DM to moderator
+
+An alternative would be to encourage users to send DM to moderators. This has the
+following drawbacks:
+
+1. This requires moderators to accept every single DM of they wish to deal with bad content. We assume
+  that many moderators will not want to do that.
+2. Sending DMs to people one is not familiar with is considered impolite by many users. Many users
+  will therefore not do it and rather use the current Report Content button, with the issues
+  listed at the start of this document.
+3. If several users report the same content, moderators may find themselves bombarded
+  with large number of DMs, which makes tracking bad content more complicated.
+
+### Send DM to moderator, with UX
+
+A variant would be to implement a "Report Content to Room Moderators" button as a mechanism that
+sends a DM to all moderators in the room. This solves one of the pain points above but still
+has the following drawbacks:
+
+1. This requires moderators to accept every single DM of they wish to deal with bad content. We assume
+  that many moderators will not want to do that.
+2. If several users report the same content, moderators may find themselves bombarded
+  with large number of DMs, which makes tracking bad content more complicated.
+
+### Reporting room
+
+Another alternative would be to create (dynamically), for each room `r`, another room `r_reports` dedicated to reporting events
+in `r`. The room `r_reports` would let each member of `r` send message to `r_reports` but only moderators of `r`
+could read messages in `r_reports`. This would simplify the work of federation and ensure that reports are not
+lost in case of ill-timed network problems but would raise a number of difficulties:
+
+1. What user creates `r_reports` and on which server?
+2. What happens whenever a user becomes a moderator/loses PL in `r`?
+3. Why should the moderators of `r` accept invites to this unknown room `r_reports`?
+4. When an event is reported and the room `r_reports` already exists, how does the homeserver determine that the event
+  should be reported in this same `r_reports`?
+
+### Reporting room, revisited
+
+A variant of the previous idea would be to create, for each room `room` and each report `rep`, a new room `room_report`.
+Once the room is created, only moderators of `room` have access to this room `room_report`. This would similarly
+simplify the work of federation and ensure that reports are not lost in case of ill-timed network problems but
+would raise a number of difficulties:
+
+1. What user creates `room_reports` and on which server?
+2. Why should the moderators of `room` accept invites to this unknown room `room_reports`?
+3. If the room is subject to much abuse or if many users report the same event, moderators quickly risk ending
+   up with hundreds of distinct abuse rooms. This may end up being very counter-productive.
+
+## Security considerations
+
+If a user reports bad content (e.g. political propaganda) to an moderator who actually
+sees this content as good, there is a risk of reprisal against the reporting user. We consider this
+risk fairly low.
+
+A graver risk may appear if a user notices bad content (e.g. terrorism, child sexual
+abuse, illicit substance trade) on a room actually dedicated to this bad content and accidentally reports
+it to the room moderator instead of the homeserver administrator and/or the authorities.
+
+We believe that the risk can be mitigated/eliminated by having the client UX explain carefully the
+difference between sending to a homeserver administrator, to a room moderator or to the authorities.
+
+
+## Unstable prefix
+
+During experimentation,
+
+- `m.server_notice.content_report` will be prefixed into `org.matrix.msc2938.content_report`;
+- `target` will be prefixed into `org.matrix.msc2938.target`;