Long quotes in replies

[progval]

Describe the bug

Seen earlier today (10:42 UTC) on #matrix-irc on Libera

10:42:43 <jess[m]> > <@ilmari:ilmari.org> `!listrooms` says
10:42:43 <jess[m]> > > You are connected, but not joined to any channels.
10:42:43 <jess[m]> >
10:42:43 <jess[m]> > but `/whois` from the IRC side says
10:42:43 <jess[m]> >
10:42:44 <jess[m]> > > There is no such nick ilmari_
10:42:44 <jess[m]> still having problems?

To Reproduce

No idea. Possibly just replying to a multi-line message.

Expected behavior

Should probably be just this:

<jess[m]> ilmari_: still having problems?

[Half-Shot]

That looks more like @jesopo quoted a message rather than a reply?

[progval]

I found another example:

{
  "content": {
    "body": "> <@ilmari:ilmari.org> I tried to join now, and my HS logged this: \n> ```\n> 2022-05-31 13:16:37,121 - synapse.util.retryutils - 222 - INFO - POST-1985617- Connection to matrix.org was unsuccessful (<class 'synapse.api.errors.RequestSendFailed'>(Failed to send request: ResponseNeverReceived: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>])); backoff now 600000\n> 2022-05-31 13:16:37,121 - synapse.federation.federation_client - 683 - WARNING - POST-1985617- Failed to send_join via matrix.org: Failed to send request: ResponseNeverReceived: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>]\n> 2022-05-31 13:16:37,122 - synapse.http.server - 95 - INFO - POST-1985617- <XForwardedForRequest at 0x7f44969d8cd0 method='POST' uri='/_matrix/client/r0/join/%23_oftc_%23debian%3Amatrix.org' clientproto='HTTP/1.0' site='8008'> SynapseError: 502 - Failed to send_join via any server\n> 2022-05-31 13:16:37,125 - synapse.access.http.8008 - 427 - INFO - POST-1985617- 152.37.111.239 - 8008 - {@ilmari:ilmari.org} Processed request: 125.448sec/0.002sec (0.020sec, 0.002sec) (0.012sec/0.050sec/5) 68B 502 \"POST /_matrix/client/r0/join/%23_oftc_%23debian%3Amatrix.org HTTP/1.0\" \"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0\" [0 dbevts]\n> ```\n\nfederation will initially fail to join massive rooms regardless if they are bridged or not ",
    "format": "org.matrix.custom.html",
    "formatted_body": "<mx-reply><blockquote><a href=\"https://matrix.to/#/!pSjsXGiqgurMQWoWnf:half-shot.uk/$Ui7cHVM0Xz-xHeQ1DhLHhvDQqs9H_LJTgWasonKrcVA\">In reply to</a> <a href=\"https://matrix.to/#/@ilmari:ilmari.org\">@ilmari:ilmari.org</a><br><p>I tried to join now, and my HS logged this:</p>\n<pre><code>2022-05-31 13:16:37,121 - synapse.util.retryutils - 222 - INFO - POST-1985617- Connection to matrix.org was unsuccessful (&lt;class 'synapse.api.errors.RequestSendFailed'&gt;(Failed to send request: ResponseNeverReceived: [&lt;twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.&gt;])); backoff now 600000\n2022-05-31 13:16:37,121 - synapse.federation.federation_client - 683 - WARNING - POST-1985617- Failed to send_join via matrix.org: Failed to send request: ResponseNeverReceived: [&lt;twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.&gt;]\n2022-05-31 13:16:37,122 - synapse.http.server - 95 - INFO - POST-1985617- &lt;XForwardedForRequest at 0x7f44969d8cd0 method='POST' uri='/_matrix/client/r0/join/%23_oftc_%23debian%3Amatrix.org' clientproto='HTTP/1.0' site='8008'&gt; SynapseError: 502 - Failed to send_join via any server\n2022-05-31 13:16:37,125 - synapse.access.http.8008 - 427 - INFO - POST-1985617- 152.37.111.239 - 8008 - {@ilmari:ilmari.org} Processed request: 125.448sec/0.002sec (0.020sec, 0.002sec) (0.012sec/0.050sec/5) 68B 502 &quot;POST /_matrix/client/r0/join/%23_oftc_%23debian%3Amatrix.org HTTP/1.0&quot; &quot;Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0&quot; [0 dbevts]\n</code></pre>\n</blockquote></mx-reply>federation will initially fail to join massive rooms regardless if they are bridged or not",
    "m.relates_to": {
      "m.in_reply_to": {
        "event_id": "$Ui7cHVM0Xz-xHeQ1DhLHhvDQqs9H_LJTgWasonKrcVA"
      }
    },
    "msgtype": "m.text"
  },
  "origin_server_ts": 1654003301663,
  "sender": "@hifi:vi.fi",
  "type": "m.room.message",
  "unsigned": {},
  "event_id": "$qpiDW1-Ad-8vaO-VvEjvG79DAr_rsVOfVPw3FmAy5EI",
  "room_id": "!pSjsXGiqgurMQWoWnf:half-shot.uk"
}

gives this on IRC:

13:21:44 <hifi[m]> > <@ilmari:ilmari.org> I tried to join now, and my HS logged this:... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/cd43f10d19c5c03f8850acb9f8b467d645a4e430)

and the pastebin's content is:

> <@ilmari:ilmari.org> I tried to join now, and my HS logged this: 
> ```
> 2022-05-31 13:16:37,121 - synapse.util.retryutils - 222 - INFO - POST-1985617- Connection to matrix.org was unsuccessful (<class 'synapse.api.errors.RequestSendFailed'>(Failed to send request: ResponseNeverReceived: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>])); backoff now 600000
> 2022-05-31 13:16:37,121 - synapse.federation.federation_client - 683 - WARNING - POST-1985617- Failed to send_join via matrix.org: Failed to send request: ResponseNeverReceived: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>]
> 2022-05-31 13:16:37,122 - synapse.http.server - 95 - INFO - POST-1985617- <XForwardedForRequest at 0x7f44969d8cd0 method='POST' uri='/_matrix/client/r0/join/%23_oftc_%23debian%3Amatrix.org' clientproto='HTTP/1.0' site='8008'> SynapseError: 502 - Failed to send_join via any server
> 2022-05-31 13:16:37,125 - synapse.access.http.8008 - 427 - INFO - POST-1985617- 152.37.111.239 - 8008 - {@ilmari:ilmari.org} Processed request: 125.448sec/0.002sec (0.020sec, 0.002sec) (0.012sec/0.050sec/5) 68B 502 "POST /_matrix/client/r0/join/%23_oftc_%23debian%3Amatrix.org HTTP/1.0" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0" [0 dbevts]
> ```

federation will initially fail to join massive rooms regardless if they are bridged or not 

[progval]

Just got a better example without pastebining or markdown:

{
  "content": {
    "body": "> <@ilynica:matrix.org> Hello! So I’m new to this matrix thing with Literally Zero knowledge about programming and terminologies. like nada. like a kid who doesn’t even know what a cpu is lol\n> But I do plan on building a community here for all of my subscribers from patreon. My question is, is it possible to have a room without letting anyone see the participant list, and not letting anyone dm each other?\n> I want to keep old patrons who cancelled and prevent them from begging other patrons to share early access content for free lol\n\nthere's no way currently to do that and it would probably be pretty difficult to get that working with how matrix works to my knowledge (I dont fully understand it so dont quote me on that lmao) just generally from what I've seen people who know it more then me say ",
    "format": "org.matrix.custom.html",
    "formatted_body": "<mx-reply><blockquote><a href=\"https://matrix.to/#/!OGEhHVWSdvArJzumhm:matrix.org/$l0lzyEnSVhLBAh3fmPW3DcxIO6paHDO2yYJx8ASYrDw?via=matrix.org&via=libera.chat&via=wolfo.tech\">In reply to</a> <a href=\"https://matrix.to/#/@ilynica:matrix.org\">@ilynica:matrix.org</a><br>Hello! So I’m new to this matrix thing with Literally Zero knowledge about programming and terminologies. like nada. like a kid who doesn’t even know what a cpu is lol<br/>But I do plan on building a community here for all of my subscribers from patreon. My question is, is it possible to have a room without letting anyone see the participant list, and not letting anyone dm each other?<br/>I want to keep old patrons who cancelled and prevent them from begging other patrons to share early access content for free lol</blockquote></mx-reply>there's no way currently to do that and it would probably be pretty difficult to get that working with how matrix works to my knowledge (I dont fully understand it so dont quote me on that lmao) just generally from what I've seen people who know it more then me say",
    "m.relates_to": {
      "m.in_reply_to": {
        "event_id": "$l0lzyEnSVhLBAh3fmPW3DcxIO6paHDO2yYJx8ASYrDw"
      }
    },
    "msgtype": "m.text",
    "org.matrix.msc1767.message": [
      {
        "body": "> <@ilynica:matrix.org> Hello! So I’m new to this matrix thing with Literally Zero knowledge about programming and terminologies. like nada. like a kid who doesn’t even know what a cpu is lol\n> But I do plan on building a community here for all of my subscribers from patreon. My question is, is it possible to have a room without letting anyone see the participant list, and not letting anyone dm each other?\n> I want to keep old patrons who cancelled and prevent them from begging other patrons to share early access content for free lol\n\nthere's no way currently to do that and it would probably be pretty difficult to get that working with how matrix works to my knowledge (I dont fully understand it so dont quote me on that lmao) just generally from what I've seen people who know it more then me say ",
        "mimetype": "text/plain"
      },
      {
        "body": "<mx-reply><blockquote><a href=\"https://matrix.to/#/!OGEhHVWSdvArJzumhm:matrix.org/$l0lzyEnSVhLBAh3fmPW3DcxIO6paHDO2yYJx8ASYrDw?via=matrix.org&via=libera.chat&via=wolfo.tech\">In reply to</a> <a href=\"https://matrix.to/#/@ilynica:matrix.org\">@ilynica:matrix.org</a><br>Hello! So I’m new to this matrix thing with Literally Zero knowledge about programming and terminologies. like nada. like a kid who doesn’t even know what a cpu is lol<br/>But I do plan on building a community here for all of my subscribers from patreon. My question is, is it possible to have a room without letting anyone see the participant list, and not letting anyone dm each other?<br/>I want to keep old patrons who cancelled and prevent them from begging other patrons to share early access content for free lol</blockquote></mx-reply>there's no way currently to do that and it would probably be pretty difficult to get that working with how matrix works to my knowledge (I dont fully understand it so dont quote me on that lmao) just generally from what I've seen people who know it more then me say",
        "mimetype": "text/html"
      }
    ]
  },
  "origin_server_ts": 1654026332442,
  "sender": "@ninchuka:envs.net",
  "type": "m.room.message",
  "unsigned": {},
  "event_id": "$ocE6sxY5_6GEgt3wC3xGapZTFx60oZWtF8-W8pfsark",
  "room_id": "!OGEhHVWSdvArJzumhm:matrix.org"
}

shows on IRC like this:

19:45:55 <ninchuka[m]> > <@ilynica:matrix.org> Hello! So I’m new to this matrix thing with Literally Zero knowledge about programming and terminologies. like nada. like a kid who doesn’t even know what a cpu is lol
19:45:55 <ninchuka[m]> > But I do plan on building a community here for all of my subscribers from patreon. My question is, is it possible to have a room without letting anyone see the participant list, and not letting anyone dm each other?
19:45:55 <ninchuka[m]> > I want to keep old patrons who cancelled and prevent them from begging other patrons to share early access content for free lol
19:45:55 <ninchuka[m]> there's no way currently to do that and it would probably be pretty difficult to get that working with how matrix works to my knowledge (I dont fully understand it so dont quote me on that lmao) just generally from what I've seen people who know it more then me say 

[progval]

It seems some messages in particular cause fallbacks to be sent for anyone replying to it. example from #matrix on libera:

16:59:06 <dpprpl[m]> Hi guys. Does anyone know if there is some software or service for helpdesk with matrix support or a bridge for it?
16:59:06 <dpprpl[m]> I want to propose matrix for corporate communication at my job and it would be one of the reasons to use it.
16:59:06 <dpprpl[m]> Our support uses jivo chat and it has an API, but I doubt I will get funding to develop our own bridge
17:04:03 <dpprpl[m]> Was my message deleted? 
17:21:29 <Adminion[m]> <Adminion[m]> "There is custom css for Element,..." <- It requires modding Element Web/Desktop.
17:23:16 <dpprpl[m]> Hi guys. I'm new here
17:23:16 <Adminion[m]> <farribeiro> "<span style="color: red"> This a..." <- I do not believe that inline CSS is supported.
17:23:17 <Adminion[m]> > <@dpprpl:matrix.org> Hi guys. Does anyone know if there is some software or service for helpdesk with matrix support or a bridge for it?
17:23:17 <Adminion[m]> > I want to propose matrix for corporate communication at my job and it would be one of the reasons to use it.
17:23:17 <Adminion[m]> > Our support uses jivo chat and it has an API, but I doubt I will get funding to develop our own bridge
17:23:17 <Adminion[m]> What features does your helpdesk software need to support? Help tickets?
17:23:18 <AustinHuang[m]> > <@dpprpl:matrix.org> Hi guys. Does anyone know if there is some software or service for helpdesk with matrix support or a bridge for it?
17:23:18 <AustinHuang[m]> > I want to propose matrix for corporate communication at my job and it would be one of the reasons to use it.
17:23:18 <AustinHuang[m]> > Our support uses jivo chat and it has an API, but I doubt I will get funding to develop our own bridge
17:23:18 <AustinHuang[m]> https://element.io/blog/element-launches-chatterbox/
17:23:28 <Adminion[m]> dpprpl[m]: Ir was not.
17:23:30 <Adminion[m]> > <@dpprpl:matrix.org> Was my message deleted?
17:23:31 <Adminion[m]>  * It was not.
17:23:36 <dpprpl[m]> Adminion[m]: Sorry, I guess it lagged 
17:23:37 <Adminion[m]> > <@dpprpl:matrix.org> Hi guys. Does anyone know if there is some software or service for helpdesk with matrix support or a bridge for it?
17:23:37 <Adminion[m]> > I want to propose matrix for corporate communication at my job and it would be one of the reasons to use it.
17:23:37 <Adminion[m]> > Our support uses jivo chat and it has an API, but I doubt I will get funding to develop our own bridge
17:23:37 <Adminion[m]> You can checkout [honoroit](https://gitlab.com/etke.cc/honoroit), a helpdesk bot.
17:23:37 <ilmari[m]> > <@dpprpl:matrix.org> Hi guys. Does anyone know if there is some software or service for helpdesk with matrix support or a bridge for it?
17:23:37 <ilmari[m]> > I want to propose matrix for corporate communication at my job and it would be one of the reasons to use it.
17:23:37 <dpprpl[m]> <Adminion[m]> "> <@dpprpl:matrix.org> Hi guys..." <- Tickets and also statistics about support employees work. 
17:23:37 <ilmari[m]> > Our support uses jivo chat and it has an API, but I doubt I will get funding to develop our own bridge
17:23:37 <ilmari[m]> Element just launched [Chatterbox](https://element.io/blog/element-launches-chatterbox/) which is initially just an embedded chat box for your website, but because each chat is just a Matrix room, you can integrate whatever with that. See also the [Matrix Live](https://matrix.org/blog/2022/07/08/this-week-in-matrix-2022-07-08) video.

I don't see anything weird in the original message, though:

{
  "content": {
    "body": "Hi guys. Does anyone know if there is some software or service for helpdesk with matrix support or a bridge for it?\nI want to propose matrix for corporate communication at my job and it would be one of the reasons to use it.\nOur support uses jivo chat and it has an API, but I doubt I will get funding to develop our own bridge",
    "format": "org.matrix.custom.html",
    "formatted_body": "Hi guys. Does anyone know if there is some software or service for helpdesk with matrix support or a bridge for it?<br />I want to propose matrix for corporate communication at my job and it would be one of the reasons to use it.<br />Our support uses jivo chat and it has an API, but I doubt I will get funding to develop our own bridge",
    "msgtype": "m.text"
  },
  "origin_server_ts": 1658249905612,
  "sender": "@dpprpl:matrix.org",
  "type": "m.room.message",
  "unsigned": {
    "age": 27519
  },
  "event_id": "$NhKC9OtnlfGJJi-KARnDsU0TdfNuq7B31Ydjr-fNEKE",
  "room_id": "!OGEhHVWSdvArJzumhm:matrix.org"
}

[progval]

Actually it is pretty easy to reproduce, one just needs to send "foo\nbar" as a message and reply "baz".

On the Matrix side, this gives:

{
  "content": {
    "body": "foo\nbar",
    "msgtype": "m.text",
    "org.matrix.msc1767.text": "foo\nbar"
  },
  "origin_server_ts": 1660047285973,
  "sender": "@pinkie:matrix.org",
  "type": "m.room.message",
  "unsigned": {
    "age": 119,
    "transaction_id": "m1660047285785.2"
  },
  "event_id": "$h0DORYmWAeEXWhnmvQNOJYUjm75dzi1YMWLsVqEDtjg",
  "room_id": "!RPsqPQfSxxItkiItKd:libera.chat"
}

followed by

{
  "content": {
    "body": "> <@pinkie:matrix.org> foo\n> bar\n\nbaz",
    "format": "org.matrix.custom.html",
    "formatted_body": "<mx-reply><blockquote><a href=\"https://matrix.to/#/!RPsqPQfSxxItkiItKd:libera.chat/$h0DORYmWAeEXWhnmvQNOJYUjm75dzi1YMWLsVqEDtjg?via=libera.chat&via=matrix.org\">In reply to</a> <a href=\"https://matrix.to/#/@pinkie:matrix.org\">@pinkie:matrix.org</a><br>foo<br/>bar</blockquote></mx-reply>baz",
    "m.relates_to": {
      "m.in_reply_to": {
        "event_id": "$h0DORYmWAeEXWhnmvQNOJYUjm75dzi1YMWLsVqEDtjg"
      }
    },
    "msgtype": "m.text",
    "org.matrix.msc1767.message": [
      {
        "body": "> <@pinkie:matrix.org> foo\n> bar\n\nbaz",
        "mimetype": "text/plain"
      },
      {
        "body": "<mx-reply><blockquote><a href=\"https://matrix.to/#/!RPsqPQfSxxItkiItKd:libera.chat/$h0DORYmWAeEXWhnmvQNOJYUjm75dzi1YMWLsVqEDtjg?via=libera.chat&via=matrix.org\">In reply to</a> <a href=\"https://matrix.to/#/@pinkie:matrix.org\">@pinkie:matrix.org</a><br>foo<br/>bar</blockquote></mx-reply>baz",
        "mimetype": "text/html"
      }
    ]
  },
  "origin_server_ts": 1660047289826,
  "sender": "@pinkie:matrix.org",
  "type": "m.room.message",
  "unsigned": {
    "age": 665,
    "transaction_id": "m1660047289403.3"
  },
  "event_id": "$OWp5yajpZqxr50kr7fNWgd_B6Mu_GfCZsaJh2fUFu_A",
  "room_id": "!RPsqPQfSxxItkiItKd:libera.chat"
}

and on the IRC side:

12:14:46 <val-m> foo
12:14:46 <val-m> bar
12:14:54 <val-m> > <@pinkie:matrix.org> foo
12:14:54 <val-m> > bar
12:14:54 <val-m> baz

[progval]

By the way, when the first message has two newlines, it sends this to IRC:

12:13:48 <val-m> foo
12:13:48 <val-m> bar
12:13:53 <val-m> > <@pinkie:matrix.org> foo
12:13:53 <val-m> > 
12:13:53 <val-m> > bar
12:13:53 <val-m> baz

(the empty line is omitted from the original message, but not from the fallback in the reply)

[funderscore1]

This issue is still reproducible today:

<+f_[mtrx]> foo
<+f_[mtrx]> bar
<@f_> yes?
<+f_[mtrx]> f_[mtrx]: > <@funderscore:nova.astraltech.org> foo
<+f_[mtrx]> > bar
<+f_[mtrx]> baz

On matrix side:

{
  "type": "m.room.message",
  "sender": "@funderscore:nova.astraltech.org",
  "content": {
    "msgtype": "m.text",
    "body": "foo\nbar",
    "m.mentions": {}
  },
  "origin_server_ts": 1718386798145,
  "unsigned": {
    "age": 102,
    "transaction_id": "m1718386800329.1"
  },
  "event_id": "$IKarqi1nz6lw1NqujAQJ4GQBOLjHFUx1-jQ52rliKE0",
  "room_id": "!twUYbWkeMlxafAwfQP:matrix.org"
}

followed by

{
  "type": "m.room.message",
  "sender": "@funderscore:nova.astraltech.org",
  "content": {
    "msgtype": "m.text",
    "body": "> <@funderscore:nova.astraltech.org> foo\n> bar\n\nbaz",
    "format": "org.matrix.custom.html",
    "formatted_body": "<mx-reply><blockquote><a href=\"https://matrix.to/#/!twUYbWkeMlxafAwfQP:matrix.org/$IKarqi1nz6lw1NqujAQJ4GQBOLjHFUx1-jQ52rliKE0?via=matrix.org&via=nova.astraltech.org&via=envs.net\">In reply to</a> <a href=\"https://matrix.to/#/@funderscore:nova.astraltech.org\">@funderscore:nova.astraltech.org</a><br>foo<br/>bar</blockquote></mx-reply>baz",
    "m.mentions": {},
    "m.relates_to": {
      "m.in_reply_to": {
        "event_id": "$IKarqi1nz6lw1NqujAQJ4GQBOLjHFUx1-jQ52rliKE0"
      }
    }
  },
  "origin_server_ts": 1718386824672,
  "unsigned": {
    "age": 70,
    "transaction_id": "m1718386826677.2"
  },
  "event_id": "$RbhUkeZdigw7WZXt-l5tMymmv8hvPxwzluAgp5yCXD0",
  "room_id": "!twUYbWkeMlxafAwfQP:matrix.org"
}

[funderscore1]

Hello, I did some investigation on this issue.

In textForReplyEvent, src/bridge/MatrixHandler.ts, lines 1301-1308:

        const replyText = (body: string): string => {
            const REPLY_REGEX = /> <(.*?)>(.*?)\n\n([\s\S]*)/;
            const match = REPLY_REGEX.exec(body);
            if (match === null || match.length !== 4) {
                return body;
            }
            return match[3];
        };

The REPLY_REGEX only matches this:

> <@user:matrix.org> message

reply

but doesn't match this:

> <@user:matrix.org> line1
> line2

reply

Hence, why the end result that's sent on IRC is:

<user2[m]> <user[m]> "line1..." <- > <@user:matrix.org> line1
<user2[m]> > line2
<user2[m]> reply