Outline client cannot use passwords with certain non-ASCII or symbolic characters

[Edisonliiii]

Describe the bug
[Client Side]
Testing clients: IOS 12.4.9 latest version Outline
MacOS Big Sur 11.0.1 latest version Outline
URL was generated via ShadowsocksX-NG 1.9.4(1), it could be successful decoded via echo '[HASH]' | ase64 --decode.

[Server Side]
ss-manager --manager-address /var/run/shadowsocks-manager.sock -c [PATH_TO_CONF]

Once I paste the URL generated by ssr-ng, it could not connect to the server.

Expected behavior
It should have connected to the serve smoothly.
If you use systemctl to initialize the shadowsocks-libev, and then generate URL via ssr-ng, and finally paste to Outline for Mac, there is no issue at all.

Screenshots

[Edisonliiii]

I think there is a certain type of conflict that ss-server and ss-manager handle the connection differently, which has been ignored by Outline. I just blindly guess, my support is just because no matter ss-server/ss-manager, ssr-ng could connect to it perfectly.

The issue was kinda bizarre. If ss-server & ss-manager use the same mechanism under the hood, then it's even more strange.

[Edisonliiii]

I know ss-manager is basically using ss-server to listen incoming requests, then why they act differently when eating incoming requests sent by Outline?

Same server, one ss run via systemctl, the other is run via ss-manager, the first is no problem, the second just can not connect.

[Edisonliiii]

Just figure this out. The problem is Outline can not decode the password with non-ASCII characters in it.

For example, if you password was something like ˜∂¶∂¶™……≤çµ, it would never be able to connect to the server side through your Outline client.

Please fix this issue.

[Edisonliiii]

Alright! Real funny, seems like could not include ? or , as well.
I tried password begin with ? and include , somewhere, and it report in terms of the source code

"server-add-invalid": "{openLine}Invalid access key.{closeLine}Enter a valid access key (ss://) to add a server.",

Please fix.

[fortuna]

@Edisonliiii Can you share a concrete example that works? If can be fake.

[ohnorobo]

I tested this by using some of the sample urls from the shadowsocksconfig tests and attempting to add them as servers in my outline android client.

Here's a comparison table

Success	Type	IP version	Password
Yes	SIP002	4	latin
Yes	SIP002	6	latin
Yes	SIP002	4	non-latin
Yes	Legacy	4	latin
No	Legacy	6	latin
No	Legacy	4	non-latin

Even the failing cases have successful parse tests in shadowsocksconfig, so there's probably some additional checking code in outline-client for legacy ss:// uris that is overzealous.

The error I'm seeing looks like this:

@Edisonliiii were you failing when trying to add the servers in your outline client at all, or were they adding correctly, but then getting an error when you clicked 'connect'?

[fortuna]

@ohnorobo What happens with the ˜∂¶∂¶™……≤çµ and one with ??

[fortuna]

The test at https://github.com/Jigsaw-Code/outline-shadowsocksconfig/blob/e491bd06e98b1b9f3c4745e8eb66c503c490745d/src/shadowsocks_config.spec.ts#L366 is broken

The example url ss://YmYtY2ZiOuWwj-a0nuS4jeihpeWkp-a0nuWQg-iLpkAxOTIuMTY4LjEwMC4xOjg4ODg#Foo%20Bar is not valid. The base64 encoding shouldn't have -.

The serialization is also broken:
https://github.com/Jigsaw-Code/outline-shadowsocksconfig/blob/e491bd06e98b1b9f3c4745e8eb66c503c490745d/src/shadowsocks_config.spec.ts#L247
It's outputting dashes.

[fortuna]

@ohnorobo I have the impression we need to use Base64.{atob,btoa}() instead of Base64.{encode,decode}():
https://www.npmjs.com/package/js-base64#decode-vs-atob-and-encode-vs-btoa-

Ignore this. btoa/atob break with multi-byte characters. We need to encode the string into bytes before usting atob/btoa, which I think encode/decode already does? But something is wrong, since we shouldn't have those dashes

[ohnorobo]

The dashes are coming from using the base64 urlsafe encoding. https://tools.ietf.org/html/rfc4648#section-5

We just switched to it here. Jigsaw-Code/outline-shadowsocksconfig@d677252

Both the urlsafe (using - and _) and non-urlsafe (using + and /) encodings can be decoded by the library we're using. See en/decoding examples here: https://github.com/dankogai/js-base64#synopsis