reconnect when handshake failed.

[vwxyzh]

When app server failed to handshake with service, it will stop connect.
But the app server will running to a strange state:

1. app server is alive
2. app server never connect to service, even issues in service are gone.
3. customer must restart the app server to reconnect with service, and in most case customer have to restart it manually.

Solutions:

1. stop app server when all server connections are broken.
   Big impact when service has some temporary issue.
2. retry to connect service for ever.

[davidfowl]

Add a comment explaining that this is exponential back off up to 5.

[davidfowl]

When would this happen?

[vwxyzh]

Handshake failed.

[davidfowl]

Sure, when does that happen?

[davidfowl]

BTW I think this change is good I just want to understand the reasoning. Is it pre-emptive or did we see issues here?

[vicancy]

@davidfowl The issue #222 (comment) here, the server connections dropped, and never came back until a new deployment. So we are thinking it might be more robust for the server side to always try connecting

[davidfowl]

This is specifically this case right:

azure-signalr/src/Microsoft.Azure.SignalR.Common/ServiceConnectionBase.cs

Line 159 in dcaffb4

// False means we got a HandshakeResponseMessage with error. Will take below actions:

When does this happen?

[vwxyzh]

And we want to return more error in handshake, e.g. reach rate limit for free tiers, and when customer upgrade it should be recover.
Currently, when customer reaching rate limit, service will disconnect the connection without any useful message (PR #236 want to send a useful message to app server), and SDK will try to reconnect service in a very high rate (delay 0~1s * 5 connection per hub * hub count).

[vwxyzh]

And I think this case:

azure-signalr/src/Microsoft.Azure.SignalR.Common/ServiceConnectionBase.cs

Lines 167 to 174 in dcaffb4

	catch (Exception ex)
	{
	Log.FailedToConnect(_logger, ex);
	await DisposeConnection();
	await Task.Delay(ReconnectInterval);
	}

should also return false, instead of reconnecting with a short delay.

For example, if my access key is wrong, SDK will try to connect service, and always get an exception, web socket never connected, and retry in a high rate.

[davidfowl]

For example, if my access key is wrong, SDK will try to connect service, and always get an exception, web socket never connected, any retry in a high rate.

I agree with most of what you're saying. I'd like to see some tests for these cases.