Merge pull request #950 from rustyrussell/guilt/soft-errors

Really: BOLT 1: introduce warning messages, reduce requirements to send (hard) errors

Author: TheBlueMatt

00-introduction.md

@@ -132,6 +132,20 @@ See [BOLT #11: Invoice Protocol for Lightning Payments](11-payment-encoding.md)
    * _See related: [closing transaction](#closing-transaction), [funding transaction](#funding-transaction), [penalty transaction](#penalty-transaction)_
    * _See types: [revoked commitment transaction](#revoked-commitment-transaction)_
 
+* #### *Fail the channel*:
+  * This is a forced close of the channel. Very early on (before
+  opening), this may not require any action but forgetting the
+  existence of the channel.  Usually it requires signing and
+  broadcasting the latest commitment transaction, although during
+  mutual close it can also be performed by signing and broadcasting a
+  mutual close transaction.  See [BOLT #5](05-onchain.md#failing-a-channel).
+
+* #### *Close the connection*:
+  * This means closing communication with the peer (such as closing
+  the TCP socket).  It does not imply closing any channels with the
+  peer, but does cause the discarding of uncommitted state for
+  connections with channels: see [BOLT #2](02-peer-protocol.md#message-retransmission).
+
 * #### *Final node*:
    * The final recipient of a packet that is routing a payment from an *[origin node](#origin-node)* through some number of *[hops](#hop)*. It is also the final *[receiving peer](#receiving-peer)* in a chain.
    * _See category: [node](#node)_

01-messaging.md

@@ -17,7 +17,7 @@ All data fields are unsigned big-endian unless otherwise specified.
   * [Fundamental Types](#fundamental-types)
   * [Setup Messages](#setup-messages)
     * [The `init` Message](#the-init-message)
-    * [The `error` Message](#the-error-message)
+    * [The `error` and `warning` Messages](#the-error-and-warning-messages)
   * [Control Messages](#control-messages)
     * [The `ping` and `pong` Messages](#the-ping-and-pong-messages)
   * [Appendix A: BigSize Test Vectors](#appendix-a-bigsize-test-vectors)
@@ -285,11 +285,11 @@ The receiving node:
   - upon receiving unknown _odd_ feature bits that are non-zero:
     - MUST ignore the bit.
   - upon receiving unknown _even_ feature bits that are non-zero:
-    - MUST fail the connection.
+    - MUST close the connection.
   - upon receiving `networks` containing no common chains
-    - MAY fail the connection.
+    - MAY close the connection.
   - if the feature vector does not set all known, transitive dependencies:
-    - MUST fail the connection.
+    - MUST close the connection.
 
 #### Rationale
 
@@ -306,7 +306,7 @@ support a single network, the `networks` fields avoids nodes
 erroneously believing they will receive updates about their preferred
 network, or that they can open channels.
 
-### The `error` Message
+### The `error` and `warning` Messages
 
 For simplicity of diagnosis, it's often useful to tell a peer that something is incorrect.
 
@@ -316,7 +316,11 @@ For simplicity of diagnosis, it's often useful to tell a peer that something is
    * [`u16`:`len`]
    * [`len*byte`:`data`]
 
-The 2-byte `len` field indicates the number of bytes in the immediately following field.
+1. type: 1 (`warning`)
+2. data:
+   * [`channel_id`:`channel_id`]
+   * [`u16`:`len`]
+   * [`len*byte`:`data`]
 
 #### Requirements
 
@@ -331,24 +335,31 @@ The fundee node:
     - MUST use `temporary_channel_id` in lieu of `channel_id`.
 
 A sending node:
-  - when sending `error`:
-    - MUST fail the channel referred to by the error message.
   - SHOULD send `error` for protocol violations or internal errors that make channels unusable or that make further communication unusable.
   - SHOULD send `error` with the unknown `channel_id` in reply to messages of type `32`-`255` related to unknown channels.
+  - when sending `error`:
+    - MUST fail the channel(s) referred to by the error message.
+    - MAY set `channel_id` to all zero to indicate all channels.
+  - when sending `warning`:
+    - MAY set `channel_id` to all zero if the warning is not related to a specific channel.
+  - MAY close the connection after sending.
   - MAY send an empty `data` field.
   - when failure was caused by an invalid signature check:
     - SHOULD include the raw, hex-encoded transaction in reply to a `funding_created`, `funding_signed`, `closing_signed`, or `commitment_signed` message.
-  - when `channel_id` is 0:
-    - MUST fail all channels with the receiving node.
-    - MUST close the connection.
-  - MUST set `len` equal to the length of `data`.
 
 The receiving node:
   - upon receiving `error`:
-    - MUST fail the channel referred to by the error message, if that channel is with the sending node.
-  - if no existing channel is referred to by the message:
+    - if `channel_id` is all zero:
+      - MUST fail all channels with the sending node.
+    - otherwise:
+      - MUST fail the channel referred to by `channel_id`, if that channel is with the sending node.
+  - upon receiving `warning`:
+    - SHOULD log the message for later diagnosis.
+    - MAY disconnect.
+    - MAY reconnect after some delay to retry.
+    - MAY attempt `shutdown` if permitted at this point.
+  - if no existing channel is referred to by `channel_id`:
     - MUST ignore the message.
-  - MUST truncate `len` to the remainder of the packet (if it's larger).
   - if `data` is not composed solely of printable ASCII characters (For reference: the printable character set includes byte values 32 through 126, inclusive):
     - SHOULD NOT print out `data` verbatim.
 
@@ -359,6 +370,11 @@ if the connection is simply dropped, then the peer may retry the
 connection. It's also useful to describe protocol violations for
 diagnosis, as this indicates that one peer has a bug.
 
+On the other hand, overuse of error messages has lead to
+implementations ignoring them (to avoid an otherwise expensive channel
+break), so the "warning" message was added to allow some degree of
+retry or recovery for spurious errors.
+
 It may be wise not to distinguish errors in production settings, lest
 it leak information — hence, the optional `data` field.
 
@@ -394,7 +410,7 @@ A node sending a `ping` message:
   - MUST NOT set `ignored` to sensitive data such as secrets or portions of initialized
 memory.
   - if it doesn't receive a corresponding `pong`:
-    - MAY terminate the network connection,
+    - MAY close the network connection,
       - and MUST NOT fail the channels in this case.
 
 A node sending a `pong` message:
@@ -410,7 +426,7 @@ A node receiving a `ping` message:
 
 A node receiving a `pong` message:
   - if `byteslen` does not correspond to any `ping`'s `num_pong_bytes` value it has sent:
-    - MAY fail the channels.
+    - MAY close the connection.
 
 ### Rationale
 

02-peer-protocol.md

@@ -392,7 +392,8 @@ The sender:
 
 The recipient:
   - if `signature` is incorrect OR non-compliant with LOW-S-standard rule<sup>[LOWS](https://github.com/bitcoin/bitcoin/pull/6769)</sup>:
-    - MUST fail the channel.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
 
 #### Rationale
 
@@ -438,7 +439,8 @@ The sender MUST set:
 
 The recipient:
   - if `signature` is incorrect OR non-compliant with LOW-S-standard rule<sup>[LOWS](https://github.com/bitcoin/bitcoin/pull/6769)</sup>:
-    - MUST fail the channel.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - MUST NOT broadcast the funding transaction before receipt of a valid `funding_signed`.
   - on receipt of a valid `funding_signed`:
     - SHOULD broadcast the funding transaction.
@@ -484,7 +486,7 @@ A non-funding node (fundee):
   transaction after a timeout of 2016 blocks.
 
 From the point of waiting for `funding_locked` onward, either node MAY
-fail the channel if it does not receive a required response from the
+send an `error` and fail the channel if it does not receive a required response from the
 other node after a reasonable timeout.
 
 #### Rationale
@@ -558,14 +560,15 @@ A sending node:
 
 A receiving node:
   - if it hasn't received a `funding_signed` (if it is a funder) or a `funding_created` (if it is a fundee):
-    - SHOULD fail the connection
+    - SHOULD send an `error` and fail the channel.
   - if the `scriptpubkey` is not in one of the above forms:
-    - SHOULD fail the connection.
+    - SHOULD send a `warning`.
   - if it hasn't sent a `funding_locked` yet:
     - MAY reply to a `shutdown` message with a `shutdown`
   - once there are no outstanding updates on the peer, UNLESS it has already sent a `shutdown`:
     - MUST reply to a `shutdown` message with a `shutdown`
   - if both nodes advertised the `option_upfront_shutdown_script` feature, and the receiving node received a non-zero-length `shutdown_scriptpubkey` in `open_channel` or `accept_channel`, and that `shutdown_scriptpubkey` is not equal to `scriptpubkey`:
+    - MAY send a `warning`.
     - MUST fail the connection.
 
 #### Rationale
@@ -647,7 +650,8 @@ The sending node:
 The receiving node:
   - if the `signature` is not valid for either variant of closing transaction
   specified in [BOLT #3](03-transactions.md#closing-transaction) OR non-compliant with LOW-S-standard rule<sup>[LOWS](https://github.com/bitcoin/bitcoin/pull/6769)</sup>:
-    - MUST fail the connection.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - if `fee_satoshis` is equal to its previously sent `fee_satoshis`:
     - SHOULD sign and broadcast the final closing transaction.
     - MAY close the connection.
@@ -673,7 +677,8 @@ The receiving node:
           - MUST propose a `fee_satoshis` in the overlap between received and (about-to-be) sent `fee_range`.
   - otherwise, if `fee_satoshis` is not strictly between its last-sent `fee_satoshis`
   and its previously-received `fee_satoshis`, UNLESS it has since reconnected:
-    - SHOULD fail the connection.
+    - SHOULD send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - otherwise, if the receiver agrees with the fee:
     - SHOULD reply with a `closing_signed` with the same `fee_satoshis` value.
   - otherwise:
@@ -888,6 +893,7 @@ An offering node:
   - MUST NOT offer an HTLC with a timeout deadline before its `cltv_expiry`.
   - if an HTLC which it offered is in either node's current
   commitment transaction, AND is past this timeout deadline:
+    - SHOULD send an `error` to the receiving peer (if connected).
     - MUST fail the channel.
 
 A fulfilling node:
@@ -896,6 +902,7 @@ A fulfilling node:
   - MUST fail (and not forward) an HTLC whose fulfillment deadline is already past.
   - if an HTLC it has fulfilled is in either node's current commitment
   transaction, AND is past this fulfillment deadline:
+    - SHOULD send an `error` to the offering peer (if connected).
     - MUST fail the channel.
 
 ### Adding an HTLC: `update_add_htlc`
@@ -959,19 +966,24 @@ been received). It MUST continue incrementing instead.
 
 A receiving node:
   - receiving an `amount_msat` equal to 0, OR less than its own `htlc_minimum_msat`:
-    - SHOULD fail the channel.
+    - SHOULD send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - receiving an `amount_msat` that the sending node cannot afford at the current `feerate_per_kw` (while maintaining its channel reserve and any `to_local_anchor` and `to_remote_anchor` costs):
-    - SHOULD fail the channel.
+    - SHOULD send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - if a sending node adds more than receiver `max_accepted_htlcs` HTLCs to
     its local commitment transaction, OR adds more than receiver `max_htlc_value_in_flight_msat` worth of offered HTLCs to its local commitment transaction:
-    - SHOULD fail the channel.
+    - SHOULD send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - if sending node sets `cltv_expiry` to greater or equal to 500000000:
-    - SHOULD fail the channel.
+    - SHOULD send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - MUST allow multiple HTLCs with the same `payment_hash`.
   - if the sender did not previously acknowledge the commitment of that HTLC:
     - MUST ignore a repeated `id` value after a reconnection.
   - if other `id` violations occur:
-    - MAY fail the channel.
+    - MAY send a `warning` and close the connection, or send an
+      `error` and fail the channel.
 
 The `onion_routing_packet` contains an obfuscated list of hops and instructions for each hop along the path.
 It commits to the HTLC by setting the `payment_hash` as associated data, i.e. includes the `payment_hash` in the computation of HMACs.
@@ -1057,13 +1069,16 @@ A node:
 
 A receiving node:
   - if the `id` does not correspond to an HTLC in its current commitment transaction:
-    - MUST fail the channel.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - if the `payment_preimage` value in `update_fulfill_htlc`
   doesn't SHA256 hash to the corresponding HTLC `payment_hash`:
-    - MUST fail the channel.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - if the `BADONION` bit in `failure_code` is not set for
   `update_fail_malformed_htlc`:
-    - MUST fail the channel.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - if the `sha256_of_onion` in `update_fail_malformed_htlc` doesn't match the
   onion it sent:
     - MAY retry or choose an alternate error response.
@@ -1122,12 +1137,15 @@ fee changes).
 A receiving node:
   - once all pending updates are applied:
     - if `signature` is not valid for its local commitment transaction OR non-compliant with LOW-S-standard rule <sup>[LOWS](https://github.com/bitcoin/bitcoin/pull/6769)</sup>:
-      - MUST fail the channel.
+      - MUST send a `warning` and close the connection, or send an
+        `error` and fail the channel.
     - if `num_htlcs` is not equal to the number of HTLC outputs in the local
     commitment transaction:
-      - MUST fail the channel.
+      - MUST send a `warning` and close the connection, or send an
+        `error` and fail the channel.
   - if any `htlc_signature` is not valid for the corresponding HTLC transaction OR non-compliant with LOW-S-standard rule <sup>[LOWS](https://github.com/bitcoin/bitcoin/pull/6769)</sup>:
-    - MUST fail the channel.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - MUST respond with a `revoke_and_ack` message.
 
 #### Rationale
@@ -1181,9 +1199,10 @@ A sending node:
 
 A receiving node:
   - if `per_commitment_secret` is not a valid secret key or does not generate the previous `per_commitment_point`:
-    - MUST fail the channel.
+    - MUST send an `error` and fail the channel.
   - if the `per_commitment_secret` was not generated by the protocol in [BOLT #3](03-transactions.md#per-commitment-secret-requirements):
-    - MAY fail the channel.
+    - MAY send a `warning` and close the connection, or send an
+      `error` and fail the channel.
 
 A node:
   - MUST NOT broadcast old (revoked) commitment transactions,
@@ -1225,12 +1244,15 @@ The node _not responsible_ for paying the Bitcoin fee:
 
 A receiving node:
   - if the `update_fee` is too low for timely processing, OR is unreasonably large:
-    - SHOULD fail the channel.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - if the sender is not responsible for paying the Bitcoin fee:
-    - MUST fail the channel.
+    - MUST send a `warning` and close the connection, or send an
+      `error` and fail the channel.
   - if the sender cannot afford the new fee rate on the receiving node's
   current commitment transaction:
-    - SHOULD fail the channel,
+    - SHOULD send a `warning` and close the connection, or send an
+      `error` and fail the channel.
       - but MAY delay this check until the `update_fee` is committed.
 
 #### Rationale
@@ -1357,10 +1379,10 @@ A node:
     - if `next_commitment_number` is not 1 greater than the
   commitment number of the last `commitment_signed` message the receiving
   node has sent:
-      - SHOULD fail the channel.
+      - SHOULD send an `error` and fail the channel.
     - if it has not sent `commitment_signed`, AND `next_commitment_number`
     is not equal to 1:
-      - SHOULD fail the channel.
+      - SHOULD send an `error` and fail the channel.
   - if `next_revocation_number` is equal to the commitment number of
   the last `revoke_and_ack` the receiving node sent, AND the receiving node
   hasn't already received a `closing_signed`:
@@ -1372,10 +1394,10 @@ A node:
   - otherwise:
     - if `next_revocation_number` is not equal to 1 greater than the
     commitment number of the last `revoke_and_ack` the receiving node has sent:
-      - SHOULD fail the channel.
+      - SHOULD send an `error` and fail the channel.
     - if it has not sent `revoke_and_ack`, AND `next_revocation_number`
     is not equal to 0:
-      - SHOULD fail the channel.
+      - SHOULD send an `error` and fail the channel.
 
  A receiving node:
   - if `option_static_remotekey` or `option_anchors` applies to the commitment
@@ -1387,7 +1409,7 @@ A node:
       - SHOULD fail the channel.
     - otherwise:
       - if `your_last_per_commitment_secret` does not match the expected values:
-        - SHOULD fail the channel.
+        - SHOULD send an `error` and fail the channel.
   - otherwise, if it supports `option_data_loss_protect`:
     - if `next_revocation_number` is greater than expected above, AND
     `your_last_per_commitment_secret` is correct for that
@@ -1398,7 +1420,7 @@ A node:
         should the sending node broadcast its commitment transaction on-chain.
     - otherwise (`your_last_per_commitment_secret` or `my_current_per_commitment_point`
     do not match the expected values):
-      - SHOULD fail the channel.
+      - SHOULD send an `error` and fail the channel.
 
 A node:
   - MUST NOT assume that previously-transmitted messages were lost,

03-transactions.md

@@ -484,7 +484,8 @@ contribute to fees.
 
 A node:
   - if the resulting fee rate is too low:
-    - MAY fail the channel.
+    - MAY send a `warning` and close the connection, or send an
+      `error` and fail the channel.
 
 ## Dust Limits
 
@@ -2507,4 +2508,3 @@ before subtraction of:
 ![Creative Commons License](https://i.creativecommons.org/l/by/4.0/88x31.png "License CC-BY")
 <br>
 This work is licensed under a [Creative Commons Attribution 4.0 International License](http://creativecommons.org/licenses/by/4.0/).
-

05-onchain.md

@@ -74,8 +74,8 @@ A node:
   reorganizations.
   - upon the funding transaction being spent, if the channel is NOT already
   closed:
+    - MAY send a descriptive `error`.
     - SHOULD fail the channel.
-    - MAY send a descriptive error packet.
   - SHOULD ignore invalid transactions.
 
 ## Rationale