Commit(s) pushed to ElementsProject/lightning #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
whitslack
changed the title
Repository owner
locked and limited conversation to collaborators
Oct 24, 2019
cdecker
force-pushed
the
master
branch
2 times, most recently
from
bef4422 to
af4eec7
Compare
Changelog-None Signed-off-by: Lagrang3 <[email protected]>
Disable channels with HTLC min violations so that we don't hit them twice when computing routes. Changelog-None Signed-off-by: Lagrang3 <[email protected]>
that can be useful for us in the mcf.c main loop. Changelog-None Signed-off-by: Lagrang3 <[email protected]>
by a small amount if the deliver amount is less than the requested amount by X. This step saves runtime by avoiding calling an extra MCF and it helps us solve a small percentage of cases where the only available routes have HTLCmin that is bigger than X. Changelog-None Signed-off-by: Lagrang3 <[email protected]>
We use a wrapper around the MCF solver that takes care of finding the
best linearization parameters and fixing the flow values to meet the
htlc_min and htlc_max constraints.
We have reworked the current implementation and made it a bit more
similar to renepay's version.
Out of 50000 simulated payment situations distributed accross payment
amounts of 1e2, 1e3, 1e4, 1e5 and 1e6 sats, we find that 133 failed
cases in the master branch turn to success with the current changes,
while only 3 success cases in the master are not solved by the changes.
master
+-------+------+
| S | F |
+---+-------+------+
| S | 46329 | 133 |
changes +---+-------+------+
| F | 3 | 3535 |
+---+-------+------+
Out of the 133 cases that flipped from failure to success the failed
reasons were:
122 -> "Could not find route without excessive cost"
5 -> "We couldn't quite afford it"
5 -> "Amount *msat below minimum"
1 -> tripped an HTLC min check
Changelog-None.
Signed-off-by: Lagrang3 <[email protected]>
and remove unused helper functions. Changelog-None Signed-off-by: Lagrang3 <[email protected]>
that new flows respect the HTLC min/max constraints. Changelog-None Signed-off-by: Lagrang3 <[email protected]>
make use of standard keys for project and dependeny specification - provide sources to run uv build so that it can refer local packages - using hatchling for build as is stock build option - use optional-dependencies.dev for dev-dependencies - add hatch targets for packages and includes where unclear Changelog-Update: use uv with hatchling instead of poetry
allows for namespace sharing in a virtual environment otherwise gets overriden by pyln-proto-grpc's pyln folder
Add pytest.ini with pythonpath = . configuration to ensure modules are properly discovered when running tests. Also add empty __init__.py file to make the tests directory a proper Python package.
Replace poetry with uv for managing Python dependencies and running commands across CI workflow. - Add astral-sh/setup-uv@v5 action to install uv - Replace all poetry run commands with uv run - Remove poetry-specific installation steps - Update Python setup in multiple jobs
add openssl installation and flags to install grpcio-tools
Remove unnecessary global declarations across multiple test files. This change improves code quality by eliminating redundant global statements for variables that are already accessible in their respective scopes. Add proper type annotation for fees_from_status in test_closing.py and import the required typing modules. These changes maintain the same functionality while making the code cleaner and more compliant with Python best practices.
Extract package versions from pyproject.toml directly instead of using poetry commands. Use `uv run` to execute flake8, pytest and other Python tools consistently. Add new make commands for uv builds
add documentation for release pipeline changes
update commands and usage for uv
missing libffi fails coincurve installation
Updates tests/rkls_github_canned_server.py to use markupsafe.escape instead of the deprecated flask.escape function. The flask.escape function is now deprecated and will be removed in a future Flask release, so this change ensures forward compatibility.
hatchling build requires license file but the build that coincurve 21.0.0 uses doesn’t account for that in `build_hatch.py` this was added to prevent need of `cffi` as a runtime dependency but we can probably live without it until it gets fixed. change MR: ofek/coincurve#176 reported issue: ofek/coincurve#187
Now you can grep for 'sqlite3 version' and see where we would like to update. Debian 11 (Bullseye) and Ubuntu 20.04 (Focal) ship with SQLite 3.31.1. RHEL 9 ships with 3.34.1. Fedora 38+ uses SQLite 3.40+. Unfortunately, RHEL8 ships with 3.26.0, and is still on maintenance Support (security fixes, no new features): runs until May 31, 2029. Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
This is RHEL8's version. Signed-off-by: Rusty Russell <[email protected]> Changelog-Changed: build: we now require sqlite3 version 3.26 or above (released 2018-12-01).
It was breaking a lot, due to Cargo dependencies not being published, no surprise there, we are just publishing them with the changes it is complaining about.
Run version script
Fixes error in current reproducible builds: ``` error: failed to parse lock file at: /build/Cargo.lock Caused by: lock file version `4` was found, but this version of Cargo does not understand this lock file, perhaps Cargo needs to be updated? ```
Fixes current error: ``` ERROR: Invalid requirement: 'Updating dependencies': Expected end or semicolon (after name and no valid version specifier) ```
Fixes conflict error in Release action when trying to fetch the commit and tag both together. Like ``` fatal: Cannot fetch both da7d305 and refs/tags/v25.09rc1 to refs/tags/v25.09rc1 ``` Changelog-None.
If you have run recent master, upgrade will fail with:
```
Cannot migrate account database version 18
```
The final migration is:
```
/* We used to send anchors to the wallet, but set ignored tag. Now we send
* them to external. */
{SQL("UPDATE chain_events"
" SET account_id = (SELECT id FROM accounts WHERE name = 'external')"
" WHERE account_id = (SELECT id FROM accounts WHERE name = 'wallet')"
" AND ignored = 1"), NULL},
```
Which is harmless (we do that upgrade ourselves on migration, but if it's done already that will have no effect).
Signed-off-by: Rusty Russell <[email protected]>
``` lightningd: FATAL SIGNAL 6 (version v25.09rc1-1-ga00ed81) 0x5c9e848ca050 send_backtrace common/daemon.c:33 0x5c9e848ca249 crashdump common/daemon.c:78 0x7f451664532f ??? ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 0x7f451669eb2c __pthread_kill_implementation ./nptl/pthread_kill.c:44 0x7f451669eb2c __pthread_kill_internal ./nptl/pthread_kill.c:78 0x7f451669eb2c __GI___pthread_kill ./nptl/pthread_kill.c:89 0x7f451664527d __GI_raise ../sysdeps/posix/raise.c:26 0x7f45166288fe __GI_abort ./stdlib/abort.c:79 0x5c9e84893ac3 migrate_from_account_db wallet/account_migration.c:500 0x5c9e848943f7 db_migrate wallet/db.c:1139 ... #5 0x0000555555615ac4 in migrate_from_account_db (ld=0x555555999238, db=0x55555599b158) at wallet/account_migration.c:500 500 abort(); (gdb) p ev->tag $1 = 0x555555a4fbb8 "journal_entry" ``` Signed-off-by: Rusty Russell <[email protected]>
added uv lines
replace rc1
Recent versions of urllib3 fail certificate verification if certificates lack the Authority Key Identifier or Key Usages extensions: ``` SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Missing Authority Key Identifier (_ssl.c:1032) SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: CA cert does not include key usage extension (_ssl.c:1032) ``` Luckily, rcgen offers parameters in its CertificateParams structure to add these extensions. Let's use them. Changelog-Fixed: Certificates auto-generated by grpc-plugin, rest-plugin, and wss-proxy-plugin now include the required Authority Key Identifier and Key Usages extensions.
In practice, we were too strict. Here's Phoenix paying my node via another node:
```
2025-08-25T13:57:53.311Z DEBUG 02...-chan#216: Failing HTLC because of an invalid payload (TLV 10 pos 103): cltv_expiry 911816 > payment_constraint 911721
```
We add 6 blocks, but this is supposed to be the *max* allowed. Increase it to 1008, to allow shadow padding. Here are the CLTV delays across advertized channels in the network: most are far less than this:
Count Delay
1 0
1 4
3 12
899 18
602 20
1 22
3 24
1 25
9 26
5 29
72 30
54 32
4352 34
2 35
6 36
10 37
27 38
8 39
15256 40
94 42
102 44
13 45
20 46
87 48
230 50
100 51
14 55
316 60
29 64
30 65
24 68
82 69
9 70
779 72
13 77
243 78
1 79
26793 80
1 82
6 83
1 84
18 85
1 86
9 87
16 88
101 89
2 90
11 96
113 99
9527 100
41 112
34 118
132 119
403 120
24 128
232 140
138 142
14303 144
2 145
41 150
6 160
3 172
8 174
27 180
4 190
297 200
99 210
34 216
219 222
15 240
105 288
17 300
7 336
1 360
19 400
24 420
26 432
20 450
1 480
12 500
1 720
1 850
1 1000
1 1002
1 1144
1 1192
5 1201
1 1444
1 1795
1 1900
1 2016
Signed-off-by: Rusty Russell <[email protected]>
Changelog-Changed: Protocol: Offers on nodes with only private channels are now payable (i.e. no more blinded path errors!).
Fixes: #7718
@nepet noted that Valgrind complained. Nobody really cares though? TL;DR: if channel isn't enabled, estimate isn't set. Signed-off-by: Rusty Russell <[email protected]> Changelog-None: CI only
Reported-by: grubles Signed-off-by: Rusty Russell <[email protected]>
In line channeld/channeld_wiregen.c:832 `*added+i` is not a tal object hence the instruction in common/htlc_wire.c:200 `tal_arr(ctx, struct tlv_field, 0);` crashes CLN. This is fixed by stating that added_htlc is a a varsize_type. Logs: 2025-08-16T02:25:28.640Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v25.05-200-g79b959b)V ... 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:95 (call_error) 0x54f6bc 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:169 (check_bounds) 0x54f75a 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:178 (to_tal_hdr) 0x54f782 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:193 (to_tal_hdr_or_null) 0x54f7c7 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:471 (tal_alloc_) 0x54ffe4 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:517 (tal_alloc_arr_) 0x5500c4 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: common/htlc_wire.c:200 (fromwire_len_and_tlvstream) 0x48d63d 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: common/htlc_wire.c:234 (fromwire_added_htlc) 0x48dd23 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: channeld/channeld_wiregen.c:832 (fromwire_channeld_got_commitsig) 0x4c61fa 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:2377 (peer_got_commitsig) 0x4549cb 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/channel_control.c:1552 (channel_msg) 0x4140fe 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/subd.c:560 (sd_msg_read) 0x461513 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:60 (next_plan) 0x544885 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:422 (do_plan) 0x544cea 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:439 (io_ready) 0x544d9d 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:455 (io_loop) 0x54665d 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x42d220 2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1487 (main) 0x43280f gdb inspection: 830 *added = num_added ? tal_arr(ctx, struct added_htlc, num_added) : NULL; 831 for (size_t i = 0; i < num_added; i++) 832 fromwire_added_htlc(&cursor, &plen, *added + i); (gdb) p i $3 = 1 Changelog-None: crash introduced this release. Signed-off-by: Lagrang3 <[email protected]> [ Added test, removed Changelog --RR ]
This wasn't committed, so CI is failing. Signed-off-by: Rusty Russell <[email protected]>
Sorry :( We chose to update our own few custom plugin notifications by manually creating the deprecated fields and adding the new ones, rather than having lightningd fix them up. But this didn't apply to other plugins which might issue their own notifications: in particular, this hit @daywalker90. Simply documenting this is lazy, but we're close to release and I don't expect anyone else to be affected. Reported-by: @daywalker90 Signed-off-by: Rusty Russell <[email protected]>
Prior to 23.05, we used this tag to mark onchain to-self inputs we didn't
wait for (because they were too small). This fixes migration if that happened
(and we are debating whether we should re-introduce this!).
```
lightningd: FATAL SIGNAL 6 (version v25.09rc2)
0x100c8683 send_backtrace
common/daemon.c:33
0x100c876f crashdump
common/daemon.c:78
0x7fffb2080493 ???
???:0
0x7fffb1ab0cac ???
__pthread_kill_implementation+0x1bc:0
0x7fffb1a48a5b ???
__GI_raise+0x2b:0
0x7fffb1a2a3db ???
__GI_abort+0x153:0
0x100935b7 migrate_from_account_db
wallet/account_migration.c:424
0x10093ff7 db_migrate
wallet/db.c:1139
0x10096763 db_setup
wallet/db.c:1185
0x100a1bcb wallet_new
wallet/wallet.c:223
0x1004485f main
lightningd/lightningd.c:1311
0x7fffb1a2aba3 ???
__libc_start_call_main+0x93:0
0x7fffb1a2adeb ???
__libc_start_main_alias_1+0x1ab:0
0xffffffffffffffff ???
???:0
lightningd: Died with signal 6
```
Signed-off-by: Rusty Russell <[email protected]>
Fixes: #8484
Note that this can only happen if you ran a master commit before rc1: ``` 2025-08-21T10:03:03.255Z **BROKEN** lightningd: bookkeper migration: Accessing a null column e.ignored/15 in query SELECT e.id, e.account_id, a.name, e.origin, e.tag, e.credit, e.debit, e.output_value, e.currency, e.timestamp, e.blockheight, e.utxo_txid, e.outnum, e.spending_txid, e.payment_id, e.ignored, e.stealable, e.ev_desc, e.spliced, a.closed_count, a.peer_id, a.we_opened FROM chain_events e LEFT OUTER JOIN accounts a ON e.account_id = a.id ORDER BY e.timestamp, e.id; ``` Signed-off-by: Rusty Russell <[email protected]>
If they ran off master, currency can be null: ``` 2025-08-21T10:03:04.566Z **BROKEN** lightningd: bookkeper migration: Accessing a null column e.currency/7 in query SELECT e.id, e.account_id, a.name, e.tag, e.credit, e.debit, e.fees, e.currency, e.payment_id, e.part_id, e.timestamp, e.ev_desc, e.rebalance_id FROM channel_events e LEFT OUTER JOIN accounts a ON a.id = e.account_id ORDER BY e.timestamp, e.id; ``` So allow this, but *also* check if it's a different currency and skip. This won't happen: you had to manually inject events in a different currency. Signed-off-by: Rusty Russell <[email protected]>
Lisa *told* me about this on review, and I ignored it. Fool: took an extra day to get the account.db which was triggering this so I could see the problem. Signed-off-by: Rusty Russell <[email protected]>
Now accounts are not transient, we can't use them as temporary parents: ``` lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: MEMLEAK: 0x5e3bf926ce28 lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: label=plugins/bkpr/sql.c:109:struct chain_event lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: alloc: lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: ccan/ccan/tal/tal.c:488 (tal_alloc_) lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: plugins/bkpr/sql.c:109 (chain_events) lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: plugins/bkpr/sql.c:197 (chain_events_from_sql) lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: plugins/bkpr/recorder.c:352 (find_chain_event_by_id) lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: plugins/bkpr/channelsapy.c:120 (fillin_apy_acct_details) lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: plugins/bkpr/channelsapy.c:172 (compute_channel_apys) lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: plugins/bkpr/bookkeeper.c:220 (getblockheight_done) lightningd-2 2025-08-28T02:36:35.629Z **BROKEN** plugin-bookkeeper: plugins/libplugin.c:1134 (handle_rpc_reply) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/libplugin.c:1438 (rpc_read_response_one) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/libplugin.c:1462 (rpc_conn_read_response) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/io/io.c:60 (next_plan) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/io/io.c:422 (do_plan) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/io/io.c:439 (io_ready) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/io/poll.c:455 (io_loop) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/libplugin.c:2564 (plugin_main) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/bkpr/bookkeeper.c:1547 (main) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ../csu/libc-start.c:360 (__libc_start_main_impl) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: steal: lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/tal/tal.c:559 (tal_steal_) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/bkpr/recorder.c:361 (find_chain_event_by_id) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/bkpr/channelsapy.c:120 (fillin_apy_acct_details) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/bkpr/channelsapy.c:172 (compute_channel_apys) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/bkpr/bookkeeper.c:220 (getblockheight_done) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/libplugin.c:1134 (handle_rpc_reply) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/libplugin.c:1438 (rpc_read_response_one) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/libplugin.c:1462 (rpc_conn_read_response) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/io/io.c:60 (next_plan) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/io/io.c:422 (do_plan) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/io/io.c:439 (io_ready) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ccan/ccan/io/poll.c:455 (io_loop) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/libplugin.c:2564 (plugin_main) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/bkpr/bookkeeper.c:1547 (main) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: ../csu/libc-start.c:360 (__libc_start_main_impl) lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: parents: lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/bkpr/account.c:51:struct account lightningd-2 2025-08-28T02:36:35.630Z **BROKEN** plugin-bookkeeper: plugins/bkpr/bookkeeper.c:1546:struct bkpr ``` Signed-off-by: Rusty Russell <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR exists only to generate email notifications upon pushes to
ElementsProject/lightning:master.