feat: Add reservations expiration tracking to DataAPI #1733

dmanc · 2025-07-07T22:30:05Z

Why are these changes needed?

Will change base branch to master after #1709 is merged.

Reservation expiration tracking queries a subgraph to get the latests reservations for all accounts. The metrics are updated on every scrape to the /metrics endpoint.

Sample output on Sepolia

# HELP eigenda_reservation_time_until_expiry_seconds Time until reservation expiration in seconds
# TYPE eigenda_reservation_time_until_expiry_seconds gauge
eigenda_reservation_time_until_expiry_seconds{account="0x41fa832fad553c3b92976344d44b3548517679ac"} 9.88159285152387e+08
eigenda_reservation_time_until_expiry_seconds{account="0x7ccfa1e81b8d1d85552e860a3f3f63c94bd58259"} 2.3241885152387e+07
eigenda_reservation_time_until_expiry_seconds{account="0xae2c15d8354d14c05fe38733bd33ddeb00ab71ed"} 9.222690152387e+06
eigenda_reservation_time_until_expiry_seconds{account="0xc212b67c76586979cd736ea3e9738bb7bbf01b04"} 9.242137152387e+06
eigenda_reservation_time_until_expiry_seconds{account="0xcd1161b78f01da838ce0d42ec750891ec8708f1d"} 9.242137152387e+06
eigenda_reservation_time_until_expiry_seconds{account="0xd9309b3cf1b7dbf59f53461c2a66e2783dd1766f"} 5.012028152387e+06
eigenda_reservation_time_until_expiry_seconds{account="0xe25c3dec16f989793d230dc48c17388d53b4ac6e"} 2.3241885152387e+07
# HELP eigenda_reservations_active Number of active reservations
# TYPE eigenda_reservations_active gauge
eigenda_reservations_active 7

Checks

I've made sure the tests are passing. Note that there might be a few flaky tests, in that case, please comment that they are not relevant.
I've checked the new test coverage and the coverage percentage didn't drop.
Testing Strategy
- Unit tests
- Integration tests
- This PR is not tested :(

pschork · 2025-07-07T22:40:27Z

disperser/dataapi/subgraph_client.go

@@ -97,6 +98,10 @@ type (
 		// The operatorIds of nonsigners for the batch.
 		NonSigners []string
 	}
+	Reservation struct {


Should we also throw reservation symbols in here? Would allow us to monitor total allocated capacity.

Yup I can add that

+1. How are the metrics meant to be used? I'm assuming this is purely for us internally to add grafana alerts when a reservation is about to expire?

Would we ever want to expose reservations info on the blob explorer? If so then we might as well read all the info exposed per reservation by the contract/subgraph?

First use case is for reservation expiration tracking internally but we can extend the scope to any blob explorer requirements in future PRs.

samlaf · 2025-07-08T04:06:55Z

disperser/cmd/dataapi/flags/flags.go

+		//We need the socket address of the subgraph payments api to pull the subgraph data from.
+		Usage:    "the socket address of the subgraph payments api",


is this really a socket address? aka ip:port? Or is it a full URL with http endpoint mapping to specific satsuma resource?

It's just a full URL of an http endpoint

no need to do in this PR but austin has raised this in the past and I agree, "socket address" typically mans "ip:port" pair. At least in offchain code/comments we should mention its a full URL

I think all we need to do here is change the usage text for these subgraph flags to
Usage: "the URL of the subgraph payments api", and we're golden.

disperser/cmd/dataapi/config.go

disperser/dataapi/metrics.go

samlaf · 2025-07-08T04:42:35Z

disperser/dataapi/subgraph_client.go

@@ -97,6 +98,10 @@ type (
 		// The operatorIds of nonsigners for the batch.
 		NonSigners []string
 	}
+	Reservation struct {


+1. How are the metrics meant to be used? I'm assuming this is purely for us internally to add grafana alerts when a reservation is about to expire?

Would we ever want to expose reservations info on the blob explorer? If so then we might as well read all the info exposed per reservation by the contract/subgraph?

disperser/dataapi/subgraph_client.go

samlaf · 2025-07-08T04:49:21Z

tools/ejections/cmd/main.go

@@ -77,7 +77,7 @@ func RunScan(ctx *cli.Context) error {
 	if chainState == nil {
 		return errors.New("failed to create chain state")
 	}
-	subgraphApi := subgraph.NewApi(config.SubgraphEndpoint, config.SubgraphEndpoint)
+	subgraphApi := subgraph.NewApi(config.SubgraphEndpoint, config.SubgraphEndpoint, config.SubgraphEndpoint)


This kind of shows that the NewApi config/constructor is generic for no reasons. Are there ever cases where the different subgraphs don't use the same endpoint? This looks weird that we're just passing the same endpoint 3 times. If we don't ever need the different subgraphs to have different endpoints, then prob best to consolidate them into a single endpoint field like in this config

This was weird when I saw it but seems like this specific ejection tool thing only needs the middle one but this object needs all three. Open to suggestions on refactoring but feel like it's gonna be a big change.

oh I see. Fine with just adding a comment then.

yea the ejection tool is my work. Passing duplicate urls is a hack since NewApi is an abstraction over multiple subgraph clients (ejection only needs one) and I didn't want to justify yak shaving a refactor of 2yo code assumptions built for dependency injection because some esoteric ejection reporting tool needed to call a subgraph 💀

I'm totally fine with these decisions, just think they should be documented in the code itself.

disperser/dataapi/subgraph/queries.go

disperser/dataapi/v2/reservation_collector.go

The base branch was changed.

disperser/dataapi/v2/reservation_collector.go

pschork · 2025-07-19T05:26:44Z

disperser/cmd/dataapi/flags/flags.go

+		//We need the socket address of the subgraph payments api to pull the subgraph data from.
+		Usage:    "the socket address of the subgraph payments api",


I think all we need to do here is change the usage text for these subgraph flags to
Usage: "the URL of the subgraph payments api", and we're golden.

pschork · 2025-07-19T05:45:42Z

tools/ejections/cmd/main.go

@@ -77,7 +77,7 @@ func RunScan(ctx *cli.Context) error {
 	if chainState == nil {
 		return errors.New("failed to create chain state")
 	}
-	subgraphApi := subgraph.NewApi(config.SubgraphEndpoint, config.SubgraphEndpoint)
+	subgraphApi := subgraph.NewApi(config.SubgraphEndpoint, config.SubgraphEndpoint, config.SubgraphEndpoint)


yea the ejection tool is my work. Passing duplicate urls is a hack since NewApi is an abstraction over multiple subgraph clients (ejection only needs one) and I didn't want to justify yak shaving a refactor of 2yo code assumptions built for dependency injection because some esoteric ejection reporting tool needed to call a subgraph 💀

* Initialize subgraph * Initialize subgraph * Initialize subgraph * Save payment subgraph state * fix * Add reservation collector to dataapi * Add query examples + more test coverage * Remove docker compose * Add scripts for updating abi and adds a CI check when subgraph tests run * fix flags * fix lint error * Fix rebase issues * Inject the registry * wrap errors * expose more info from reservation data * fmt * rename updateCounts to updateMetrics * Document subgraph.NewApi in ejections tool

dmanc requested review from pschork and samlaf July 7, 2025 22:30

dmanc changed the title ~~Add reservations expiration tracking to DataAPI~~ feat: Add reservations expiration tracking to DataAPI Jul 7, 2025

pschork previously approved these changes Jul 7, 2025

View reviewed changes

samlaf reviewed Jul 8, 2025

View reviewed changes

Base automatically changed from payments-subgraph to master July 8, 2025 20:09

dmanc added 11 commits July 9, 2025 15:55

Initialize subgraph

9aa0908

Initialize subgraph

36a2577

Initialize subgraph

2502dfb

Save payment subgraph state

662501a

fix

8856b8f

Add reservation collector to dataapi

234ec08

Add query examples + more test coverage

01a6556

Remove docker compose

8b6db4e

Add scripts for updating abi and adds a CI check when subgraph tests run

0884b05

fix flags

eba93d5

fix lint error

d5efc06

dmanc force-pushed the dataapi-reservations branch from 3545ebd to d5efc06 Compare July 9, 2025 23:01

Fix rebase issues

ca40476

github-advanced-security bot found potential problems Jul 9, 2025

View reviewed changes

disperser/dataapi/v2/reservation_collector.go Fixed Show fixed Hide fixed

dmanc added 6 commits July 9, 2025 16:25

Inject the registry

6aae980

Merge branch 'master' into dataapi-reservations

5134612

wrap errors

e023781

expose more info from reservation data

68084d2

fmt

fdde579

rename updateCounts to updateMetrics

65c828a

dmanc marked this pull request as ready for review July 19, 2025 01:41

dmanc requested review from samlaf and pschork July 19, 2025 01:42

samlaf previously approved these changes Jul 19, 2025

View reviewed changes

pschork previously approved these changes Jul 19, 2025

View reviewed changes

dmanc added 2 commits July 21, 2025 21:36

Document subgraph.NewApi in ejections tool

fdedee7

Merge branch 'master' into dataapi-reservations

390aaeb

dmanc dismissed stale reviews from pschork and samlaf via 390aaeb July 22, 2025 04:36

samlaf approved these changes Jul 22, 2025

View reviewed changes

dmanc merged commit 9a95396 into master Jul 22, 2025
18 of 19 checks passed

dmanc deleted the dataapi-reservations branch July 22, 2025 04:57

		//We need the socket address of the subgraph payments api to pull the subgraph data from.
		Usage: "the socket address of the subgraph payments api",

feat: Add reservations expiration tracking to DataAPI #1733

feat: Add reservations expiration tracking to DataAPI #1733

Uh oh!

Conversation

dmanc commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Checks

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmanc Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dmanc commented Jul 7, 2025 •

edited

Loading

dmanc Jul 8, 2025 •

edited

Loading