Skip to content

HIP-1056 Add block stream poller #10250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
Feb 6, 2025
Merged

HIP-1056 Add block stream poller #10250

merged 12 commits into from
Feb 6, 2025

Conversation

xin-hedera
Copy link
Contributor

@xin-hedera xin-hedera commented Jan 27, 2025 •
edited
Loading

Description:

This PR adds block stream poller

  • Add BlockStreamPoller
  • Disable RecordFileDownloader bean if block stream poller is enabled
  • Update configuration.md
  • Update StreamFileProvider to support downloading block stream files without listing
  • Update MetricsExecutionInterceptor to support block stream file path
  • Refactor stream file provider tests

Related issue(s):

Fixes #9895

Notes for reviewer:

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

@xin-hedera xin-hedera added this to the 0.123.0 milestone Jan 27, 2025
@xin-hedera xin-hedera self-assigned this Jan 27, 2025
@xin-hedera xin-hedera added enhancement Type: New feature downloader Area: S3 downloader labels Jan 27, 2025
@codecov Codecov
Copy link

codecov bot commented Jan 27, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 96.73913% with 3 lines in your changes missing coverage. Please review.

Project coverage is 92.21%. Comparing base (883b425) to head (b85807a).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
...rter/downloader/provider/S3StreamFileProvider.java 89.47% 1 Missing and 1 partial ⚠️
...va/com/hedera/mirror/common/domain/StreamFile.java 0.00% 1 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff              @@
##               main   #10250      +/-   ##
============================================
+ Coverage     92.18%   92.21%   +0.03%     
- Complexity     8005     8024      +19     
============================================
  Files           980      982       +2     
  Lines         33444    33502      +58     
  Branches       4218     4225       +7     
============================================
+ Hits          30830    30894      +64     
+ Misses         1612     1608       -4     
+ Partials       1002     1000       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@xin-hedera xin-hedera marked this pull request as ready for review January 28, 2025 16:30
@xin-hedera xin-hedera requested a review from a team as a code owner January 28, 2025 16:30
@xin-hedera
code smell
ea1c0ce 
Signed-off-by: Xin Li <[email protected]>
jnels124
jnels124 reviewed Jan 30, 2025
View reviewed changes
@xin-hedera
address feedback
699e255 
Signed-off-by: Xin Li <[email protected]>
@xin-hedera
match if missing
3550000 
Signed-off-by: Xin Li <[email protected]>
edwin-greene
edwin-greene previously approved these changes Jan 30, 2025
View reviewed changes
Copy link
Contributor

@edwin-greene edwin-greene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

jnels124
jnels124 previously approved these changes Jan 30, 2025
View reviewed changes
Copy link
Contributor

@jnels124 jnels124 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

steven-sheehy
steven-sheehy requested changes Jan 31, 2025
View reviewed changes
@xin-hedera xin-hedera dismissed stale reviews from jnels124 and edwin-greene via ea59d61 February 3, 2025 05:46
steven-sheehy
steven-sheehy reviewed Feb 4, 2025
View reviewed changes
@steven-sheehy steven-sheehy modified the milestones: 0.123.0, 0.124.0 Feb 4, 2025
@sonarqubecloud SonarQubeCloud
Copy link

sonarqubecloud bot commented Feb 6, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

steven-sheehy
steven-sheehy approved these changes Feb 6, 2025
View reviewed changes
Copy link
Contributor

@steven-sheehy steven-sheehy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

jnels124
jnels124 approved these changes Feb 6, 2025
View reviewed changes
Copy link
Contributor

@jnels124 jnels124 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xin-hedera xin-hedera merged commit d58b526 into main Feb 6, 2025
36 checks passed
@xin-hedera xin-hedera deleted the 9895-hip-1056-poller branch February 6, 2025 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jnels124 jnels124 jnels124 approved these changes

@steven-sheehy steven-sheehy steven-sheehy approved these changes

@edwin-greene edwin-greene Awaiting requested review from edwin-greene

Assignees

@xin-hedera xin-hedera

Labels
downloader Area: S3 downloader enhancement Type: New feature
Projects
None yet
Milestone
0.124.0
Development

Successfully merging this pull request may close these issues.

HIP-1056 Poll for block files
4 participants
@xin-hedera @jnels124 @steven-sheehy @edwin-greene