Skip to content

Conversation

jtuglu1
Copy link
Contributor

@jtuglu1 jtuglu1 commented Aug 12, 2025

I'm seeing 404 errors when deleting unused segments from UI:

curl '<router-url>/druid/indexer/v1/datasources/<datasource>/intervals/1000-01-01_2025-08-11' -v -X DELETE

vs this succeeds (url which is documented)

curl '<router-url>/druid/coordinator/v1/datasources/<datasource>/intervals/1000-01-01_2025-08-11' -v -X DELETE

Looks like this kill datasource handle in the UI does not call the same prefix in the docs.

Description

Prevents 404s in the UI when attempting to delete unused segments.

Release note

Fix 404 issue when deleting unused segments from UI


Key changed/added classes in this PR
  • MyFoo
  • OurBar
  • TheirBaz

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@jtuglu1 jtuglu1 force-pushed the fix-broken-kill-unused-segments-endpoint branch from 1012ade to ddae037 Compare August 12, 2025 22:09
@jtuglu1 jtuglu1 marked this pull request as ready for review August 13, 2025 00:03
@gianm
Copy link
Contributor

gianm commented Aug 13, 2025

This changed in #18172, along with a bunch of other APIs, as part of the work to move segment metadata management to the Overlord (started in #17545). The change may have been over-zealous, since it looks like this particular API did not move to the Overlord.

@gianm
Copy link
Contributor

gianm commented Aug 13, 2025

@kfaraz was it intentional that this API did not move? IMO, it's odd for this one to remain on the Coordinator, given that GET and DELETE on /druid/coordinator/v1/datasources/{datasource} (apparently its parent) did move to the Overlord.

@kfaraz
Copy link
Contributor

kfaraz commented Aug 13, 2025

@gianm , at the time of the original PR #17545 , we only moved APIs that mark segments as used or unused
as they were the only ones needed for consistent behaviour with segment metadata caching enabled on the Overlord.

The list of updated APIs was called out in the original PR as well as the docs PR #17999 .
But I must have missed it during the review of web-console changes in #18172.

Move this API?

I did have plans to eventually move this API to the Overlord too as it just calls overlordClient.runKillTask().
But I was undecided on the API path to use and some other details.

it's odd for this one to remain on the Coordinator, given that GET and DELETE on /druid/coordinator/v1/datasources/{datasource} (apparently its parent) did move to the Overlord.

  • DELETE /coordinator/datasources/{datasource}/intervals/{interval} launches a kill task
  • Whereas other DELETE /druid/coordinator/v1/datasources/... APIs mark segments as unused
  • Using DELETE method is also somewhat incorrect because a successful response here doesn't indicate
    successful deletion of any resource, just that a task was submitted to the Overlord.
  • Given that the API just translates to calling /task on the Overlord with a simple payload as below,
    do we even need this API?
{
"type": "kill",
"dataSource": "<your-datasource>",
"interval": "<your-kill-interval>"
}

Suggested next steps:

  • Mark this coordinator API as deprecated in Druid 35.
  • Delete all of the deprecated coordinator APIs in Druid 36.
  • Update web-console to call Overlord /task API with the payload suggested above.
  • This will be useful in the future, when we expose other kill task options on the web-console such as specifying versions to kill etc.
  • I don't think this warrants a patch release.
  • We can call it out as a known issue in the docs and suggest mitigation steps ("Submit a JSON task" option or enable auto-kill on the Coordinator or embedded kill on the Overlord).

@gianm , @jtuglu1 , please let me know what you think.

Copy link
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the change locally.
Approving this for now so that it can be included in 34.0.1

We can do the long term fix later.

@jtuglu1 jtuglu1 merged commit 75afb0a into apache:master Aug 14, 2025
11 checks passed
@jtuglu1 jtuglu1 added this to the 34.0.1 milestone Aug 15, 2025
@jtuglu1 jtuglu1 deleted the fix-broken-kill-unused-segments-endpoint branch August 22, 2025 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants