Skip to content

Add job tagging to API #2774

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
May 7, 2024
Merged

Add job tagging to API #2774

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0d44966
add job tagging to API
davidsharp7 Mar 17, 2024
d114c3f
fix various tests that were failing
davidsharp7 Mar 19, 2024
fd76893
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 Mar 19, 2024
655bf3a
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 Mar 21, 2024
02e7484
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 Apr 5, 2024
58f40cf
Merge branch 'main' into api/add_job_tags
davidsharp7 Apr 17, 2024
1d3f819
fix merge commit
davidsharp7 Apr 17, 2024
057bf14
fix liniting
davidsharp7 Apr 17, 2024
4d1550a
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 Apr 17, 2024
da04b41
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 Apr 18, 2024
3f429f9
Merge branch 'main' into api/add_job_tags
davidsharp7 Apr 23, 2024
911549b
update based on pr feedback
davidsharp7 Apr 25, 2024
2750d66
Merge branch 'api/add_job_tags' of https://github.com/davidsharp7/mar…
davidsharp7 Apr 25, 2024
2b8ae9a
lint tag test code
davidsharp7 Apr 25, 2024
b73c6b3
lint tag code
davidsharp7 Apr 25, 2024
ca332d6
fix log typo
davidsharp7 Apr 25, 2024
322ed34
fix logging
davidsharp7 Apr 26, 2024
d127354
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 Apr 26, 2024
a64c2f6
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 Apr 28, 2024
4517062
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 Apr 30, 2024
f726170
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 May 2, 2024
4065cb5
Merge branch 'MarquezProject:main' into api/add_job_tags
davidsharp7 May 7, 2024
4c18168
update based on PR feedback.
davidsharp7 May 7, 2024
10703c1
correct db field error
davidsharp7 May 7, 2024
bf6621e
Merge branch 'main' into api/add_job_tags
wslulciuc May 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions api/src/main/java/marquez/api/DatasetResource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@ public Response tagField(
throwIfNotExists(namespaceName, datasetName);
throwIfNotExists(namespaceName, datasetName, fieldName);
log.info(
"Tagging field '{}' for dataset '{}' with '{}'.",
"Tagging field '{}' on dataset '{}' with '{}'.",
fieldName.getValue(),
datasetName.getValue(),
tagName.getValue());
Expand Down Expand Up @@ -270,7 +270,7 @@ public Response deleteTagField(
throwIfNotExists(namespaceName, datasetName);
throwIfNotExists(namespaceName, datasetName, fieldName);
log.info(
"Deleting Tag '{}' from field '{}' on dataset '{}' in namepspace '{}'.",
"Deleting Tag '{}' from field '{}' on dataset '{}' in namespace '{}'.",
tagName.getValue(),
fieldName.getValue(),
datasetName.getValue(),
Expand Down
42 changes: 42 additions & 0 deletions api/src/main/java/marquez/api/JobResource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
import marquez.common.models.JobName;
import marquez.common.models.NamespaceName;
import marquez.common.models.RunId;
import marquez.common.models.TagName;
import marquez.common.models.Version;
import marquez.db.JobFacetsDao;
import marquez.db.JobVersionDao;
Expand Down Expand Up @@ -273,6 +274,47 @@ public Response getRunFacets(
return Response.ok(facets).build();
}

@Timed
@ResponseMetered
@ExceptionMetered
@POST
@Path("/namespaces/{namespace}/jobs/{job}/tags/{tag}")
@Produces(APPLICATION_JSON)
public Response updatetag(
@PathParam("namespace") NamespaceName namespaceName,
@PathParam("job") JobName jobName,
@PathParam("tag") TagName tagName) {
throwIfNotExists(namespaceName);
throwIfNotExists(namespaceName, jobName);

jobService.updateJobTags(namespaceName.getValue(), jobName.getValue(), tagName.getValue());
Job job =
jobService
.findJobByName(namespaceName.getValue(), jobName.getValue())
.orElseThrow(() -> new JobNotFoundException(jobName));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this should be quite unlikely given updateJobTags would presumably throw if the job didn't exist? A case of, we get an optional back so we should do something beyond an unqualified get() on it?

Copy link
Member Author

@davidsharp7 davidsharp7 Apr 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah understand what you mean -it will fall over before it gets to this point so why bother? Mainly as that seems to be the de-facto pattern for a lot of the code i.e

execute something -> retrieve object (job, dataset etc) else throw an error.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following our code style!

return Response.ok(job).build();
}

@ResponseMetered
@ExceptionMetered
@DELETE
@Path("/namespaces/{namespace}/jobs/{job}/tags/{tag}")
@Produces(APPLICATION_JSON)
public Response deletetag(
@PathParam("namespace") NamespaceName namespaceName,
@PathParam("job") JobName jobName,
@PathParam("tag") TagName tagName) {
throwIfNotExists(namespaceName);
throwIfNotExists(namespaceName, jobName);

jobService.deleteJobTags(namespaceName.getValue(), jobName.getValue(), tagName.getValue());
Job job =
jobService
.findJobByName(namespaceName.getValue(), jobName.getValue())
.orElseThrow(() -> new JobNotFoundException(jobName));
return Response.ok(job).build();
}

@Value
static class JobVersions {
@NonNull
Expand Down
Loading