Skip to content

Conversation

collado-mike
Copy link
Collaborator

Problem

Part one of #1946
Implements the migration script and changes to JobDao and RunDao to support symlinked jobs.

Note: All database schema changes require discussion. Please link the issue for context.

Checklist

  • You've signed-off your work
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • You've updated the CHANGELOG.md with details about your change under the "Unreleased" section (if relevant, depending on the change, this may not be necessary)
  • You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)

@collado-mike collado-mike requested a review from wslulciuc April 12, 2022 23:49
@collado-mike collado-mike added this to the 0.22.0 milestone Apr 12, 2022
@collado-mike collado-mike changed the title Job symlink Migration script and DAO updates for job renaming and symlinking Apr 12, 2022
@codecov
Copy link

codecov bot commented Apr 12, 2022

Codecov Report

Merging #1947 (6b29b55) into main (35e0a45) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 6b29b55 differs from pull request most recent head 1cab6b7. Consider uploading reports for the commit 1cab6b7 to get more accurate results

@@            Coverage Diff            @@
##               main    #1947   +/-   ##
=========================================
  Coverage     78.09%   78.09%           
- Complexity      944      945    +1     
=========================================
  Files           193      193           
  Lines          5249     5251    +2     
  Branches        418      418           
=========================================
+ Hits           4099     4101    +2     
  Misses          705      705           
  Partials        445      445           
Impacted Files Coverage Δ
api/src/main/java/marquez/db/Columns.java 81.81% <ø> (ø)
api/src/main/java/marquez/db/OpenLineageDao.java 96.26% <ø> (ø)
api/src/main/java/marquez/db/RunDao.java 92.59% <ø> (ø)
api/src/main/java/marquez/db/models/JobRow.java 100.00% <ø> (ø)
api/src/main/java/marquez/db/JobDao.java 100.00% <100.00%> (ø)
...src/main/java/marquez/db/mappers/JobRowMapper.java 72.00% <100.00%> (+1.16%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

public static final String SCHEMA_LOCATION = "schema_location";

/* JOB ROW COLUMNS */
public static final String SYMLINK_TARGET_UUID = "symlink_target_uuid";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note, I can imagine the renaming of datasets also requiring a symlink target column.

Copy link
Member

@wslulciuc wslulciuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being able to rename jobs using symlinks has been on the roadmap for a while! Great work, @collado-mike! Also, thanks for being so thorough with testing 💯 🥇

@collado-mike collado-mike enabled auto-merge (squash) April 20, 2022 17:27
@collado-mike collado-mike merged commit 2c4a4db into main Apr 20, 2022
@collado-mike collado-mike deleted the job_symlink branch April 20, 2022 17:36
jonathanpmoraes pushed a commit to nubank/NuMarquez that referenced this pull request Feb 6, 2025
…quezProject#1947)

* Add migration script to create job symlink target

Signed-off-by: Michael Collado <[email protected]>

* Update JobDao to find job symlink targets and add tests

Signed-off-by: Michael Collado <[email protected]>

* Update RunDao to fetch runs for jobs with symlinks

Signed-off-by: Michael Collado <[email protected]>

* Update changelog

Signed-off-by: Michael Collado <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants