Skip to content

feat(catalog): Implement MemoryCatalog's table update/ commit path #1290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

DerGut
Copy link

@DerGut DerGut commented May 3, 2025

Which issue does this PR close?

Relates to #700

This is a first attempt at implementing the commit path for the MemoryCatalog. Happy about any feedback!

What changes are included in this PR?

This PR implements the MemoryCatalog::update_table method. It also adds a MetadataLocation struct to avoid some re-validations of the location format.
I'm very open to removing the MetadataLocation or moving it's addition to a separate PR.

Are these changes tested?

So far, I've included unit tests to verify

  • whether a table update that was committed via the MemoryCatalog can be observed in the updated table
  • that in the case of two competing updates, the later commit fails if it has conflicting requirements
  • that an update fails if the table isn't known to the catalog
  • parsing logic of the MetadataLocation

DerGut added 9 commits May 3, 2025 22:47
…tates

Signed-off-by: DerGut <jannik.steinmann@gmx.de>
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
@DerGut DerGut marked this pull request as ready for review May 4, 2025 14:20
@DerGut DerGut changed the title Implement MemoryCatalog's table update/ commit path feat(catalog) Implement MemoryCatalog's table update/ commit path May 5, 2025
@DerGut DerGut changed the title feat(catalog) Implement MemoryCatalog's table update/ commit path feat(catalog): Implement MemoryCatalog's table update/ commit path May 5, 2025
@jonathanc-n
Copy link
Contributor

Thanks for this pr! I do not think we should move forward with this yet before our transaction API is finished. This is a nice start though and can probably be added later and tested alongside the rest of the transaction functionality.

@jonathanc-n
Copy link
Contributor

What do you think? @liurenjie1024 @Fokko

@DerGut
Copy link
Author

DerGut commented May 12, 2025

Works for me, happy to iterate on it later if needed 👍

Is there a ticket for pending transaction API improvements? I couldn't find one.

@jonathanc-n
Copy link
Contributor

#700 This holds most of the improvements for updates, deletes, etc. You can possibly look into the different pull requests that are contributing to this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants