Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADDED] ObjectStore API in jetstream package #1450

Merged
merged 4 commits into from
Jan 12, 2024

Conversation

piotrpio
Copy link
Collaborator

This PR adds ObjectStore API to jetstream package.

It is for the most part a port of object.go and object_test.go to the new package, without many changes. All object store tests moved from nats package pass in the new package (after adding context and changing structs packages)

Notable changes:

  • All interfaces and structs ported to jetstream package
  • Put() and Watch() methods use the legacy push consumers to prevent permission issues when moving to new API
  • API now utilises context.Context extensively in line with the rest of the package
  • Usage of typed errors

Signed-off-by: Piotr Piotrowski piotr@synadia.com

@piotrpio piotrpio requested a review from Jarema November 2, 2023 15:54
Copy link
Member

@Jarema Jarema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good and playes nicely with the new jetstream package.
There are few things to adresss.

objAllMetaPreTmpl = "$O.%s.M.>" // $O.<bucket>.M.> // meta stream subject
objChunksPreTmpl = "$O.%s.C.%s" // $O.<bucket>.C.<object-nuid> // chunk message subject
objMetaPreTmpl = "$O.%s.M.%s" // $O.<bucket>.M.<name-encoded> // meta message subject
objNoPending = "0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

num, instead of no?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a "0", so this means "no pending"

streamsReq := streamsRequest{
Subject: fmt.Sprintf(objAllChunksPreTmpl, "*"),
}
go func() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably should cancel the goroutine in case of ctx.Done.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of context timeout we will get canceled / deadline exceeded error on page, err := l.streamInfos(ctx, streamsReq) in line 1244 (when fetching new page) so I don't think it's necessary to add another guard here.

@piotrpio piotrpio requested a review from Jarema December 12, 2023 11:01
Copy link
Member

@Jarema Jarema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

piotrpio and others added 4 commits January 12, 2024 10:41
Signed-off-by: Piotr Piotrowski <piotr@synadia.com>
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
Signed-off-by: Piotr Piotrowski <piotr@synadia.com>
Signed-off-by: Piotr Piotrowski <piotr@synadia.com>
@piotrpio piotrpio force-pushed the add-object-store-to-jetstream-pkg branch from b306903 to ff8804c Compare January 12, 2024 09:41
Copy link
Member

@Jarema Jarema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@piotrpio piotrpio merged commit c8f56b8 into main Jan 12, 2024
1 of 2 checks passed
@piotrpio piotrpio mentioned this pull request Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants