Storage refactor API and docstring tweaks #569
Conversation
zhilingc
requested review from
davidheryanto,
khorshuheng,
pradithya and
woop
as code owners
zhilingc
requested review from
ches
and removed request for
pradithya and
davidheryanto
There was a problem hiding this comment.
Thank you for the follow up!
I think the API is improved over the proto objects, it's simpler, and HistoricalRetrievalResult has clear and justified purpose as a value object. 👍
And the code looks nicer to boot!
@woop was right that "historical retrieval" doesn't quite roll off the tongue—more syllables and characters—but I have to say I like how it reads in code here. I think I would functionally understand this quicker as a newcomer to the code base.
About the elephant in the room question of the PR description, I guess that discussion would be better held on some other thread. It'll help me to get the big picture from the roll-up PR, now that I've seen a couple of its constituents.
| * @param error error that occurred | ||
| * @return {@link HistoricalRetrievalResult} | ||
| */ | ||
| public static HistoricalRetrievalResult errorResult(String id, Exception error) { |
There was a problem hiding this comment.
"Stutters" as they say in Go at the call site, if static imports aren't used:
var result = HistoricalRetrievalResult.errorResult("guid", err)
// vs. maybe
var result = HistoricalRetrievalResult.error("guid", err)There was a problem hiding this comment.
Actually I wonder how it'd look if you dropped Result from the class name altogether… Since it's representative of a (possibly unstarted) process to produce results, but doesn't actually directly contain the result data.
There was a problem hiding this comment.
Actually, it always represents a completed retrieval request. I'll update it to your former suggestion.
There was a problem hiding this comment.
Okay, in that case it makes sense.
I feel a little reserved though about committing to the pattern of out-of-band file-based result data as API, i.e. getFileUris(). Not sure if large data sets will ever be streamed through RPC responses, but either of these options compromise the possibility for user jobs doing retrieval and processing the results to leverage data locality in systems where that could be achieved—see discussion starting at #482 (comment)
There was a problem hiding this comment.
To be clear, I think there's nothing actionable about my prior comment for this PR, I think it's a bigger picture discussion for #567.
storage/api/src/main/java/feast/storage/api/write/WriteResult.java
Outdated
Show resolved
Hide resolved
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ches, zhilingc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
zhilingc
force-pushed
the
storage-refactor
branch
from
bd1b39e
to
5328a1b
Compare
|
The metrics namespacing commit got brought into this PR now, not sure that was intended? |
|
Crap. Forgot i rebased before :( Let me remove it |
zhilingc
force-pushed
the
storage-refactor-fixes
branch
from
75091af
to
d9f6b39
Compare
|
/retest |
|
/lgtm |
* API and docstring tweaks * Fix javadoc linting errors * Apply spotless * Fix javadoc formatting * Drop result from HistoricalRetrievalResult constructors
* API and docstring tweaks * Fix javadoc linting errors * Apply spotless * Fix javadoc formatting * Drop result from HistoricalRetrievalResult constructors
* API and docstring tweaks * Fix javadoc linting errors * Apply spotless * Fix javadoc formatting * Drop result from HistoricalRetrievalResult constructors
…567) * Add storage interfaces, basic file structure (#529) * Add storage interfaces, basic file structure * Apply spotless, add comments * Move parseResponse and isEmpty to response object * Make changes to write interface to be more beam-like * Pass feature specs to the retriever * Pass feature specs to online retriever * Add FeatureSetRequest * Add mistakenly removed TestUtil * Add mistakenly removed TestUtil * Add BigQuery storage (#546) * Add Redis storage implementation (#547) * Add Redis storage * Remove staleness check; can be checked at the service level * Remove staleness related tests * Add dependencies to top level pom * Clean up code * Change serving and ingestion to use storage API (#553) * Change serving and ingestion to use storage API * Remove extra exclusion clause * Storage refactor API and docstring tweaks (#569) * API and docstring tweaks * Fix javadoc linting errors * Apply spotless * Fix javadoc formatting * Drop result from HistoricalRetrievalResult constructors * Change pipeline to use DeadletterSink API (#586) * Add better code docs to storage refactor (#601) * Add better code documentation, make GetFeastServingInfo independent of retriever * Make getStagingLocation method of historical retriever * Apply spotless * Clean up dependencies, remove exclusions at serving (#607) * Clean up OnlineServingService code (#605) * Clean up OnlineServingService code to be more readable * Revert Metrics * Rename storage API packages to nouns
What this PR does / why we need it:
Addressing most of @ches's comments here: #529
since that PR has already been merged.
The elephant in the room is unfortunately storage-api modules having Beam dependencies, which serving should not have. The two options currently on the table are:
Does this PR introduce a user-facing change?: