Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Access] Add script execution to Access API #4791

Merged
merged 29 commits into from
Oct 18, 2023

Conversation

peterargue
Copy link
Contributor

Closes: #4781

Add local script execution to access API

Comment on lines +376 to 381
var coded fvmerrors.CodedError
if fvmerrors.As(err, &coded) {
// general FVM/ledger errors
if coded.Code().IsFailure() {
return rpc.ConvertError(err, "failed to execute script", codes.Internal)
}
Copy link
Contributor Author

@peterargue peterargue Oct 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is correct for detecting a cadence error locally.

Currently, we assume any error from fvm is "InvalidArgument":

value, err := h.engine.ExecuteScriptAtBlockID(ctx, req.GetScript(), req.GetArguments(), blockID)
if err != nil {
// return code 3 as this passes the litmus test in our context
return nil, status.Errorf(codes.InvalidArgument, "failed to execute script: %v", err)
}

it seems like we could do better. I'm ok just leaving the logic the same for now though.

@peterargue peterargue force-pushed the petera/4781-integrate-local-script-exec branch from 55dea45 to c249e94 Compare October 4, 2023 18:02
@peterargue peterargue self-assigned this Oct 6, 2023
Base automatically changed from gregor/script-execution/execution-module to master October 6, 2023 16:45
@codecov-commenter
Copy link

codecov-commenter commented Oct 6, 2023

Codecov Report

Attention: 196 lines in your changes are missing coverage. Please review.

Comparison is base (d953312) 55.57% compared to head (47e2282) 55.85%.
Report is 116 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4791      +/-   ##
==========================================
+ Coverage   55.57%   55.85%   +0.27%     
==========================================
  Files         872      944      +72     
  Lines       82036    87767    +5731     
==========================================
+ Hits        45591    49019    +3428     
- Misses      32930    35058    +2128     
- Partials     3515     3690     +175     
Flag Coverage Δ
unittests 55.85% <52.54%> (+0.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
engine/access/rpc/backend/backend.go 75.54% <100.00%> (+2.17%) ⬆️
...dule/state_synchronization/indexer/indexer_core.go 87.41% <ø> (ø)
module/metrics/noop.go 0.00% <0.00%> (ø)
module/execution/scripts.go 68.33% <50.00%> (-3.60%) ⬇️
module/metrics/transaction.go 0.00% <0.00%> (ø)
engine/access/rpc/backend/script_executor.go 0.00% <0.00%> (ø)
engine/access/rpc/backend/config.go 0.00% <0.00%> (ø)
engine/access/rpc/backend/backend_scripts.go 74.66% <79.53%> (+7.42%) ⬆️
engine/access/rpc/backend/backend_accounts.go 46.78% <37.64%> (-11.11%) ⬇️

... and 85 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

if err != nil {
b.log.Error().Err(err).Msgf("failed to get account at blockID: %v", latestBlockID)
b.log.Debug().Err(err).Msgf("failed to get account at blockID: %v", sealedBlockID)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why changing to debug?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could fail for a lot of reasons that aren't errors occurring on the node, so I figure it's better to produce fewer error level messages in that case.

engine/access/rpc/backend/backend_scripts.go Show resolved Hide resolved
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to convert account message: %v", err)
if errors.Is(err, ErrDataNotAvailable) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can use convertScriptExecutionError here too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not directly since it has some additional error type handling done in convertScriptExecutionError that doesn't apply to getting accounts, but I could extract the common bits out to a separate function

module/execution/scripts.go Show resolved Hide resolved
module/execution/scripts.go Show resolved Hide resolved
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to convert account message: %v", err)
if errors.Is(err, ErrDataNotAvailable) {
return nil, status.Errorf(codes.OutOfRange, "data for block height %d is not available", height)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does it work that when the height is for past sporks, we query historical AN?

Copy link
Contributor Author

@peterargue peterargue Oct 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, it works the same as it does today, except instead of a state commitment not found error, we return data for block height x is not available. The client will need to first check if the height they are querying for is within the current spork

Co-authored-by: Gregor G. <75445744+sideninja@users.noreply.github.com>

// get the latest sealed header
latestHeader, err := b.state.Sealed().Head()
sealed, err := b.state.Sealed().Head()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just want to double-check, this state used here is the same state as used by script executor right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the script executor uses state directly, just headers. but they are both populated by the follower engine

@@ -760,7 +759,7 @@ func (builder *FlowAccessNodeBuilder) BuildExecutionSyncComponents() *FlowAccess
return nil, err
}

builder.ScriptExecutor = scripts
builder.ScriptExecutor.InitReporter(builder.ExecutionIndexer, scripts)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that we don't create the script executor until its dependencies are ready, this eliminates the case where ScriptExecutor's method is called before the indexer and scripts model are passed in.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The challenge is that we want the API to run before the checkpoint loads. So there needs to be some way for the API to check if scripts is available. I used the approach of wrapping it in engine/access/rpc/backend/script_executor.go, but am open to other approaches if you have other ideas.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 I've used the same approach for GetRegisterValues

@peterargue peterargue enabled auto-merge October 18, 2023 20:25
@peterargue peterargue added this pull request to the merge queue Oct 18, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 18, 2023
@peterargue peterargue added this pull request to the merge queue Oct 18, 2023
Merged via the queue into master with commit ac256c1 Oct 18, 2023
35 of 36 checks passed
@peterargue peterargue deleted the petera/4781-integrate-local-script-exec branch October 18, 2023 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Script Execution] Implement RPC APIs for script execution
5 participants