Blocks don't get finalized in light client

[h4x3rotab]

Reproduce

1. Compile the latest Substrate and node-template
2. Start a two-node local testnet and begin to produce blocks

# full node 1
./target/debug/node-template \
    --base-path /tmp/alice \
    --chain=local \
    --alice \
    --node-key 0000000000000000000000000000000000000000000000000000000000000001 \
    --validator
# full node 2
./target/debug/node-template \
    --base-path /tmp/bob \
    --bootnodes /ip4/127.0.0.1/tcp/30333/p2p/QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR \
    --chain=local \
    --bob \
    --port 30334 \
    --validator

3. Start a light node

./target/debug/node-template \
    --base-path /tmp/bob2 \
    --bootnodes /ip4/127.0.0.1/tcp/30333/p2p/QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR \
    --chain=local \
    --port 30335 --light

Expected behavior

Both the full node and the light node should sync and finalize the blocks.

Actual behavior

The light node doesn't finalize any block. A screenshot is attached. The left side is a full node and the right side is the light node.

[h4x3rotab]

Added a log at ChainSync::on_blocks_processed to print the block import result. Both the full node and the light node output similar messages (formatted):

substrate_network::protocol::sync  import result:
0xb631…62c4 => Ok(ImportedUnknown(
    58,
    ImportedAux {
        clear_justification_requests: false,
        needs_justification: false,
        bad_justification: false,
        needs_finality_proof: false,
        is_new_best: true
    }, Some(PeerId("QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR"))))

[svyatonik]

Most probably this is because justification is only generated when GRANDPA authorities set changes, and it doesn't change that fast (there are only 33 blocks on your screenshot). And the only way to finalize block on light client is to receive justification (as a part of finality proof).

Some time ago, justifications were also generated every N blocks, but it was removed, iirc.

So it is the expected behavior atm. Not sure i we can improve it. What are your thoughts, @rphmeier @andresilva ?

[h4x3rotab]

@svyatonik Does it mean I can trust all the imported block even if it's higher than the last finalized block, or I have to wait until the next authorities set change in a light client?

[svyatonik]

If by trust you mean whether the block can be reverted, then no - you can't trust. We could switch to the other fork any time before block is finalized. You could trust in what the imported header has passed the other layer of verification - consensus over authorship engine (Aura, I hope BABE in future). In that sense you can trust that the header has been generated and properly signed by the expected authority (but not the 2/3).

[h4x3rotab]

Thanks for the answer. This design introduces a notable delay when a light client user wait for the finalization of a block. I suspect the delay may be a trouble for a lot of use cases. Will there be an option to let the light client request finality proof (or block justification when there's no authorities set change) more frequently, e.g. every block?

[h4x3rotab]

According to paritytech/polkadot-sdk#571, the current implementation is intended but may be changed in the future. I'm not sure how much overhead it will add to announce block justification but at least we can provide a real use case for the need:

We are working on a blockchain-TEE (Trusted Execution Environment) hybrid system. TEE is offline. So we need to bridge the chain state to the TEE side. We must validate the imported data inside the TEE in order to make sure the block won't be reverted. So we are trying to implement a Substrate light client in TEE. We have also contacted with our friend project substraTEE (by SCS team) and I belive they have the same requirement.

[h4x3rotab]

Closing. We built our own light validation client in TEE and produce the block justification on a full node with "justification_period: 1" set.