Chunk explorer

[holisticode]

This PR aims at introducing the possibility to explore chunk references.

It adds a explore command for the main binary. This would print all hashes (alphabetically sorted) for a given file to STDOUT

The idea is that the command be used to generate a list of hashes.

This is all up for discussion and not meant to be opened upstream yet.

[nonsense]

@holisticode I think we should split this PR in two:

1. the explore part seems fine to me and we can probably merge soon. Isn't swarm references or swarm hashes a better name, rather than explore?
2. the chunk-explore does not belong in swarm-smoke IMO and also it is not efficient to start a new process for every single chunk. The same way explore returns a list of many references, we should have a command that checks many references, not just one, and returns structured output.

[holisticode]

@holisticode I think we should split this PR in two:

the explore part seems fine to me and we can probably merge soon. Isn't swarm references or swarm hashes a better name, rather than explore?

Agreed, so I understand you are suggesting to use references or hashes as command extension to the swarm binary right?

the chunk-explore does not belong in swarm-smoke IMO and also it is not efficient to start a new process for every single chunk.

Agree here too. I used the swarm-smoke binary as a short-cut to reuse the functionality of setting up endpoints, cluster and nodes to the query. But as we can't use ws for this currently anyway, it has no reason to stay there at all.

The same way explore returns a list of many references, we should have a command that checks many references, not just one, and returns structured output.

Also agree. Are you saying as a command extension to the swarm binary? Separate binary? What is your suggestion to query multiple nodes (all nodes in a cluster namespace for example)?

[nonsense]

@holisticode

Agreed, so I understand you are suggesting to use references or hashes as command extension to the swarm binary right?

Yes, one of these two names is better than explore IMO :)

Agree here too. I used the swarm-smoke binary as a short-cut to reuse the functionality of setting up endpoints, cluster and nodes to the query. But as we can't use ws for this currently anyway, it has no reason to stay there at all.

We should modify the swarm-smoke to take a list of hosts instead of generating them based on cluster, from, to, scheme, etc. and then we can target the ws endpoint from within the k8s cluster.

Also agree. Are you saying as a command extension to the swarm binary? Separate binary? What is your suggestion to query multiple nodes (all nodes in a cluster namespace for example)?

RPC call, like hasChunks that you can query over ws would do the trick. Then you can iterate over all ws endpoints and check all nodes.

[nonsense]

@holisticode the description says:

a `chunk-explore` command option for the `swarm-smoke` binary., which queries the `has` RPC endpoint of nodes if the locally store a certain chunk. This second command has to be refined yet and further discussed, as it is not possible currently to query RPC endpoints via `ws` at the moment for cluster deployments. Maybe a different way to query nodes .

It is possible to query cluster deployments via ws so maybe update this part when you submit upstream.

[holisticode]

Reopened upstream as ethereum/go-ethereum#19008