Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

status api lists undialable peer ids #414

Closed
andrew opened this issue Aug 27, 2021 · 3 comments · Fixed by #966
Closed

status api lists undialable peer ids #414

andrew opened this issue Aug 27, 2021 · 3 comments · Fixed by #966
Assignees
@olizilla
Labels
kind/bug A bug in existing code (including security flaws) P3 Low: Not priority right now

Comments

@andrew
Copy link

andrew commented Aug 27, 2021

url: https://api.web3.storage/status/bafybeibwtnukjkvxkbh3r4gza274w6qfomquekqqf7wddezytz5csylnja

response:

{
  "cid": "bafybeibwtnukjkvxkbh3r4gza274w6qfomquekqqf7wddezytz5csylnja",
  "created": "2021-08-27T15:25:42.720612Z",
  "dagSize": 11245446,
  "pins": [
    {
      "status": "Pinned",
      "updated": "2021-08-27T15:28:54.410267Z",
      "peerId": "12D3KooWJeRQfPbiv5U2RqQ9yK3qijbNxKarKEGLMkGrfXJuZ2Bo",
      "peerName": "web3-storage-sv15-2",
      "region": "US-CA"
    },
    {
      "status": "Pinned",
      "updated": "2021-08-27T15:28:54.410267Z",
      "peerId": "12D3KooWMbibcXHwkSjgV7VZ8TMfDKi6pZvmi97P83ZwHm9LEsvV",
      "peerName": "web3-storage-dc13",
      "region": "US-DC"
    },
    {
      "status": "Pinned",
      "updated": "2021-08-27T15:28:54.410267Z",
      "peerId": "12D3KooWLWFUri36dmTkki6o9PwfQNwGb2gsHuKD5FdcwzCXYnwc",
      "peerName": "web3-storage-am6-2",
      "region": "NL"
    }
  ],
  "deals": []
}

I tried to connect to the listed peer ids using my ipfs node and all three failed with the same error:

$ ipfs swarm connect /p2p/12D3KooWMbibcXHwkSjgV7VZ8TMfDKi6pZvmi97P83ZwHm9LEsvV
Error: connect 12D3KooWMbibcXHwkSjgV7VZ8TMfDKi6pZvmi97P83ZwHm9LEsvV failure: routing: not found

The peer ids that do have the content according to the dht:

$ ipfs dht findprovs bafybeibwtnukjkvxkbh3r4gza274w6qfomquekqqf7wddezytz5csylnja
12D3KooWEDMw7oRqQkdCJbyeqS5mUmWGwTp8JJ2tjCzTkHboF6wK
12D3KooWGBWx9gyUFTVQcKMTenQMSyE2ad9m7c9fpjS4NMjoDien
12D3KooWSnniGsyAF663gvHdqhyfJMCjWJv54cGSzcPiEMAfanvU
12D3KooWNuoVEfVLJvU3jWY2zLYjGUaathsecwT19jhByjnbQvkj
12D3KooWPdXVdo7b1GUf4yWGPAtphDdqMby4bYh9q1csYniHAZKx
@andrew andrew added kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization labels Aug 27, 2021
@atopal atopal added P2 Medium: Good to have, but can wait until someone steps up and removed need/triage Needs initial labeling and prioritization labels Sep 2, 2021
@atopal atopal added P3 Low: Not priority right now and removed P2 Medium: Good to have, but can wait until someone steps up labels Nov 24, 2021
@vasco-santos
Copy link
Contributor

In this case

The PeerId here is the Cluster PeerId and not the IPFS PeerId which address can be found at https://github.com/web3-storage/web3.storage/blob/main/PEERS.

For instance, in this example 12D3KooWMbibcXHwkSjgV7VZ8TMfDKi6pZvmi97P83ZwHm9LEsvV is the cluster ID of web3-storage-dc13, but its IPFS peerId is 12D3KooWSnniGsyAF663gvHdqhyfJMCjWJv54cGSzcPiEMAfanvU

@vasco-santos vasco-santos mentioned this issue Jan 24, 2022
Closed
@vasco-santos
Copy link
Contributor

Created ipfs-cluster/ipfs-cluster#1554 to see if we can include the PeerId of the actual IPFS Node. An alternative is to perform a follow up cluster request per upload and get the list of peers of a cluster and inspect the response.

Note: This will need a small migration window as we currently store the PeerId of the Cluster in our Database and previous rows would be linked with the old one (per SQL ON CONFLICT clauses)

@olizilla olizilla self-assigned this Feb 7, 2022
olizilla added a commit that referenced this issue Feb 8, 2022
@olizilla
fix: use ipfs peerId as pin location
0b9aee5 
Cluster now provides the peerId for the underlying ipfs node in pin status response, so we fix things to use that as our pin location peer id. Previously we used the cluster peerId which should only be used for internal cluster admin. With this change users will be able to use the peerId in the /status response to connect to the ipfs node that has their stuff via `ipfs swarm connect <peerId>`

- Fix toPin to use ipfsPeerId
- Update tests to verify that status response for new content has a peerId of the ipfs node for one of the cluster nodes.
- Update ipfs-cluster client to [4.0.0](https://github.com/nftstorage/ipfs-cluster/releases/tag/v4.0.0)
- Update ipfs-cluster in docker-compose to [0.14.5-rc1](https://github.com/ipfs/ipfs-cluster/releases/tag/v0.14.5-rc1)
- Tweak test hooks to not wait for things to shut down before printing the test results overview, to reduce debugging friction.

Fixes: #414

TODO:
- [] db migration script to update PinLocation peerIds
- [] verify cluster infra is updated to >= ipfs-cluster >= 0.14.5-rc1
- [] schedule read-only maintenance

License: (Apache-2.0 AND MIT)
Signed-off-by: Oli Evans <oli@tableflip.io>
@olizilla olizilla mentioned this issue Feb 8, 2022
Merged
3 tasks
@alanshaw
Copy link
Member

Maintenance scheduled to fix this on 2022-02-24 @ 14:30 UTC

alanshaw pushed a commit that referenced this issue Feb 24, 2022
@olizilla
fix: use ipfs peerId as pin location (#966)
5ae212d 
Cluster now provides the peerId for the underlying ipfs node in pin status response, so we fix things to use that as our pin location peer id. Previously we used the cluster peerId which should only be used for internal cluster admin. With this change users will be able to use the peerId in the /status response to connect to the ipfs node that has their stuff via `ipfs swarm connect <peerId>`

- Fix toPin to use ipfsPeerId
- Update tests to verify that status response for new content has a peerId of the ipfs node for one of the cluster nodes.
- Update ipfs-cluster client to [4.0.0](https://github.com/nftstorage/ipfs-cluster/releases/tag/v4.0.0)
- Update ipfs-cluster in docker-compose to [0.14.5-rc1](https://github.com/ipfs/ipfs-cluster/releases/tag/v0.14.5-rc1)
- Tweak test hooks to not wait for things to shut down before printing the test results overview, to reduce debugging friction.

Fixes: #414

TODO:
- [] db migration script to update PinLocation peerIds
- [] verify cluster infra is updated to >= ipfs-cluster >= 0.14.5-rc1
- [] schedule read-only maintenance

License: (Apache-2.0 AND MIT)
Signed-off-by: Oli Evans <oli@tableflip.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@olizilla olizilla

Labels
kind/bug A bug in existing code (including security flaws) P3 Low: Not priority right now
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: use ipfs peerId as pin location web3-storage/web3.storage
5 participants
@andrew @olizilla @alanshaw @atopal @vasco-santos