TF Pinning Service

IPFS Pinning Service over TF Quantum-Safe Storage
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Compile
3. Usage
• Environment Variables
• Docker
• Docker Compose
4. Interacting With The Pinning Service
• Using the HTTP API
• Using the IPFS CLI
• Using the IPFS Desktop GUI app
5. API Specs
6. COMPLIANCE TEST SUITE
7. Roadmap
8. Contributing
9. License
10. Contact
11. Acknowledgments

About The Project

IPFS pinning service backed by ThreeFold decentralized grid and Quantum Safe Filesystem.

(back to top)

Built With

• Go
• IPFS
• QSFS
• Vue

(back to top)

Getting Started

Threefold pinning service is a IPFS pinning service that complies with the IPFS Pinning Service API specification. it combines the content addressing capabilities of IPFS, the decentralized, Peer To Peer Internet infrastructure of Threefold, and Threefold's distributed and ultra-secure storage backend QSFS(Quantum-Safe Storage Filesystem) in order to achieve a powerful decentralized storage solution, wrapping the power of these systems into an easy-to-use service with simple rest API.

Prerequisites

• An IPFS cluster with at least one IPFS peer.
  • For development you can spin up a local IPFS Cluster instance. setup instructions.
  • For setup IPFS and IPFS Cluster on a production environment see here
• The tfpin web service binary.
  • You can download it from releases or compile from the source code. If you need to compile from the source code, you will need also:
    • An installation of Go 1.16 or later. installation instructions
    • Git client

Compile

To compile tfpin binary from the source code, follow below instructions

1 - Clone the repository:

• Open Terminal, Change the current working directory to the location where you want the cloned directory, and type

git clone https://github.com/threefoldtech/tf-pinning-service.git

2- Compile:

• Change to ./tf-pinning-service and type

make build

Then find the compiled binary file in ./bin for current os and ./bin/linux_amd64 for linux os.

Other make scripts available.

make run        # run the web server

make build_run  # compile and run the web server

Make sure to set correctly the required environment variables. see below usage section.

(back to top)

Usage

1- set the required environment variables (see next section), then run the compiled binary

TFPIN_CLUSTER_HOSTNAME="cluster-host-name" TFPIN_CLUSTER_USERNAME="usname" TFPIN_CLUSTER_PASSWORD="password" ./tfpin

2- You will need API token to interact with the service, for development run the script below to add a token of your choice (the development of the real sign-up flow blocked on the the chain part for now):

go run ./scripts/add_test_tokens.go BestTokenEver
Token `BestTokenEver` stored successfully.

Environment Variables

Environment Variable	Description	Default Value
TFPIN_CLUSTER_HOSTNAME		127.0.0.1
TFPIN_CLUSTER_PORT		9097
TFPIN_CLUSTER_USERNAME		""
TFPIN_CLUSTER_PASSWORD		""
TFPIN_CLUSTER_REPLICA_MIN		-1 (Pin on all cluster IPFS peers)
TFPIN_CLUSTER_REPLICA_MAX		-1 (Pin on all cluster IPFS peers)
TFPIN_DB_DSN		pins.db
TFPIN_DB_LOG_LEVEL		1 (silent)
TFPIN_SERVER_ADDR		:8000
TFPIN_SERVER_LOG_LEVEL		3 (warn)
TFPIN_AUTH_HEADER_KEY		Authorization

Docker

Building Image

building binaries with version information and other metadata will improve the monitoring, logging, and debugging processes by adding identifying information to help track the builds over time. this why you should provide a value for the build arg VERSION.

VERSION=$([ ! -z "$(git tag --points-at HEAD)" ] && git tag --points-at HEAD || git rev-parse --short HEAD) # use current tag, if none, use commit hash
docker build --build-arg VERSION="$VERSION" -t abouelsaad/tfpinsvc .

Running the container

docker run -d --name tf-pin-svc --env TFPIN_CLUSTER_HOSTNAME="HOSTNAME" --env TFPIN_CLUSTER_PORT="9094" --env TFPIN_CLUSTER_USERNAME="USERNAME" --env TFPIN_CLUSTER_PASSWORD="PASSWORD" --env TFPIN_SERVER_ADDR=":80" -p 8000:80 abouelsaad/tfpinsvc

Adding test token

docker exec tf-pin-svc ./add_test_tokens BestTokenEver

Docker Compose

For development and testing, with one command you can spin up

• an IPFS cluster, and
• Threefold pinning service

which is already configured to communicate, so you just need to get/add a test token, and start interacting with the service.

docker compose up -d
docker exec tf-pin-svc ./add_test_tokens BestTokenEver

(back to top)

Interacting With The Pinning Service

To interact with the pinning service you can use any of:

• IPFS Desktop or IPFS Web UI installation instructions
• IPFS CLI installation instructions
• Any http client, like curl

See Use pinning service instructions

The threefold pinning service endpoint for all requests is https://[hostname]/api/v1/pins

Using the HTTP API

Add a pin

curl -X POST 'https://[HOSTNAME]/api/v1/pins' \
  --header 'Accept: */*' \
  --header 'Authorization: Bearer <YOUR_AUTH_TOKEN>' \
  --header 'Content-Type: application/json' \
  -d '{
  "cid": "<CID_TO_BE_PINNED>",
  "name": "PreciousData.pdf"
}'

List successful pins

curl -X GET 'https://[HOSTNAME]/api/v1/pins' \
  --header 'Accept: */*' \
  --header 'Authorization: Bearer <YOUR_AUTH_TOKEN>'

Delete a pin

curl -X DELETE 'https://[HOSTNAME]/api/v1/pins/<REQUEST_ID>' \
  --header 'Accept: */*' \
  --header 'Authorization: Bearer <YOUR_AUTH_TOKEN>'

Using the IPFS CLI

The IPFS CLI can be used to maintain pins by first adding the threefold pinning service.

ipfs pin remote service add tfpinsvc https://[HOSTNAME]/api/v1/ <YOUR_AUTH_TOKEN>

Add a pin

ipfs pin remote add --service=tfpinsvc --name=<PIN_NAME> <CID>

List successful pins

ipfs pin remote ls --service=tfpinsvc

Delete a pin

ipfs pin remote rm --service=tfpinsvc --cid=<CID>

Using the IPFS Desktop GUI app

see here

(back to top)

API Specs

Threefold pinning service is compatible with the IPFS Pinning Service API (1.0.0) OpenAPI spec. see here

(back to top)

COMPLIANCE TEST SUITE

This tool should be used as compliance test suite for IPFS Pinning Service API Spec. https://github.com/ipfs-shipyard/pinning-service-compliance

(back to top)

Roadmap

• simple ipfs pining service server compatible with this OpenAPI spec (https://github.com/ipfs/pinning-services-api-spec).
• registration using smart contract
• payment
• Use QSFS as blocks storage layer

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the Apache License. See LICENSE for more information.

(back to top)

Contact

Threefold - @threefold_io

Project Link: https://github.com/threefoldtech/tf-pinning-service

(back to top)

Acknowledgments

• Pinning services api spec
• Work with pinning services
• Threefold quantum safe filesystem

(back to top)