Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

feat: ipns locally #1400

Merged
merged 34 commits into from
Aug 29, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
1723ffd
feat: wip
vasco-santos Jun 13, 2018
70e5b0a
feat: wip
vasco-santos Jun 13, 2018
ad27451
wip
vasco-santos Jun 19, 2018
d819d10
code review fixes
vasco-santos Jun 20, 2018
8a06024
added ipns package, cli tests and logs
vasco-santos Jun 30, 2018
72dbb61
added resolve recursive parameter
vasco-santos Jul 2, 2018
d9c8836
added cache
vasco-santos Jul 3, 2018
22e1bc8
added proper logging, errors and code refactor
vasco-santos Jul 3, 2018
7e19583
added cli and http tests
vasco-santos Jul 5, 2018
7bcc648
updated tests
vasco-santos Jul 16, 2018
fab0b76
fix: updated core arguments to an options object
vasco-santos Jul 20, 2018
bc4a9ee
fix: core returns and refactor tests
vasco-santos Jul 21, 2018
7f2aa3d
fix: core with camelCase responses
vasco-santos Jul 23, 2018
fdf3b90
fix: add initializeKeyspace to init
vasco-santos Jul 24, 2018
8204f85
wip
vasco-santos Jul 27, 2018
61f4d4e
wip
vasco-santos Aug 6, 2018
08a952b
added republisher
vasco-santos Aug 7, 2018
ba7585d
chore: updated readme
vasco-santos Aug 8, 2018
6e3c435
chore: added tests
vasco-santos Aug 8, 2018
b2253e1
chore: upgrade dependency of ipns
vasco-santos Aug 9, 2018
17bcdcb
fix: lint
vasco-santos Aug 9, 2018
1523579
fix: code review
vasco-santos Aug 9, 2018
04bf892
fix: tests
vasco-santos Aug 10, 2018
0515553
fix: republish cancelable
vasco-santos Aug 10, 2018
36e51fb
chore: added tests for path
vasco-santos Aug 10, 2018
87e9d91
fix: lint
vasco-santos Aug 10, 2018
df4a276
fix: code review
vasco-santos Aug 10, 2018
49725ae
chore: refactored errors
vasco-santos Aug 10, 2018
ca25516
chore: upgrade interface-ipfs-core
vasco-santos Aug 10, 2018
23b5fbc
fix: initial rebroadcast delay
vasco-santos Aug 10, 2018
6bb6c97
fix: initial rebroadcast delay
vasco-santos Aug 10, 2018
7cd8fc5
chore: upgrade ipns dependency
vasco-santos Aug 27, 2018
84ee2c8
fix: added not implemented error for routing on ipns getPublished
vasco-santos Aug 28, 2018
77caec1
fix: republish working for starting and stopping multiple times
vasco-santos Aug 28, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ You can check the development status at the [Waffle Board](https://waffle.io/ipf
- [Core API](#core-api)
- [Files](#files)
- [Graph](#graph)
- [Name](#name)
- [Crypto and Key Management](#crypto-and-key-management)
- [Network](#network)
- [Node Management](#node-management)
Expand Down Expand Up @@ -545,6 +546,12 @@ The core API is grouped into several areas:
- [`ipfs.pin.ls([hash], [options], [callback])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/PIN.md#pinls)
- [`ipfs.pin.rm(hash, [options], [callback])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/PIN.md#pinrm)

### Name

- [name](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/NAME.md)
- [`ipfs.name.publish(value, [options, callback])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/NAME.md#namepublish)
- [`ipfs.name.resolve(value, [options, callback])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/NAME.md#nameresolve)

#### Crypto and Key Management

- [key](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/KEY.md)
Expand Down Expand Up @@ -837,6 +844,8 @@ Listing of the main packages used in the IPFS ecosystem. There are also three sp
| [`ipld`](//github.com/ipld/js-ipld) | [![npm](https://img.shields.io/npm/v/ipld.svg?maxAge=86400&style=flat)](//github.com/ipld/js-ipld/releases) | [![Dep](https://david-dm.org/ipld/js-ipld.svg?style=flat)](https://david-dm.org/ipld/js-ipld) | [![Build Status](https://ci.ipfs.team/buildStatus/icon?job=ipld/js-ipld/master)](https://ci.ipfs.team/job/ipld/job/js-ipld/job/master/) | [![Coverage Status](https://codecov.io/gh/ipld/js-ipld/branch/master/graph/badge.svg)](https://codecov.io/gh/ipld/js-ipld) |
| [`ipld-dag-pb`](//github.com/ipld/js-ipld-dag-pb) | [![npm](https://img.shields.io/npm/v/ipld-dag-pb.svg?maxAge=86400&style=flat)](//github.com/ipld/js-ipld-dag-pb/releases) | [![Dep](https://david-dm.org/ipld/js-ipld-dag-pb.svg?style=flat)](https://david-dm.org/ipld/js-ipld-dag-pb) | [![Build Status](https://ci.ipfs.team/buildStatus/icon?job=ipld/js-ipld-dag-pb/master)](https://ci.ipfs.team/job/ipld/job/js-ipld-dag-pb/job/master/) | [![Coverage Status](https://codecov.io/gh/ipld/js-ipld-dag-pb/branch/master/graph/badge.svg)](https://codecov.io/gh/ipld/js-ipld-dag-pb) |
| [`ipld-dag-cbor`](//github.com/ipld/js-ipld-dag-cbor) | [![npm](https://img.shields.io/npm/v/ipld-dag-cbor.svg?maxAge=86400&style=flat)](//github.com/ipld/js-ipld-dag-cbor/releases) | [![Dep](https://david-dm.org/ipld/js-ipld-dag-cbor.svg?style=flat)](https://david-dm.org/ipld/js-ipld-dag-cbor) | [![Build Status](https://ci.ipfs.team/buildStatus/icon?job=ipld/js-ipld-dag-cbor/master)](https://ci.ipfs.team/job/ipld/job/js-ipld-dag-cbor/job/master/) | [![Coverage Status](https://codecov.io/gh/ipld/js-ipld-dag-cbor/branch/master/graph/badge.svg)](https://codecov.io/gh/ipld/js-ipld-dag-cbor) |
| **Name** |
| [`ipns`](//github.com/ipfs/js-ipns) | [![npm](https://img.shields.io/npm/v/ipns.svg?maxAge=86400&style=flat)](//github.com/ipfs/js-ipns/releases) | [![Dep](https://david-dm.org/ipfs/js-ipns.svg?style=flat-square)](https://david-dm.org/ipfs/js-ipns) | [![Build Status](https://ci.ipfs.team/buildStatus/icon?job=ipfs/js-ipns/master)](https://ci.ipfs.team/job/ipfs/job/js-ipns/job/master/) | [![Coverage Status](https://codecov.io/gh/ipfs/js-ipns/branch/master/graph/badge.svg)](https://codecov.io/gh/ipfs/js-ipns) |
| **Repo** |
| [`ipfs-repo`](//github.com/ipfs/js-ipfs-repo) | [![npm](https://img.shields.io/npm/v/ipfs-repo.svg?maxAge=86400&style=flat)](//github.com/ipfs/js-ipfs-repo/releases) | [![Dep](https://david-dm.org/ipfs/js-ipfs-repo.svg?style=flat)](https://david-dm.org/ipfs/js-ipfs-repo) | [![Build Status](https://ci.ipfs.team/buildStatus/icon?job=ipfs/js-ipfs-repo/master)](https://ci.ipfs.team/job/ipfs/job/js-ipfs-repo/job/master/) | [![Coverage Status](https://codecov.io/gh/ipfs/js-ipfs-repo/branch/master/graph/badge.svg)](https://codecov.io/gh/ipfs/js-ipfs-repo) |
| **Exchange** |
Expand Down
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@
"ipld": "~0.17.3",
"ipld-dag-cbor": "~0.12.1",
"ipld-dag-pb": "~0.14.6",
"ipns": "~0.1.3",
"is-ipfs": "~0.4.2",
"is-pull-stream": "~0.0.0",
"is-stream": "^1.1.0",
Expand All @@ -132,6 +133,7 @@
"libp2p-keychain": "~0.3.1",
"libp2p-mdns": "~0.12.0",
"libp2p-mplex": "~0.8.0",
"libp2p-record": "~0.5.1",
"libp2p-secio": "~0.10.0",
"libp2p-tcp": "~0.12.0",
"libp2p-webrtc-star": "~0.15.3",
Expand Down Expand Up @@ -164,6 +166,7 @@
"pull-zip": "^2.0.1",
"read-pkg-up": "^4.0.0",
"readable-stream": "2.3.6",
"receptacle": "^1.3.2",
"stream-to-pull-stream": "^1.7.2",
"tar-stream": "^1.6.1",
"temp": "~0.8.3",
Expand Down
5 changes: 5 additions & 0 deletions src/cli/bin.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@ const cli = yargs
type: 'string',
default: ''
})
.option('local', {
desc: 'Run the command locally, instead of using the daemon',
type: 'boolean',
default: false
})
.epilog(utils.ipfsPathHelp)
.demandCommand(1)
.fail((msg, err, yargs) => {
Expand Down
20 changes: 20 additions & 0 deletions src/cli/commands/name.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
'use strict'

/*
IPNS is a PKI namespace, where names are the hashes of public keys, and
the private key enables publishing new (signed) values. In both publish
and resolve, the default name used is the node's own PeerID,
which is the hash of its public key.
*/
module.exports = {
command: 'name <command>',

description: 'Publish and resolve IPNS names.',

builder (yargs) {
return yargs.commandDir('name')
},

handler (argv) {
}
}
47 changes: 47 additions & 0 deletions src/cli/commands/name/publish.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
'use strict'

const print = require('../../utils').print

module.exports = {
command: 'publish <ipfsPath>',

describe: 'Publish IPNS names.',

builder: {
resolve: {
describe: 'Resolve given path before publishing. Default: true.',
default: true
},
lifetime: {
alias: 't',
describe: 'Time duration that the record will be valid for. Default: 24h.',
default: '24h'
},
key: {
alias: 'k',
describe: 'Name of the key to be used or a valid PeerID, as listed by "ipfs key list -l". Default: self.',
default: 'self'
},
ttl: {
describe: 'Time duration this record should be cached for (caution: experimental).',
default: ''
}
},

handler (argv) {
const opts = {
resolve: argv.resolve,
lifetime: argv.lifetime,
key: argv.key,
ttl: argv.ttl
}

argv.ipfs.name.publish(argv.ipfsPath, opts, (err, result) => {
if (err) {
throw err
}

print(`Published to ${result.name}: ${result.value}`)
})
}
}
41 changes: 41 additions & 0 deletions src/cli/commands/name/resolve.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
'use strict'

const print = require('../../utils').print

module.exports = {
command: 'resolve [<name>]',

describe: 'Resolve IPNS names.',

builder: {
nocache: {
alias: 'n',
describe: 'Do not use cached entries. Default: false.',
default: false
},
recursive: {
alias: 'r',
recursive: 'Resolve until the result is not an IPNS name. Default: false.',
default: false
}
},

handler (argv) {
const opts = {
nocache: argv.nocache,
recursive: argv.recursive
}

argv.ipfs.name.resolve(argv.name, opts, (err, result) => {
if (err) {
throw err
}

if (result && result.path) {
print(result.path)
} else {
print(result)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to succeed and not have a path property?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that case happens when the daemon is offline

})
}
}
1 change: 1 addition & 0 deletions src/core/components/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,4 @@ exports.key = require('./key')
exports.stats = require('./stats')
exports.mfs = require('./mfs')
exports.resolve = require('./resolve')
exports.name = require('./name')
18 changes: 12 additions & 6 deletions src/core/components/init.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ module.exports = function init (self) {
opts.log = opts.log || function () {}
const config = defaultConfig()
let privateKey

waterfall([
// Verify repo does not yet exist.
(cb) => self._repo.exists(cb),
Expand All @@ -75,14 +76,14 @@ module.exports = function init (self) {
peerId.create({ bits: opts.bits }, cb)
}
},
(keys, cb) => {
(peerId, cb) => {
self.log('identity generated')
config.Identity = {
PeerID: keys.toB58String(),
PrivKey: keys.privKey.bytes.toString('base64')
PeerID: peerId.toB58String(),
PrivKey: peerId.privKey.bytes.toString('base64')
}
privateKey = peerId.privKey
if (opts.pass) {
privateKey = keys.privKey
config.Keychain = Keychain.generateOptions()
}
opts.log('done')
Expand All @@ -102,14 +103,19 @@ module.exports = function init (self) {
cb(null, true)
}
},
// add empty unixfs dir object (go-ipfs assumes this exists)
(_, cb) => {
if (opts.emptyRepo) {
return cb(null, true)
}

const tasks = [
// add empty unixfs dir object (go-ipfs assumes this exists)
(cb) => self.object.new('unixfs-dir', cb)
(cb) => {
waterfall([
(cb) => self.object.new('unixfs-dir', cb),
(emptyDirNode, cb) => self._ipns.initializeKeyspace(privateKey, emptyDirNode.toJSON().multihash, cb)
], cb)
}
]

if (typeof addDefaultAssets === 'function') {
Expand Down
167 changes: 167 additions & 0 deletions src/core/components/name.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
'use strict'

const debug = require('debug')
const promisify = require('promisify-es6')
const waterfall = require('async/waterfall')
const parallel = require('async/parallel')
const human = require('human-to-milliseconds')
const crypto = require('libp2p-crypto')
const errcode = require('err-code')

const log = debug('jsipfs:name')
log.error = debug('jsipfs:name:error')

const utils = require('../utils')
const path = require('../ipns/path')

const keyLookup = (ipfsNode, kname, callback) => {
if (kname === 'self') {
return callback(null, ipfsNode._peerInfo.id.privKey)
}

const pass = ipfsNode._options.pass

waterfall([
(cb) => ipfsNode._keychain.exportKey(kname, pass, cb),
(pem, cb) => crypto.keys.import(pem, pass, cb)
], (err, privateKey) => {
if (err) {
log.error(err)
return callback(errcode(err, 'ERR_CANNOT_GET_KEY'))
}

return callback(null, privateKey)
})
}

module.exports = function name (self) {
return {
/**
* IPNS is a PKI namespace, where names are the hashes of public keys, and
* the private key enables publishing new (signed) values. In both publish
* and resolve, the default name used is the node's own PeerID,
* which is the hash of its public key.
*
* @param {String} value ipfs path of the object to be published.
* @param {Object} options ipfs publish options.
* @param {boolean} options.resolve resolve given path before publishing.
* @param {String} options.lifetime time duration that the record will be valid for.
This accepts durations such as "300s", "1.5h" or "2h45m". Valid time units are
"ns", "ms", "s", "m", "h". Default is 24h.
* @param {String} options.ttl time duration this record should be cached for (NOT IMPLEMENTED YET).
* This accepts durations such as "300s", "1.5h" or "2h45m". Valid time units are
"ns", "ms", "s", "m", "h" (caution: experimental).
* @param {String} options.key name of the key to be used or a valid PeerID, as listed by 'ipfs key list -l'.
* @param {function(Error)} [callback]
* @returns {Promise|void}
*/
publish: promisify((value, options, callback) => {
if (typeof options === 'function') {
callback = options
options = {}
}

options = options || {}
const resolve = !(options.resolve === false)
const lifetime = options.lifetime || '24h'
const key = options.key || 'self'

if (!self.isOnline()) {
const errMsg = utils.OFFLINE_ERROR

log.error(errMsg)
return callback(errcode(errMsg, 'OFFLINE_ERROR'))
}

// TODO: params related logic should be in the core implementation

// Normalize path value
try {
value = utils.normalizePath(value)
} catch (err) {
log.error(err)
return callback(err)
}

parallel([
(cb) => human(lifetime, cb),
// (cb) => ttl ? human(ttl, cb) : cb(),
(cb) => keyLookup(self, key, cb),
// verify if the path exists, if not, an error will stop the execution
(cb) => resolve.toString() === 'true' ? path.resolvePath(self, value, cb) : cb()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tasks can all be done in parallel

], (err, results) => {
if (err) {
log.error(err)
return callback(err)
}

// Calculate lifetime with nanoseconds precision
const pubLifetime = results[0].toFixed(6)
const privateKey = results[1]

// TODO IMPROVEMENT - Handle ttl for cache
// const ttl = results[1]
// const privateKey = results[2]

// Start publishing process
self._ipns.publish(privateKey, value, pubLifetime, callback)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be using the resolved value here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don’t understand the resolved value. The result of: path.resolvePath(self, value, cb)? If so, it returns a Dag.Node for ipfs or the result of resolving an ipns path.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g.

  1. /ipns/ipfs.io resolves to /ipfs/QmYNQJoKGNHTpPxCBPh9KkDpaExgd2duMa3aF6ytMpHdao
  2. /ipfs/Qmcxw9wCZTPm4MArAitog7amDqE2qLd6EKz2prpUvykeKG/gif/unicorn-pug.jpg resolves to
    /ipfs/QmRU85XyXTfFHRetAKSzj6jNGHUNNkrsDAqJqa5k15DvP4

Expectations:

  1. ipfs name publish --resolve=true /ipns/ipfs.io publishes /ipfs/QmYNQJoKGNHTpPxCBPh9KkDpaExgd2duMa3aF6ytMpHdao
  2. ipfs name publish --resolve=false /ipns/ipfs.io publishes /ipns/ipfs.io
  3. ipfs name publish --resolve=true /ipfs/Qmcxw9wCZTPm4MArAitog7amDqE2qLd6EKz2prpUvykeKG/gif/unicorn-pug.jpg publishes /ipfs/QmRU85XyXTfFHRetAKSzj6jNGHUNNkrsDAqJqa5k15DvP4
  4. ipfs name publish --resolve=false /ipfs/Qmcxw9wCZTPm4MArAitog7amDqE2qLd6EKz2prpUvykeKG/gif/unicorn-pug.jpg publishes /ipfs/Qmcxw9wCZTPm4MArAitog7amDqE2qLd6EKz2prpUvykeKG/gif/unicorn-pug.jpg

Are my expectations incorrect? Also, there might be a bug in go-ipfs... ipfs/kubo#5368

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does the same thing as go-ipfs does: // verify the path exists

The docs could be better though 😕

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I understand now!

})
}),

/**
* Given a key, query the DHT for its best value.
*
* @param {String} name ipns name to resolve. Defaults to your node's peerID.
* @param {Object} options ipfs resolve options.
* @param {boolean} options.nocache do not use cached entries.
* @param {boolean} options.recursive resolve until the result is not an IPNS name.
* @param {function(Error)} [callback]
* @returns {Promise|void}
*/
resolve: promisify((name, options, callback) => {
if (typeof options === 'function') {
callback = options
options = {}
}

options = options || {}
const nocache = options.nocache && options.nocache.toString() === 'true'
const recursive = options.recursive && options.recursive.toString() === 'true'

const local = true // TODO ROUTING - use self._options.local

if (!self.isOnline() && !local) {
const errMsg = utils.OFFLINE_ERROR

log.error(errMsg)
return callback(errcode(errMsg, 'OFFLINE_ERROR'))
}

// TODO: params related logic should be in the core implementation

if (local && nocache) {
const error = 'cannot specify both local and nocache'

log.error(error)
return callback(errcode(new Error(error), 'ERR_NOCACHE_AND_LOCAL'))
}

// Set node id as name for being resolved, if it is not received
if (!name) {
name = self._peerInfo.id.toB58String()
}

if (!name.startsWith('/ipns/')) {
name = `/ipns/${name}`
}

const resolveOptions = {
nocache,
recursive,
local
}

self._ipns.resolve(name, self._peerInfo.id, resolveOptions, callback)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless all of the above parameter validation is specific to the ipfs.name.resolve API, I'd move it into self._ipns.resolve.

Copy link
Member Author

@vasco-santos vasco-santos Aug 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also used here, for resolving ipns paths

})
}
}
Loading