Sprint Sep 7

[ghost]

Sprint Goals

• container.camp
• go-ipfs 0.4.0

Sprint Discussions

See https://github.com/ipfs/pm#sprint-discussion-schedule

Please take notes in a separate pad, if you can, and link it here.

Sprint Deliverables

• Add yourself below, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

@whyrusleeping

• assist in building of shp tool
• pull relevant changes out of mfs PR
• fix swarm notification failures
• go1.5 sadhack failure fix
• clean up goroutine leaks in tests
  • made some progress, not as much as i'd like
• tool to manually make bitswap requests
  • 'ipfs block get' to make a single request
  • 'ipfs bitswap unwant' to cancel
• investigate a timeout in FindProvs
• finish private-blocks PR
• rebase dev0.4.0 again
• go-peerstream bug
• fix ipns fuse tests

@diasdavid

• kad-record-store
• plug kad-record-store into distributed-record-store
• node-libp2p
• attend WebRTC WG F2F Meeting remotely
• attend container.camp
• upgrade modules to be "engines" : { "node" : "^4.0.0" },
  • add engine libp2p/website#1
  • update jsonld dependency to work with node v4 ipld/js-ipld-dag-cbor#4
  • update dependencies to work with nodev4 libp2p/interface-record-store#1
  • update deps to be node v4 compatible libp2p/js-iprs-record#1
  • upgrade deps to be node v4 libp2p/js-libp2p-kad-record-store#1
  • add engine, all good libp2p/js-libp2p-kad-routing#1
  • libp2p/js-libp2p-distributed-record-store@a991524
  • daviddias/remap-keys@7150c2b
  • daviddias/js-merkledag-store@dd3f44c
  • libp2p/js-libp2p-bootstrap@70fcf8c
  • libp2p/js-libp2p@c1e9f5c
  • libp2p/js-peer-info@5317037
  • daviddias/node-ipfs-logger@b031891
  • libp2p/js-peer-id@171a561
  • multiformats/js-multistream-select@cda1c1b
  • libp2p/js-libp2p-switch@139ac6e
  • adding engine to default to Node.js v4 ipfs-inactive/starlog#24

@lgierth

• denylist daemon implement access control lists for gateway kubo#1551 https://github.com/ipfs/gateway-dmca-denylist
• storage user for @mikolalysenko Provision resources for hosting OSM infra#84
• h.ipfs.io for access via cjdns gateway: make /ipfs,/ipns,/api and CORS work with h.ips.io infra#78
• update gateway links on website Gateway links and https:// ipfs-inactive/website#40
• gateway: caching headers
• fix CORS headers for chrome CORS returning all, instead of just one kubo#1659
• chat.ipfs.io
• HN outfall: full disks Stop keeping container logs infra#29 what happened to repo gc? infra#80
• pushed to next sprint
  • ipfs.io self-hosted blog post
  • peer discovery using cjdns WIP: discover peers from cjdns kubo#1316
  • switch-ping IPE<->cjdns https://github.com/lgierth/ipe
  • check out the ruby bindings https://github.com/Fryie/ipfs-ruby

@RichardLitt

• Check in with krl about the front end for starlog
• Take a look at the ipfs distro page and see if it can be touched up.
• Prepare a spec for an ops job
• Check out the IPFS app and see if you can get it up, running, and clear without help or a walkthrough.
• See about setting DNS records for DO programmatically.
• Take a look at node-ipfs. PR review from daviddias.

@chriscool

• import sub-lib of iptb so go doesnt whine about importing a binary kubo#1644 vendor go-sleep as iptb

@mildred

See: #30 (comment)

• write tests that can extract links and other information from the new format
• implement ipld using these tests
• If time allows, find a way to get the algorithm extract information from the protobuf format (either special case or implement a generic method)
• If time allows, find out if that is enough to plug in IPLD inside the core IPFS code.

[daviddias]

SPRINT CHECKIN

• built kad-record-store
• attended container.camp (@jbenet did a great closing talk btw!!! starship is awesome)
• attended W3C WebRTC WG Seattle Meeting
• pushed on Node.js v4.0.0 for our Node.js repos through a bunch of PR (let me know if I missed any)
  • add engine libp2p/website#1
  • update jsonld dependency to work with node v4 ipld/js-ipld-dag-cbor#4
  • update dependencies to work with nodev4 libp2p/interface-record-store#1
  • update deps to be node v4 compatible libp2p/js-iprs-record#1
  • upgrade deps to be node v4 libp2p/js-libp2p-kad-record-store#1
  • add engine, all good libp2p/js-libp2p-kad-routing#1
  • libp2p/js-libp2p-distributed-record-store@a991524
  • daviddias/remap-keys@7150c2b
  • daviddias/js-merkledag-store@dd3f44c
  • libp2p/js-libp2p-bootstrap@70fcf8c
  • libp2p/js-libp2p@c1e9f5c
  • libp2p/js-peer-info@5317037
  • daviddias/node-ipfs-logger@b031891
  • libp2p/js-peer-id@171a561
  • multiformats/js-multistream-select@cda1c1b
  • libp2p/js-libp2p-switch@139ac6e
  • adding engine to default to Node.js v4 ipfs-inactive/starlog#24

++

• Learned about a possible side-channel attack to ECDSA with secp256k1 curves, proposed a change for IPRS to mitigate that - need for clarification on how a Record is really structured specs#24 (comment) This would require me to change the DRS code, but better now than later
• I'll need to push for a new thing, a WebCrypto module that abstracts away the browser WebCryptoAPI and the crypto module in Node.js, so we can have isomorphic code (otherwise it is going to be a mess). Fedor validated the Idea :) Node.js crypto and the WebCrypto API nodejs/node#2833 (comment)
• Revisited swarm design, concluded I need to break it a lil more and follow two new interfaces
  • abstract-transport for a transport that offers a stream interface and that can be built on top of another stream too (for example, putting crypto stream on top of a plain text stream and or add things like congestion mechanism if needed for non reliable transports)
  • abstract-socket for a dial/listen interface, this will generate things like libp2p-tcp, libp2p-udp, libp2p-webrtc, etc
• people have been asking for bitswap, which I was waiting to have a libp2p all polished before shipping any code, but something we should consider is start migrating the network layer in go (even if a separate repo) to libp2p, so that when the Node.js one is ready, the Go one will be compatible from the start and bitswap can be interoperable.
• started playing with libp2p website https://github.com/diasdavid/libp2p-website

[whyrusleeping]

@whyrusleeping

• assist in building of shp tool
• pull relevant changes out of mfs PR
  • still working on branch locally, mostly done
• fix swarm notification failures
  • very stubborn and random, needs more work
• go1.5 sadhack failure fix
  • also fixed gobuilder at the same time!
• clean up goroutine leaks in tests
  • made some progress, not as much as i'd like
  • kill thirdparty/waitable for a solution that doesnt use extra goroutines
• tool to manually make bitswap requests
  • 'ipfs block get' to make a single request
  • 'ipfs bitswap unwant' to cancel (PR ready)
• investigate a timeout in FindProvs (didnt get to it, wasnt high priority)
• finish private-blocks PR (ready for CR again)
• rebase dev0.4.0 again
  • fix a few test failures that popped up after repeated rebasing
• go-peerstream bug
• fix ipns fuse tests
• flatfs put retry PR (allow put to try again if it fails due to too many open files go-datastore#29)
  • going to extract logging from go-ipfs to use here
• fix nat spam issue by caching external port mappings
• wrote a tar importer and exporter command for ipfs
• kill thirdparty/waitable

[RichardLitt]

@RichardLitt

• Check in with krl about the front end for starlog. Worked on this, some PRs and set up, have more open PRs to do. Going to wait before doing more UX things until it is a bit more mature.
• Take a look at the ipfs distro page and see if it can be touched up. Done, working on building a longer document about what I see are the pros and cons of the current page, what can be touched up
• Prepare a spec for an ops job not done yet. Currently just trying to do everything I can, need to plan a bit better and see how I can help more. At the moment I am focusing on making contributing easier for new members (based on my own issues getting this up and running), and on general READMEs and open PRs
• Check out the IPFS app and see if you can get it up, running, and clear without help or a walkthrough. Done, but would like to do it again. A lot of the problems I've run into are educational - I think we should have an overarching textbook about IPFS. For instance, I'm still not quite clear what a Merkledag thing is. My solution to this is to research everything I'm confused about and see if I can make an easy-to-understand FAQ with terms and explanations. Onerous, but I think worth it.
• See about setting DNS records for DO programmatically. Did the lion's share of this, see https://github.com/RichardLitt/set-dns-records. Currently I need access to DigitalOcean to test it, though.
• Take a look at node-ipfs. PR review from daviddias. Reworked the readme, added a contribute section. Going to take a look at some of the swarm modules soon, been busy fighting other fires.

[ghost]

Pad for Sprint Sep 14: https://etherpad.mozilla.org/CBVep5omJj

[jbenet]

@jbenet update

(sorry i failed to add it last time)

this last week:

• we saw fantastic news spreading thanks to kyledrake's awesome post. still handling lots of inbound requests, messages, discussions, etc. Great response! 👏 👏
• i was focused most of the week on finishing starship, and giving a talk at container.camp. it went great! many people were very impressed with ipfs, and asked all about it afterwards. good stuff.
• I got through some amt of CR/merging, but i'm way behind. catching up over the next few days.
• starship / container.camp
  • make dkr-shp tool
  • test dkr-shp tool
  • test with a small network
  • make demo
  • write talk
  • give talk at container.camp
• other
  • [~] catching up on PRs and discussions
  • lots of CR and discussions
  • made https://github.com/ipfs/install-go-ipfs which should simplify installation in various platforms.
• outreach:
  • container.camp
  • PR requests
  • twitters

[krl]

worked on starlog documentation, and the edit-feature, both are more or less complete.

firefox support polyfill is still not working in gateway, but firefox dev works fine. It's probably just a command line switch to make sure the polyfills are not bundled up, and some rewriting hackery to make it load it off ipfs.

i would like to flesh out the issues, and what features should be available (automatic check for updated feeds, supporting quoting), for crypto-quoting, we're blocked on ipns records. for proper management of the log, keystore blocks.

[chriscool]

ipfs/kubo#1644 was closed because it was not needed, so I worked instead on ipfs/kubo#1692

[mildred]

@mildred

last week:

• write tests that can extract links and other information from the new format (pending in Implement stripping LD directives from IPLD Node, and transforming IPLD to JSON-LD ipld/go-ipld-deprecated#7)
• implement ipld using these tests (pending in Implement stripping LD directives from IPLD Node, and transforming IPLD to JSON-LD ipld/go-ipld-deprecated#7)
• If time allows, find a way to get the algorithm extract information from the protobuf format (either special case or implement a generic method)
• If time allows, find out if that is enough to plug in IPLD inside the core IPFS code.

[ghost]

• i'm still getting distracted by all the little "can you quickly do x for me" things, need to say no more often, and let people make gh-issues for the stuff that i might do in another sprint.
• also i'm still the only deployer :) need to bug people more.
• completed
  • h.ipfs.io for access via cjdns gateway: make /ipfs,/ipns,/api and CORS work with h.ips.io infra#78
  • update gateway links on website Gateway links and https:// ipfs-inactive/website#40
  • chat.ipfs.io
  • HN outfall: full disks Stop keeping container logs infra#29 what happened to repo gc? infra#80
• work in progress
  • [~] gateway-dmca-denylist
  • [~] storage user for @mikolalysenko Provision resources for hosting OSM infra#84
• not started
  • gateway: caching headers
  • fix CORS headers for chrome https://github.com/ipfs/go-ipfs/issues/1659`O
  • ipfs.io self-hosted blog post
  • peer discovery using cjdns WIP: discover peers from cjdns kubo#1316
  • switch-ping IPE<->cjdns https://github.com/lgierth/ipe
  • check out the ruby bindings https://github.com/Fryie/ipfs-ruby

[davidar]

Didn't realise I was in a sprint until this morning (and I'm generally asleep at 3AM, btw :) so here's stuff I've done recently (mostly under ipfs/archives):

• pushed http://chat.ipfs.io live (thanks @lgierth )
  • still some outstanding issues, but it works pretty well for me (@Luzifer has also kindly tested it)
• pushed a demo of the IA book reader, and had a brief conversation with @brewsterkahle about it, which was cool :)
• pushed for closer integration between ipfs and matrix, resulting in some brainstorming from both teams
• broached the subject of using dat to push datasets onto ipfs
• struck a deal with BASE BASE ipfs-inactive/archives#3 that they will release all their data into the Creative Commons if we can help with extracting fulltext links from the metadata (if anyone wants to help with this, I'd really appreciate it)
• continued pushing OSM-on-ipfs forward
• made some notes about aggregation Aggregation --> CRDTs discussion notes#40 which seems to be one of the main sore points in getting "dynamic" websites onto ipfs on a large scale
• argued with people about IPLD and the path meta-scheme naming :)

Upcoming:

• the arXiv mirror is this close to going public, which I think will make a great demonstration of the power of ipfs in helping the Open Access movement, and hopefully enable further cool stuff to be done to highlight the benefits of Creative Commons licensing for academic papers
• continue plodding along with ipfs/archive and matrix integration, and continue giving input into how to make ipfs a great platform for people to host lots of cool stuff in a distributed way :)

[jbenet]

@davidar no worries at all, was just giving you space to talk about the awesome stuff you've been up to if you wanted to :)

—
Sent from Mailbox

On Tue, Sep 15, 2015 at 9:14 AM, David A Roberts notifications@github.com
wrote:

Didn't realise I was in a sprint until this morning (and I'm generally asleep at 3AM, btw :) so here's stuff I've done recently (mostly under ipfs/archives):

• pushed http://chat.ipfs.io live (thanks @lgierth )
  • still some outstanding issues, but it works pretty well for me (@Luzifer has also kindly tested it)
• pushed a demo of the IA book reader, and had a brief conversation with @brewsterkahle about it, which was cool :)
• pushed for closer integration between ipfs and matrix, resulting in some brainstorming from both teams
• broached the subject of using dat to push datasets onto ipfs
• made some notes about aggregation Aggregation --> CRDTs discussion notes#40 which seems to be one of the main sore points in getting "dynamic" websites onto ipfs on a large scale
• argued with people about IPLD and the path meta-scheme naming :)
  Upcoming:
• the arXiv mirror is this close to going public, which I think will make a great demonstration of the power of ipfs in helping the Open Access movement, and hopefully enable further cool stuff to be done to highlight the benefits of Creative Commons licensing for academic papers

- continue plodding along with ipfs/archive and matrix integration, and continue giving input into how to make ipfs a great platform for people to host lots of cool stuff in a distributed way :)

Reply to this email directly or view it on GitHub:
#31 (comment)

[davidar]

@jbenet I wasn't complaining, thanks for thinking of me :)

[ghost]

Who made the pad link a big heading? I like it, thanks!

[daviddias]

@lgierth aahah it was me :)