Use IPFS to host a Blog (or similar type of website)

[daviddias]

I'm specially interested in figure out a way (or ways) that we can start hosting Blog type of webpages on IPFS, without having to create a new framework for blogging, meaning that I'm focusing on statically generated blogs or blogs that compiled to a static generated version.

Even though a statically generated blog is a static website, it has some extra challenges to the normal static webpage hosting, simply because of the expectations that we have from a blog nowadays, such as searching for post tags or blog posts by date.

I'm writing this issue to put the ideas and discussions we've been having about it in one place, so that it also easier to see the progress.

These statically generated blog should be accessible and correctly rendered if loaded from a ipfs.io/ipfs/hash, ipfs.io/ipns/hash or somedomain.com/. It is important to this in mind, because the the fact that we might have some basePath means that loading assets (css, fonts, images) on a webpage through root path ('/'), becomes impossible. What this means is that we need a way to make the urls of the assets of a web page to be loaded through a path that is relative to itself ('../css' or '../../img/collection1' for eg).

Some options:

• precompile the whole blog
  • use harp to compile the blog. Checking if it is possible if it can handle the resolving of the basePath basePath kennethormandy/hb-casper#3
• have a native web app to do the blog generation
  • make this native webapp be a IPFS hosted WebComponent that loads posts and styling from somewhere else
  • make harp to be a browser app?
• leverage the MerkleDAG to create a more interesting data structure for the blog
  • https://github.com/ipfs/starlog

Some more context:

• Handling multiple paths for the same content ipfs-inactive/starlog#31

[edrex]

The only way to allow serving the same site tree under each of ipfs.io/ipfs/hash, ipfs.io/ipns/hash, and somedomain.com/ is to teach the build tool to use relative URLs for all <a>, <link>, <style>, <script>, etc tags. This includes rewriting intralinks in the post text if the text could be viewed at some other path such as an index page.

Setting the base tag and using absolute paths each require knowing at publish time what the base path will be.

I use Harp and I hit this when I set up the IPNS gateway proxy thing a couple of months ago. I'll report here if I find a way to do relative paths nicely with Harp. There are a ton of issues about that on the Harp tracker.

[edrex]

Conversion of absolute paths (path-absolute in the URI RFC) to relative could be done as a post-processing step, provided as a subcommand or flag of some standard IPFS static site publishing tool (diasdavid/ipscend).

An optional list of prefix patterns (^\/ by default) would allow converting sites with interlinks

• including the host: https://mysite.net/style.css
• or with a non-root base URL: /mysite/style.css

A quick search, https://github.com/tactivos/grunt-cdn does something similar.

[ghost]

The gateway's directory index currently uses relative paths (it's all a bit hairy)

[jbshirk]

https://github.com/Jermolene/TiddlyWiki5 (built on Node.js) should work well. Here is the default (documentation) site:

http://127.0.0.1:8080/ipns/Qme67efGcE9iX6r8Wk6CrHUsmWckLbnMHQ19g5YK15doUY/tw5.htm

This of course is read-only now, but for a single user it should be straightforward to deploy and update on IPNS. For multiple users, people can fork the site from Github, staging edits to be pulled and merged by the owner.

Since TW5 can use subfolders (wiki folders) this would be a better way to organize as well as have multiple subsites with different themes, etc. Even with a single page app, merges should work fine, as "tiddlers" (pages) are stored as div elements). Tiddler edit versions remain in the html app. Of course there are also categories, tags, and all kinds of useful plugins.

In my first experiment, I forked the whole TW5 repository, which is everything needed to build from NPM, and this offers many other capabilities, such as loading individual tiddlers, html, .md, and JSON as separate files, as well as updating TW5 itself. I deleted that fork, as I am nowhere near ready to deal with all that; I just wanted to see the site load from IPFS, and it does!

On a related question, I was wondering if using FUSE (such as MacFUSE) would automate the process of
ipfs add -r <sitedir>
ipfs pin add -r <sitehash>
ipfs name publish <sitehash>

My understanding is that currently IPFS FUSE is read-only. But is this where it is going?

[RichardLitt]

The only way to allow serving the same site tree under each of ipfs.io/ipfs/hash, ipfs.io/ipns/hash, and somedomain.com/ is to teach the build tool to use relative URLs for all <a>, <link>, <style>, <script>, etc tags. This includes rewriting intralinks in the post text if the text could be viewed at some other path such as an index page.

Isn't this what Jekyll on GitHub Pages does? Would we be able to implement Jekyll static blog posts through an IPFS hash? I'm curious.

[edrex]

@jbshirk I'm definitely interested in an IPFS backend for TW5, especially one with one file per tiddler modeled after the NodeJS backend. There's also interest [1] [2] from the Smallest Federated Wiki community (TW5's architectural soul brother, CC @WardCunningham @jasonkgreen).

My personal focus will remain on ironing out the basic static site publishing workflow. Experiments at closing the editing loop in the browser are sure to be worthwhile, but with the Javascript IPFS implementation a relativistic moving target (out of hell) anything that gets out in front of it is likely to be crushed into a fine interstellar powder.

My understanding is that currently IPFS FUSE is read-only. But is this where it is going?

This works (ref ipfs-inactive/faq#64):

ipfs daemon  --writable --mount-ipfs ~/Store/ipfs  --mount-ipns ~/Store/ipns
vim ~/Store/ipns/current/index.html

In my very limited testing writing to the IPNS FUSE implementation results in system instability, at least on OS X.

[jbenet]

yeah fuse is annoyingly unstable in osx at the moment. if anyone out there wants to help on this it would be much appreciated, we dont have the bandwidth to fix that right now but it would be very useful.

much of all of this can be done through APIs btw-- whether the cli or using https://github.com/ipfs/js-ipfs-api -- and yes the end goal is to have a node in the browser itself. much progress is happening here, and if people want it to come faster, help out @diasdavid

[Kubuxu]

Hugo (https://gohugo.io) has RelativeURLs option to make all paths really relative.

[hackergrrl]

Tiny lightweight IPFS blog CLI tool: https://github.com/noffle/ipfs-blog

[kevincox]

I'd like to point out that using relative URLs for assets is actually a bad idea because it plays poorly with the distributed system that websites are.

For example imagine this website:

<!-- index.html -->
<link rel=stylesheet href=style.css>

/* style.css */
html { background: green; }

That seems great. But now you push an update to add a button.

<!-- index.html -->
<link rel=stylesheet href=style.css>
<button>

/* style.css */
html { background: green; }
button { background: red; }

Oops, users who visited your webpage have the old CSS cached.

You may try renaming the CSS asset as is standard practice but now you have a new problem.

HTTP/2 404
ipfs resolve -r /ipfs/QmXCoP3YLkcqL2LnJ4X2JfWi1RsF7AnuT2z4Lwhk6Z5T1U/style.css: no link named "style.css" under QmXCoP3YLkcqL2LnJ4X2JfWi1RsF7AnuT2z4Lwhk6Z5T1U

Users with index.html cached (or who loaded it as the site was updating) will fail to look up the old stylesheet. This problem will also recur any time the name of your assets change.

The real solution is to use absolute links for your assets.

<!-- index.html -->
<link rel=stylesheet href=ipfs://QmdqnZxmTKUgqRM6WiqNoyd2bQezpkrjFxHpVmG5fQPB5p/index.css crossorigin="" integrity="sha256-bN/i4E0zpSXNZ1zYZx+rSb8fZCZJIL8TYFBHZ6E64GY=">
<button>

But of course this doesn't work for most users because very few have ifps:// protocol support. So the solution today is to hardcode a gateway into your blog. But of course now its lifetime is tied to the lifetime of the gateway that you picked.

(The other solution is to include old assets files in the new directory until they are expired. This does solve the problem but now each root hash contains more than necessary and you deployment process has to include old versions which is non-trivial).

So the summary:

• Links to top level items such as HTML pages, or anything loaded externally should be relative.
• Links to "assets" which are never supposed to be bookmarked or navigated to directly should be absolute.

[WardCunningham]

Wiki has always distinguished internal and external links. Federated wiki extends the semantics of internal links to include what is an ordered search of contributing authors (sites) when an internal link cannot be resolved locally. We've come to call this mechanism "collaborative links".

Our node/coffeescript implementation has flirted with IPFS with some success but not so much that it became standard practice. We've learned from the experience and completed a more aggressive implementation on top of DAT that evolves with it. This sharpened our understanding of the separation between client and server. We have since sketched implementation strategies for hosting within Scuttlebutt that look promising.

As we approach our tenth year, we are actively considering incorporating what we have learned and even upping our ambitions for the next decade. An emerging theme is the "plasticity" of our content especially when that represents computations and similar complex cooperative relationships.

We expect to depend heavily on javascript in some form but will most likely leave behind some frameworks and dialects that have served us well in the past. Similarly, we have resorted to domain names as identity and recognize that some or various forms of cryptographic identity will at least supplement domain names in our federation.

I mention all this because we would both be served by staying informed regarding each other's work.

[Stebalien]

I'd like to point out that using relative URLs for assets is actually a bad idea because it plays poorly with the distributed system that websites are.

This has nothing to do with relative paths. Browsers cache based on the absolute path and the cache policy for the resource. Whether or not you use an absolute path in your HTML is not relevant.

[kevincox]

You are right, but that completely ignores the issue.

If you are using a custom domain, or a subdomain IPNS gateway you need to reference a path on a different domain. You can not do this with relative links. The link itself isn't the problem, but is an indicator that the problem hasn't been solved.

So:

1. You need the URL of the asset to change so that the browser doesn't use a stale copy.
2. IPFS directories can't change, so you can't add new items.
3. Therefore you need to link to a new IPFS directory.
4. Assuming a subdomain-style gateway to link to a new directory you need to link to a different domain.
5. Therefore you can't use relative links.