Skip to content

meedstrom/org-node

Repository files navigation

org-node

MELPA MELPA Stable

News 1.8.0

Some code was spun out into a new library el-job, so that’s a new dependency. Update your package repositories.

Background

What’s all this

I like org-roam but found it too slow, so I made quickroam. And that idea spun off into this package, a standalone thing. It may also be easier to pick up than org-roam.

  • If you were using org-roam, there is nothing to migrate. You can use both packages. It’s the same on-disk format: “notes” are identified by their org-id.

    With optional shims, you can even skip syncing the org-roam DB and continue using its rich backlinks buffer!

    In pursuit of being “just org-id”, this package has no equivalent setting to org-roam-directory – it just looks up org-id-locations.

  • If you were not using org-roam, maybe think of it as somewhat like org-recent-headings tooled-up to the extent that you won’t need other methods of browsing, as long as you give IDs to all objects of interest.

    If you were the sort of person to prefer ID-links over file links or any other type of link, you’re in the right place! Now you can rely on IDs, and—if you want—stop worrying about filenames, directories and subtree hierarchies. As long as you’ve assigned an ID to a heading or file, you can find it later.

What’s a “node”?

My life can be divided into two periods ”before org-roam” and ”after org-roam”. I crossed a kind of gap once I got a good way to link between my notes. It’s odd to remember when I just relied on browsing subtrees and filesystem directories – what a strange way to work!

I used to lose track of things I had written, under some forgotten heading in a forgotten file in a forgotten directory. The org-roam method let me find and build on my own work, instead of recreating it all the time.

At the core, all the “notetaking packages” (orgrr/zk/zetteldeft/org-roam/denote/howm/minaduki/…) try to help you with this: make it easy to link between notes and explore them.

Right off the bat, that imposes two requirements: a method to search for notes, since you can’t link to something you can’t search for, and a design-choice about what kinds of things should turn up as search hits. What’s a “note”?

Just searching for Org files is too coarse. Just searching for any subtree anywhere brings in too much clutter.

Here’s what org-roam invented. It turns out that if you limit the search-hits to just those files and subtrees you’ve deigned to assign an org-id – which roughly maps to everything you’ve ever thought it was worth linking to – it filters out the noise excellently.

Once a subtree has an ID you can link to, it’s a “node” because it has joined the wider graph, the network of linked nodes. I wish the English language had more distinct sounds for the words “node” and “note”, but to clarify, I’ll say “ID-node” when the distinction matters.

Features

A comparison of three similar systems, which permit relying on org-id and don’t lock you into the all-too-seductive (for developers) concept of “one-note-per-file”.

Featureorg-roamorg-nodeorg-super-links
Backlinksyesyesyes
Node search and insertyesyes– (suggests org-ql)
Node aliasesyesyes
Node exclusionyeslimitednot applicable
Refileyesyes
Rich backlinks bufferyesyes (org-roam’s)
Customize how backlinks shownyesyes (org-roam’s)yes
Reflinksyesyes (as backlinks)
Ref searchyesyes (as aliases)not applicable
Org 9.5 @citations as refsyesyesnot applicable
Support org-ref v3yeslimitednot applicable
Support org-ref v2yesnot applicable
Work thru org-roam-captureyesyes?
Work thru org-captureyes?
Daily-nodesyesyes
Node seriesyes
Show backlinks in same windowyesyes
Cooperate with org-super-linksyesnot applicable
Fix link descriptionsyes
List dead linksyes
Rename file when title changesyes
Warn about duplicate titlesyes
Principled “related-section”yes
Untitled notes
Support roam: linksyes– (WONTFIX)not applicable
Can have separate note pilesyes– (WONTFIX)not applicable
Some query-able cacheEmacSQLhash tables
Async cache rebuildyesnot applicable
Time to cache my 3000 nodes2m 48s0m 01snot applicable
Time to save file w/ 400 nodes5–10sinstant?
Time to open minibuffer1–3sinstantnot applicable

Setup

Install

Assuming your package manager knows about MELPA, add this initfile snippet:

(use-package org-node
  :after org
  :config (org-node-cache-mode))

If you are an org-roam user, you’ll want the following module as well. Check its README to make org-node work with org-roam side-by-side.

(use-package org-node-fakeroam
  :defer)

An update broke things?

You can rollback to a version that used to work. How to rollback.

Quick start

If you’re new to these concepts, fear not. The main things for day-to-day operation are two verbs: “find” and “insert”.

Pick some short keys and try them out.

(keymap-set global-map "M-s M-f" #'org-node-find)
(keymap-set org-mode-map "M-s M-i" #'org-node-insert-link)

To browse config options, type M-x customize-group RET org-node RET.

Final tip: there’s no separate command for creating a new node! Reuse one of the commands above, and type the name of a node that doesn’t exist. Try it and see what happens!

Backlinks

What are backlinks?

Backlinks are the butter on the bread of your notes. If you’ve ever seen a “What links here” section on some webpage, that’s exactly what it is. Imagine seeing that, all the time. The following sections outline two general ways to do so.

Backlink solution 1: Borrow org-roam’s backlink buffer

As a Roam user, you can keep using M-x org-roam-buffer-toggle.

TIP: If it has been slow, or saving files has been slow, org-node-fakeroam gives you ways to speed it up.

TIP: If you have not done so yet, I recommend binding some short key sequences. I spent many months waffling on where to bind them, so here’s an example:

;; Either this...
(keymap-set org-mode-map "M-s M-r" #'org-roam-buffer-toggle)
(keymap-set global-map "M-s M-d" #'org-roam-buffer-display-dedicated)

;; ...or just this for a different behavior
(keymap-set global-map "M-s M-r" #'org-node-fakeroam-show-buffer)

Backlink solution 2: Print inside the file

I rarely have the screen space to display a backlink buffer. Because it needs my active involvement to keep visible, I go long periods seeing no backlinks. This solution can be a great complement (or even stand alone).

Option 2A: Let org-node manage a :BACKLINKS: property

For a first-time run, type M-x org-node-backlink-fix-all-files. (Don’t worry if you change your mind; undo with M-x org-node-backlink-regret.)

Then enable the following global mode, which keeps these properties updated.

(org-node-backlink-mode)

NOTE 1: To be clear, this never generates new IDs. That’s your own business. This only adds/edits :BACKLINKS: properties, and no backlink will appear that correspond to a link if the context for that link has no ID among any outline ancestor.

NOTE 2: By default, the setting org-node-backlink-aggressive is nil, so that stale backlinks are not cleaned until you carry out some edits under an affected heading and then save the file, which fixes that heading’s :BACKLINKS: property. Flip the setting if you’d like it to be more proactive:

(setq org-node-backlink-aggressive t)

NOTE 3: People who prefer to hard-wrap text instead of enabling visual-line-mode or similar may not find this way of displaying backlinks very scalable, since Org places properties on a single logical line.

Option 2B: Let org-super-links manage a :BACKLINKS:...:END: drawer

I think the following should work. Totally untested, let me know!

(add-hook 'org-node-insert-link-hook #'org-node-convert-link-to-super)

Bad news: this is currently directed towards people who used org-super-links from the beginning, or people who are just now starting to assign IDs, as there is not yet a command to add new BACKLINKS drawers in bulk to preexisting nodes. (org-super-links#93)

Misc

Manage org-id-locations

I find unsatisfactory the config options in org-id (Why? See Taking ownership of org-id), so org-node gives you an extra way to feed data to org-id. That helps ensure that you never run into “ID not found” situations.

Example setting:

(setq org-node-extra-id-dirs
      '("~/org/"
        "~/Syncthing/"
        "/mnt/stuff/"))

Do a M-x org-node-reset and see if it can find your notes now.

Undoing a Roam hack

If you have org-roam loaded, opening a link can sometimes send you to an outdated file path due to a line in org-roam-id.el that causes org-id to preferentially look up the org-roam DB instead of org-id’s own table!

Either revert that with the following snippet, or if Fakeroam can cover your needs, simply delete the Roam DB (at “~/.emacs.d/org-roam.db”).

;; Undo a Roam override
(with-eval-after-load 'org-roam-id
  ;; Default for Org 9.1 through 9.7+
  (org-link-set-parameters
   "id" :follow #'org-id-open :store #'org-id-store-link-maybe))

Exclude uninteresting nodes

One user had over a thousand project-nodes, but only just began to do a knowledge base, and wished to avoid seeing the project nodes.

This could work by, for example, excluding a “project” tag or any note that has a TODO state:

(setq org-node-filter-fn
      (lambda (node)
        (not
         (or (org-node-get-todo node)
             (member "project"
                     (org-node-get-tags-with-inheritance node))
             (assoc "ROAM_EXCLUDE" (org-node-get-properties node))))))

You can also use a whitelist approach, allowing only nodes from a certain directory “my-wiki”:

(setq org-node-filter-fn
      (lambda (node)
        (and (string-search "/my-wiki/" (org-node-get-file-path node))
             (not (assoc "ROAM_EXCLUDE" (org-node-get-properties node))))))

Limitation: ROAM_EXCLUDE

Let’s say you have a big archive file, fulla IDs, and you want all the nodes within out of sight. Putting a :ROAM_EXCLUDE: t at the top won’t do it, because unlike in org-roam, child ID nodes of an excluded node are not excluded! The org-node-filter-fn applies its ruleset to each node in isolation.

However, nodes in isolation do still have inherited tags. So you can use that, or the file name or outline path.

Really, filename? A big selling point of IDs is that you avoid depending on filenames, it’s often pragmatic to let up on purism just a bit :-) It works well for me to filter out any file or directory that happens to contain “archive” in the name:

(setq org-node-filter-fn
      (lambda (node)
        (not (string-search "archive" (org-node-get-file-path node)))))

Or put something like #+filetags: :wiki_exclude: at the top of each file, and set:

(setq org-node-filter-fn
      (lambda (node)
        (not (member "wiki_exclude"
                     (org-node-get-tags-with-inheritance node))))))

Org-capture

You may have heard that org-roam has its own special set of capture templates: the org-roam-capture-templates.

People who understand the magic of capture templates, they may take this in stride. Me, I never felt confident using a second-order abstraction over an already leaky abstraction I didn’t fully understand.

Can we just use vanilla org-capture? That’d be less scary. The answer is yes!

The secret sauce is (function org-node-capture-target):

(setq org-capture-templates
      '(("i" "Capture into ID node"
         plain (function org-node-capture-target) nil
         :empty-lines-after 1)

        ("j" "Jump to ID node"
         plain (function org-node-capture-target) nil
         :jump-to-captured t
         :immediate-finish t)

        ;; Sometimes handy after `org-node-insert-link', to
        ;; make a stub you plan to fill in later, without
        ;; leaving the current buffer for now
        ("s" "Make quick stub ID node"
         plain (function org-node-capture-target) nil
         :immediate-finish t)))

With that done, you can optionally configure the everyday commands org-node-find & org-node-insert-link to outsource to org-capture when they try to create new nodes:

(setq org-node-creation-fn #'org-capture)

Completion-at-point

To complete words at point into known node titles:

(org-node-complete-at-point-mode)
(setq org-roam-completion-everywhere nil) ;; Prevent Roam's variant

Any analogue to org-roam-node-display-template?

To customize how the nodes look in the minibuffer, configure org-node-affixation-fn:

M-x customize-variable RET org-node-affixation-fn RET

You may also want to set org-node-alter-candidates to t.

Grep

If you have Ripgrep installed on the computer and consult installed on Emacs, you can use this command to grep across all your Org files at any time.

(keymap-set global-map "M-s M-g" #'org-node-grep) ;; Requires consult

This is can be a power tool for mass edits. Say you want to rename some Org tag :math: to :Math: absolutely everywhere. Then you could follow a procedure such as:

  1. Use org-node-grep and type :math:
  2. Use embark-export (from embark)
  3. Use wgrep-change-to-wgrep-mode (from wgrep)
  4. Do a query-replace (M-%) to replace all :math: with :Math:
  5. Type C-c C-c to apply the changes

Let org-open-at-point detect refs

Say there’s a link to a web URL, and you’ve forgotten you also have a node listing that exact URL in its ROAM_REFS property.

Wouldn’t it be nice if, clicking on that link, you automatically visit that node first instead of being sent to the web? Here you go:

(add-hook 'org-open-at-point-functions
          #'org-node-try-visit-ref-node)

Limitation: TRAMP

Working with files over TRAMP is unsupported for now. Org-node tries to be very fast, often nulling file-name-handler-alist, which TRAMP needs.

The best way to change this is to file an issue to show you care :-)

Limitation: Encryption

Encrypted nodes probably won’t be found. As above, file an issue.

Limitation: Unique titles

If two ID-nodes exist with the same title, one of them disappears from minibuffer completions.

That’s just the nature of completion. Much can be said for embracing the uniqueness constraint, and org-node will print messages calling attention to any collisions.

Anyway… there’s a workaround. Assuming you leave org-node-affixation-fn at its default setting, adding this to initfiles tends to do the trick:

(setq org-node-alter-candidates t)

This lets you match against the node outline path and not only the title, which resolves most conflicts given that the most likely source of conflict is subheadings in disparate files, that happen to be named the same. Some people make this trick part of their workflow.

NB: for users of org-node-complete-at-point-mode, this workaround won’t help those completions. With some luck you’ll rarely insert the wrong link, but it’s worth being aware of. (#62)

Limitation: Org-ref

Org-node supports the Org 9.5 @citations, but not fully the aftermarket org-ref &citations that emulate LaTeX look-and-feel, since it nearly doubles my scan time if I amend org-link-plain-re to match all of org-ref-cite-types.

What works is bracketed Org-ref v3 citations that start with “cite”, e.g. [[citep:...]], [[citealt:...]], [[citeauthor:...]], since org-node-parser.el is able to pick them up for free. What doesn’t work is e.g. [[bibentry:...]] since it doesn’t start with “cite”, nor plain citep:... since it is not wrapped in brackets.

If you need more of Org-ref, you have at least two options:

Toolbox

Basic commands:

  • org-node-find
  • org-node-insert-link
  • org-node-insert-transclusion
  • org-node-insert-transclusion-as-subtree
  • org-node-visit-random
  • org-node-refile
  • org-node-series-dispatch
    • Browse node series – see README
  • org-node-extract-subtree
    • A bizarro counterpart to org-roam-extract-subtree. Export the subtree at point into a file-level node, leave a link in the outline parent of where the subtree was, and show the new file as current buffer.
  • org-node-nodeify-entry
    • (Trivial) Give an ID to the subtree at point, and run the hook org-node-creation-hook
  • org-node-insert-heading
    • (Trivial) Shortcut for org-insert-heading + org-node-nodeify-entry
  • org-node-grep
    • (Requires consult) Grep across all known Org files.
  • org-node-fakeroam-show-roam-buffer
    • A different way to invoke the Roam buffer: display the buffer or refresh it if it was already visible. And a plot twist, if it was not visible, do not refresh until the second invocation.
      • Useful if you have disabled the automatic redisplay, because the Roam command org-roam-buffer-toggle is not meant for that.

Rarer commands:

  • org-node-lint-all
    • Can help you fix a broken setup: it runs org-lint on all known files and generates a report of syntax problems, for you to correct manually. Org-node assumes all files have valid syntax, though many of the problems reported by org-lint are survivable.
  • org-node-rewrite-links-ask
    • Look for link descriptions that got out of sync with the corresponding node title, then prompt at each link to update it
  • org-node-rename-file-by-title
    • Auto-rename the file based on the current #+title
      • Also works as an after-save-hook! Does nothing as such until you configure org-node-renames-allowed-dirs.
      • Please note that if your filenames have datestamp prefixes, it is important to get org-node-datestamp-format right or it may clobber a pre-existing datestamp.
  • org-node-list-dead-links
    • List links where the destination ID could not be found
  • org-node-list-reflinks
    • List citations and non-ID links
      • Can be interesting for seeing which links have an associated node and which don’t (usually, most don’t)
  • org-node-backlink-fix-all-files
    • Update BACKLINKS property in all nodes
  • org-node-list-feedback-arcs
  • org-node-rename-asset-and-rewrite-links
    • Interactively rename an asset such as an image file and try to update all Org links to them. Requires wgrep.
      • NOTE: For now, it only looks for links inside the root directory that it prompts you for, and sub and sub-subdirectories and so on – but won’t find a link outside that root directory.

        Like if you have Org files under /mnt linking to assets in /home, those links won’t be updated. Neither if you choose ~/org/subdir as the root directory will links in ~/org/file.org be updated.

Experimental: Node series

Do you already know about “daily-notes”? Then get started with a keybinding such as:

(keymap-set global-map "M-s s" #'org-node-series-dispatch)

and configure org-node-series-defs. See wiki.

What are series?

It’s easiest to explain series if we use “daily-notes” (or “dailies”) as an example of a series.

Roam’s idea of a “daily-note” is the same as an org-journal entry: a file/entry where the title is just today’s date.

You don’t need software for that basic idea, only to make it extra convenient to navigate them and jump back and forth in the series.

Thus, fundamentally, any “journal” or “dailies” software are just operating on a sorted series to navigate through. You could have series about, let’s say, historical events, Star Trek episodes, your school curriculum…

You may be taken aback that defining a new series requires writing 5 lambdas, but once you get the hang of it, you can often reuse those lambdas.

Future

A future version will likely bring convenient wrappers that let you define a series in 1-2 lines.

It’s also possible we just redesign this completely. Input welcome. How would you like to define a series? Where should the information be stored?

Appendix

Appendix I: Rosetta stone

API cheatsheet between org-roam and org-node.

Actionorg-roamorg-node
Get ID near point(org-roam-id-at-point)(org-id-get nil nil nil t)
Get node at point(org-roam-node-at-point)(org-node-at-point)
Get list of files(org-roam-list-files)(org-node-list-files)
Prompt user to pick a node(org-roam-node-read)(org-node-read)
Get backlink objects(org-roam-backlinks-get NODE)(org-node-get-id-links-to NODE)
Get reflink objects(org-roam-reflinks-get NODE)(org-node-get-reflinks-to NODE)
Get title(org-roam-node-title NODE)(org-node-get-title NODE)
Get title of file where NODE is(org-roam-node-file-title NODE)(org-node-get-file-title NODE)
Get title or name of file where NODE is(org-node-get-file-title-or-basename NODE)
Get name of file where NODE is(org-roam-node-file NODE)(org-node-get-file-path NODE)
Get ID(org-roam-node-id NODE)(org-node-get-id NODE)
Get tags(org-roam-node-tags NODE)(org-node-get-tags-with-inheritance NODE)
Get local tags(org-node-get-tags-local NODE)
Get outline level(org-roam-node-level NODE)(org-node-get-level NODE)
Get whether this is a subtree(=< 0 (org-roam-node-level NODE))(org-node-get-is-subtree NODE)
Get char position(org-roam-node-point NODE)(org-node-get-pos NODE)
Get properties(org-roam-node-properties NODE)(org-node-get-properties NODE), only includes explicit properties
Get subtree TODO state(org-roam-node-todo NODE)(org-node-get-todo NODE)
Get subtree SCHEDULED(org-roam-node-scheduled NODE)(org-node-get-scheduled NODE)
Get subtree DEADLINE(org-roam-node-deadline NODE)(org-node-get-deadline NODE)
Get subtree priority(org-roam-node-priority NODE)(org-node-get-priority NODE)
Get outline-path(org-roam-node-olp NODE)(org-node-get-olp NODE)
Get ROAM_REFS(org-roam-node-refs NODE)(org-node-get-refs NODE)
Get ROAM_ALIASES(org-roam-node-aliases NODE)(org-node-get-aliases NODE)
Get ROAM_EXCLUDE(assoc "ROAM_EXCLUDE" (org-node-get-properties NODE)), no inheritance
Ensure fresh data(org-roam-db-sync)(org-node-cache-ensure t t)

Appendix II: How to rollback

Instructions to downgrade to an older version, let’s say 1.6.2:

With vc-use-package (built into Emacs 30+):

(use-package org-node
  :vc (:fetcher github :repo "meedstrom/org-node"
       :rev "6ed833d1025d5163137904f2a1f6d80d930610b4"))

With Quelpa:

(use-package org-node
  :quelpa (org-node :fetcher github :repo "meedstrom/org-node"
                    :commit "6ed833d1025d5163137904f2a1f6d80d930610b4"))

With Elpaca as follows. Note that recipe changes only take effect after you do M-x elpaca-delete and it re-clones – the idea is that Elpaca users will prefer to do it manually inside the cloned repo.

(use-package org-node
  :ensure (:fetcher github :repo "meedstrom/org-node"
           :tag "1.6.2"))

With Straight, either create a Straight lockfile to pin a specific commit, or use the “v1.6” or “v1.5” branches:

(use-package org-node
  :straight (org-node :type git :host github :repo "meedstrom/org-node"
                      :branch "v1.6"))

Match fakeroam version

The extension org-node-fakeroam has been tightly coupled with specific versions of org-node, but hopefully I’ve made it warn about having the wrong version. Then you can use a similar procedure as above.

Just in case—

  • For org-node v1.5, use fakeroam v1.3
  • For org-node v1.6, use fakeroam v1.4
  • For org-node v1.7+, use latest

What version is installed?

This info depends on your package manager. I do not embed version in the source code.

  • With Elpaca, M-x elpaca-info RET org-node RET
  • With the built-in package.el, look inside the autogenerated file org-node-pkg.el
  • With Quelpa: ?
  • With Straight: ?