Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: New embark-org package #502

Closed
oantolin opened this issue May 5, 2022 · 52 comments
Closed

RFC: New embark-org package #502

oantolin opened this issue May 5, 2022 · 52 comments

Comments

@oantolin
Copy link
Owner

oantolin commented May 5, 2022

I decided to write an embark-org package because I imagine many people have had to resort to writing their own target finders and action keymaps, so it's probably worth having a single coordinated effort. ac79525

The current version just has targets for table cells (which do triple duty as standing in for the entire column or row too), entire tables, and org links. I plan to add more stuff before putting this on GNU ELPA and would love some suggestions for both target types and for actions.

Obviously it's a little weird this initial version doesn't have anything for headings, but it's not that bad because Embark already has generic support for outline headings, we'd just need to add org specific actions (like changing todo keyword, priority or tags, I guess).

(The support for links is maybe a little more complex than you'd expect, so I wrote a rationale for it in comments in the source code, in case you're curious.)

So what targets are missing? I'd say headings and dates have useful actions. What about source blocks?

I'm not against writing some custom actions (I threw in a "copy as markdown" action for links requested long ago by @hmelman, for example), but as usual would love to mainly focus on exisitng org commands.

@oantolin
Copy link
Owner Author

oantolin commented May 5, 2022

To be honest, I only use the table cell target to copy the contents of a single cell to the kill-ring, but even just for that it's worth it. 😛

@oantolin
Copy link
Owner Author

oantolin commented May 5, 2022

Footnotes have a few reasonable actions too!

@minad
Copy link
Contributor

minad commented May 5, 2022

Cool. I look forward to try this out. Org-menu seems also useful for inspiration? The copy as MD is a good ootb addition. I also have sth like this in my config.

@oantolin
Copy link
Owner Author

oantolin commented May 5, 2022

Maybe the copy as markdown action should be less trusting of the user and explicitly check that you are using it on a URL. The current version will happily produce "Markdown" links for any link type at all.

@oantolin
Copy link
Owner Author

oantolin commented May 5, 2022

Org-menu seems also useful for inspiration?

Yes! Thank you for mentioning it, @sheijk had even suggested that his package be used as inspiration, but it had slipped my mind.

@minad
Copy link
Contributor

minad commented May 5, 2022

Copy as markdown is also useful for the markup. I would trust the user and bind it to the region map.

@oantolin
Copy link
Owner Author

oantolin commented May 5, 2022

Oh, you mean use the markdown exporter on a region. Yes, good idea. I currently just have a small function for links.

@oantolin
Copy link
Owner Author

oantolin commented May 5, 2022

More generally there are a bunch of org commands that are useful for regions. I think embark-org should add a prefix key to the region map.

@minad
Copy link
Contributor

minad commented May 6, 2022

;; The reason for this is left as an exercise to the reader.
;; Solution: Na ryvfc gnetrg znl cebzcg gur hfre sbe fbzrguvat!

I love this! Having obfuscated solutions in code comments is indeed a good use case for the rot13 action 😆

@minad
Copy link
Contributor

minad commented May 6, 2022

I just looked at the code a bit and I have only a few minor comments:

  • You may want to mark the macros as private embark-org--define-...
  • Regarding the link naming conventions, I would use org-link-file instead of org-file-link, to make the inheritance slightly more obvious, but well it makes the names "less english".
  • The package header should be changed such that it can be released on ELPA - fsf assignment. I assume you will release this as part of embark on ELPA (not as a separate package package embark-org).

About your keymap design - One alternative is to support merging multiple keymaps via entries like this the embark-keymap-alist:

...
(embark-org-file-link embark-org-link-map embark-file-map)
(file . embark-file-map)
(embark-org-link . embark-org-link-map)
...

I think this should be very easy to do in embark.el directly and you avoid the inheritance ceremony in embark-org.el. Maybe this proves to be useful elsewhere? Obvious downside is that we don't have an extra dedicated keymap in case the user wants to add more specific actions, e.g., for actions which make only sense for org-file-links. But if that's needed the user could still create an additional dedicated keymap and register it too.

@minad
Copy link
Contributor

minad commented May 6, 2022

About the loading of embark-org - I would suggest to add a with-eval-after-load block after provide such that the package becomes automatically available if the user runs embark-act (or other autoloaded commands) from an Org buffer.

(provide 'embark)

(with-eval-after-load 'org
  (require 'embark-org))

Alternatively the user puts a nested with-eval-after-load in the configuration (or the equivalent use-package statements):

(with-eval-after-load 'org
   (with-eval-after-load 'embark
      (require 'embark-org)))

@oantolin
Copy link
Owner Author

oantolin commented May 6, 2022

I'm glad you liked the use of ro13. 😛

I'm not too keen on allowing multiple keymaps in the embark-keymap-alist, mainly because of the issue you mentioned, that there is no obvious place to add new bindings. I don't want to force users to figure out how to make such a keymap and register it themselves, I think you can go a long way in cofniguring your Emacs with just define-key or use-package's :bind and not know how to create a new keymap that inherits from two others. But also, I think it makes it more difficult to examine Embark's keymaps from outside embark-act (if your are in embark-act it's easy to look at the current keymap, of course). I often use embark-bindings-in-keymap to remind myself of soemthing about the actions for a given type, and it's very handy that they are all in a single keymap.

@oantolin
Copy link
Owner Author

oantolin commented May 6, 2022

About the FSF assignment, Stefan Monnier already wrote to me about that. 🙂 GNU ELPA has a copyright check feature and having that file with the incorrect copyright keeps GNU ELPA from building the tarball for my package. I told him I was hoping to hold off on publishing, so he told me to create an .elpaignore file. He wasn't sure if that would skip the copyright check for the ignored file or not, and apparently it doesn't, but he said he wants to change that. The end result is that he specifically asked me refrain from giving the FSF copyright before he fixes the issue. 🙂

@oantolin
Copy link
Owner Author

oantolin commented May 6, 2022

I'm still undecided about whether to bundle embark-org with embark, or to make it a separate package, actually.

@oantolin
Copy link
Owner Author

oantolin commented May 6, 2022

I understand your point about the names, but org-file-link just sounds much more natural to me than org-link-file. And the name does make it clear that it is a type of org-link to human readers, so I think that's good enough.

@minad
Copy link
Contributor

minad commented May 6, 2022 via email

@oantolin
Copy link
Owner Author

oantolin commented May 6, 2022

I guess I can't think of any advantage. I had just visualized it as separate for no good reason! 😆

@minad
Copy link
Contributor

minad commented May 6, 2022 via email

@oantolin
Copy link
Owner Author

oantolin commented May 7, 2022

I tried your suggestion, @minad, of generalizing embark-keymap-alist to allow lists of keymaps, and it does look slightly nicer.

I've also added a general Org target finder, and actions for source blocks.

@minad
Copy link
Contributor

minad commented May 7, 2022

Looks good! Now we have a hell of new targets 👀

@minad
Copy link
Contributor

minad commented May 7, 2022

There is also a org-table-convert-region in the region map. Would it make sense to move that action to embark-org such that it is merged in only if you are in an org buffer? But it seems also generally useful. Generally I wonder if there are more targets which should be available immediately even outside of Org buffers, e.g., links. But this requires us to write our own target finder since Org is not yet loaded.

@oantolin
Copy link
Owner Author

oantolin commented May 7, 2022

Looks good! Now we have a hell of new targets 👀

And you'll notice I only enabled 9 out of 54 Org objects and elements! (I have no idea what the difference between an object and an element is, org-element-context happily returns both kinds.)

@oantolin
Copy link
Owner Author

oantolin commented May 7, 2022

I think org-table-convert-region is generally useful and like it in the region map.

I also wondered about Org links in non-Org buffers. I was thinking of this compromise: keep the link target finder in embark-org, so you only have it active once you've loaded Org, but make the target finder work in all buffers (the initial version did, I'd just have to bring that one back!) and change the org-open-at-point and org-insert-link actions to their global versions.

One thing that gave me pause is that I'm not sure org-open-at-point-global can really replace org-open-at-point: if you use it in an Org buffer does it really do all that org-open-at-point does?

@minad
Copy link
Contributor

minad commented May 7, 2022

And you'll notice I only enabled 9 out of 54 Org objects and elements!

Yes, I noticed. It is a nice implementation and we can easily extend it in the future :)

One thing that gave me pause is that I'm not sure org-open-at-point-global can really replace org-open-at-point: if you use it in an Org buffer does it really do all that org-open-at-point does?

I don't know. In practice I usually don't use the global command in Org buffers.

@oantolin
Copy link
Owner Author

oantolin commented May 7, 2022

@hmelman, you are a big Embark user in text modes: what do you think about the following issue with headings? The org parse includes in the bounds it reports for a headline the entire subtree., while the general outline heading support we have in Embark includes only the line of the heading. For the purposes of navigation or promoting/demoting headings that does not matter, but it does make a difference of course in the "copy to kill-ring" or "delete" actions. Should we settle on just one of the options or should we have both targets, so that people can quickly copy just the heading line, or cycle to the next target and copy the entire subtree?

@hmelman
Copy link

hmelman commented May 7, 2022

I'm only an occasional org-mode user, though I use markdown-mode a lot. In markdown-mode I'd expect an embark heading target to just be the heading, not the whole subtree. I'd expect this in other modes with headings too.

I think for org-mode embark should do what org users expect. If other org commands typically operate on the sub-tree do that, even if it's different from other text modes. I would think that the ability to copy just the heading would be useful, but I'm unclear on some org usage such as would this include properties, drawers, keywords or other org-isms.

@oantolin
Copy link
Owner Author

oantolin commented May 7, 2022

In markdown I believe the defun target is a subtree? Or is just the heading and the body text before the next section or subsection heading? At any rate, in Markdown you have both heading and "heading+some body text" targets, do you use both?

@hmelman
Copy link

hmelman commented May 7, 2022

Yes, in a heading the first target is Heading, then Identifier and then Defun (which is the subtree). I honestly don't use embark much in markdown headings. I'd use T for titlecase-line in a heading. The movement stuff I'd use other markdown bindings for (I use M-S-arrows for promote/demote and move subtree up/down) and hide/show I use S-TAB and C-TAB all of these I've made standard for me via outline-minor-mode.

@minad
Copy link
Contributor

minad commented May 7, 2022

@oantolin Are you okay with publishing your embark-org package at some point via MELPA? This could happen before an ELPA release, it just requires a small adjustment in the recipe: https://github.com/melpa/melpa/blob/cbaf88bb3789691a2f733369c271d33b8e959f6b/recipes/embark#L3

@minad
Copy link
Contributor

minad commented May 22, 2022

The embark and embark-consult packages are specified separately in ELPA. In order to include embark-org.el in the embark package, the ELPA declarations must be adjusted:

https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/elpa-packages#n223

  1. Adjust MELPA recipe
  2. Adjust ELPA declaration
  3. Remove .elpaignore

@jakanakaevangeli
Copy link
Contributor

Two minor comments from me:

With an org link whose description contains bold text and with point on this
bold text, for example [[file:/tmp/][tmp *dir*]] with point before the letter
r, the embark org target finder fails to recognize the surrounding org link.

This can be fixed with something like

   (when-let (((derived-mode-p 'org-mode 'org-agenda-mode))
-             (element (org-element-context))
-             ((memq (car element) embark-org--types))
+             (element (org-element-lineage (org-element-context) embark-org--types t))
              (begin (org-element-property :begin element))

Another feature request would be for an ability to cycle through ancestors. For
example, when the point is on a link inside a table cell, embark-cycle could
cycle between the org link, table cell and whole the table.

This could also be implemented with the help of org-element-lineage, but
probably less cleanly, because as far as I know, a single target finder cannot
yet return multiple candidates. Perhaps we could have about three target
finders, each one returning a more distant ancestor:

  1. embark-org-target-element-context:
    (org-element-lineage (org-element-context) embark-org--types t)
  2. embark-org-target-element-context-parent:
    (org-element-lineage (org-element-lineage (org-element-context) embark-org--types t) embark-org--types)

And so on for the grand-parent. Not very pretty and efficient, I know.

@minad
Copy link
Contributor

minad commented May 23, 2022

This could also be implemented with the help of org-element-lineage, but
probably less cleanly, because as far as I know, a single target finder cannot
yet return multiple candidates. Perhaps we could have about three target
finders, each one returning a more distant ancestor:

It seems to me that this is a sufficient reason to support target finders which return multiple candidates. Such finders are needed for complex markup with nested elements and potentially also for treesitter in the future.

The reason why we didn't add multiple-target finders was that the semantics were not clear. The question back then was if these multiple targets have the same priority and are supposed to be acted on at the same time, which would have then muddied the distinction of embark-act and embark-act-all. But in this case we are talking about something else, a single target finder returning nested targets with decreasing priorities.

I don't see a reason why we shouldn't add this, since it is an obvious generalization and more of a technical detail which facilitates the target finder implementation. In contrast, adding a bunch of child, parent, grand-parent, grand-grand-parent finders is neither pretty nor efficient as you wrote.

@oantolin
Copy link
Owner Author

I've been thinking about allowing target finders and target transformers to return multiple targets for a while anyway. It would even simplify some of the default configuration. It's an easy change, I just didn't want to do it if it wasn't necessary. So far I've made do without it somewhat begrudgingly (I'd like for example that if an Elisp symbol names both a variable and a function to get both targets) but this is another great use case, @jakanakaevangeli.

@oantolin
Copy link
Owner Author

oantolin commented May 23, 2022

Note that you do currently get both table cell and whole-table targets you can cycle between. But I do agree it would be great to be able to cycle among all ancestors.

@minad
Copy link
Contributor

minad commented May 23, 2022

If supporting multiple targets simplifies the default implementation, the change is justified. Looking forward to see if this holds indeed or if Embark will get more complicated. As usual I am happy to see simplification and consolidation. :)

@oantolin
Copy link
Owner Author

oantolin commented May 23, 2022

In my mind I separate the implementation of Embark from the default configuration of Embark. I only claimed allowing multiple target would simplify (and even improve) the default configuration. I don't think the change will make much of a difference to the implementation, conceptually a mapcar becomes a mapcan, and that's it (it's slightly more complicated since there is no actual mapcar call).

@minad
Copy link
Contributor

minad commented May 23, 2022

I don't think the change will make much of a difference to the implementation, conceptually a mapcar becomes a mapcan, and that's it (it's slightly more complicated since there is no actual mapcar call).

Okay, I understand. The implementation will get slightly complicated, but this small change will already be amortized when you manage to simplify only two or three target finders. They belong to the configuration, but Embark overall will still get simpler. In the case of embark-org having these child, parent, grand-parent finders wouldn't be really meaningful, while having multi-target finders is a natural generalization.

@hrehfeld
Copy link

In org mode there are quite a few things to do with clipboard content:

  • yank
  • org-download
  • org-cliplink
  • ?

some of which are only valid for certain types of clipboard content. Would embark be a good choice to dispatch on that?

@minad
Copy link
Contributor

minad commented Sep 13, 2022

@oantolin Btw, what about the embark-org package? When will you make this available?

@oantolin
Copy link
Owner Author

Good question! There are three things I'd like to do to it right now, of which the first two should probably be done before a release:

  1. Tweak the target priorities so that the targets end up ordered by size, which I think is the least confusing order possible.

  2. Change the link copying bindings. I've grown to dislike always typing w t, which seems to be the most common thing I want. I'd like an unadorned w to copy the inner target (or full target?) and to have the other four copying options under a prefix, but it's not clear what that prefix should be. I guess capital W is the most reasonable choice, although it does shadow copying the relative path for file links.

  3. Add org-specific actions for headings and maybe the heading target which counterintuitively includes both the heading and the text under it (even just for copying that could be pretty useful).

The third can wait since it adds new functionality rather than changing existing functionality.

Any thoughts on link copying? I keep rethinking it and not finding a super-ergonomic solution. There's already a long complicated comment in the source explaining the current design, but I wish I could do slightly better.

@minad
Copy link
Contributor

minad commented Sep 13, 2022

  1. Agree
  2. W sounds like a good choice. Or change wt to ww and ww to wf?
  3. Makes sense

Anyway I just wondered if there are some truly serious issues. Details can always be sorted out step by step. I expect that we are going to add more and more special actions over time inspired by org-menu.

@oantolin
Copy link
Owner Author

I went with your suggestion for link copying, after trying it for a couple of days.

@akurth
Copy link

akurth commented Nov 27, 2022

Could this be removed from .elpaignore eventually, as (I guess several) users currently have to install it manually?

Thanks for considering.

@oantolin
Copy link
Owner Author

Sure, I'm dragging my feet too much. I'll do it now. @minad also sent me a strong hint that I should do that, in the form of a PR... 🙄

@minad
Copy link
Contributor

minad commented Nov 27, 2022

Yeah, finally! ;)

@oantolin
Copy link
Owner Author

I guess any further discussion of embark-org can happen in specific issues.

@oantolin
Copy link
Owner Author

@jakanakaevangeli I decided to go ahead with target finders that can return multiple targets and now you can cycle between targets in the entire org-element-lineage. Thanks for suggesting it a while ago and for telling me about the lineage function.

@minad
Copy link
Contributor

minad commented Dec 27, 2022

I've forgotten that we had already discussed the multi target idea, so I wrote some critical comment here: #562 (comment). The arguments given here are all good, but it is still unclear to me if there is an actual simplification thanks to that. The whole table finder could be removed in 5f5b005. Additionally we get a generalized behavior since the ancestor elements are all available now. Which actual Org syntax elements are affected by this?

@oantolin
Copy link
Owner Author

oantolin commented Dec 27, 2022

Which actual Org syntax elements are affected by this?

Chains of length 3 I often have are link < list item < plain list and link < table cell < table. Before this change you could only target the link!

@oantolin
Copy link
Owner Author

it is still unclear to me if there is an actual simplification

It's not simpler than the previous code but I believe it is simpler and more general than code I would have had to write to get this functionality without the multi-valued targets. I think it's OK to have multi-valued targets even if the vast majority of target finders are single-valued. Just like I wound up liking the multi-valued keymap change you proposed above. This new change feels very similar in spirit.

@minad
Copy link
Contributor

minad commented Dec 27, 2022

To me there is a difference. The multi-valued keymaps seem like a pure technicality to save you from defining all these intermediate keymaps without any deeper conceptual implications. It is not that I don't see the point of the multi-target generalization. It makes sense, also a transformer from symbol to function+variable. Probably my concerns are unfounded. At least we didn't go with the generalization where a target finder could return multiple targets to be acted on at once. 😅

@oantolin
Copy link
Owner Author

I agree it the multi-valued target finder extension has a higher cognitive weight than the multi-keymap extension, but I don't feel it's that much higher, the implementation was pretty straightforward and the two uses, for Org and for Emacs Lisp symbols, seem to me to justify it. I may come to regret this, of course, but for now I think it's worth it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants