Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packages with subdirectories and non-elisp files are not handled properly #4

Closed
alem0lars opened this issue Nov 22, 2014 · 14 comments
Closed

Comments

@alem0lars
Copy link
Contributor

If a specifies in its recipe the files to be installed, some of them seems to be ignored.
For example the SLIME package from melpa specifies:

(slime :repo "slime/slime"
       :fetcher github
       :files ("*.el"
               ("lib" "lib/hyperspec.el")
               "swank"
               "*.lisp"
               "*.asd"
               ("contrib" "contrib/*" (:exclude "contrib/test"))
               "doc/slime.texi"
               "doc/slime.info"
               "doc/dir"
               "ChangeLog"))

Which lists the files that should be included.

However when I install the app-emacs/slime package I only get these files:

slime-autoloads.el
slime.el
slime.elc
slime-tests.el
slime-tests.elc

In fact, starting emacs with slime now gives me the error: Could not open load file: lib/hyperspec.

@jauhien
Copy link
Owner

jauhien commented Nov 22, 2014

Yes, this is a big issue. gs-elpa does not handle additional files at the moment. Do you know, how does the package-el itself handle this kind of things? If so, could you provide me links to the relevant information? If no, I will do a research and fix it, but when I will have enough free time (and this issue is in my todo for quite a long time already).

@jauhien jauhien changed the title Some packages aren't correctly installed Additional files and other custom installation issues of elisp packages are not handled by gs-elpa Nov 22, 2014
@jauhien
Copy link
Owner

jauhien commented Nov 22, 2014

@alem0lars: which version of SLIME do you need? Try 2.0_p20130930 or even 9999 from the lisp overlay.

@jauhien
Copy link
Owner

jauhien commented Nov 22, 2014

@alem0lars: see also https://bugs.gentoo.org/show_bug.cgi?id=530148 for SLIME version bump.

@alem0lars
Copy link
Contributor Author

I couldn't find a reference to the :files entry in package.el.

Tell me if I'm wrong, but that information is in the recipe not in the repository that hosts the package itself. This is because it's not a package specific thing but a package archiver specific thing.
It just tells to MELPA what to keep when generating the package from the sources.

So it shouldn't have nothing to do with the package manager too, which takes an already packaged thing and loads it.


So, let's sum up:

  1. The informations that we aren't considering aren't package specific
  2. They are defined inside the recipes in the MELPA repository, e.g. https://github.com/milkypostman/melpa/blob/master/recipes/slime
  3. We can and should consider the MELPA way (not package.el)

Considering only the :files thing for a moment (in my opinion it's the most urgent requirement, because it prevents packages from working).

In the MELPA source code, one interesting file is package-build.el:

  • :files is read using: (plist-get config :files).
  • I think this is the function where builds the paths:
(defun pb/expand-source-file-list (dir config)
  "Shorthand way to expand paths in DIR for source files listed in CONFIG."
  (mapcar 'car (package-build-expand-file-specs dir (pb/config-file-list config))))
  • It clones the repository (according to :fetcher)
    For example, the github fetcher is implemented with the following functions:
(defun pb/checkout-github (name config dir)
  "Check package NAME with config CONFIG out of github into DIR."
  (let* ((url (format "git://github.com/%s.git" (plist-get config :repo))))
    (pb/checkout-git name (plist-put (copy-sequence config) :url url) dir)))

(defun pb/checkout-git (name config dir)
  ; .... omitted code ...
  (pb/run-process nil "git" "clone" repo dir)))
  ; .... omitted code ...
  (apply 'pb/run-process dir "git" "log" "--first-parent" "-n1" "--pretty=format:'\%ci'"
               (pb/expand-source-file-list dir config))

What if we just clone the entire repository as it does?

@jauhien
Copy link
Owner

jauhien commented Nov 22, 2014

MELPA is used by package.el package manager then. And it somehow handles these issues. Or am I wrong?

The other way to proceed: we can create other ebuild generator, gs-melpa or whatever that will take the recipes, not the ELPA format files. Or add new kind of repos to gs-elpa. This possibility seems interesting for me. The issue here is that usage of recipes can significantly slow down ebuilds generation, but it can be handled by server-side automatic generation of ebuilds or some information for them. Some kind of MELPA, but for ebuilds. The similar way to roverlay or gs-pypi.

Note, that at the moment gs-elpa works with the info provided by MELPA format files, so it has no access to recipes.

in my opinion it's the most urgent requirement, because it prevents packages from working

The majority of packages work, as they are pure elisp packages. At least, all that I needed so far worked without any problems. )

@jauhien
Copy link
Owner

jauhien commented Nov 24, 2014

Ok, it seems package.el just copies all the additional files, so it should be quite easy to fix. I have just tried to install slime with it. I will read package.el code and then do the relevant changes in gs-elpa. May be on this weekend if I have enough free time.

@alem0lars
Copy link
Contributor Author

Ok, thanks :)

jauhien added a commit that referenced this issue Nov 30, 2014
@jauhien
Copy link
Owner

jauhien commented Nov 30, 2014

@alem0lars: please, check if my last commit to gs-elpa fixes the problem. To do so, unmask live gs-elpa, emerge it and resync the repo:

sudo flaggie app-portage/gs-elpa '+**'
sudo emerge -va -1 app-portage/gs-elpa
sudo layman -s melpa-stable

After it, please, try to emerge SLIME or whatever package with multiple files you need from melpa-stable (or change melpa-stable to melpa if you are using it).

@jauhien
Copy link
Owner

jauhien commented Nov 30, 2014

It seems I am doing the same as package.el now. As I compile all the elisp files in the package archive and then install all the subdirectories.

@jauhien jauhien changed the title Additional files and other custom installation issues of elisp packages are not handled by gs-elpa Packages with subdirectories and non-elisp files are not handled properly Nov 30, 2014
@jauhien
Copy link
Owner

jauhien commented Dec 4, 2014

Changes commited to the tree as 0.1.3 release. Please, test if they fix your problem.

@alem0lars
Copy link
Contributor Author

I've just installed SLIME. Now the files under contrib are installed too, but swank and lib aren't copied, which cause errors.

@jauhien
Copy link
Owner

jauhien commented Dec 4, 2014

It copies all the files:

jauhien@zcj ~ % equery f app-emacs/slime
 * Searching for slime in app-emacs ...
 * Contents of app-emacs/slime-2.11:
/usr
/usr/share
/usr/share/emacs
/usr/share/emacs/site-lisp
/usr/share/emacs/site-lisp/site-gentoo.d
/usr/share/emacs/site-lisp/site-gentoo.d/50slime-gentoo.el
/usr/share/emacs/site-lisp/slime
/usr/share/emacs/site-lisp/slime/ChangeLog
/usr/share/emacs/site-lisp/slime/contrib
/usr/share/emacs/site-lisp/slime/contrib/ChangeLog
/usr/share/emacs/site-lisp/slime/contrib/Makefile
/usr/share/emacs/site-lisp/slime/contrib/README.md
/usr/share/emacs/site-lisp/slime/contrib/bridge.el
/usr/share/emacs/site-lisp/slime/contrib/bridge.elc
/usr/share/emacs/site-lisp/slime/contrib/inferior-slime.el
/usr/share/emacs/site-lisp/slime/contrib/inferior-slime.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-asdf.el
/usr/share/emacs/site-lisp/slime/contrib/slime-asdf.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-autodoc.el
/usr/share/emacs/site-lisp/slime/contrib/slime-autodoc.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-banner.el
/usr/share/emacs/site-lisp/slime/contrib/slime-banner.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-c-p-c.el
/usr/share/emacs/site-lisp/slime/contrib/slime-c-p-c.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-cl-indent.el
/usr/share/emacs/site-lisp/slime/contrib/slime-cl-indent.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-clipboard.el
/usr/share/emacs/site-lisp/slime/contrib/slime-clipboard.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-compiler-notes-tree.el
/usr/share/emacs/site-lisp/slime/contrib/slime-compiler-notes-tree.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-editing-commands.el
/usr/share/emacs/site-lisp/slime/contrib/slime-editing-commands.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-enclosing-context.el
/usr/share/emacs/site-lisp/slime/contrib/slime-enclosing-context.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-fancy-inspector.el
/usr/share/emacs/site-lisp/slime/contrib/slime-fancy-inspector.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-fancy-trace.el
/usr/share/emacs/site-lisp/slime/contrib/slime-fancy-trace.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-fancy.el
/usr/share/emacs/site-lisp/slime/contrib/slime-fancy.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-fontifying-fu.el
/usr/share/emacs/site-lisp/slime/contrib/slime-fontifying-fu.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-fuzzy.el
/usr/share/emacs/site-lisp/slime/contrib/slime-fuzzy.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-highlight-edits.el
/usr/share/emacs/site-lisp/slime/contrib/slime-highlight-edits.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-hyperdoc.el
/usr/share/emacs/site-lisp/slime/contrib/slime-hyperdoc.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-indentation.el
/usr/share/emacs/site-lisp/slime/contrib/slime-indentation.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-mdot-fu.el
/usr/share/emacs/site-lisp/slime/contrib/slime-mdot-fu.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-media.el
/usr/share/emacs/site-lisp/slime/contrib/slime-media.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-motd.el
/usr/share/emacs/site-lisp/slime/contrib/slime-motd.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-mrepl.el
/usr/share/emacs/site-lisp/slime/contrib/slime-mrepl.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-package-fu.el
/usr/share/emacs/site-lisp/slime/contrib/slime-package-fu.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-parse.el
/usr/share/emacs/site-lisp/slime/contrib/slime-parse.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-presentation-streams.el
/usr/share/emacs/site-lisp/slime/contrib/slime-presentation-streams.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-presentations.el
/usr/share/emacs/site-lisp/slime/contrib/slime-presentations.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-references.el
/usr/share/emacs/site-lisp/slime/contrib/slime-references.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-repl.el
/usr/share/emacs/site-lisp/slime/contrib/slime-repl.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-sbcl-exts.el
/usr/share/emacs/site-lisp/slime/contrib/slime-sbcl-exts.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-scheme.el
/usr/share/emacs/site-lisp/slime/contrib/slime-scheme.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-scratch.el
/usr/share/emacs/site-lisp/slime/contrib/slime-scratch.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-snapshot.el
/usr/share/emacs/site-lisp/slime/contrib/slime-snapshot.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-sprof.el
/usr/share/emacs/site-lisp/slime/contrib/slime-sprof.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-trace-dialog.el
/usr/share/emacs/site-lisp/slime/contrib/slime-trace-dialog.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-tramp.el
/usr/share/emacs/site-lisp/slime/contrib/slime-tramp.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-typeout-frame.el
/usr/share/emacs/site-lisp/slime/contrib/slime-typeout-frame.elc
/usr/share/emacs/site-lisp/slime/contrib/slime-xref-browser.el
/usr/share/emacs/site-lisp/slime/contrib/slime-xref-browser.elc
/usr/share/emacs/site-lisp/slime/contrib/swank-arglists.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-asdf.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-c-p-c.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-clipboard.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-fancy-inspector.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-fuzzy.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-goo.goo
/usr/share/emacs/site-lisp/slime/contrib/swank-hyperdoc.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-ikarus.ss
/usr/share/emacs/site-lisp/slime/contrib/swank-indentation.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-jolt.k
/usr/share/emacs/site-lisp/slime/contrib/swank-kawa.scm
/usr/share/emacs/site-lisp/slime/contrib/swank-larceny.scm
/usr/share/emacs/site-lisp/slime/contrib/swank-listener-hooks.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-media.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-mit-scheme.scm
/usr/share/emacs/site-lisp/slime/contrib/swank-mlworks.sml
/usr/share/emacs/site-lisp/slime/contrib/swank-motd.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-mrepl.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-package-fu.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-presentation-streams.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-presentations.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-r6rs.scm
/usr/share/emacs/site-lisp/slime/contrib/swank-repl.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-sbcl-exts.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-snapshot.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-sprof.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-trace-dialog.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank-util.lisp
/usr/share/emacs/site-lisp/slime/contrib/swank.rb
/usr/share/emacs/site-lisp/slime/dir
/usr/share/emacs/site-lisp/slime/lib
/usr/share/emacs/site-lisp/slime/lib/hyperspec.el
/usr/share/emacs/site-lisp/slime/lib/hyperspec.elc
/usr/share/emacs/site-lisp/slime/metering.lisp
/usr/share/emacs/site-lisp/slime/nregex.lisp
/usr/share/emacs/site-lisp/slime/sbcl-pprint-patch.lisp
/usr/share/emacs/site-lisp/slime/slime-autoloads.el
/usr/share/emacs/site-lisp/slime/slime-tests.el
/usr/share/emacs/site-lisp/slime/slime-tests.elc
/usr/share/emacs/site-lisp/slime/slime.el
/usr/share/emacs/site-lisp/slime/slime.elc
/usr/share/emacs/site-lisp/slime/slime.info
/usr/share/emacs/site-lisp/slime/start-swank.lisp
/usr/share/emacs/site-lisp/slime/swank
/usr/share/emacs/site-lisp/slime/swank-loader.lisp
/usr/share/emacs/site-lisp/slime/swank.asd
/usr/share/emacs/site-lisp/slime/swank.lisp
/usr/share/emacs/site-lisp/slime/swank/abcl.lisp
/usr/share/emacs/site-lisp/slime/swank/allegro.lisp
/usr/share/emacs/site-lisp/slime/swank/backend.lisp
/usr/share/emacs/site-lisp/slime/swank/ccl.lisp
/usr/share/emacs/site-lisp/slime/swank/clisp.lisp
/usr/share/emacs/site-lisp/slime/swank/cmucl.lisp
/usr/share/emacs/site-lisp/slime/swank/corman.lisp
/usr/share/emacs/site-lisp/slime/swank/ecl.lisp
/usr/share/emacs/site-lisp/slime/swank/gray.lisp
/usr/share/emacs/site-lisp/slime/swank/lispworks.lisp
/usr/share/emacs/site-lisp/slime/swank/match.lisp
/usr/share/emacs/site-lisp/slime/swank/mkcl.lisp
/usr/share/emacs/site-lisp/slime/swank/rpc.lisp
/usr/share/emacs/site-lisp/slime/swank/sbcl.lisp
/usr/share/emacs/site-lisp/slime/swank/scl.lisp
/usr/share/emacs/site-lisp/slime/swank/source-file-cache.lisp
/usr/share/emacs/site-lisp/slime/swank/source-path-parser.lisp
/usr/share/emacs/site-lisp/slime/xref.lisp

Are you using 0.1.3 version from the tree? Have you resynced your overlay with layman? Please, show me the build log.

@alem0lars
Copy link
Contributor Author

Oh sorry you're right 😄
Re-synced, re-emerged and now it works fine!

Thanks a lot for the support 👍

@jauhien
Copy link
Owner

jauhien commented Oct 6, 2015

Ok, seems to work now. Closing.

@jauhien jauhien closed this as completed Oct 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants