Test Pelias vagrant setup on Windows

[dianashk]

No description provided.

[easherma]

I'm doing this atm.

[rowanwins]

I'm currently attempting this

• Windows 10
• Vagrant 1.8.5
• VirtualBox 5.0.26

It seems to get through the vast bulk of the process although I do run into an error around the elasticsearch plugin install analysis-icu

==> default: ---- Begin output of node scripts/create_index.js ----
==> default: STDOUT:
==> default: STDERR: 1 required plugin(s) are not installed on the node(s) shown above.
==> default: you must install the plugins before continuing with the installation.
==> default:
==> default: you can install the missing packages on 'pelias' [127.0.0.1] with the following command(s):
==> default:
==> default:  sudo /usr/local/elasticsearch/bin/plugin install analysis-icu
==> default:
==> default: note: some plugins may require you to restart elasticsearch.
==> default:
==> default: please install mandatory plugins before continuing.
==> default: ---- End output of node scripts/create_index.js ----
==> default: Ran node scripts/create_index.js returned 1
==> default: [2016-09-05T01:50:51+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.

I was able to manually install the missing dependency on the box and then rerun the provisioning and all worked fine. Im also not sure if this is just a general issue with the repo or something windows specific.

On another note thanks for open-sourcing Pelias, it's a fantastic piece of work!

[DavidShillito-Tactician]

I am trying this and get the same error. I know nothing abut Linux, sudo etc. How do I "manually install the missing dependency on the box and then rerun the provisioning "?

i.e. sudo /usr/local/elasticsearch/bin/plugin install analysis-icu

I can see the Virtual Box VM running but if I open it and tap in it I get a login prompt and have to provide a username and password, which I do not know.

And If I did get there and entered the above command, how would I next rerun the provisioning? The only command I issued to start the whole process created the VM so I expect rerunning it will replace the VM in which I just issued the sudo command.

[rowanwins]

Hi @DavidShillito-Tactician

So the username & password should be 'vagrant' and 'vagrant'.

Once I was in I ran sudo /usr/local/elasticsearch/bin/plugin install elasticsearch/elasticsearch-analysis-icu as per the documentation here.

After that back on your regular machine with a cmd prompt open in your pelias directory (eg where you'd run vagrant up from) run the following

curl -XDELETE http://localhost:9200/pelias
vagrant provision

Hope that helps :)

[DavidShillito-Tactician]

Hi @rowanwins
Thanks for your quick and helpful reply. The login was successful, as was the install, but the XDELETE was not,. It complained that the index has missing. I decided to carry on assuming that the original failure had not got to the point where the index was created, The vagrant provision command failed after a minute or two with output that ended as follows:

==> default: [2016-09-08T01:48:08+00:00] INFO: runit_service[pelias-api] restart
ed
==> default:
==> default: - restart service runit_service[pelias-api]
==> default:
==> default: Running handlers:
==> default: [2016-09-08T01:48:08+00:00] ERROR: Running exception handlers
==> default: Running handlers complete
==> default: [2016-09-08T01:48:08+00:00] ERROR: Exception handlers complete
==> default: [2016-09-08T01:48:08+00:00] FATAL: Stacktrace dumped to /var/chef/c
ache/chef-stacktrace.out
==> default: Chef Client failed. 8 resources updated in 63.409649694 seconds
==> default: [2016-09-08T01:48:08+00:00] ERROR: execute[node scripts/create_inde
x.js](pelias::schema line 35) had an error: Mixlib::ShellOut::ShellCommandFaile
d: Expected process to exit with [0], but received '1'
==> default: ---- Begin output of node scripts/create_index.js ----
==> default: STDOUT: --------------
==> default: create index
==> default: --------------
==> default: STDERR: IndexCreationException[[pelias] failed to create index]; ne
sted: ElasticsearchIllegalArgumentException[failed to find char filter type [icu
_normalizer] for [nfkc_normalizer]]; nested: NoClassSettingsException[Failed to
load class setting [type] with value [icu_normalizer]]; nested: ClassNotFoundExc
eption[org.elasticsearch.index.analysis.icunormalizer.IcuNormalizerCharFilterFac
tory];
==> default: ---- End output of node scripts/create_index.js ----
==> default: Ran node scripts/create_index.js returned 1
==> default: [2016-09-08T01:48:08+00:00] FATAL: Chef::Exceptions::ChildConvergeE
rror: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.

[rowanwins]

hmm it looks like some of the other elasticsearch components might be having issues but im not sure why that might be sorry.

[easherma]

Personally, I'd recommend installing pelias module by module. You could
still use vagrant to set up a VM to do so on, if you'd like.

Install instructions here:
https://github.com/pelias/pelias-doc/blob/master/installing.md

It looks like there is some sort of error with create_index.js . That's the
script run to set up the elasticsearch schema for pelias.

On Thu, Sep 8, 2016 at 5:46 PM, Rowan Winsemius notifications@github.com
wrote:

hmm it looks like some of the other elasticsearch components might be
having issues but im not sure why that might be sorry.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#11 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AKDTT1sPGWZuzJQVauEragu-zkl9knSkks5qoJBWgaJpZM4FcJgE
.

Eric Sherman
224-402-0004

[DavidShillito-Tactician]

I was unable to install successfully using those or any other instructions that I could find.
It seems that some of the files are out-of-date and the user is expected to understand how to for example, adjust everything to work with the latest version of ElasticSearch.

My own skillset is not Unix-based so I am unable to fill in the gaps in the instructions.
All of the tooling used in this project - beyond Java and JavaScript - is new to me.

Has anyone considered putting out a pre-built VirtualBox VM that just works?
Downloading such a file, although it would be several GB, would be a small price to pay compared with the time wasted so far.

[rowanwins]

Hi @DavidShillito-Tactician

The trouble with anything prebuilt is that it becomes out of date as soon as it is built. The Pelias guys are very active in updating their projects so it's little wonder that something like a Vagrant VM gets let behind.

Having said that it would be nice if there was something that helped people get up and running a bit quicker. I'm currently working on a fairly simple Pelias setup, this evening I've had a quick crack at trying to turning it into a more generic pelias-starter repo which may help get you going. Note that this is literally an hours work in an attempt to get something up there so there is probably a bug or two in there and Im sure there are more optimal ways to set things up.

Cheers

[DavidShillito-Tactician]

Thanks but that is still not clear to me. I am away from the system until tomorrow but I am not clear if I perform your commands in the host system (Windows 10 Professional) or the VM (Ubuntu).
And I am not sure if the ElasticSearch installed in the VM Is the right version. It is in a directory called ElasticSearch 1.7.something but some of the articles state that 2.4 is required. And when I go to the ElasticSearch site to download it the instruction again assume too much knowledge.

[easherma]

I understand the frustration, I went through the same process. To look at
it in a more positive, if you learn how to set up pelias 'from scratch',
you can file issues and ask for support when you run into out of date info
or other bugs. These issues will get adressed much more directly since the
modules are used directly. The Vagrant environment sounds nice in theory,
but in my experience its one more level of abstraction.

The api team is on their gitter chat fairly often. I usually keep an eye on
it too; I don't work for Mapzen, but if you run into specific roadblocks
installing modules I'd pop in there and ask for help!

Elasticsearch 2.4 is required, the engine recently was upgraded to use 2.4.
I'm not sure where the Vagrant environment is at in that process, though.

On Tuesday, September 13, 2016, DavidShillito-Tactician <
notifications@github.com> wrote:

Thanks but that is still not clear to me. I am away from the system until
tomorrow but I am not clear if I perform your commands in the host system
(Windows 10 Professional) or the VM (Ubuntu).
And I am not sure if the ElasticSearch installed in the VM Is the right
version. It is in a directory called ElasticSearch 1.7.something but some
of the articles state that 2.4 is required. And when I go to the
ElasticSearch site to download it the instruction again assume too much
knowledge.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#11 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AKDTT4Vvf8fSbzZoS-LbjO1AIskJPuP2ks5qpo5dgaJpZM4FcJgE
.

Eric Sherman
224-402-0004

[hannesj]

I think the PR #56 is meant to fix that.

[orangejulius]

Hey @DavidShillito-Tactician and @rowanwins,
Sorry for the delayed response on the part of the Pelias team. Somehow none of us noticed this conversation happening, and I just discovered I wasn't watching this repo on Github, but now I am!

The state of the Vagrant image has been frustrating to us too for a while, and we've talked a lot about what to do about it internally. The problem is this: internally at Mapzen we have our own set of chef scripts that are used to deploy Pelias in our production environment for Mapzen Search. These scripts roughly consists of two parts: the part that pertains directly to Pelias and essentially represents the full set of knowledge of how to set up and deploy Pelias, and the standard, boring, stuff that any company has when they have something in production. The first part can and should be open source, but unfortunately it lives in the same (private) repository as the standard, boring parts that are unfortunately tied a little too closely to credentials and whatnot.

We haven't been very good about duplicating the work we do privately in this repo, and as all of us on the Pelias team set up Pelias manually on our own machines, we don't really notice when we break the Vagrant image. At some point, we need to go through and untangle those two parts, put the pelias specific chef scripts in their own (open) repository, and reconfigure the Vagrant image to use them. Then, when we make changes in Pelias, updating them for our production environment will also update them for the Vagrant image.

Due to the nature of the work, we can't even open source a bunch of code and ask the community for help untangling and rearranging it, since the untangling part has to be done by us. For the last few months, we've discussed when we should do this many times, and each time we've chosen to prioritize adding new features or fixing bugs in Pelias over improving the Vagrant image.

But in the last few weeks we've shipped a string of changes that each do their part in making the Vagrant image even more broken, and now we've decided to dive into this untangling after the work to import Who's on First venues and postal codes. That should be done in a week or two!

The other thing is that we don't really like chef that much. We have a lot more to learn about it, but it can be clunky, and slow. In the long term we are interested in finding something else that allows our production and development environments to share set up code, while being easy for people new to the project to get started with. Docker of course is mentioned a lot, but we haven't decided at all. We'll tackle that someday...

TLDR: we hope to give the Vagrant image some serious revamping in the next few weeks. Until then, use the official install docs, which are more up to date and also teach you much more about how Pelias works, so when issues do arise there's a better chance of fixing them. In the very long term we are looking to move off chef if we can find something nicer. Of course always feel free to keep opening issues, asking questions in our Gitter chat, etc.

[DavidShillito-Tactician]

Thank you all for taking the time to explain things. I think I need to wait for the revamped Vagrant image that you suggest may become available in a few weeks.

[dianashk]

Hey, all! The Vagrant setup has been deprecated, so this issue will be closed.
Thanks for all your feedback and support. ❤️