Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Help accessing Eclipse Che with Minikube and KVM on external machine #16414

Closed
gnoejuan opened this issue Mar 21, 2020 · 11 comments
Closed

Help accessing Eclipse Che with Minikube and KVM on external machine #16414

gnoejuan opened this issue Mar 21, 2020 · 11 comments
Labels
area/install Issues related to installation, including offline/air gap and initial setup kind/question Questions that haven't been identified as being feature requests or bugs.

Comments

@gnoejuan
Copy link

gnoejuan commented Mar 21, 2020

Summary


tl;dr

6HbM5xCD

This is my setup, what is the proper way of using Minikube (for KVM acceleration)? Or perhaps is Minikube the wrong approach? Should I deploy a different solution for k8s? I'm open to installing a new operating system on the k8s machine (such as k3os or any other k8s focused OS). I had already backed up all my data.


Things I've tried

! Node may be unable to resolve external DNS records ! VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository
chectl doesn't thow any errors, but I'm not seeing any bandwidth activity.
image

I was able to quickly understand Docker, but with k8s, I feel like I'm way in over my head.

Relevant information

@gnoejuan gnoejuan added the kind/question Questions that haven't been identified as being feature requests or bugs. label Mar 21, 2020
@che-bot che-bot added the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Mar 21, 2020
@tolusha tolusha added area/install Issues related to installation, including offline/air gap and initial setup and removed status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. labels Mar 23, 2020
@ibuziuk
Copy link
Member

ibuziuk commented Mar 23, 2020

@tolusha will you be able to take a look at that one?

@tolusha
Copy link
Contributor

tolusha commented Mar 23, 2020

That's very interesting task, but unfortunately I don't have an appropriate environment.
May be someone else have already faced that.

@ibuziuk
Copy link
Member

ibuziuk commented Mar 24, 2020

@l0rd @davidfestal @benoitf maybe you can take a look at this setup and suggest smth

@gnoejuan
Copy link
Author

At this point, KVM acceleration isn't important anymore. I believe the most productive way is to focus on a remote bare metal install. Example environments would be an AWS EC2 instance or Vultr which doesn't appear to offer a managed K8s solution.

I would still use the setup pictured above, but I hope a tutorial that applies to a hosted bare metal server (without managed k8s) would also apply to a local machine.

@skabashnyuk
Copy link
Contributor

@gnoejuan would you like to try with crc(local openshift 4 ) there is a guide for it https://gist.github.com/tmckayus/8e843f90c44ac841d0673434c7de0c6a ?

@l0rd
Copy link
Contributor

l0rd commented Mar 26, 2020

We use minikube running on a remote machine for our e2e tests. Maybe @rhopp @dmytro-ndp have some suggestions.

@ultrafab
Copy link

Hello @gnoejuan, i've just finish to setup something i think similar to your needs, running completely inside a private network.

This is the abstract, let me know if i can give you more informations:

  • Create a KVM Virtual Machine with Ubuntu 18.04 server. (I've done cause i've had an already setup KVM host, but i think you can skip and use a bare 18.04 installation on a phisical machine).
  • Installing Openshift 3.11 on the virtual (or phisical) machine (https://www.techrepublic.com/article/how-to-install-openshift-origin-on-ubuntu-18-04/)
  • Configure a dnsmasq service on another network server (in my case i've installed it on the KVM Host machine hosting the VM with Openshift), and configure it to resolve a local domain (your-che-domain.lan) pointing to the Openshift machine IP. It's important that you setup a wildcard domain, this way it will be able to resolve also the dynamic subdomains created by Che.
  • THIS IS IMPORTANT (without it everything keeps failing as documented here: How to run Che on Openshift 3.11 in a local network #16417) create a static /etc/resolv.conf file insider your Openshift VM, pointing to the IP of the dnsmasq host. This way Openshift, when starting, will pass your private DNS/Domain settings to Openshift.
  • Add your dnsmasq machine ip to the DNS list of every client that needs to connect to Che.

Let me know if i can provide more insight.

@gnoejuan
Copy link
Author

@gnoejuan would you like to try with crc(local openshift 4 ) there is a guide for it gist.github.com/tmckayus/8e843f90c44ac841d0673434c7de0c6a ?

For this one, I'm stuck at AttributeError: 'NoneType' object has no attribute 'typeattributes'

image

But it looks like it should have been fixed here: SELinuxProject/selinux#81 (comment)

Hello @gnoejuan, i've just finish to setup something i think similar to your needs, running completely inside a private network.

This is the abstract, let me know if i can give you more informations:

* Create a KVM Virtual Machine with Ubuntu 18.04 server. (I've done cause i've had an already setup KVM host, but i think you can skip and use a bare 18.04 installation on a phisical machine).

* Installing Openshift 3.11 on the virtual (or phisical) machine ([techrepublic.com/article/how-to-install-openshift-origin-on-ubuntu-18-04](https://www.techrepublic.com/article/how-to-install-openshift-origin-on-ubuntu-18-04/))

* Configure a dnsmasq service on another network server (in my case i've installed it on the KVM Host machine hosting the VM with Openshift), and configure it to resolve a local domain (your-che-domain.lan) pointing to the Openshift machine IP. It's important that you setup a wildcard domain, this way it will be able to resolve also the dynamic subdomains created by Che.

* THIS IS IMPORTANT (without it everything keeps failing as documented here: #16417) create a static `/etc/resolv.conf` file insider your Openshift VM, pointing to the IP of the dnsmasq host. This way Openshift, when starting, will pass your private DNS/Domain settings to Openshift.

* Add your dnsmasq machine ip to the DNS list of every client that needs to connect to Che.

Let me know if i can provide more insight.

Here, I get so close. It had stopped right at "Starting Che." It times out, and gives a directory for the logs /tmp/chexxxxxxx. When I navigate to the dir, it doesn't exist.

I'm curious if this is a problem with installing on Ubuntu 18.04.4 LTS for both solutions. I am working on downloading Ubuntu 16.04.6 LTS

@renannprado
Copy link

@gnoejuan could you get around the AttributeError: 'NoneType' object has no attribute 'typeattributes' issue?

@gnoejuan
Copy link
Author

gnoejuan commented Jul 6, 2020

@gnoejuan could you get around the AttributeError: 'NoneType' object has no attribute 'typeattributes' issue?

I was unable to get around the issue. It has been awhile since I was tackling CRC method to deploy a k8s cluster. My recollection was the recommended Ubuntu version at the time was lower than 18.04, but the KVM version that came with Ubuntu 16 was too old. Researching how to upgrade KVM only returned results on installing QEMU/KVM.

image

Ubuntu 18.04 LTS or newer and Debian 10 or newer are not officially supported and may require manual set up of the host machine.

https://code-ready.github.io/crc/

After trying RancherOS (Supposedly due to be obsoleted rancher/os#3000 (comment)), k3OS never seemed to resolve an IP address, Fedora's CoreOS required learning another technology called Ignition. . . .

After researching Kubernetes more thoroughly, it appears the fastest/easiest path to a local Kubernetes server is to use kubectl.

Personally, I chose to use microk8s on the k8s machine pictured above.

Microk8s guide: https://microk8s.io/docs/working-with-kubectl

tl;dr: on k8s machine: microk8s config and copy and paste or microk8s config > config and find a way to retrieve the file to your client machine.

Assuming you don't have a .kube in your home directory on the client machine. I.E the machine you want to access Kubernetes from.

cd $HOME
mkdir .kube
cd .kube

put the config file from earlier in the .kube folder

chectl server:start -p k8s -a helm -b mainhub.lan

where mainhub.lan is your k8s machine.

Minikube guide: https://medium.com/@kuljeetsinghkhurana/access-minikube-using-kubectl-from-remote-machine-2b0eeefad9cb

In theory, chectl server:start -p k8s -a helm -b mainhub.lan should work here too, but I haven't tested it.

Although this experience has dragged me into the world of Kubernetes kicking and screaming, I enjoyed the learning experience.

@gnoejuan gnoejuan closed this as completed Jul 6, 2020
@emircanerkul
Copy link

➜  minikube chectl server:start -p k8s -a helm -b mainhub.lan
Error: Nonexistent flags: -p, -a, -b

I'm also looking for answer;
#22989

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/install Issues related to installation, including offline/air gap and initial setup kind/question Questions that haven't been identified as being feature requests or bugs.
Projects
None yet
Development

No branches or pull requests

9 participants