No tags yet.
After installing and uninstalling multiple permutations of software, I had pretty much corrupted my Kubernetes cluster and kNative installation. So, I'm going to reinstall from scratch to clean stuff up.
Kuberenetes cluster installation instructions to follow with be here.
I run the Docker install, even though I had previously installed Docker and this goes into an upgrade protocol. I accept the prompt to automatically restart Docker.
Trying to run kubeadm init on what will become my master, I am warned that kubelet is a previous version, so I run "sudo apt install kubelet" on both my master and my first node to get it up to the latest version.
This is sufficient to get the master up-and-running. On the first node, I have to perform a couple of extra steps. Specifically, I had to execute the commands below to let the current user have access to kubectl without having to do "sudo".
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/kubelet.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
sudo chmod 777 /var/lib/kubelet/pki/kubelet-client-current.pem
After I got through setting up all of the worker nodes, I noticed they still didn't have worker node labels, so I ran "kubectl label node kubernetes-fritzymac node-role.kubernetes.io/worker=worker" to properly label each of them. In this case, "kubernetes-fritzymac" is the name I chose for one of my nodes.
So, now that I've got a totally clean Kubernetes cluster back, I'm going to go ahead and install kNative on it using the instructions here.
The first decision point in the instructions is the choice of networking layer. I choose Istio, as described in these instructions.
There is a choice to be made between installing with and without sidecar injection. Now, I know Istio well enough to feel like I'm going to want sidecars, but... it says to do it without sidecar injection unless you really know of a reason otherwise. And, I'm going to guess it will be easier to add sidecars later than to remove them if I don't want them, so I'll go without sidecar injection for now.
I notice that Istio is on 1.7 now, and I had been working on 1.3.4 - so, hopefully some nice improvements in here. :-). Reading the instructions on installing istioctl, it looks like the sidecar injection preference is set on a namespace by namespace level, so... shouldn't be too much of a hassle.
I go through and install the Istio sample application even, because I want something to let me know that that much of the installation has worked, at least. It works fine. So, I go on to install Kiali. It looks like this method also installs Prometheus, Grafana and Jaeger. I can't seem to get to any of their user interfaces, but I'm not really focused on that, so I will go back and continue the rest of the kNative install!
Now, I took a bit of a... larger step here than I had expected. In following the instructions down to the bottom, they wind up telling you how to install all of Istio when all I had really wanted to install was istioctl. So, I've got an Istio installation running now - quite unexpectedly, following all of these instructions. I loop back to the kNative instructions and hope for the best.
At one point, it tells me to save the information below for later use.
There is some stuff next in the instructions about how to access your application using different kinds of DNS. It looks like this won't become relevant until I have my first app up-and-running but, when I get to that point, I'll have to use the temporary DNS method, as I have no external IP.
The next decision I'm asked to make is what kind of messaging layer I want. In line with the sample application I have been creating, I want a Kafka queue for my messages. So, I choose that and am directed to follow these instructions to get Kafka up and running on my cluster.
For my tiny cluster, I reduce the number of zookeeper replicas to 1 in the configuration file. Installing Kafka seems to go surprisingly smoothly. :-)
All of that seems to run fine to conclusion, also. I install all the observability stuff - even though I've not been able to get access to the Grafana, Prometheus, etc. dashboards yet.
To close this out, let me try one of the kNative Kafka examples. If these work in their entirety, I'll believe myself properly configured and ready to try to continue moving my Flashcards app over to this.
And those don't work at all. :-(. What to do this far into the process when nothing works?
I decide to actually go for a complete Kubernetes uninstall and reinstall... betting on the proposition that the second time you do something, it always goes much faster. :-)
Tear down seems to go fine. Reinstalling: I skip the upgrades, since I just upgraded to all the latest versions. I start right from "sudo kubeadm init --pod-network-cidr=10.244.0.0/16". It is blocked by some leftover files from the last cluster, so I do "sudo rm -rf /etc/kubernetes/manifests" and retry. Port 10250 is still in use, so I reboot.
That was insufficient. Some steps I missed last time I reset the cluster that I need to run this time are "sudo kubeadm reset" and "sudo rm -rf $HOME/.kube/config" and "sudo rm -rf /var/lib/etcd". That allows me to run to completion on the master... all of about 10 minutes spent getting there - much faster than the initial teardown and recreation of the master!
This works fine on my first node, also - running the command I was given by the master when I created it. Once again, I am getting errors about not being allowed to access kubectl without using sudo because of the private key file. But, I decide to just ignore that. This time, though, I'm only creating 1 node before I try the rest. I'll go back and add the second node if-and-when this one works fine!
It turns out that the errors even on the root node are because I copied an old kubelet.conf for my configuration file for kubectl instead of the new admin.conf. I realized this when I got tons of errors on step 1 of kNative setup about permissioning. Once I copied the correct file - all worked well (for this step, at least).
Unlike last time, this time I only perform enough of the instructions here to get istioctl available on my command line.
I get an error trying to install Istio without sidecar injection that indicates I don't have enough privileges. So, I try it with "sudo", and now istioctl isn't in my path. :-(. I su to root. Repeat the instructions for adding istioctl to my path and retry. It complains that the port is 0 and should be above 1024, with some references to 80 and 443. I see that those ports are referenced in the YAML file, so wonder if maybe I could/should set those to higher port numbers - like 8080 and 8443. I give it a whirl and re-run. Ah yes... I'm seeing output now that I definitely didn't get the last time I tried this! :-)
Unfortunately, as you can see -- it still blows up midway through the process.
I try deleting the SSL port. Maybe that is some issue, since we don't have any kind of certificate.
As a last dash attempt, I decide to try to install kNative via helm - which appears to still be on my system. I follow the directions here.
Sadly, this results in failure, also. kNative doesn't work with Helm 3. So, I upgraded to Helm 3. Then, it didn't like the remnants of the past several install attempts - so I removed all of those. Finally, it wasn't able to find some web hook that is a part of the install.
So... kNative installation on bare metal, from scratch... problematic at this point. Rather an entire day wasted. :-(