0%

Install Kubernetes by using kubeadm

学习Kubernetes是一件很痛苦的事情,你需要很多基础,而且还要学会科学上网,还要有耐心,安装一个Kubernetes更是让人头痛的事情,在对Kubernetes不了解的情况下,安装Kubernetes成了学习这门技术的门坎。

本文尽可能详细地写清楚Kubernetes Master的安装过程及环境准备,后续会有Kubernetes worker node的安装配置过程文章。

一起共勉。

一、配置环境

1. 对所有Kubernetes Node配置IP Forward

参照:http://blog.sanlea.com/2019/12/27/Fix-ip-forward-setting-on-CentOS-7-8/

2. 关闭swap

1
$ swapoff -a

修改/etc/fstab,注释掉swap分区。

3. 关闭Firewall

1
2
$ systemctl stop firewalld.service
$ systemctl disable firewalld.service

4. 关闭SELinux

1
$ setenforce 0

修改/etc/selinux/config,将SELINUX配置为disabled:

1
SELINUX=disabled

5. 配置Kubernetes RPM源,安装kuberet / kubeadm / kubectl

创建/etc/yum.repos.d/kubernetes.repo文件:

1
2
3
4
5
6
7
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

安装相关应用 kuberet / kubeadm / kubectl

1
2
$ yum install -y kubelet kubeadm kubectl
$ systemctl enable kubelet && systemctl start kubelet

二、安装 Kubernetes Master

1. 配置

1
$ kubeadm config print init-defaults > config.yaml

这时在config.yaml文件中的内容如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 1.2.3.4
bindPort: 6443
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: master
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.17.0
networking:
dnsDomain: cluster.local
serviceSubnet: 10.96.0.0/12
scheduler: {}

这个文件会成为初始化Kubernetes Master的配置文件,我们只需要关心以下配置即可:

API Server

1
2
3
localAPIEndpoint:
advertiseAddress: 1.2.3.4
bindPort: 6443

advertiseAddress是对外暴露的IP地址,一般为master所在的物理主机IP地址:

我的master主机的IP地址是192.168.55.55,所以就配置成这样:

1
2
3
localAPIEndpoint:
advertiseAddress: 192.168.55.55
bindPort: 6443

Docker Repository

这里配置获取Kubernetes Images的仓库:

1
imageRepository: k8s.gcr.io

因为Google太邪恶了,被墙,所以k8s.gcr.io是不可访问的,会导致Kubernetes Images无法下载,这时,我们需要配置成国内的Repository,这里我们使用阿里的Repository:

1
imageRepository: registry.aliyuncs.com/google_containers

Service IP Range

这里配置Service的IP范围,这个可以使用默认的:

1
2
3
4
networking:
dnsDomain: cluster.local
podSubnet: 10.100.0.0/16
serviceSubnet: 10.101.0.0/16

2. 下载Kubernetes Images

1
$ kubeadm config images pull --config=config.yaml

3. 安装Kubernetes master

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ kubeadm init --config=config.yaml

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.55.55:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:cca05032d2bc49abc05dba8de6fbbc2780b5498042be8f9416c7e141df84ce6b

按输出说明,保存kubectl配置:

1
2
3
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

最好复制保存上面的命令输出,以备后续使用。

当然,我们仍可以通过使用命令来再次上面的参数:

获取Token

1
2
3
4
5
$ kubeadm token list

# 输出:
# TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
# abcdef.0123456789abcdef 22h 2018-11-10T14:24:51Z authentication,signing <none> system:bootstrappers:kubeadm:default-node-token

获取sha256

1
2
3
4
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

# 输出:
# 9fcb02a0f4ab216866f87986106437b7305474850f0de81b9ac9c36a468f7c67

Kubernetes 默认是不把Master节点作为工作节点的,所以所有创建的Pod都不会部署到Master节点上。但是一般开发环境这样做就有点浪费了,所以,可以执行下面的命令来将Master变成工作节点:

1
$ kubectl taint nodes --all node-role.kubernetes.io/master-

4. 安装CNI网络插件

为了简单起见,方便入门,我们使用不需要配置即可使用的Calico插件:

安装网络插件可以让coredns等Kubernetes核心组件正常运行,所以一般都需要装,哪怕是单机模式。

下载配置文件:

1
$ wget https://docs.projectcalico.org/v3.11/manifests/calico.yaml

这里一定要记住版本不要错,因为在v3.11版本之前的版会出现一些兼容问题,所以请使用最新的版本。

修改配置文件,让Calico知道我为Kubernetes设置的Pod 的网段:

1
2
- name: CALICO_IPV4POOL_CIDR
value: "10.100.0.0/16"

执行安装:

1
$ kubectl apply -f calico.yaml

5. 验证安装:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ kubectl get pods -n kube-system

NAME READY STATUS RESTARTS AGE
calico-kube-controllers-648f4868b8-9b629 1/1 Running 0 50m
calico-node-6wjzl 1/1 Running 0 47m
calico-node-mq2bb 1/1 Running 0 50m
coredns-9d85f5447-bn9pg 1/1 Running 0 50m
coredns-9d85f5447-m7gn8 1/1 Running 0 50m
etcd-master 1/1 Running 0 50m
kube-apiserver-master 1/1 Running 0 50m
kube-controller-manager-master 1/1 Running 0 50m
kube-proxy-lw6nw 1/1 Running 0 50m
kube-proxy-zdh6s 1/1 Running 1 47m
kube-scheduler-master 1/1 Running 0 50m

如果能看到上面的输出结果,一般认为Kubernetes Master已经安装成功了。

查看节点

1
2
3
4
$ kubectl get nodes

NAME STATUS ROLES AGE VERSION
master Ready master 16h v1.17.0

验证kube-apiserver, kube-controller-manager, kube-scheduler, pod network 是否正常

1
2
3
4
5
6
7
8
9
10
11
12
# 部署一个 Nginx Deployment,包含两个Pod
# https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
kubectl create deployment nginx --image=nginx:alpine
kubectl scale deployment nginx --replicas=2

# 验证Nginx Pod是否正确运行,并且会分配10.244.开头的集群IP
kubectl get pods -l app=nginx -o wide

# 输出如下:
# NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
# nginx-65d5c4f7cc-7pzgp 1/1 Running 0 88s 10.244.1.2 ubuntu2 <none>
# nginx-65d5c4f7cc-l2h26 1/1 Running 0 82s 10.244.1.3 ubuntu2 <none>

验证kube-proxy是否正常

1
2
3
4
5
6
7
8
9
10
11
12
13
# 以 NodePort 方式对外提供服务 https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/
kubectl expose deployment nginx --port=80 --type=NodePort

# 查看集群外可访问的Port
kubectl get services nginx

# 输出如下:
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# nginx NodePort 10.110.142.125 <none> 80:30092/TCP 7s

# 可以通过任意 NodeIP:Port 在集群外部访问这个服务,本示例中部署的2台集群IP分别是192.168.0.8和192.168.0.7
curl http://192.168.0.8:30092
curl http://192.168.0.7:30092

验证一下dns, pod network是否正常

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# 运行Busybox并进入交互模式
kubectl run -it curl --image=radial/busyboxplus:curl

# 输入`nslookup nginx`查看是否可以正确解析出集群内的IP,已验证DNS是否正常
[ root@curl-5cc7b478b6-tlf46:/ ]$ nslookup nginx

# 输出如下:
# Server: 10.96.0.10
# Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
#
# Name: nginx
# Address 1: 10.110.142.125 nginx.default.svc.cluster.local

# 通过服务名进行访问,验证kube-proxy是否正常
[ root@curl-5cc7b478b6-tlf46:/ ]$ curl http://nginx/

# 输出如下:
# <!DOCTYPE html> ---省略

# 分别访问一下2个Pod的内网IP,验证跨Node的网络通信是否正常
[ root@curl-5cc7b478b6-tlf46:/ ]$ curl http://10.244.1.2/
[ root@curl-5cc7b478b6-tlf46:/ ]$ curl http://10.244.1.3/

OK,一切正常的话,Kubernetes Master就已经安装成功了。

三、复位重置

如果在安装过程中遇到什么问题,导致过程回不去了或者想全新再来一次,其实并不需要重新安装操作系统,仅只需要执行以下命令即可:

1
kubeadm reset

四、参考来源

https://www.cnblogs.com/RainingNight/p/using-kubeadm-to-create-a-cluster-1-12.html