k8s-1.16高可用集群部署

2020/01/10 posted in  Kubernetes

机器要求

高可用集群一般用于生产环境,官方推荐至少需要3台master节点机器,4台node节点机器

Hostname IP Role
ucloud-bj-k8s-master-01 10.9.142.180 Master Node
ucloud-bj-k8s-node-01 10.9.165.222 Worker Node
ucloud-bj-k8s-node-02 10.9.127.58 Worker Node
ucloud-bj-K8s-node-03 10.9.57.4 Worker Node
ucloud-bj-K8s-node-04 10.9.174.192 Worker Node

安装准备

  1. 禁用Swap
    k8s为了使容器的调度更符合机器的实际资源情况,k8s建议关闭内存交换

    swapoff -a
    

    同时删除/etc/fstab中swap那条记录

    当然,如果你的机器资源确实不多,需要利用swap,那么你可以不关闭swap交换空间,通过如下参数告诉k8s开启swap

    kubelet --fail-swap-on=false ...
    
  2. 端口开放(生产环境)

Docker安装(可选)

由于k8s需要安装指定docker18.06版本,所以如果你的版本不对,可以先卸载重新安装

sudo apt-get remove docker docker-engine docker-ce docker.io

安装docker

# 从 Ubuntu 的存储库安装 Docker:
apt-get update
apt-get install -y docker.io

# 或者从 Docker 的 Ubuntu 或 Debian 镜像仓库中安装 Docker CE 18.06:

## 安装环境准备。
apt-get update && apt-get install apt-transport-https ca-certificates curl software-properties-common

## 下载 GPG 密钥。
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

## 添加 docker apt 镜像仓库。
add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

## 安装 docker。
apt-get update && apt-get install docker-ce=18.06.0~ce~3-0~ubuntu

# 设置守护进程。
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

mkdir -p /etc/systemd/system/docker.service.d

# 重启 docker。
systemctl daemon-reload
systemctl restart docker

安装 kubelet,kubeadm,kubectl

添加阿里源密钥

curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - 

国内源(阿里)

cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF

安装

apt-get update
apt-get install -y kubelet=1.16.2-00 kubeadm=1.16.2-00 kubectl=1.16.2-00
apt-mark hold kubelet kubeadm kubectl
    

安装Master节点

kubeadm init --control-plane-endpoint "k8s-api.youxuetong.com:6443" --image-repository registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --upload-certs

如果安装完成,最后后输出如下内容

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join k8s-master:6443 --token vxszss.bboqeevhypvt0sxl \
    --discovery-token-ca-cert-hash sha256:56205646be3a53103e175d544dcd27cc82317c93042763cab20745334d8cb782 \
    --control-plane

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join k8s-master:6443 --token vxszss.bboqeevhypvt0sxl \
    --discovery-token-ca-cert-hash sha256:56205646be3a53103e175d544dcd27cc82317c93042763cab20745334d8cb782

部署CNI网络

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml

接入其他Master节点

在其他master节点机器上执行

  kubeadm join k8s-master:6443 --token vxszss.bboqeevhypvt0sxl \
    --discovery-token-ca-cert-hash sha256:56205646be3a53103e175d544dcd27cc82317c93042763cab20745334d8cb782 \
    --control-plane

如果提示token失效,请利用下面的命令去第一个master节点重新生成token

kubeadm token create --print-join-command

接入其他Node节点

kubeadm join k8s-master:6443 --token vxszss.bboqeevhypvt0sxl \
    --discovery-token-ca-cert-hash sha256:56205646be3a53103e175d544dcd27cc82317c93042763cab20745334d8cb782

安装完成

查看各节点状态

ubuntu@ucloud-bj-k8s-master-01:~$ kubectl get nodes -o wide
NAME                      STATUS   ROLES    AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
ucloud-bj-k8s-master-01   Ready    master   26d   v1.16.2   10.9.142.180   <none>        Ubuntu 16.04.6 LTS   4.4.0-142-generic   docker://18.6.0
ucloud-bj-k8s-master-02   Ready    master   26d   v1.16.2   10.9.175.27    <none>        Ubuntu 16.04.6 LTS   4.4.0-142-generic   docker://18.6.0
ucloud-bj-k8s-master-03   Ready    master   26d   v1.16.2   10.9.91.143    <none>        Ubuntu 16.04.6 LTS   4.4.0-142-generic   docker://18.6.0
ucloud-bj-k8s-node-01     Ready    <none>   26d   v1.16.2   10.9.165.222   <none>        Ubuntu 16.04.6 LTS   4.4.0-142-generic   docker://18.6.0
ucloud-bj-k8s-node-02     Ready    <none>   26d   v1.16.2   10.9.127.58    <none>        Ubuntu 16.04.6 LTS   4.4.0-142-generic   docker://18.6.0
ucloud-bj-k8s-node-03     Ready    <none>   26d   v1.16.2   10.9.57.4      <none>        Ubuntu 16.04.6 LTS   4.4.0-142-generic   docker://18.6.0
ucloud-bj-k8s-node-04     Ready    <none>   26d   v1.16.2   10.9.174.192   <none>        Ubuntu 16.04.6 LTS   4.4.0-142-generic   docker://18.6.0