kubeadm安装

一：硬件环境准备
三台机器，计划为：一台master，两台node

序     ip    系统版本    hostname    配置    节点类型
1    192.168.137.61    CentOS 7.4.1611 (Core)    master61    2核2G    Master
2    192.168.137.62    CentOS 7.4.1611 (Core)    node62    2核2G    node
3    192.168.137.63    CentOS 7.4.1611 (Core)    node63    2核2G    node
二：系统软件环境预置
1.设置hosts
vim /etc/hosts
加入以下内容
192.168.137.61 master61
192.168.137.62 node62
192.168.137.63 node63
hostnamectl set-hostname master61
hostnamectl set-hostname node62
hostnamectl set-hostname node63

yum install -y ipvsadm

yum install -y wget vim net-tools lrzsz

如果以下命令执行不过去，可以尝试手动输入，可能复制 页代码的问题

2.关闭防火墙
[root@vm210 ~]# systemctl stop firewalld
[root@vm210 ~]# systemctl disable firewalld
[root@vm210 ~]# systemctl status firewalld
● firewalld.service – firewalld – dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)
[root@vm210 ~]# 
3.安装docker
使用官方安装脚本自动安装
安装命令如下：

curl -fsSL https://get.docker.com | bash -s docker –mirror Aliyun
也可以使用国内 daocloud 一键安装命令：

curl -sSL https://get.daocloud.io/docker | sh
systemctl enable –now docker

4.配置yum源
vim /etc/yum.repos.d/kubernetes.repo

加入以下内容
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
5.设置SELinux
vim /etc/selinux/config
添加如下SELINUX=disabled 
注释掉SELINUX=enforcing,SELINUXTYPE=targeted 
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing – SELinux security policy is enforced.
#     permissive – SELinux prints warnings instead of enforcing.
#     disabled – No SELinux policy is loaded.
#SELINUX=enforcing
# SELINUXTYPE= can take one of three values:
#     targeted – Targeted processes are protected,
#     minimum – Modification of targeted policy. Only selected processes are protected.
#     mls – Multi Level Security protection.
#SELINUXTYPE=targeted 
SELINUX=disabled 
6.关闭swap内存
使用swap会影响性能。kubelet禁用swap

1）系统级零时关闭

? swapoff -a ，重启后失效

2）系统级全部关闭

? vi /etc/fstab，注释掉swap那一行

? 需要重启。重启后不失效

?

#
# /etc/fstab
# Created by anaconda on Mon Dec  2 21:02:22 2019
#
# Accessible filesystems, by reference, are maintained under ‘/dev/disk’
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root /                       xfs     defaults        0 0
UUID=b232659c-bd84-46f0-928b-a46d55500934 /boot                   xfs     defaults        0 0
#/dev/mapper/centos-swap swap  
7.设置iptables
解决iptables而导致流量无法正确路由的问题

cat  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl –system
三：Master节点安装kubeadm
1.安装kubelet 和kubeadm以及kubectl
yum install -y kubelet-1.20.0 kubeadm-1.20.0 kubectl-1.20.0
systemctl enable kubelet

yum install -y kubelet kubeadm kubectl –disableexcludes=kubernetes
systemctl enable –now kubelet
2.启动docker
systemctl enable docker && systemctl start docker
3.下载所需要的镜像
for i in `kubeadm config images list`; do 
  imageName=${i#k8s.gcr.io/}
  docker pull registry.aliyuncs.com/google_containers/$imageName
  docker tag registry.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
  docker rmi registry.aliyuncs.com/google_containers/$imageName
done;
否则造成下面：

error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.20.15: output: Error response from daemon: Get “https://k8s.gcr.io/v2/”: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1

4.更改kubelet的参数
vim /etc/sysconfig/kubelet

改为如下参数
KUBELET_EXTRA_ARGS=–cgroup-driver=systemd
5.kubeadm初始化
kubeadm init

完成之后有如下结果
To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run “kubectl apply -f [podnetwork].yaml” with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm init
 –apiserver-advertise-address=192.168.137.61
 –image-repository registry.aliyuncs.com/google_containers
 –kubernetes-version v1.20.0
 –service-cidr=10.96.0.0/12
 –pod-network-cidr=10.244.0.0/16
 –ignore-preflight-errors=all

错（kubelete无法起动）：
[kubelet-check] It seems like the kubelet isn’t running or healthy.
[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz’ failed with error: Get “http://localhost:10248/healthz”: dial tcp [::1]:10248: connect: connection refused.
created /etc/docker/daemon.json and added below:

{
    “exec-opts”: [“native.cgroupdriver=systemd”]
}
Then

 sudo systemctl daemon-reload
 sudo systemctl restart docker
 sudo systemctl restart kubelet
Run kubeadm init or kubeadm join again.

kubeadm join 192.168.137.61:6443 –token uxc0ef.5t33r5ryhjf2hf32
        –discovery-token-ca-cert-hash sha256:46e0b740b4121b8db76437d7979c7cfcba2ffd4d535b3469ed136822f3ec86fd 

最后的命令需要在node节点中执行，从而加入的k8s集群

依据提示执行如下命令
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
四：node节点安装kubeadm
1.安装kubeadm kubelet
yum install -y kubelet-1.20.0 kubeadm-1.20.0

yum -y install kubeadm kubelet
2.启动docker
systemctl enable docker && systemctl start docker
3.下载所需要的镜像
for i in `kubeadm config images list`; do 
  imageName=${i#k8s.gcr.io/}
  docker pull registry.aliyuncs.com/google_containers/$imageName
  docker tag registry.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
  docker rmi registry.aliyuncs.com/google_containers/$imageName
done;
4.更改kubelet的参数
vim /etc/sysconfig/kubelet

改为如下参数
KUBELET_EXTRA_ARGS=–cgroup-driver=systemd
5)加入master
token来自master节点执行kubeinit的结果
kubeadm join 192.168.137.61:6443 –token uxc0ef.5t33r5ryhjf2hf32
        –discovery-token-ca-cert-hash sha256:46e0b740b4121b8db76437d7979c7cfcba2ffd4d535b3469ed136822f3ec86fd 
master中执行命令如下：

kubeadm token create –print-join-command
discovery-token-ca-cert-hash : 用于Node验证Master身份
根据CA的公钥证书数据来计算出hash值
openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -pubkey | openssl rsa -pubin -outform DER 2>/dev/null | sha256sum | cut -d’ ‘ -f1

这里的计算结果，跟join加入的discovery-token-ca-cert-hash后面接的结果是一样的，一致就可以说加入正确
————————————————

如果加入不进去，提示下面，应该是kubelete启动不了造成的：

[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz’ failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.

kubeadm init

kubeadm reset 
rm -rf /etc/cni/net.d
rm -rf $HOME/.kube/config
rm -rf /etc/kubernetes/

或者看上面kubeadm init 故障  或者可能是redhat-release版本不同，请yum update -y

kubernetes:[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz‘ failed_zJay-L’s Blog-CSDN博客
[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz’ failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.背景：搭建k8s高可用集群，由于自己瞎操作，在初始化master（master-1）时操作失败，于是又换了一个master（maste
https://blog.csdn.net/rookie23rook/article/details/114369501
： accepts at most 1 arg(s), received 3   可以尝试手动输入，可能复制 页代码的问题

kubernetes node节点加入容器 [ERROR FileContent–proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forw
输入 : sysctl -w net.ipv4.ip_forward=1 就可以解决这个问题了

五：安装 络插件
master上安装calico.yaml，解决STATUS notReady：  

wget https://docs.projectcalico.org/v3.8/manifests/calico.yaml –no-check-certificate

kubectl apply -f calico.yaml

kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml
curl https://docs.projectcalico.org/manifests/calico-etcd.yaml -o calico.yaml
六：查询k8s集群状态
1.查询node节点
[root@vm210 k8s]# kubectl get nodes
NAME    STATUS   ROLES    AGE     VERSION
vm210   Ready    master   58m     v1.16.3
vm211   Ready       21m     v1.16.3
vm212   Ready       6m29s   v1.16.3
The connection to the server localhost:8080 was refused – did you specify the right host or port决
问题分析

环境变量
原因：kubernetes master没有与本机绑定，集群初始化的时候没有绑定，此时设置在本机的环境变量即可解决问题。
问题图片

解决方式

步骤一：设置环境变量

    source /etc/profile

2.查询pods状态
root@vm210 k8s]# kubectl  get pods –namespace=kube-system
NAME                                      READY   STATUS            RESTARTS   AGE
calico-kube-controllers-55754f75c-7wvrb   1/1     Running           0          6m20s
calico-node-9x82m                         1/1     Running           0          6m20s
calico-node-gn5qh                         1/1     Running           0          6m20s
calico-node-h8kvz                         0/1     PodInitializing   0          6m20s
coredns-5644d7b6d9-h9sn2                  1/1     Running           0          59m
coredns-5644d7b6d9-pwfl5                  1/1     Running           0          59m
etcd-vm210                                1/1     Running           0          58m
kube-apiserver-vm210                      1/1     Running           0          58m
kube-controller-manager-vm210             1/1     Running           0          58m
kube-proxy-6hjk2                          1/1     Running           0          22m
kube-proxy-bcmhh                          1/1     Running           0          7m31s
kube-proxy-bt9rn                          1/1     Running           0          59m
kube-scheduler-vm210                      1/1     Running           0          58m
3.查询pod具体状态
kubectl –namespace=kube-system describe pod

kubectl –namespace=kube-system describe pod calico-node-h8kvz
4.master节点也可以像node节点一样调度pod
kubectl taint nodes –all node-role.kubernetes.io/master-

[root@vm210 k8s]# kubectl taint nodes –all node-role.kubernetes.io/master-
node/vm210 untainted
taint “node-role.kubernetes.io/master” not found
taint “node-role.kubernetes.io/master” not found
5.查询k8s版本

[root@vm210 k8s]# kubectl version
Client Version: version.Info{Major:”1″, Minor:”16″, GitVersion:”v1.16.3″, GitCommit:”b3cbbae08ec52a7fc73d334838e18d17e8512749″, GitTreeState:”clean”, BuildDate:”2019-11-13T11:23:11Z”, GoVersion:”go1.12.12″, Compiler:”gc”, Platform:”linux/amd64″}
Server Version: version.Info{Major:”1″, Minor:”16″, GitVersion:”v1.16.3″, GitCommit:”b3cbbae08ec52a7fc73d334838e18d17e8512749″, GitTreeState:”clean”, BuildDate:”2019-11-13T11:13:49Z”, GoVersion:”go1.12.12″, Compiler:”gc”, Platform:”linux/amd64″}
至此， 通过kubeadm工具就实现了Kubernetes集群的快速搭建。 如果安装失败， 则可以执行kubeadm reset命令将主机恢复原状， 重新执行kubeadm init，或者kubeadm join命令， 再次进行安装。

安装的过程中，若有任何问题，欢迎添加vx：xydjun 。大家一起交流探讨

7. 测试kubernetes集群
验证Pod工作
验证Pod 络通信
验证DNS解析
在Kubernetes集群中创建一个pod，验证是否正常运行：

kubectl create deployment nginx –image=nginx
kubectl expose deployment nginx –port=80 –type=NodePort
kubectl get pod,svc

访问地址：http://NodeIP:Port

8. 部署 Dashboard
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.3/aio/deploy/recommended.yaml

默认Dashboard只能集群内部访问，修改Service为NodePort类型，暴露到外部：

vim recommended.yaml

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    – port: 443
      targetPort: 8443
      nodePort: 30001
  selector:
    k8s-app: kubernetes-dashboard

kubectl apply -f recommended.yaml

kubectl get pods -n kubernetes-dashboard

NAME                                         READY   STATUS    RESTARTS   AGE
dashboard-metrics-scraper-7b59f7d4df-6qglx   1/1     Running   0          21h
kubernetes-dashboard-5dbf55bd9d-hv78v        1/1     Running   6          21h

 访问地址：https://NodeIP:30001

创建service account并绑定默认cluster-admin管理员集群角色：

“`

创建用户
kubectl create serviceaccount dashboard-admin -n kube-system

用户授权
kubectl create clusterrolebinding dashboard-admin –clusterrole=cluster-admin –serviceaccount=kube-system:dashboard-admin

获取用户Token
kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk ‘/dashboard-admin/{print $1}’)

“` 使用输出的token登录Dashboard。

9. 切换容器引擎为Containerd
https://kubernetes.io/zh/docs/setup/production-environment/container-runtimes/#containerd

1、配置先决条件

cat overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
设置必需的 sysctl 参数，这些参数在重新启动后仍然存在。
# Setup required sysctl params, these persist across reboots.
cat net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
# Apply sysctl params without reboot
sudo sysctl –system
2、安装containerd

yum install -y yum-utils device-mapper-persistent-data lvm2

yum-config-manager
  –add-repo
  https://download.docker.com/linux/centos/docker-ce.repo

yum update -y && sudo yum install -y containerd.io

mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml

systemctl restart containerd

3、修改配置文件

vim /etc/containerd/config.toml

[plugins.”io.containerd.grpc.v1.cri”]

sandbox_image = “registry.aliyuncs.com/googlecontainers/pause:3.2”
[plugins.”io.containerd.grpc.v1.cri”.containerd.runtimes.runc.options]

SystemdCgroup = true

[plugins.”io.containerd.grpc.v1.cri”.registry.mirrors.”docker.io”]

           endpoint = [“https://b9pmyelo.mirror.aliyuncs.com”]

systemctl restart containerd

4、配置kubelet使用containerd

 vim /etc/sysconfig/kubelet

KUBELET_EXTRA_ARGS=–container-runtime=remote –container-runtime-endpoint=unix:///run/containerd/containerd.sock –cgroup-driver=systemd

or:

echo “KUBELET_EXTRA_ARGS=–container-runtime=remote –container-runtime-endpoint=unix:///run/containerd/containerd.sock –cgroup-driver=systemd” > /etc/sysconfig/kubelet

systemctl restart kubelet

5、验证

kubectl get node -o wide

k8s-node1 xxx containerd://1.4.4 “`

课堂笔记
“` 怎么查看容器日志kubectl logs -n kube-system

怎么查看容器事件kubectl describe pod -n kube-system

calico无法拉取镜像解决办法/p>

grep image calico.yaml

image: calico/cni:v3.15.1 image: calico/pod2daemon-flexvol:v3.15.1 image: calico/node:v3.15.1

docker pull xxx docker save calico/cni:v3.15.1 > cni.tar docker load

init失败或者情况环境可以使用： kubeadm reset

为什么部署 络组件Q1：每个docker主机创建的容器ip可能冲突Q2：容器1访问容器2，容器1怎么知道容器2在哪个docker主机Q3：容器1访问容器2数据包怎么传输过去/p>

1、k8s现在可以使用docker嘛可以。 2、dockershim什么时候被移除预计1.23版本。 3、docker还值的学习嘛值得。

kubectl get pods –show-labels # 查看资源标签 kubectl get pod -l app=web # 根据标签筛选资源 
 

文章知识点与官方知识档案匹配，可进一步学习相关知识云原生入门技能树容器(docker)安装docker8577 人正在系统学习中