在两台不同厂商的云服务器上部署k8s集群，遇到一些问题。在此进行下总结。
首先要网络能够互通，我是通过添加虚拟网卡的方式
lsmod | grep ip_vs #检查是否有开启
#临时开启ip_vs
for i in $(ls /lib/modules/$(uname -r)/kernel/net/netfilter/ipvs|grep -o "^[^.]*");do echo $i; /sbin/modinfo -F filename $i >/dev/null 2>&1 && /sbin/modprobe $i; done
#永久开启
ls /lib/modules/$(uname -r)/kernel/net/netfilter/ipvs|grep -o "^[^.]*" >> /etc/modules
CENTOS7 增加虚拟网卡
确认内核是否有tun模块
# modinfo tun
yum install tunctl -y
如果找不到
vim /etc/yum.repos.d/nux-misc.repo
[nux-misc]
name=Nux Misc
baseurl=http://li.nux.ro/download/nux/misc/el7/x86_64/
enabled=0
gpgcheck=1
gpgkey=http://li.nux.ro/download/nux/RPM-GPG-KEY-nux.ro
# yum --enablerepo=nux-misc install tunctl
---------------------------------------
tunctl -t tap0 -u root
tap0 是虚拟网卡名字
ifconfig tap0 172.168.1.1 netmask 255.255.255.0 promisc
172.168.1.1 是ip地址
查看
ifconfig tap0
然后建立虚拟IP，如果有节点不是公网内网分离则不需要
#ifconfig查看网卡有哪些
#临时建立虚拟网卡，注意如果多个服务器都是公网内网分离，则都需要建立虚拟网卡
ifconfig eth1:0 公网IP netmask 255.255.255.0 up
#删除
ifconfig eth1:0 down
#永久建立
vim /etc/network/interfaces
auto eth1:0
iface eth1:0 inet static
name Ethernet alias LAN card
address 192.168.33.149
netmask 255.255.255.0
broadcast 192.168.33.255
/etc/init.d/networking restart
#ifconfig查看网卡有哪些 #临时建立虚拟网卡，注意如果多个服务器都是公网内网分离，则都需要建立虚拟网卡 ifconfig eth1:0 公网IP netmask 255.255.255.0 up #删除 ifconfig eth1:0 down #永久建立 vim /etc/network/interfaces auto eth1:0 iface eth1:0 inet static name Ethernet alias LAN card address 192.168.33.149 netmask 255.255.255.0 broadcast 192.168.33.255 /etc/init.d/networking restart
观察参数修改是否生效命令：
sysctl --system
安装docker
安装k8s相关组件
修改kubelet的文件
vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
# 注意，这步很重要，如果不做，节点仍然会使用内网IP注册进集群
# 在末尾添加参数 --node-ip=公网IP
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS --node-ip=公网ip
vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
# 注意，这步很重要，如果不做，节点仍然会使用内网IP注册进集群
# 在末尾添加参数 --node-ip=公网IP
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS --node-ip=公网ip
创建master节点
部署dashboard时一直处于Pending状态
kubectl describe pod kubernetes-dashboard-5947dc95db-n42cs -n kubernetes-dashboard
1、node节点的taint（污点）和toleration（容忍）
使用kubeadm初始化的集群，出于安全考虑Pod不会被调度到Master Node上，也就是说Master Node不参与工作负载。这是因为当前的master节点被打上了node-role.kubernetes.io/master:NoSchedule的污点：
[root@master] ~$ kubectl describe nodes master.hanli.com |grep -E '(Roles|Taints)'
Roles: master
Taints: node-role.kubernetes.io/master:NoSchedul因为这里搭建的是测试环境，内存资源不太足，可以选择去掉这个污点使master来承担一些工作负载：
先看一下taint命令的语法格式
kubectl taint node [node] key=value[effect] 其中[effect] 可取值: [ NoSchedule | PreferNoSchedule | NoExecute ]
NoSchedule: 一定不能被调度
PreferNoSchedule: 尽量不要调度，实在没有地方调度的情况下，才考虑可以调度过来
NoExecute: 不仅不会调度, 还会立即驱逐Node上已有的Pod
现在去掉污点：key的值node-role.kubernetes.io，value为空
[root@master] ~$ kubectl taint nodes master.hanli.com node-role.kubernetes.io/master-
node/master.hanli.com untainted修改后查看Taints字段状态：
-
[root@master] ~$ kubectl describe nodes master.hanli.com |grep -E '(Roles|Taints)' -
Roles: <none> -
Taints: <none>
如果要重新使master打上污点，即恢复Master 的不可调度状态，执行如下命令：
-
[root@master] ~$ kubectl taint nodes master.hanli.com node-role.kubernetes.io/master=:NoSchedule -
node/master.hanli.com tainted
这里要注意的是如果你丢掉了“=”，会报 error: at least one taint update is required错误
[root@master] ~$ kubectl taint nodes master.hanli.com node-role.kubernetes.io/master:NoSchedule
error: at least one taint update is required
除了直接去除污点，我们还可以在pod的yaml 配置文件中添加tolerations 字段来容忍 节点的污点
例如，如果你想让你的pod可以被调度到master节点上，在 pod 的 spec 中添加如下设置即可：
tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Exists"
effect: "NoSchedule"
