[Kubernetes] kubeadm으로 클러스터 생성하기 With NCP (2)

2022-05-15

저번 포스팅에서는 사전 작업 및 설치 과정을 보여드렸습니다.

이어서 이번에는 Master Node/Worker Node에서의 설치 작업, 그리고 Node 끼리 연결하고 그 결과를 확인하는 마무리 과정을 보여드리겠습니다~

3. Master Node Settings

1) Control Components 설치

kube-apiserver, kube-controller-manager, kube-scheduler, kube-proxy, pause, etcd, coredns
아래 설치에서 flags를 사용하여 repository나 설치 버전 변경이 가능합니다.
- –image-repository string (Default: k8s.gcr.io)
- –kubernetes-version string (Default: stable-1)

root@master-node1:~# kubeadm config images pull
	[config/images] Pulled k8s.gcr.io/kube-apiserver:v1.23.6
	[config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.23.6
	[config/images] Pulled k8s.gcr.io/kube-scheduler:v1.23.6
	[config/images] Pulled k8s.gcr.io/kube-proxy:v1.23.6
	[config/images] Pulled k8s.gcr.io/pause:3.6
	[config/images] Pulled k8s.gcr.io/etcd:3.5.1-0
	[config/images] Pulled k8s.gcr.io/coredns/coredns:v1.8.6

2) Kubeadm Init

Master Node를 setup해 주기 위해 kubeadm init 명령어를 통해 Master Node를 초기화해 줍니다.
아래 출력문에서는 초기화 과정을 보여 주면서 다음 진행할 작업을 친절하게 출력해 줍니다 ^^

root@master-node1:~# kubeadm init
	[init] Using Kubernetes version: v1.23.6
	[preflight] Running pre-flight checks
	[preflight] Pulling images required for setting up a Kubernetes cluster
	[preflight] This might take a minute or two, depending on the speed of your internet connection
	[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
	[certs] Using certificateDir folder "/etc/kubernetes/pki"
	[certs] Generating "ca" certificate and key
	[certs] Generating "apiserver" certificate and key
	[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernet   es.default.svc.cluster.local master-node1] and IPs [10.96.0.1 10.41.2.156]
	[certs] Generating "apiserver-kubelet-client" certificate and key
	[certs] Generating "front-proxy-ca" certificate and key
	[certs] Generating "front-proxy-client" certificate and key
	[certs] Generating "etcd/ca" certificate and key
	[certs] Generating "etcd/server" certificate and key
	[certs] etcd/server serving cert is signed for DNS names [localhost master-node1] and IPs [10.41.2.156 127.0.0.1 ::1]
	[certs] Generating "etcd/peer" certificate and key
	[certs] etcd/peer serving cert is signed for DNS names [localhost master-node1] and IPs [10.41.2.156 127.0.0.1 ::1]
	[certs] Generating "etcd/healthcheck-client" certificate and key
	[certs] Generating "apiserver-etcd-client" certificate and key
	[certs] Generating "sa" key and public key
	[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
	[kubeconfig] Writing "admin.conf" kubeconfig file
	[kubeconfig] Writing "kubelet.conf" kubeconfig file
	[kubeconfig] Writing "controller-manager.conf" kubeconfig file
	[kubeconfig] Writing "scheduler.conf" kubeconfig file
	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
	[kubelet-start] Starting the kubelet
	[control-plane] Using manifest folder "/etc/kubernetes/manifests"
	[control-plane] Creating static Pod manifest for "kube-apiserver"
	[control-plane] Creating static Pod manifest for "kube-controller-manager"
	[control-plane] Creating static Pod manifest for "kube-scheduler"
	[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
	[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kuberne   tes/manifests". This can take up to 4m0s
	[apiclient] All control plane components are healthy after 7.503627 seconds
	[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
	[kubelet] Creating a ConfigMap "kubelet-config-1.23" in namespace kube-system with the configuration for the kubelets    in the cluster
	NOTE: The "kubelet-config-1.23" naming of the kubelet ConfigMap is deprecated. Once the UnversionedKubeletConfigMap f   eature gate graduates to Beta the default name will become just "kubelet-config". Kubeadm upgrade will handle this tr   ansition transparently.
	[upload-certs] Skipping phase. Please see --upload-certs
	[mark-control-plane] Marking the node master-node1 as control-plane by adding the labels: [node-role.kubernetes.io/ma   ster(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
	[mark-control-plane] Marking the node master-node1 as control-plane by adding the taints [node-role.kubernetes.io/mas   ter:NoSchedule]
	[bootstrap-token] Using token: 65z741.dd70q5p4d17ch3cv
	[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
	[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
	[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long te   rm certificate credentials
	[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bo   otstrap Token
	[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
	[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
	[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
	[addons] Applied essential addon: CoreDNS
	[addons] Applied essential addon: kube-proxy
	
	Your Kubernetes control-plane has initialized successfully!
	
	To start using your cluster, you need to run the following as a regular user:
	
	  mkdir -p $HOME/.kube
	  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
	  sudo chown $(id -u):$(id -g) $HOME/.kube/config
	
	Alternatively, if you are the root user, you can run:
	
	  export KUBECONFIG=/etc/kubernetes/admin.conf
	
	You should now deploy a pod network to the cluster.
	Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
	  https://kubernetes.io/docs/concepts/cluster-administration/addons/
	
	Then you can join any number of worker nodes by running the following on each as root:
	
	kubeadm join 10.41.2.156:6443 --token 65z741.dd70q5p4d17ch3cv \
	        --discovery-token-ca-cert-hash sha256:a6949071dd2204e2ae577f890f5c58b330b0055ee14773f702647895be919c0a

3) Kubectl 설정

kubectl을 사용하기 위해 디렉토리 생성 및 소유권을 변경해 줍니다.

root@master-node1:~# mkdir -p $HOME/.kube
root@master-node1:~# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@master-node1:~# sudo chown $(id -u):$(id -g) $HOME/.kube/config

4. 네트워크 플러그인 설치_Master Node

pod 간의 통신을 위해 네트워크 플러그인 설치가 필요합니다.
네트워크 플러그인 종류
- Weave
  kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
- Flannel
  kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
- Calico
  kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

Calico 설치

  root@master-node1:~# kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml**
  	configmap/calico-config created
  	....중략....
  	daemonset.apps/calico-node created
  	serviceaccount/calico-node created
  	deployment.apps/calico-kube-controllers created
  	serviceaccount/calico-kube-controllers created
  	Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
  	poddisruptionbudget.policy/calico-kube-controllers created

5. Worker Node Settings

1) Kubeadm Join

아까 Master Node 초기화 작업 진행하면서 보여준 출력문을 그대로 Worker Node에 입력해 줍니다.
kubeadm join 명령어를 통해 Worker Node를 Master Node에 연결해 주는 작업입니다.

root@worker-node1:~# kubeadm join 10.41.2.156:6443 --token 0toaxd.7se6f4jxz6zlu1qr --discovery-token-ca-cert-hash sha256:a6949071dd2204e2ae577f890f5c58b330b0055ee14773f702647895be919c0a
	[preflight] Running pre-flight checks
	[preflight] Reading configuration from the cluster...
	[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
	W0503 01:50:32.857613   16706 utils.go:69] The recommended value for "resolvConf" in "KubeletConfiguration" is: /run/systemd/resolve/resolv.conf; the provided value is: /run/systemd/resolve/resolv.conf
	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
	[kubelet-start] Starting the kubelet
	[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
	
	This node has joined the cluster:
	* Certificate signing request was sent to apiserver and a response was received.
	* The Kubelet was informed of the new secure connection details.
	
	Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

2) 각 노드 상태 확인

Node끼리 잘 연결이 되었는지 상태를 확인할 수 있는 명령어는 여럿 있는데 그 중 저는 아래 세 가지로 확인해 보았습니다.

(1) Kubectl get nodes

Master Node에서 아래 명령어를 입력하면 Node끼리 잘 연결이 되었는지 확인이 가능합니다.

root@master-node1:~# kubectl get node
	NAME           STATUS   ROLES                  AGE     VERSION
	master-node1   Ready    control-plane,master   56m     v1.23.6
	worker-node1   Ready    <none>                 5m30s   v1.23.6

(2) Kubectl get cs

component의 상태를 확인할 수 있는 명령어인데 1.19 버전부터는 사용하지 않는다고 합니다.

root@master-node1:~# kubectl get cs
	Warning: v1 ComponentStatus is deprecated in v1.19+
	NAME                 STATUS    MESSAGE                         ERROR
	scheduler            Healthy   ok
	controller-manager   Healthy   ok
	etcd-0               Healthy   {"health":"true","reason":""}

(3) API 구성 요소 확인

kubernetes component는 kube-system라는 namespace에 설치되어 kube-system에서 확인 가능합니다.

root@master-node1:~# kubectl get po -o custom-columns=POD:metadata.name,NODE:spec.nodeNam  e --sort-by spec.nodeName -n kube-system
POD                                       NODE
calico-kube-controllers-7c845d499-crnbg   master-node1
kube-controller-manager-master-node1      master-node1
calico-node-svp7t                         master-node1
coredns-64897985d-b6g7z                   master-node1
coredns-64897985d-w2rcv                   master-node1
etcd-master-node1                         master-node1
kube-apiserver-master-node1               master-node1
kube-proxy-86k76                          master-node1
kube-scheduler-master-node1               master-node1
calico-node-j8d59                         worker-node1
kube-proxy-j257x                          worker-node1

참고 사이트

Docker 공식 문서
- Docker storage driver : https://docs.docker.com/storage/storagedriver/select-storage-driver/
- Install Docker Engine on Ubuntu : https://docs.docker.com/engine/install/ubuntu/
Kubernetes 공식 문서
- 버전 차이(skew) 정책 : https://kubernetes.io/ko/releases/version-skew-policy/
- Creating a cluster with kubeadm : https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#version-skew-policy
NCP 가이드
- 리눅스 서버 생성 : https://www.ncloud.com/guideCenter/guide/1

Share on

Twitter Facebook LinkedIn

[Kubernetes] kubeadm으로 클러스터 생성하기 With NCP (2)

3. Master Node Settings

1) Control Components 설치

2) Kubeadm Init

3) Kubectl 설정

4. 네트워크 플러그인 설치_Master Node

5. Worker Node Settings

1) Kubeadm Join

2) 각 노드 상태 확인

(1) Kubectl get nodes

(2) Kubectl get cs

(3) API 구성 요소 확인

참고 사이트

Share on

Leave a comment

You may also enjoy

[AEKS2] 6주차 - EKS Security: Kyverno

[AEKS2] 6주차 - EKS Security: K8S 보안 위협

[AEKS2] 6주차 - EKS Security: EKS IRSA & Pod Identity

[AEKS2] 6주차 - EKS Security: EKS API 인증 모드