[Database/Kubernetes/DOIK] AWS EC2에 Vanilla Kubernetes 실습 환경 배포
이번에 Database Operator In Kubernetes (DOIK) 스터디에 참여하게 되었다.
스터디에서 notion과 slack으로 관련 자료를 공유해 주시는데 좋은 자료가 많아서 행복하다,,😚
이번에 새롭게 알게 된 사실은 차별 IT용어가 있어서 지양하는 단어가 몇 있다는 것이다.
그 중 하나는 Master Node인데, 차별 IT 용어로 master slave를 비권장하므로 Master Node 대신 Control Plane 사용을 권장한다고 한다.
그래서 나도 이제부터 Control Plane이라고 칭할 것 이다!
이번 포스팅에서는 스터디에서 제공해 준 CloudFormation 템플릿으로 EC2에 Vanilla Kubernetes를 설치한 과정을 보여줄 것이다.
** Vanilla란, 프로그램에 아무것도 추가하지 않은 순정 상태의 소프트웨어를 가리키는 속어다.
0. 전체 구성도
- Kubernetes 버전 v1.23.6 , Calico CNI(CrossSubnet 모드, ENI S/D Uncheck) , CRI(Docker), StorageClass(local-path, hostpath)
- EC2 Spec : Ubuntu 22.04, t3.medium, EBS gp3 40GiB
- 네트워크 대역 : Public Subnet 2개, Private Subnet 2개
- Control Plane 1대, Worker Node 3대 구성
| Hostname | IPv4 |
|---|---|
| k8s-m | 192.168.10.10 |
| k8s-w1 | 192.168.10.101 |
| k8s-w2 | 192.168.10.102 |
| k8s-w3 | 192.168.20.103 |
1. Cloudfarmation으로 자동 배포
0단계 : EC2 접속에 사용하는 SSH Key 생성
참고 ) 1단계: EC2 SSH 키 페어 생성
- AWS Management Console에서 AWS 리전 선택 :
ap-northeast-2 (서울) - 서비스 > EC2 > 네트워크 및 보안 > 키 페어 > 키 페어 생성
- 이름 :
doik-ec2-access-key-ap-northeast-2/ 키 페어 유형 :RSA(공개키) / 프라이빗 키 파일 형식 :.pem(OpenSSH와 함께 사용) > 키 페어 생성
1단계 : 템플릿 지정
- 스터디에서 제공해 준 템플릿
- Amazon S3 URL
2단계 : 스택 세부 정보 지정
- 스택 이름 :
myk8s(기본) - 파라미터 :
- ««< EC2 Node »»>
KeyName: 0단계에서 생성한 KeetPair 선택SgIngressCidr: EC2 인스턴스를 접속할 수 있는 IP 주소 입력 (나의 public IP/32)
- ««< EC2 Node »»>
3단계 : 스택 옵션 구성
- 패스
4단계 : 검토
-
내용 확인 > 스택 생성
-
인스턴스 생성 확인
2. Cluster 생성 및 Component 설치
(1) 설치 후, 기본 setup
- Control Plane 접속
-
EC2에서 생성한 Private Key로 ssh 접속
gain@LAPTOP-NGE5O25S:/mnt/c/Users/User/Downloads$ ssh -i ./myk8s.pem ubuntu@13.125.127.182 Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.15.0-1005-aws x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage System information as of Fri May 27 18:20:23 KST 2022 System load: 0.3359375 Processes: 143 Usage of /: 9.0% of 38.60GB Users logged in: 0 Memory usage: 21% IPv4 address for docker0: 172.17.0.1 Swap usage: 0% IPv4 address for ens5: 192.168.10.10 27 updates can be applied immediately. 21 of these updates are standard security updates. able Last login: Fri May 27 16:07:30 2022 from 125.178.215.17 (🐤 |kubernetes-admin@kubernetes:default) root@k8s-m:~# -
처음에는 접속 오류를 겪었다.. 아래 포스팅 참고
-
-
Control Plane 설정 로그 확인
(🍎 |kubernetes-admin@kubernetes:default) root@k8s-m:~# sudo tail -f /var/log/cloud-init-output.log No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. /nfs4-share <world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) >>>> K8S Controlplane Config End <<<< Cloud-init v. 22.1-14-g2e17a0d6-0ubuntu1~22.04.5 finished at Sun, 22 May 2022 12:24:19 +0000. Datasource DataSourceEc2Local. Up 181.88 seconds - CNI/StorageClass 등 설치 스크립트 실행
- Control Plane SSH 종료 후 다시 SSH 접속
(🍎 |kubernetes-admin@kubernetes:default) root@k8s-m:~# exa -bghHliSR DOIK/ inode Permissions Links Size Blocks User Group Date Modified Name 516217 drwxr-xr-x 2 - - root root 22 May 21:23 1 516239 .rw-r--r-- 1 55 8 root root 22 May 21:23 README.md DOIK/1: inode Permissions Links Size Blocks User Group Date Modified Name 516232 .rw-r--r-- 1 782 8 root root 22 May 21:23 1-3pods.yaml 516233 .rw-r--r-- 1 213Ki 432 root root 22 May 21:23 calico-crosssubnet-v3.22.2.yaml 516234 .rwx------ 1 1.2Ki 8 root root 22 May 21:23 final.sh 516235 .rwx------ 1 2.3Ki 8 root root 22 May 21:23 init.sh 516236 .rwx------ 1 1.9Ki 8 root root 22 May 21:23 master.sh 516237 .rw-r--r-- 1 4.1Ki 16 root root 22 May 21:23 metrics-server.yaml 516238 .rwx------ 1 971 8 root root 22 May 21:23 worker.sh (🍎 |kubernetes-admin@kubernetes:default) root@k8s-m:~# cd /root/DOIK/1 && ./final.sh && cd $HOME + echo '[TASK 10] Install Calico CNI' [TASK 10] Install Calico CNI + kubectl apply -f https://raw.githubusercontent.com/gasida/DOIK/main/1/calico-crosssubnet-v3.22.2.yaml //...중략... NAME: nfs-provisioner LAST DEPLOYED: Mon May 23 01:09:44 2022 NAMESPACE: kube-system STATUS: deployed REVISION: 1 TEST SUITE: None
(2) 설치 확인 작업
-
cluster 및 node 정보 확인
(🍎 |kubernetes-admin@kubernetes:default) root@k8s-m:~# kubectl cluster-info Kubernetes control plane is running at https://192.168.10.10:6443 CoreDNS is running at https://192.168.10.10:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. (🍎 |kubernetes-admin@kubernetes:default) root@k8s-m:~# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-m NotReady control-plane,master 3h40m v1.23.6 192.168.10.10 <none> Ubuntu 22.04 LTS 5.15.0-1005-aws docker://20.10.16 k8s-w1 NotReady <none> 3h39m v1.23.6 192.168.10.101 <none> Ubuntu 22.04 LTS 5.15.0-1005-aws docker://20.10.16 k8s-w2 NotReady <none> 3h39m v1.23.6 192.168.10.102 <none> Ubuntu 22.04 LTS 5.15.0-1005-aws docker://20.10.16 k8s-w3 NotReady <none> 3h39m v1.23.6 192.168.20.103 <none> Ubuntu 22.04 LTS 5.15.0-1005-aws docker://20.10.16 (🍎 |DOIK-Lab:default) root@k8s-m:~# systemctl status kubelet ● kubelet.service - kubelet: The Kubernetes Node Agent Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/kubelet.service.d └─10-kubeadm.conf Active: active (running) since Sun 2022-05-22 21:24:01 KST; 3h 50min ago Docs: https://kubernetes.io/docs/home/ Main PID: 6426 (kubelet) Tasks: 16 (limit: 4623) Memory: 39.5M CPU: 4min 17.733s CGroup: /system.slice/kubelet.service └─6426 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.co> May 23 01:09:55 k8s-m kubelet[6426]: E0523 01:09:55.691582 6426 plugins.go:752] "Error dynamically probing plugins" err="error creating>May 23 01:09:55 k8s-m kubelet[6426]: E0523 01:09:55.691782 6426 driver-call.go:262] Failed to unmarshal output for command: init, outpu>May 23 01:09:55 k8s-m kubelet[6426]: W0523 01:09:55.691796 6426 driver-call.go:149] FlexVolume: driver call failed: executable: /usr/li>May 23 01:09:55 k8s-m kubelet[6426]: E0523 01:09:55.691812 6426 plugins.go:752] "Error dynamically probing plugins" err="error creating>May 23 01:09:55 k8s-m kubelet[6426]: E0523 01:09:55.692032 6426 driver-call.go:262] Failed to unmarshal output for command: init, outpu>May 23 01:09:55 k8s-m kubelet[6426]: W0523 01:09:55.692048 6426 driver-call.go:149] FlexVolume: driver call failed: executable: /usr/li>May 23 01:09:55 k8s-m kubelet[6426]: E0523 01:09:55.692065 6426 plugins.go:752] "Error dynamically probing plugins" err="error creating>May 23 01:09:55 k8s-m kubelet[6426]: E0523 01:09:55.692552 6426 driver-call.go:262] Failed to unmarshal output for command: init, outpu>May 23 01:09:55 k8s-m kubelet[6426]: W0523 01:09:55.692563 6426 driver-call.go:149] FlexVolume: driver call failed: executable: /usr/li>May 23 01:09:55 k8s-m kubelet[6426]: E0523 01:09:55.692581 6426 plugins.go:752] "Error dynamically probing plugins" err="error creating>lines 1-23/23 (END) (🍎 |DOIK-Lab:default) root@k8s-m:~# kubectl get node -v7 I0523 01:15:39.343489 54205 loader.go:372] Config loaded from file: /root/.kube/config I0523 01:15:39.350372 54205 round_trippers.go:463] GET https://192.168.10.10:6443/api/v1/nodes?limit=500 I0523 01:15:39.350496 54205 round_trippers.go:469] Request Headers: I0523 01:15:39.350513 54205 round_trippers.go:473] Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;as=Table;v=v1beta1;g=meta.k8s.io,application/json I0523 01:15:39.350612 54205 round_trippers.go:473] User-Agent: kubectl/v1.23.6 (linux/amd64) kubernetes/ad33385 I0523 01:15:39.358562 54205 round_trippers.go:574] Response Status: 200 OK in 7 milliseconds NAME STATUS ROLES AGE VERSION k8s-m Ready control-plane,master 3h51m v1.23.6 k8s-w1 Ready <none> 3h51m v1.23.6 k8s-w2 Ready <none> 3h51m v1.23.6 k8s-w3 Ready <none> 3h51m v1.23.6 -
kube-system에 설치된 components 확인
(🍎 |DOIK-Lab:default) root@k8s-m:~# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-7c845d499-v5qfh 1/1 Running 0 6m52s kube-system calico-node-cpwhx 1/1 Running 0 6m52s kube-system calico-node-njvrx 1/1 Running 0 6m52s kube-system calico-node-sn7tg 1/1 Running 0 6m52s kube-system calico-node-x4r8b 1/1 Running 0 6m52s kube-system coredns-64897985d-856gx 1/1 Running 0 3h52m kube-system coredns-64897985d-m7l58 1/1 Running 0 3h52m kube-system etcd-k8s-m 1/1 Running 0 3h52m kube-system kube-apiserver-k8s-m 1/1 Running 0 3h52m kube-system kube-controller-manager-k8s-m 1/1 Running 0 3h52m kube-system kube-proxy-dvcnf 1/1 Running 0 3h52m kube-system kube-proxy-gxb6c 1/1 Running 0 3h52m kube-system kube-proxy-ts8ht 1/1 Running 0 3h52m kube-system kube-proxy-x6kkd 1/1 Running 0 3h52m kube-system kube-scheduler-k8s-m 1/1 Running 0 3h52m kube-system metrics-server-6c8bfb487b-r8zs2 1/1 Running 0 6m44s kube-system nfs-provisioner-nfs-subdir-external-provisioner-7d9776885d5b7h5 1/1 Running 0 6m38s local-path-storage local-path-provisioner-dbd774c5c-tjfnh 1/1 Running 0 6m44s
3. 참고 자료
- 🚀가시다님 노션🚀
Leave a comment