kubebuilder实战之二:初次体验kubebuilder

欢迎访问我的GitHub

这里分类和汇总了欣宸的全部原创(含配套源码):https://github.com/zq2599/blog_demos

系列文章链接

  1. kubebuilder实战之一:准备工作
  2. kubebuilder实战之二:初次体验kubebuilder
  3. kubebuilder实战之三:基础知识速览
  4. kubebuilder实战之四:operator需求说明和设计
  5. kubebuilder实战之五:operator编码
  6. kubebuilder实战之六:构建部署运行
  7. kubebuilder实战之七:webhook
  8. kubebuilder实战之八:知识点小记

本篇概览

本文是《kubebuilder实战》系列的第二篇,前文将kubebuilder环境准备完毕,今天咱们在此环境创建CRD和Controller,再部署到kubernetes环境并且验证是否生效,整篇文章由以下内容组成:

  1. 创建API(CRD和Controller)
  2. 构建和部署CRD
  3. 编译和运行controller
  4. 创建CRD对应的实例
  5. 删除实例并停止controller
  6. 将controller制作成docker镜像
  7. 卸载和清理

创建helloworld项目

  1. 执行以下命令,创建helloworld项目:
mkdir -p $GOPATH/src/helloworld
cd $GOPATH/src/helloworld
kubebuilder init --domain com.bolingcavalry
  1. 控制台输出类似以下内容:
[root@kubebuilder helloworld]# kubebuilder init --domain com.bolingcavalry
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.5.0
Update go.mod:
$ go mod tidy
Running make:
$ make
/root/gopath/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go build -o bin/manager main.go
Next: define a resource with:
$ kubebuilder create api
  1. 等待数分钟后创建完成,在$GOPATH/src/helloworld目录下新增以下内容,可见这是个标准的go module工程:
[root@kubebuilder ~]# tree $GOPATH/src/helloworld
/root/gopath/src/helloworld
├── bin
│   └── manager
├── config
│   ├── certmanager
│   │   ├── certificate.yaml
│   │   ├── kustomization.yaml
│   │   └── kustomizeconfig.yaml
│   ├── default
│   │   ├── kustomization.yaml
│   │   ├── manager_auth_proxy_patch.yaml
│   │   ├── manager_webhook_patch.yaml
│   │   └── webhookcainjection_patch.yaml
│   ├── manager
│   │   ├── kustomization.yaml
│   │   └── manager.yaml
│   ├── prometheus
│   │   ├── kustomization.yaml
│   │   └── monitor.yaml
│   ├── rbac
│   │   ├── auth_proxy_client_clusterrole.yaml
│   │   ├── auth_proxy_role_binding.yaml
│   │   ├── auth_proxy_role.yaml
│   │   ├── auth_proxy_service.yaml
│   │   ├── kustomization.yaml
│   │   ├── leader_election_role_binding.yaml
│   │   ├── leader_election_role.yaml
│   │   └── role_binding.yaml
│   └── webhook
│       ├── kustomization.yaml
│       ├── kustomizeconfig.yaml
│       └── service.yaml
├── Dockerfile
├── go.mod
├── go.sum
├── hack
│   └── boilerplate.go.txt
├── main.go
├── Makefile
└── PROJECT

9 directories, 30 files

创建API(CRD和Controller)

  1. 接下来要要创建资源相关的内容了,group/version/kind这三部分可以确定资源的唯一身份,命令如下:
cd $GOPATH/src/helloworld
kubebuilder create api \
--group webapp \
--version v1 \
--kind Guestbook
  1. 控制台会提醒是否创建资源(Create Resource [y/n]),输入y
  2. 接下来控制台会提醒是否创建控制器(Create Controller [y/n]),输入y
  3. kubebuilder会根据上述命令新增多个文件,如下图红框所示:
    在这里插入图片描述

构建和部署CRD

  1. kubebuilder提供的Makefile将构建和部署工作大幅度简化,执行以下命令会将最新构建的CRD部署在kubernetes上:
cd $GOPATH/src/helloworld
make install
  1. 控制台输出如下内容,提示部署成功:
[root@kubebuilder helloworld]# make install
/root/gopath/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
kustomize build config/crd | kubectl apply -f -
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.com.bolingcavalry created

编译和运行controller

  1. kubebuilder自动生成的controller源码地址是:$GOPATH/src/helloworld/controllers/guestbook_controller.go , 内容如下:
package controllers

import (
	"context"

	"github.com/go-logr/logr"
	"k8s.io/apimachinery/pkg/runtime"
	ctrl "sigs.k8s.io/controller-runtime"
	"sigs.k8s.io/controller-runtime/pkg/client"

	webappv1 "helloworld/api/v1"
)

// GuestbookReconciler reconciles a Guestbook object
type GuestbookReconciler struct {
	client.Client
	Log    logr.Logger
	Scheme *runtime.Scheme
}

// +kubebuilder:rbac:groups=webapp.com.bolingcavalry,resources=guestbooks,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=webapp.com.bolingcavalry,resources=guestbooks/status,verbs=get;update;patch

func (r *GuestbookReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
	_ = context.Background()
	_ = r.Log.WithValues("guestbook", req.NamespacedName)

	// your logic here

	return ctrl.Result{}, nil
}

func (r *GuestbookReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&webappv1.Guestbook{}).
		Complete(r)
}

  1. 本文以体验基本流程为主,不深入研究源码,所以对上面的代码仅做少量修改,用于验证是否能生效,改动如下图红框所示:
    在这里插入图片描述

  2. 执行以下命令,会编译并启动刚才修改的controller:

cd $GOPATH/src/helloworld
make run
  1. 此时控制台输出以下内容,这里要注意,controller是在kubebuilder电脑上运行的,一旦使用Ctrl+c中断控制台,就会导致controller停止:
[root@kubebuilder helloworld]# make run
/root/gopath/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
/root/gopath/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go run ./main.go
2021-01-23T20:58:35.107+0800	INFO	controller-runtime.metrics	metrics server is starting to listen	{"addr": ":8080"}
2021-01-23T20:58:35.108+0800	INFO	setup	starting manager
2021-01-23T20:58:35.108+0800	INFO	controller-runtime.manager	starting metrics server	{"path": "/metrics"}
2021-01-23T20:58:35.108+0800	INFO	controller-runtime.controller	Starting EventSource	{"controller": "guestbook", "source": "kind source: /, Kind="}
2021-01-23T20:58:35.208+0800	INFO	controller-runtime.controller	Starting Controller	{"controller": "guestbook"}
2021-01-23T20:58:35.209+0800	INFO	controller-runtime.controller	Starting workers	{"controller": "guestbook", "worker count": 1}

创建Guestbook资源的实例

  1. 现在kubernetes已经部署了Guestbook类型的CRD,而且对应的controller也已正在运行中,可以尝试创建Guestbook类型的实例了(相当于有了pod的定义后,才可以创建pod);
  2. kubebuilder已经自动创建了一个类型的部署文件:$GOPATH/src/helloworld/config/samples/webapp_v1_guestbook.yaml ,内容如下,很简单,接下来咱们就用这个文件来创建Guestbook实例:
apiVersion: webapp.com.bolingcavalry/v1
kind: Guestbook
metadata:
  name: guestbook-sample
spec:
  # Add fields here
  foo: bar
  1. 重新打开一个控制台,登录kubebuilder电脑,执行以下命令即可创建Guestbook类型的实例:
cd $GOPATH/src/helloworld
kubectl apply -f config/samples/
  1. 如下所示,控制台提示资源创建成功:
[root@kubebuilder helloworld]# kubectl apply -f config/samples/
guestbook.webapp.com.bolingcavalry/guestbook-sample created
  1. 用kubectl get命令可以看到实例已经创建:
[root@kubebuilder helloworld]# kubectl get Guestbook
NAME               AGE
guestbook-sample   112s
  1. 用命令kubectl edit Guestbook guestbook-sample编辑该实例,修改的内容如下图红框所示:
    在这里插入图片描述
  2. 此时去controller所在控制台,可以看到新增和修改的操作都有日志输出,咱们新增的日志都在里面,代码调用栈一目了然:
2021-01-24T09:51:50.418+0800	INFO	controllers.Guestbook	1. default/guestbook-sample
2021-01-24T09:51:50.418+0800	INFO	controllers.Guestbook	2. goroutine 188 [running]:
runtime/debug.Stack(0xc0002a1808, 0xc0002fc600, 0x1b)
	/root/go/src/runtime/debug/stack.go:24 +0x9f
helloworld/controllers.(*GuestbookReconciler).Reconcile(0xc0003c9dd0, 0xc0002d02f9, 0x7, 0xc0002d02e0, 0x10, 0x12f449647b, 0xc000456f30, 0xc000456ea8, 0xc000456ea0)
	/root/gopath/src/helloworld/controllers/guestbook_controller.go:49 +0x1a9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00022a480, 0x1430e00, 0xc0003e7560, 0x0)
	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:256 +0x166
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00022a480, 0xc000469600)
	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:232 +0xb0
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc00022a480)
	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:211 +0x2b
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc000292980)
	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152 +0x5f
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000292980, 0x3b9aca00, 0x0, 0x1609101, 0xc000102480)
	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153 +0x105
k8s.io/apimachinery/pkg/util/wait.Until(0xc000292980, 0x3b9aca00, 0xc000102480)
	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88 +0x4d
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:193 +0x32d

2021-01-24T09:51:50.418+0800	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "guestbook", "request": "default/guestbook-sample"}


2021-01-24T09:52:33.632+0800	INFO	controllers.Guestbook	1. default/guestbook-sample
2021-01-24T09:52:33.633+0800	INFO	controllers.Guestbook	2. goroutine 188 [running]:
runtime/debug.Stack(0xc0002a1808, 0xc0003fa5e0, 0x1b)
	/root/go/src/runtime/debug/stack.go:24 +0x9f
helloworld/controllers.(*GuestbookReconciler).Reconcile(0xc0003c9dd0, 0xc0002d02f9, 0x7, 0xc0002d02e0, 0x10, 0x1d0410fe42, 0xc000456f30, 0xc000456ea8, 0xc000456ea0)
	/root/gopath/src/helloworld/controllers/guestbook_controller.go:49 +0x1a9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00022a480, 0x1430e00, 0xc0003d24c0, 0x0)
	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:256 +0x166
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00022a480, 0xc000469600)
	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:232 +0xb0
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc00022a480)
	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:211 +0x2b
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc000292980)
	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152 +0x5f
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000292980, 0x3b9aca00, 0x0, 0x1609101, 0xc000102480)
	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153 +0x105
k8s.io/apimachinery/pkg/util/wait.Until(0xc000292980, 0x3b9aca00, 0xc000102480)
	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88 +0x4d
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:193 +0x32d

2021-01-24T09:52:33.633+0800	DEBUG	controller-runtime.controller	Successfully Reconciled	{"controller": "guestbook", "request": "default/guestbook-sample"}

删除实例并停止controller

  1. 不再需要Guestbook实例的时候,执行以下命令即可删除:
cd $GOPATH/src/helloworld
kubectl delete -f config/samples/
  1. 不再需要controller的时候,去它的控制台使用Ctrl+c中断即可;

将controller制作成docker镜像

  1. 至此,咱们已经体验过了kubebuilder的基本功能,不过实际生产环境中controller一般都会运行在kubernetes环境内,像上面这种运行在kubernetes之外的方式就不合适了,咱们来试试将其做成docker镜像然后在kubernetes环境运行;
  2. 这里有个要求,就是您要有个kubernetes可以访问的镜像仓库,例如局域网内的Harbor,或者公共的hub.docker.com,我这为了操作方便选择了hub.docker.com,使用它的前提是拥有hub.docker.com的注册帐号;
  3. 在kubebuilder电脑上,打开一个控制台,执行docker login命令登录,根据提示输入hub.docker.com的帐号和密码,这样就可以在当前控制台上执行docker push命令将镜像推送到hub.docker.com上了(这个网站的网络很差,可能要登录好几次才能成功);
  4. 执行以下命令构建docker镜像并推送到hub.docker.com,镜像名为bolingcavalry/guestbook:002
cd $GOPATH/src/helloworld
make docker-build docker-push IMG=bolingcavalry/guestbook:002
  1. hub.docker.com的网络状况不是一般的差,kubebuilder电脑上的docker一定要设置镜像加速,上述命令如果遭遇超时失败,请重试几次,此外,构建过程中还会下载诸多go模块的依赖,也需要您耐心等待,也很容易遇到网络问题,需要多次重试,所以,最好是使用局域网内搭建的Habor服务;
  2. 最终,命令执行成功后输出如下:
[root@kubebuilder helloworld]# make docker-build docker-push IMG=bolingcavalry/guestbook:002
/root/gopath/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
/root/gopath/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go test ./... -coverprofile cover.out
?   	helloworld	[no test files]
?   	helloworld/api/v1	[no test files]
ok  	helloworld/controllers	8.604s	coverage: 0.0% of statements
docker build . -t bolingcavalry/guestbook:002
Sending build context to Docker daemon  40.27MB
Step 1/14 : FROM golang:1.13 as builder
 ---> d6f3656320fe
Step 2/14 : WORKDIR /workspace
 ---> Using cache
 ---> 83d05ead1041
Step 3/14 : COPY go.mod go.mod
 ---> Using cache
 ---> ae3e15a529f4
Step 4/14 : COPY go.sum go.sum
 ---> Using cache
 ---> 082223532ccc
Step 5/14 : RUN go mod download
 ---> Using cache
 ---> bcdcfa1d65ca
Step 6/14 : COPY main.go main.go
 ---> Using cache
 ---> 81d6a629ca98
Step 7/14 : COPY api/ api/
 ---> Using cache
 ---> 75f99b174e97
Step 8/14 : COPY controllers/ controllers/
 ---> b130d9f47903
Step 9/14 : RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager main.go
 ---> Running in 768880aca19f
Removing intermediate container 768880aca19f
 ---> bb4a494d3b43
Step 10/14 : FROM gcr.io/distroless/static:nonroot
 ---> 947e6f3ed7c1
Step 11/14 : WORKDIR /
 ---> Using cache
 ---> 22cc43cef8fb
Step 12/14 : COPY --from=builder /workspace/manager .
 ---> 2137778f22c0
Step 13/14 : USER nonroot:nonroot
 ---> Running in 18295673073d
Removing intermediate container 18295673073d
 ---> f7545379ab1f
Step 14/14 : ENTRYPOINT ["/manager"]
 ---> Running in 550c47dd61dc
Removing intermediate container 550c47dd61dc
 ---> 31cb31a6b03f
Successfully built 31cb31a6b03f
Successfully tagged bolingcavalry/guestbook:002
docker push bolingcavalry/guestbook:002
The push refers to repository [docker.io/bolingcavalry/guestbook]
99035107a955: Pushed 
728501c5607d: Layer already exists 
002: digest: sha256:54f8ec88511cce5b04c5d65cc15e0f7a7b4a8afb6b235904a638bff79e3c5784 size: 739
  1. 去hub.docker.com网站看看,如下图,新镜像已经上传,这样只要任何机器只要能上网就能pull此镜像到本地使用了:
    在这里插入图片描述
  2. 镜像准备好之后,执行以下命令即可在kubernetes环境部署controller:
cd $GOPATH/src/helloworld
make deploy IMG=bolingcavalry/guestbook:002
  1. 控制台会提示各类资源被创建(rbac居多):
[root@kubebuilder ~]# cd $GOPATH/src/helloworld
[root@kubebuilder helloworld]# make deploy IMG=bolingcavalry/guestbook:002
/root/gopath/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
cd config/manager && kustomize edit set image controller=bolingcavalry/guestbook:002
kustomize build config/default | kubectl apply -f -
namespace/helloworld-system created
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.com.bolingcavalry configured
role.rbac.authorization.k8s.io/helloworld-leader-election-role created
clusterrole.rbac.authorization.k8s.io/helloworld-manager-role created
clusterrole.rbac.authorization.k8s.io/helloworld-proxy-role created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole
clusterrole.rbac.authorization.k8s.io/helloworld-metrics-reader created
rolebinding.rbac.authorization.k8s.io/helloworld-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/helloworld-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/helloworld-proxy-rolebinding created
service/helloworld-controller-manager-metrics-service created
deployment.apps/helloworld-controller-manager created
  1. 此时去看kubernetes环境的pod,发现确实已经新增了controller,如下图红框:
    在这里插入图片描述

11.细心的您应该会发现上图黄框中显示这个pod实际上有两个容器,用kubectl describe命令细看,分别是kube-rbac-proxy和manager,如下图:
在这里插入图片描述
11. 由于有两个容器,那么查看日志时就要指定其中一个了,咱们的controller对应的是manager容器,因此查看日志的命令是:

kubectl logs -f \
helloworld-controller-manager-689d4b6f5b-h9pzg \
-n helloworld-system \
-c manager
  1. 再次创建Guestbook资源的实例,依旧是kubectl apply -f config/samples/命令,再去看manager容器的日志,可见咱们修改的内容已经打印出来了:
    在这里插入图片描述

卸载和清理

  • 体验完毕后,如果想把前面创建的资源和CRD全部清理掉,可以执行以下命令:
cd $GOPATH/src/helloworld
make uninstall
  • 至此,通过kubebuilder创建Operator相关资源的基本流程,咱们已经体验过一遍了,本篇以熟悉工具和流程为主,并未体验到Operator实质性的强大功能,这些都留待后面的章节吧,咱们逐步深入学习实践;

你不孤单,欣宸原创一路相伴

  1. Java系列
  2. Spring系列
  3. Docker系列
  4. kubernetes系列
  5. 数据库+中间件系列
  6. DevOps系列

欢迎关注公众号:程序员欣宸

微信搜索「程序员欣宸」,我是欣宸,期待与您一同畅游Java世界…

已标记关键词 清除标记
©️2020 CSDN 皮肤主题: 书香水墨 设计师:CSDN官方博客 返回首页