基于K8s部署ECK(Elastic Cloud on Kubernetes)

2021/10/31 13:10:24

本文主要是介绍基于K8s部署ECK(Elastic Cloud on Kubernetes),对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!

背景
ECK,也就是Elastic Cloud on Kubernetes的缩写,可以基于K8s operator在Kubernetes集群来自动化部署、管理、编排Elasticsearch、Kibana、APM Server服务。

ECK目前的features:

Elasticsearch, Kibana 和 APM Server的部署;
TLS证书管理;
安全的Elasticsearch cluster 配置和拓扑变动管理;
使用PV和PVC;
定制K8s node上的系统配置和属性;
在本文中,Gemfield将介绍如何使用ECK来进行部署,以及如何使用一个python的例子来访问部署好的Elasticsearch服务。说来也巧,Gemfield写本文的时候,ECK 1.0正式版刚刚发布了2天。

部署
官方部署是基于K8s operator的方式进行部署的,因此只支持K8s 1.12+的kubernetes版本;

1,安装CustomResourceDefinition和operator

gemfield@civilnet:~/ES$ kubectl apply -f https://download.elastic.co/downloads/eck/1.0.0/all-in-one.yaml
customresourcedefinition.apiextensions.k8s.io/apmservers.apm.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/elasticsearches.elasticsearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/kibanas.kibana.k8s.elastic.co created
clusterrole.rbac.authorization.k8s.io/elastic-operator created
clusterrolebinding.rbac.authorization.k8s.io/elastic-operator created
namespace/elastic-system created
statefulset.apps/elastic-operator created
serviceaccount/elastic-operator created
validatingwebhookconfiguration.admissionregistration.k8s.io/elastic-webhook.k8s.elastic.co created
service/elastic-webhook-server created
secret/elastic-webhook-server-cert created
2,查看operator的log

kubectl -n elastic-system logs -f statefulset.apps/elastic-operator
3,查看新安装的CustomResourceDefinition

gemfield@civilnet:~/ES$ kubectl get CustomResourceDefinition
NAME CREATED AT
apmservers.apm.k8s.elastic.co 2020-02-06T11:17:38Z
elasticsearches.elasticsearch.k8s.elastic.co 2020-02-06T11:17:38Z
kibanas.kibana.k8s.elastic.co 2020-02-06T11:17:38Z
4,查看新安装的service

gemfield@civilnet:~/ES$ kubectl -n elastic-system get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
elastic-webhook-server ClusterIP 10.103.189.140 443/TCP 5m1s

gemfield@civilnet:~/ES$ kubectl -n elastic-system get po
NAME READY STATUS RESTARTS AGE
elastic-operator-0 1/1 Running 0 5m48s

gemfield@civilnet:~/ES$ kubectl -n elastic-system get statefulset
NAME READY AGE
elastic-operator 1/1 6m11s

gemfield@civilnet:~/ES$ kubectl -n elastic-system get ServiceAccount
NAME SECRETS AGE
default 1 7m43s
elastic-operator 1 7m43s

gemfield@civilnet:~/ES$ kubectl -n elastic-system get ValidatingWebhookConfiguration
NAME CREATED AT
elastic-webhook.k8s.elastic.co 2020-02-06T11:17:38Z

gemfield@civilnet:~/ES$ kubectl -n elastic-system get Secret
NAME TYPE DATA AGE
default-token-bx7vs kubernetes.io/service-account-token 3 8m20s
elastic-operator-token-jptwl kubernetes.io/service-account-token 3 8m20s
elastic-webhook-server-cert Opaque 2 8m20s
5,查看elasticsearch资源

当然,目前还没有

gemfield@civilnet:~/ES$ kubectl get elasticsearch
No resources found.
于是新建gemfield-elasticsearch.yaml文件,内容如下:

gemfield@civilnet:~/ES$ cat gemfield-elasticsearch.yaml
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: gemfieldes
spec:
version: 7.5.2
nodeSets:

  • name: default
    count: 1
    config:
    node.master: true
    node.data: true
    node.ingest: true
    node.store.allow_mmap: false
    volumeClaimTemplates:
    • metadata:
      name: elasticsearch-data
      spec:
      accessModes:
      • ReadWriteOnce
        resources:
        requests:
        storage: 10Gi
        storageClassName: rook-ceph-block
        这里的PVC的大小以及storageClassName都可以换成自己集群中的相关值。然后部署elasticsearch:

kubectl apply -f gemfield-elasticsearch.yaml
查看状态,这个时候就有了:

gemfield@civilnet:~/ES$ kubectl get elasticsearch
NAME HEALTH NODES VERSION PHASE AGE
gemfieldes unknown 7.5.2 ApplyingChanges 97s



gemfield@civilnet:~/ES$ kubectl get elasticsearch
NAME HEALTH NODES VERSION PHASE AGE
gemfieldes green 1 7.5.2 Ready 48s
直到HEALTH的状态为GREEN。

6,查看elasticsearch service

然后查看service:

gemfield@civilnet:~/ES$ kubectl get service gemfieldes-es-http
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gemfieldes-es-http ClusterIP 10.108.80.75 9200/TCP 83s
对于Elasticsearch、Kibana 和ApmServer这三种资源来说,K8s operator都会无了对应的service,名字为:-[es|kb|apm]-http,类型默认为ClusterIP。ClusterIP类型的service会把服务暴露在集群内网IP上,因此只能在集群内部的Pod上来访问。那怎么让外网可以访问呢?可以使用下面的选项:

apiVersion: .k8s.elastic.co/v1
kind:
metadata:
name: gemfieldes
spec:
version: 7.5.2
http:
service:
spec:
type: LoadBalancer
重点是这里的type,和K8s service的type一样。

7,ECK的TLS证书

这里的证书说的是HTTP层的,用于在不同node之间Elasticsearch内部通信的传输层的证书是ECK内部管理且不可配置的。

默认情况下,operator会维护一个用自定义的CA然后自签的证书,用于Elasticsearch, Kibana 和APM Server。这个自定义的CA、证书、私钥都存放在独立的Secret中:

gemfield@civilnet:~/ES$ kubectl get secret | grep es-http
gemfieldes-es-http-ca-internal Opaque 2 22h
gemfieldes-es-http-certs-internal Opaque 3 22h
gemfieldes-es-http-certs-public Opaque 2 22h
其中,公开的证书存放在名为-[es|kb]-http-certs-public的Secert中:

gemfield@civilnet:~/ES$ kubectl get secret gemfieldes-es-http-certs-public -o go-template=’{{index .data “tls.crt” | base64decode }}’
-----BEGIN CERTIFICATE-----
MIIDrzCCApegAwIBAgIQbXsHRaaZsLACr7MTtzSNkTANBgkqhkiG9w0BAQsFADAv

eHTZaZSMCMx8ZwF4WqFqNO0Q45GWTrk9aS0DcELppPeNl9A=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIDKTCCAhGgAwIBAgIPYzCgkK7dz6/rq4rCo9vEMA0GCSqGSIb3DQEBCwUAMC8x

5teU1SYvf8JgHJecJNPl/sZMjVmwR0I7fcHeNceaQajBrG7SBgN9j0GJX4CZ
-----END CERTIFICATE-----
你可以显式的禁用TLS:

spec:
http:
tls:
selfSignedCertificate:
disabled: true
验证服务是否已经正常运行
1,获取访问服务的用户名和密码

用户名默认为elastic,密码为bc7fzkplf8kn5266dgggvhs2,通过下面的命令获得。

gemfield@civilnet:~/ES$ kubectl get secret gemfieldes-es-elastic-user -o=jsonpath=’{.data.elastic}’ | base64 --decode
bc7fzkplf8kn5266dgggvhs2
2,使用curl命令访问服务

root@gemfield-mysql-client-5596f7ff6f-5l8c2:/# curl -k https://elastic:bc7fzkplf8kn5266dgggvhs2@gemfieldes-es-http:9200
{
“name” : “gemfieldes-es-default-0”,
“cluster_name” : “gemfieldes”,
“cluster_uuid” : “mzYCVK6GSmuxwXF8SfRXZQ”,
“version” : {
“number” : “7.5.2”,
“build_flavor” : “default”,
“build_type” : “docker”,
“build_hash” : “8bec50e1e0ad29dad5653712cf3bb580cd1afcdf”,
“build_date” : “2020-01-15T12:11:52.313576Z”,
“build_snapshot” : false,
“lucene_version” : “8.3.0”,
“minimum_wire_compatibility_version” : “6.8.0”,
“minimum_index_compatibility_version” : “6.0.0-beta1”
},
“tagline” : “You Know, for Search”
}
如何管理ECK的资源
使用request关键字和limit关键字,和K8s类似,就不细说了。

spec:
nodeSets:

  • name: default
    count: 1
    podTemplate:
    spec:
    containers:
    - name: elasticsearch
    env:
    - name: ES_JAVA_OPTS
    value: -Xms2g -Xmx2g
    resources:
    requests:
    memory: 4Gi
    cpu: 0.5
    limits:
    memory: 4Gi
    cpu: 2
    你当然可以不配置这些数据,那就使用默认的。富贵论坛默认的值如下所示:

APM Server的Requests是512MB,Limits是512MB;

Elasticsearch的Requests是2GB,Limits是2GB;

Kibana的Requests是1GB,Limits是1GB;

如何卸载ECK
1,先删除所有的elastic资源

这一步会删除所有的Elasticsearch、Kibana和APM Server的资源,包括(Pods、secrets、services等):

kubectl get namespaces --no-headers -o custom-columns=:metadata.name
| xargs -n1 kubectl delete elastic --all -n
2,再卸载operator

kubectl delete -f https://download.elastic.co/downloads/eck/1.0.0/all-in-one.yaml
使用Python访问ES
Gemfield使用的是Python3。

1,安装elasticsearch模块

pip3 install elasticsearch
2,编写脚本如下

import ssl
from datetime import datetime
from elasticsearch import Elasticsearch
#from elasticsearch.connection import create_ssl_context
#ssl_context = create_ssl_context()
#ssl_context.check_hostname = False;
#ssl_context.verify_mode = ssl.CERT_NONE

es = Elasticsearch(hosts=“https://elastic:bc7fzkplf8kn5266dgggvhs2@gemfieldes-es-http:9200/”,ca_certs=False,verify_certs=False)

print("gemfield begin: " + str(datetime.now()))
for key in range(10000000):
print(key)
es.index(index=‘messages’,doc_type=‘message’,body={‘message’: ‘gemfield is a civilnet maintainer’,‘name’:‘gemfield’,‘age’:30,‘id’:key})

print("gemfield end: " + str(datetime.now()))
这个脚本里,Gemfield禁用了https中的证书检查。

3,使用curl命令搜索结果

对ECK的扩展
本文前面部署的ECK只是一个node,在这个章节中,Gemfield将介绍如何扩展ECK。

1,node.store.allow_mmap

在这之前,需要介绍一个对ECK很重要的kernel参数:

gemfield@civilnet:~/ES$ sudo sysctl -a | grep max_map
vm.max_map_count = 65530
按照ECK的官方建议,这个值应该修改为262144:

sudo sysctl -w vm.max_map_count=262144
要持久化这个修改,还需要修改/etc/sysctl.conf文件。

这个修改后,就把gemfield-elasticsearch.yaml文件中的"node.store.allow_mmap: false"去掉,内容如下:

gemfield@civilnet:~/ES$ cat gemfield-elasticsearch.yaml
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: gemfieldes
spec:
version: 7.5.2
nodeSets:

  • name: default
    count: 1
    config:
    node.master: true
    node.data: true
    node.ingest: true
    volumeClaimTemplates:
    • metadata:
      name: elasticsearch-data
      spec:
      accessModes:
      • ReadWriteOnce
        resources:
        requests:
        storage: 10Gi
        storageClassName: rook-ceph-block
        从而可以提高性能。

2,更改node的数量

你可以通过改变count的值来更改node的数量:

gemfield@civilnet:~/ES$ cat gemfield-elasticsearch.yaml
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: gemfieldes
spec:
version: 7.5.2
nodeSets:

  • name: default
    count: 5
    config:

    3,定制elasticsearch

通过PodTemplate设置elasticsearch的资源:比如java堆的大小,K8s上cpu和ram资源等;通过http字段设置是否启动https:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: gemfieldes
spec:
version: 7.5.2
http:
tls:
selfSignedCertificate:
disabled: true
nodeSets:

  • name: default
    count: 1
    podTemplate:
    spec:
    containers:
    - name: elasticsearch
    env:
    - name: ES_JAVA_OPTS
    value: -Xms12g -Xmx12g
    resources:
    requests:
    memory: 16Gi
    limits:
    memory: 16Gi
    config:
    node.master: true
    node.data: true
    node.ingest: true
    volumeClaimTemplates:
    • metadata:
      name: elasticsearch-data
      spec:
      accessModes:
      • ReadWriteOnce
        resources:
        requests:
        storage: 100Gi
        storageClassName: rook-ceph-block


这篇关于基于K8s部署ECK(Elastic Cloud on Kubernetes)的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!


扫一扫关注最新编程教程