- Created by Terence Yim, last modified by Robin Rielley on Apr 12, 2022
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
« Previous Version 33 Current »
CDAPCDAP installation on Kubernetes was introduced in CDAP 6.2.3.
This document describes how to install CDAP on a Kubernetes cluster.
Dependencies
This section describes the infrastructure and software dependencies for operating CDAP in Kubernetes.
Kubernetes cluster
CDAP supports using Kubernetes (k8s) as the distributed resource manager. When CDAP is deployed to a k8s cluster, it spawns multiple Deployments and StatefulSets for running various CDAP services. The following diagram shows each of the CDAP services in the Kubernetes cluster:
The CDAP operator is responsible for deploying and managing all the CDAP services inside the cluster. The CDAP operator also supports managing multiple CDAP instances within the same k8s cluster. If multiple CDAP instances are deployed to the same k8s cluster, It is recommended to deploy them to different namespaces to provide better isolation.
Limitations
Currently CDAP only supports running one replica (pod) per service, except for the Preview Runner. Failure resiliency is handled by k8s to have pod restart upon failure. For pods created by StatefulSets, it relies on the infrastructure to have persistent volumes being re-mountable to the new pod, which potentially could be on a different machine.
Another limitation of operating CDAP in Kubernetes is that it does not support native compute profile. This means all user program executions are external to the Kubernetes cluster, and require a Hadoop cluster for program executions.
PostgreSQL database
CDAP needs a shared storage for its own metadata, such as deployed artifacts and applications, run histories, preferences, lineage information, and many more. Currently, CDAP supports both PostgreSQL and HBase as the metadata store. When running CDAP in Kubernetes, we recommend using PostgreSQL.
Elasticsearch
CDAP has support for metadata search, and it is backed by either Elasticsearch or HBase. In the Kubernetes environment, Elasticsearch is recommended. You can either configure CDAP to use an existing Elasticsearch cluster or run an Elasticsearch in Kubernetes by using the Elasticsearch Operator.
Hadoop Compatible File System (HCFS)
CDAP stores artifacts and runtime information through the HDFS API. Any of the HCFS implementations is supported.
Installation
This section describes the steps to deploy CDAP on Kubernetes.
Prerequisites
An operational Kubernetes cluster.
Recommended to have 64 GB of memory resources and 20 available virtual CPU for production deployment.
For better security, the Kubernetes cluster should have RBAC enabled.
Have kubectl set up to connect to the Kubernetes cluster.
A PostgreSQL database that is reachable from the Kubernetes cluster.
An Elasticsearch instance that is reachable from the Kubernetes cluster.
Refer to the Appendix section on how to set up an Elasticsearch instance inside the Kubernetes cluster.
Deploy CDAP Operator
CDAP provides a CDAP operator for easy deployment and management of CDAP in Kubernetes. You can deploy the following YAML to create all the necessary resources to have the operator running in the Kubernetes cluster, inside the cdap-system namespace.
cdap-operator.yaml
# Create operator namespace
apiVersion: v1
kind: Namespace
metadata:
name: cdap-system
labels:
name: cdap-system
control-plane: cdap-operator
---
# Create operator service account
apiVersion: v1
kind: ServiceAccount
metadata:
name: cdap-operator
namespace: cdap-system
labels:
control-plane: cdap-operator
---
# Source cdap-operator/config/rbac/cdapmaster_editor_role.yaml
# permissions to do edit cdapmasters.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cdapmaster-editor-role
rules:
- apiGroups:
- cdap.cdap.io
resources:
- cdapmasters
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- cdap.cdap.io
resources:
- cdapmasters/status
verbs:
- get
- patch
- update
---
# Source cdap-operator/config/rbac/cdapmaster_viewer_role.yaml
# permissions to do viewer cdapmasters.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cdapmaster-viewer-role
rules:
- apiGroups:
- cdap.cdap.io
resources:
- cdapmasters
verbs:
- get
- list
- watch
- apiGroups:
- cdap.cdap.io
resources:
- cdapmasters/status
verbs:
- get
---
# Source cdap-operator/config/rbac/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: cdap-operator-role
rules:
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- apps
resources:
- deployments
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- apps
resources:
- deployments/status
verbs:
- get
- patch
- update
- apiGroups:
- apps
resources:
- statefulsets
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- batch
resources:
- jobs
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- cdap.cdap.io
resources:
- cdapmasters
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- cdap.cdap.io
resources:
- cdapmasters/status
verbs:
- get
- patch
- update
---
# Source cdap-operator/config/rbac/role_binding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cdap-operator-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cdap-operator-role
subjects:
- kind: ServiceAccount
name: cdap-operator
namespace: cdap-system
---
# Source cdap-operator/config/crd/bases/cdap.cdap.io_cdapmasters.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.2.4
creationTimestamp: null
name: cdapmasters.cdap.cdap.io
spec:
group: cdap.cdap.io
names:
kind: CDAPMaster
listKind: CDAPMasterList
plural: cdapmasters
singular: cdapmaster
scope: Namespaced
validation:
openAPIV3Schema:
description: CDAPMaster is the Schema for the cdapmasters API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: "CDAPMasterSpec defines the desired state of CDAPMaster \n
Important notes: * The field name of each service MUST match the constant
values of ServiceName in constants.go as reflection is used to find
field value. * For services that are optional (i.e. may or may not be
required for CDAP to be operational), their service specification fields
are pointers. By default, these optional services are disabled. Set to
non-nil to enable them."
properties:
appFabric:
description: AppFabric is specification for the CDAP app-fabric service.
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
storageClassName:
description: StorageClassName is the name of the StorageClass for
the persistent volume used by the service.
type: string
storageSize:
description: StorageSize is specification for the persistent volume
size used by the service.
type: string
type: object
config:
additionalProperties:
type: string
description: Config is a set of configurations that goes into cdap-site.xml.
type: object
configMapVolumes:
additionalProperties:
type: string
description: ConfigMapVolumes defines a map from ConfigMap names to
volume mount path. Key is the configmap object name. Value is the
mount path. This adds ConfigMap data to the directory specified by
the volume mount path.
type: object
image:
description: Image is the docker image name for the CDAP backend.
type: string
imagePullPolicy:
description: ImagePullPolicy is the policy for pulling docker images
on Pod creation.
type: string
locationURI:
description: LocationURI is an URI specifying an object storage for
CDAP.
type: string
logLevels:
additionalProperties:
type: string
description: LogLevels is a set of logger name to log level settings.
type: object
logs:
description: Logs is specification for the CDAP logging service.
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
storageClassName:
description: StorageClassName is the name of the StorageClass for
the persistent volume used by the service.
type: string
storageSize:
description: StorageSize is specification for the persistent volume
size used by the service.
type: string
type: object
messaging:
description: Messaging is specification for the CDAP messaging service.
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
storageClassName:
description: StorageClassName is the name of the StorageClass for
the persistent volume used by the service.
type: string
storageSize:
description: StorageSize is specification for the persistent volume
size used by the service.
type: string
type: object
metadata:
description: Metadata is specification for the CDAP metadata service.
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
storageClassName:
description: StorageClassName is the name of the StorageClass for
the persistent volume used by the service.
type: string
storageSize:
description: StorageSize is specification for the persistent volume
size used by the service.
type: string
type: object
metrics:
description: Metrics is specification for the CDAP metrics service.
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
storageClassName:
description: StorageClassName is the name of the StorageClass for
the persistent volume used by the service.
type: string
storageSize:
description: StorageSize is specification for the persistent volume
size used by the service.
type: string
type: object
preview:
description: Preview is specification for the CDAP preview service.
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
storageClassName:
description: StorageClassName is the name of the StorageClass for
the persistent volume used by the service.
type: string
storageSize:
description: StorageSize is specification for the persistent volume
size used by the service.
type: string
type: object
router:
description: Router is specification for the CDAP router service.
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
replicas:
description: Replicas is number of replicas for the service.
format: int32
type: integer
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
servicePort:
description: ServicePort is the port number for the service.
format: int32
type: integer
serviceType:
description: ServiceType is the service type in kubernetes, default
is NodePort.
type: string
type: object
runtime:
description: 'Runtime is specification for the CDAP runtime service.
This is an optional service and may not be required for CDAP to be
operational. To disable this service: either omit or set the field
to nil To enable this service: set it to a pointer to a RuntimeSpec
struct (can be an empty struct)'
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
storageClassName:
description: StorageClassName is the name of the StorageClass for
the persistent volume used by the service.
type: string
storageSize:
description: StorageSize is specification for the persistent volume
size used by the service.
type: string
type: object
securitySecret:
description: SecuritySecret is secret that contains security related
configurations for CDAP.
type: string
serviceAccountName:
description: ServiceAccountName is the service account for all the service
pods.
type: string
systemappconfigs:
additionalProperties:
type: string
description: SystemAppConfigs specifies configs used by CDAP to run
system apps dynamically. Each entry is of format <filename, json app
config> which will create a separate system config file with entry
value as file content.
type: object
userInterface:
description: UserInterface is specification for the CDAP UI service.
properties:
env:
description: Env is a list of environment variables for the master
service container.
items:
description: EnvVar represents an environment variable present
in a Container.
properties:
name:
description: Name of the environment variable. Must be a C_IDENTIFIER.
type: string
value:
description: 'Variable references $(VAR_NAME) are expanded
using the previous defined environment variables in the
container and any service environment variables. If a variable
cannot be resolved, the reference in the input string will
be unchanged. The $(VAR_NAME) syntax can be escaped with
a double $$, ie: $$(VAR_NAME). Escaped references will never
be expanded, regardless of whether the variable exists or
not. Defaults to "".'
type: string
valueFrom:
description: Source for the environment variable's value.
Cannot be used if value is not empty.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
key:
description: The key to select.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the ConfigMap or its
key must be defined
type: boolean
required:
- key
type: object
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, metadata.labels, metadata.annotations,
spec.nodeName, spec.serviceAccountName, status.hostIP,
status.podIP, status.podIPs.'
properties:
apiVersion:
description: Version of the schema the FieldPath is
written in terms of, defaults to "v1".
type: string
fieldPath:
description: Path of the field to select in the specified
API version.
type: string
required:
- fieldPath
type: object
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
limits.ephemeral-storage, requests.cpu, requests.memory
and requests.ephemeral-storage) are currently supported.'
properties:
containerName:
description: 'Container name: required for volumes,
optional for env vars'
type: string
divisor:
description: Specifies the output format of the exposed
resources, defaults to "1"
type: string
resource:
description: 'Required: resource to select'
type: string
required:
- resource
type: object
secretKeyRef:
description: Selects a key of a secret in the pod's namespace
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind,
uid?'
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
type: object
required:
- name
type: object
type: array
metadata:
description: Metadata for the service.
type: object
nodeSelector:
additionalProperties:
type: string
description: NodeSelector is a selector which must be true for the
pod to fit on a node.
type: object
priorityClassName:
description: PriorityClassName is to specify the priority of the
pods for this service.
type: string
replicas:
description: Replicas is number of replicas for the service.
format: int32
type: integer
resources:
description: Resources are Compute resources required by the service.
properties:
limits:
additionalProperties:
type: string
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
type: string
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
runtimeClassName:
description: RuntimeClassName refers to a RuntimeClass object in
the node.k8s.io group, which should be used to run pods for this
service. If no RuntimeClass resource matches the named class,
pods will not be running.
type: string
serviceAccountName:
description: ServiceAccountName overrides the service account for
the service pods.
type: string
servicePort:
description: ServicePort is the port number for the service.
format: int32
type: integer
serviceType:
description: ServiceType is the service type in kubernetes, default
is NodePort.
type: string
type: object
userInterfaceImage:
description: UserInterfaceImage is the docker image name for the CDAP
UI.
type: string
required:
- locationURI
type: object
status:
description: CDAPMasterStatus defines the observed state of CDAPMaster
properties:
components:
description: Object status array for all matching objects
items:
description: ObjectStatus is a generic status holder for objects
properties:
group:
description: Object group
type: string
kind:
description: Kind of object
type: string
link:
description: Link to object
type: string
name:
description: Name of object
type: string
pdb:
description: PDB status
properties:
currenthealthy:
description: currentHealthy
format: int32
type: integer
desiredhealthy:
description: desiredHealthy
format: int32
type: integer
required:
- currenthealthy
- desiredhealthy
type: object
status:
description: 'Status. Values: InProgress, Ready, Unknown'
type: string
sts:
description: StatefulSet status
properties:
currentcount:
description: CurrentReplicas defines the no of MySQL instances
that are created
format: int32
type: integer
progress:
description: 'progress is a fuzzy indicator. Interpret as
a percentage (0-100) eg: for statefulsets, progress = 100*readyreplicas/replicas'
format: int32
type: integer
readycount:
description: ReadyReplicas defines the no of MySQL instances
that are ready
format: int32
type: integer
replicas:
description: Replicas defines the no of MySQL instances desired
format: int32
type: integer
required:
- currentcount
- progress
- readycount
- replicas
type: object
type: object
type: array
conditions:
description: Conditions represents the latest state of the object
items:
description: Condition describes the state of an object at a certain
point.
properties:
lastTransitionTime:
description: Last time the condition transitioned from one status
to another.
format: date-time
type: string
lastUpdateTime:
description: Last time the condition was probed
format: date-time
type: string
message:
description: A human readable message indicating details about
the transition.
type: string
reason:
description: The reason for the condition's last transition.
type: string
status:
description: Status of the condition, one of True, False, Unknown.
type: string
type:
description: Type of condition.
type: string
required:
- status
- type
type: object
type: array
downgradeStartTimeMillis:
description: DowngradeStartTimeMillis is the start time in milliseconds
of the downgrade process
format: int64
type: integer
imageToUse:
description: ImageToUse is the Docker image of CDAP backend the operator
uses to deploy.
type: string
observedGeneration:
description: ObservedGeneration is the most recent generation observed.
It corresponds to the Object's generation, which is updated on mutation
by the API Server.
format: int64
type: integer
upgradeStartTimeMillis:
description: UpgradeStartTimeMillis is the start time in milliseconds
of the upgrade process
format: int64
type: integer
userInterfaceImageToUse:
description: UserInterfaceImageToUse is the Docker image of CDAP UI
the operator uses to deploy.
type: string
type: object
type: object
version: v1alpha1
versions:
- name: v1alpha1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# StatefulSet for running the cdap controller
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: cdap-controller
namespace: cdap-system
labels:
control-plane: cdap-operator
spec:
selector:
matchLabels:
control-plane: cdap-operator
serviceName: cdap-operator-service
template:
metadata:
labels:
control-plane: cdap-operator
spec:
serviceAccountName: cdap-operator
containers:
- command:
- /manager
image: gcr.io/cdapio/cdap-controller:latest
name: manager
resources:
limits:
cpu: 100m
memory: 30Mi
requests:
cpu: 100m
memory: 20Mi
terminationGracePeriodSeconds: 10
Create RBAC Roles and RoleBinding
CDAP interacts with Kubernetes for configuration, service discovery, and also workload management. Deploying the following YAML file will create the necessary set of RBAC Roles and RoleBinding to the service account called cdap.
cdap-roles.yaml
# Create cdap service account apiVersion: v1 kind: ServiceAccount metadata: name: cdap --- # Create cdap role apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: cdap-role rules: - apiGroups: - "" resources: - configmaps verbs: - create - delete - get - list - patch - update - watch - apiGroups: - "" resources: - secrets verbs: - create - delete - get - list - patch - update - watch - apiGroups: - "" resources: - pods verbs: - create - get - list - watch - delete - deletecollection - apiGroups: - "" resources: - services verbs: - create - delete - get - list - patch - update - watch - apiGroups: - "" resources: - persistentvolumeclaims verbs: - deletecollection - apiGroups: - apps resources: - deployments verbs: - create - delete - get - list - patch - update - watch - apiGroups: - apps resources: - statefulsets verbs: - create - delete - get - list - patch - update - watch - apiGroups: - apps resources: - replicasets verbs: - get - list - update - watch - apiGroups: - batch resources: - jobs verbs: - create - delete - get - list - patch - update - watch --- # Create cdap RoleBinding apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: cdap-rolebinding roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: cdap-role subjects: - kind: ServiceAccount name: cdap
Prepare the secret token for CDAP
We need to set up a secret in Kubernetes to provide the cdap-security.xml file to CDAP, which will contain the PostgreSQL and Elasticsearch password. The following command assumes the database username and password are in the environment variables DB_USER and DB_PASS respectively. For Elasticsearch authentication, it expects that the username and password comes from the ES_USER and ES_PASS environment variables.
# Create the content of the cdap-security.xml
export CDAP_SECURITY=$(cat << EOF | base64 | tr -d '\n'
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>data.storage.sql.jdbc.username</name>
<value>${DB_USER}</value>
</property>
<property>
<name>data.storage.sql.jdbc.password</name>
<value>${DB_PASS}</value>
</property>
<property>
<name>metadata.elasticsearch.credentials.username</name>
<value>${ES_USER}</value>
</property>
<property>
<name>metadata.elasticsearch.credentials.password</name>
<value>${ES_PASS}</value>
</property>
</configuration>
EOF
)
# Create the secret
cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: cdap-security
type: Opaque
data:
cdap-security.xml: $CDAP_SECURITY
EOF
Deploy CDAP
Finally we are ready to deploy CDAP into the Kubernetes cluster. The following YAML provides a simple example. You will need to replace the locationURI with an HCFS compatible file system (e.g. HDFS, Google Cloud Storage, or Amazon AWS). Also, the data.storage.sql.jdbc.connection.url should be configured to point to a PostgreSQL database. Refer to cdap-default.xml for an explanation about the configurations.
cdap.yaml
apiVersion: cdap.cdap.io/v1alpha1
kind: CDAPMaster
metadata:
name: cdap
spec:
appFabric: {}
logs: {}
messaging: {}
metadata: {}
metrics: {}
preview: {}
router: {}
runtime: {}
userInterface: {}
locationURI: <hcfs-uri>
serviceAccountName: cdap
securitySecret: cdap-security
config:
enable.preview: "true"
data.storage.implementation: postgresql
data.storage.sql.jdbc.connection.url: jdbc:postgresql://<postgresql-host>/cdap
data.storage.sql.jdbc.driver.name: org.postgresql.Driver
metadata.storage.implementation: elastic
metadata.elasticsearch.cluster.hosts: https://es-cluster-es-http:9200
metadata.elasticsearch.tls.verify: "false"
hdfs.user: root
preview.max.runs: "1"
preview.poller.count: "1"
preview.runner.container.count: "5"
preview.waiting.queue.capacity: "100"
preview.waiting.queue.timeout.seconds: "3600"
provisioner.system.properties.gcp-dataproc.preferExternalIP: "true"
logLevels:
io.cdap.cdap: DEBUG
io.cdap.wrangler: DEBUG
org.apache.hadoop.hdfs.DataStreamer: ERROR
org.spark_project: WARN
You can also configure each of the CDAP services with different cpu, memory, storage, and environments. The following is a simple example that shows how to change the memory and disk size for the appFabric service.
appFabric:
env:
- name: OPTS
value: -XX:+UseG1GC -XX:+ExitOnOutOfMemoryError
resources:
requests:
memory: 8000Mi
storageSize: 200Gi
Refer to the Custom Resource Definition (CRD) for all the supported settings.
You can verify CDAP is running correctly by listing out the pods in the Kubernetes cluster.
kubectl get pods -l cdap.instance
NAME READY STATUS RESTARTS AGE cdap-cdap-appfabric-0 1/1 Running 0 114s cdap-cdap-logs-0 1/1 Running 0 2m6s cdap-cdap-messaging-0 1/1 Running 0 2m6s cdap-cdap-metadata-54db5876dc-kplkw 1/1 Running 0 2m6s cdap-cdap-metrics-0 1/1 Running 0 2m6s cdap-cdap-preview-0 1/1 Running 0 119s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-0 1/1 Running 0 79s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-1 1/1 Running 0 79s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-2 1/1 Running 0 79s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-3 1/1 Running 0 79s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-4 1/1 Running 0 79s cdap-cdap-router-987b785c9-dfcqx 1/1 Running 0 2m6s cdap-cdap-runtime-0 1/1 Running 0 2m6s cdap-cdap-service-system-pipeline-studio-6e8722c4-a064-4b2gx2tc 1/1 Running 0 17s cdap-cdap-userinterface-877f4555d-qvqmw 1/1 Running 0 2m5s
After CDAP is fully up and running, both the UI and REST can be accessed via the user-interface and router services exposed by CDAP.
kubectl get service -l cdap.instance
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE cdap-cdap-router NodePort 10.192.14.102 <none> 11015:31426/TCP 4m41s cdap-cdap-userinterface NodePort 10.192.14.106 <none> 11011:30504/TCP 4m41s
For quick testing, you can use kubectl port-foward to provide access to the CDAP service. For example, you can expose the user interface and then access it through localhost:11011 from the browser.
kubectl port-forward service/cdap-cdap-userinterface 11011
For production use cases, it is better to expose the CDAP services through a load balancer. Consult with your Kubernetes provider for how to deploy a load balancer.
Enable Authentication Service
To enable the Authentication Service in K8s environment to provide Perimeter Security, extra configurations are needed in the CDAP YAML file.
Set the following configurations in the CDAP YAML file "config:" section.
# Enable perimeter security security.enabled: "true" # A key file generated by the AuthenticationTool that is mapped into the pod via k8s secret (see below for instructions) security.data.keyfile.path: "/etc/cdap/auth/auth.key" # Disable kerberos (it is defaulted to true when security.enabled is true) kerberos.auth.enabled: "false"
Add configurations for the the authentication handler based on Configuring Managed Authentication under the "config:" section.
Use the CDAP docker image to generate an "auth.key" file.
docker run -it --rm \ --mount type=bind,source=$(pwd),target=/auth \ gcr.io/cdapio/cdap:latest \ io.cdap.cdap.security.tools.AuthenticationTool -g /auth/auth.key
Create a k8s secret from the "auth.key" file.
kubectl create secret generic cdap-auth --from-file=auth.key
Add the secret to the CDAP YAML file to map the secret into CDAP pods by adding a "secretVolumes" (same level as other options, like "config").
config: .... secretVolumes: cdap-auth: "/etc/cdap/auth"
Now, you can start CDAP with security enabled, without needing Zookeeper.
Running CDAP Programs
Starting in CDAP 6.7.0, you can run CDAP programs on Kubernetes using Spark.
Note: MapReduce and Spark Streaming engines are not supported.
To run CDAP programs on Kubernetes, as a prerequisite the following service account and role binding needs to be created as a requirement from Spark.
Create service account
kubectl create serviceaccount spark
Create role binding
kubectl create clusterrolebinding spark-role --clusterrole=edit --serviceaccount=default:spark --namespace=default
Verify by running a pipeline
Run a pipeline using CDAP UI:
Limitations
coming soon
Appendix
This section describes how to create the resources required for the CDAP installation using Google Cloud Platform.
Preparation
We will be using the standard bash shell and gcloud command line tool to perform the setup. Install Google Cloud SDK before you proceed.
Set up the following environment variables for using the gcloud command:
export PROJECT=<project-name>
export REGION=<region>
export NAME=<cluster-name>
export PROJECT_NUM=$(gcloud projects describe ${PROJECT} \
--format="value(projectNumber)")
export DB_USER=cdap
export DB_PASS=$(openssl rand -base64 14)
export DB_NAME=cdap
Kubernetes
Create a Google Container Engine (GKE) as the Kubernetes cluster. Make sure the GKE API is enabled before executing the following commands.
# Create a 6 nodes regional GKE cluster in the default network
gcloud container clusters create "${NAME}" --region "${REGION}" \
--no-enable-basic-auth --machine-type "e2-standard-4" --num-nodes "2" --scopes \
"https://www.googleapis.com/auth/cloud-platform" --enable-ip-alias --subnetwork \
"projects/${PROJECT}/regions/${REGION}/subnetworks/default" --release-channel \
"regular" --project "${PROJECT}"
# Connect to the GKE cluster
gcloud container clusters get-credentials "${NAME}" --region "${REGION}" \
--project "${PROJECT}"
Postgresql Database
Create a Google Cloud SQL instance to serve as the PostgreSQL database. Make sure the Cloud SQL API is enabled before executing the following commands:
# Create a private PostgreSQL CloudSQL instance in the default network
gcloud beta sql instances create "${NAME}" --database-version=POSTGRES_12 \
--cpu=1 --memory=4GiB --region="${REGION}" --no-assign-ip --network="default" \
--project "${PROJECT}"
# Create a PostgreSQL user
gcloud sql users create "${DB_USER}" --instance="${NAME}" --password="${DB_PASS}" \
--project "${PROJECT}"
# Create a PostgreSQL database
gcloud sql databases create "${DB_NAME}" --instance="${NAME}" --project="${PROJECT}"
# Export the DB IP address
export DB_IP=`gcloud sql instances describe "${NAME}" --project="${PROJECT}" \
--format="value(ipAddresses.ipAddress)"`
Elasticsearch in Kubernetes
We are using the Elasticsearch Operator to operate an Elasticsearch instance inside the Kubernetes cluster. You can deploy the following YAML to create all the necessary resources to have the operator running in the Kubernetes cluster, inside the elastic-system namespace:
es-operator.yaml
# Source: eck-operator/templates/operator-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: elastic-system
labels:
name: elastic-system
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
---
# Source: eck-operator/templates/service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: elastic-operator
namespace: elastic-system
labels:
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
---
# Source: eck-operator/templates/webhook.yaml
apiVersion: v1
kind: Secret
metadata:
name: "elastic-webhook-server-cert"
namespace: elastic-system
labels:
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
---
# Source: eck-operator/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: elastic-operator
namespace: elastic-system
labels:
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
data:
eck.yaml: |-
log-verbosity: 0
metrics-port: 0
container-registry: docker.elastic.co
max-concurrent-reconciles: 3
ca-cert-validity: 8760h
ca-cert-rotate-before: 24h
cert-validity: 8760h
cert-rotate-before: 24h
set-default-security-context: true
kube-client-timeout: 60s
elasticsearch-client-timeout: 180s
disable-telemetry: false
validate-storage-class: true
enable-webhook: true
webhook-name: elastic-webhook.k8s.elastic.co
---
# Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
labels:
app.kubernetes.io/instance: 'elastic-operator'
app.kubernetes.io/name: 'eck-operator-crds'
app.kubernetes.io/version: '1.3.1'
name: apmservers.apm.k8s.elastic.co
spec:
additionalPrinterColumns:
- JSONPath: .status.health
name: health
type: string
- JSONPath: .status.availableNodes
description: Available nodes
name: nodes
type: integer
- JSONPath: .status.version
description: APM version
name: version
type: string
- JSONPath: .metadata.creationTimestamp
name: age
type: date
group: apm.k8s.elastic.co
names:
categories:
- elastic
kind: ApmServer
listKind: ApmServerList
plural: apmservers
shortNames:
- apm
singular: apmserver
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: ApmServer represents an APM Server resource in a Kubernetes cluster.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ApmServerSpec holds the specification of an APM Server.
properties:
config:
description: 'Config holds the APM Server configuration. See: https://www.elastic.co/guide/en/apm/server/current/configuring-howto-apm-server.html'
type: object
count:
description: Count of APM Server instances to deploy.
format: int32
type: integer
elasticsearchRef:
description: ElasticsearchRef is a reference to the output Elasticsearch
cluster running in the same Kubernetes cluster.
properties:
name:
description: Name of the Kubernetes object.
type: string
namespace:
description: Namespace of the Kubernetes object. If empty, defaults
to the current namespace.
type: string
required:
- name
type: object
http:
description: HTTP holds the HTTP layer configuration for the APM Server
resource.
properties:
service:
description: Service defines the template for the associated Kubernetes
Service object.
properties:
metadata:
description: ObjectMeta is the metadata of the service. The
name and namespace provided here are managed by ECK and will
be ignored.
type: object
spec:
description: Spec is the specification of the service.
properties:
clusterIP:
description: 'clusterIP is the IP address of the service
and is usually assigned randomly by the master. If an
address is specified manually and is not in use by others,
it will be allocated to the service; otherwise, creation
of the service will fail. This field can not be changed
through updates. Valid values are "None", empty string
(""), or a valid IP address. "None" can be specified for
headless services when proxying is not required. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
externalIPs:
description: externalIPs is a list of IP addresses for which
nodes in the cluster will also accept traffic for this
service. These IPs are not managed by Kubernetes. The
user is responsible for ensuring that traffic arrives
at a node with this IP. A common example is external
load-balancers that are not part of the Kubernetes system.
items:
type: string
type: array
externalName:
description: externalName is the external reference that
kubedns or equivalent will return as a CNAME record for
this service. No proxying will be involved. Must be a
valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123)
and requires Type to be ExternalName.
type: string
externalTrafficPolicy:
description: externalTrafficPolicy denotes if this Service
desires to route external traffic to node-local or cluster-wide
endpoints. "Local" preserves the client source IP and
avoids a second hop for LoadBalancer and Nodeport type
services, but risks potentially imbalanced traffic spreading.
"Cluster" obscures the client source IP and may cause
a second hop to another node, but should have good overall
load-spreading.
type: string
healthCheckNodePort:
description: healthCheckNodePort specifies the healthcheck
nodePort for the service. If not specified, HealthCheckNodePort
is created by the service api backend with the allocated
nodePort. Will use user-specified nodePort value if specified
by the client. Only effects when Type is set to LoadBalancer
and ExternalTrafficPolicy is set to Local.
format: int32
type: integer
ipFamily:
description: ipFamily specifies whether this Service has
a preference for a particular IP family (e.g. IPv4 vs.
IPv6). If a specific IP family is requested, the clusterIP
field will be allocated from that family, if it is available
in the cluster. If no IP family is requested, the cluster's
primary IP family will be used. Other IP fields (loadBalancerIP,
loadBalancerSourceRanges, externalIPs) and controllers
which allocate external load-balancers should use the
same IP family. Endpoints for this Service will be of
this family. This field is immutable after creation.
Assigning a ServiceIPFamily not available in the cluster
(e.g. IPv6 in IPv4 only cluster) is an error condition
and will fail during clusterIP assignment.
type: string
loadBalancerIP:
description: 'Only applies to Service Type: LoadBalancer
LoadBalancer will get created with the IP specified in
this field. This feature depends on whether the underlying
cloud-provider supports specifying the loadBalancerIP
when a load balancer is created. This field will be ignored
if the cloud-provider does not support the feature.'
type: string
loadBalancerSourceRanges:
description: 'If specified and supported by the platform,
this will restrict traffic through the cloud-provider
load-balancer will be restricted to the specified client
IPs. This field will be ignored if the cloud-provider
does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/'
items:
type: string
type: array
ports:
description: 'The list of ports that are exposed by this
service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
items:
description: ServicePort contains information on service's
port.
properties:
appProtocol:
description: The application protocol for this port.
This field follows standard Kubernetes label syntax.
Un-prefixed names are reserved for IANA standard
service names (as per RFC-6335 and http://www.iana.org/assignments/service-names).
Non-standard protocols should use prefixed names
such as mycompany.com/my-custom-protocol. Field
can be enabled with ServiceAppProtocol feature gate.
type: string
name:
description: The name of this port within the service.
This must be a DNS_LABEL. All ports within a ServiceSpec
must have unique names. When considering the endpoints
for a Service, this must match the 'name' field
in the EndpointPort. Optional if only one ServicePort
is defined on this service.
type: string
nodePort:
description: 'The port on each node on which this
service is exposed when type=NodePort or LoadBalancer.
Usually assigned by the system. If specified, it
will be allocated to the service if unused or else
creation of the service will fail. Default is to
auto-allocate a port if the ServiceType of this
Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport'
format: int32
type: integer
port:
description: The port that will be exposed by this
service.
format: int32
type: integer
protocol:
description: The IP protocol for this port. Supports
"TCP", "UDP", and "SCTP". Default is TCP.
type: string
targetPort:
anyOf:
- type: integer
- type: string
description: 'Number or name of the port to access
on the pods targeted by the service. Number must
be in the range 1 to 65535. Name must be an IANA_SVC_NAME.
If this is a string, it will be looked up as a named
port in the target Pod''s container ports. If this
is not specified, the value of the ''port'' field
is used (an identity map). This field is ignored
for services with clusterIP=None, and should be
omitted or set equal to the ''port'' field. More
info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service'
required:
- port
type: object
type: array
publishNotReadyAddresses:
description: publishNotReadyAddresses, when set to true,
indicates that DNS implementations must publish the notReadyAddresses
of subsets for the Endpoints associated with the Service.
The default value is false. The primary use case for setting
this field is to use a StatefulSet's Headless Service
to propagate SRV records for its Pods without respect
to their readiness for purpose of peer discovery.
type: boolean
selector:
additionalProperties:
type: string
description: 'Route service traffic to pods with label keys
and values matching this selector. If empty or not present,
the service is assumed to have an external process managing
its endpoints, which Kubernetes will not modify. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/'
type: object
sessionAffinity:
description: 'Supports "ClientIP" and "None". Used to maintain
session affinity. Enable client IP based session affinity.
Must be ClientIP or None. Defaults to None. More info:
https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
sessionAffinityConfig:
description: sessionAffinityConfig contains the configurations
of session affinity.
properties:
clientIP:
description: clientIP contains the configurations of
Client IP based session affinity.
properties:
timeoutSeconds:
description: timeoutSeconds specifies the seconds
of ClientIP type session sticky time. The value
must be >0 && <=86400(for 1 day) if ServiceAffinity
== "ClientIP". Default value is 10800(for 3 hours).
format: int32
type: integer
type: object
type: object
topologyKeys:
description: topologyKeys is a preference-order list of
topology keys which implementations of services should
use to preferentially sort endpoints when accessing this
Service, it can not be used at the same time as externalTrafficPolicy=Local.
Topology keys must be valid label keys and at most 16
keys may be specified. Endpoints are chosen based on the
first topology key with available backends. If this field
is specified and all entries have no backends that match
the topology of the client, the service has no backends
for that client and connections should fail. The special
value "*" may be used to mean "any topology". This catch-all
value, if used, only makes sense as the last value in
the list. If this is not specified or empty, no topology
constraints will be applied.
items:
type: string
type: array
type:
description: 'type determines how the Service is exposed.
Defaults to ClusterIP. Valid options are ExternalName,
ClusterIP, NodePort, and LoadBalancer. "ExternalName"
maps to the specified externalName. "ClusterIP" allocates
a cluster-internal IP address for load-balancing to endpoints.
Endpoints are determined by the selector or if that is
not specified, by manual construction of an Endpoints
object. If clusterIP is "None", no virtual IP is allocated
and the endpoints are published as a set of endpoints
rather than a stable IP. "NodePort" builds on ClusterIP
and allocates a port on every node which routes to the
clusterIP. "LoadBalancer" builds on NodePort and creates
an external load-balancer (if supported in the current
cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types'
type: string
type: object
type: object
tls:
description: TLS defines options for configuring TLS for HTTP.
properties:
certificate:
description: "Certificate is a reference to a Kubernetes secret
that contains the certificate and private key for enabling
TLS. The referenced secret should contain the following: \n
- `ca.crt`: The certificate authority (optional). - `tls.crt`:
The certificate (or a chain). - `tls.key`: The private key
to the first certificate in the certificate chain."
properties:
secretName:
description: SecretName is the name of the secret.
type: string
type: object
selfSignedCertificate:
description: SelfSignedCertificate allows configuring the self-signed
certificate generated by the operator.
properties:
disabled:
description: Disabled indicates that the provisioning of
the self-signed certifcate should be disabled.
type: boolean
subjectAltNames:
description: SubjectAlternativeNames is a list of SANs to
include in the generated HTTP TLS certificate.
items:
description: SubjectAlternativeName represents a SAN entry
in a x509 certificate.
properties:
dns:
description: DNS is the DNS name of the subject.
type: string
ip:
description: IP is the IP address of the subject.
type: string
type: object
type: array
type: object
type: object
type: object
image:
description: Image is the APM Server Docker image to deploy.
type: string
kibanaRef:
description: KibanaRef is a reference to a Kibana instance running in
the same Kubernetes cluster. It allows APM agent central configuration
management in Kibana.
properties:
name:
description: Name of the Kubernetes object.
type: string
namespace:
description: Namespace of the Kubernetes object. If empty, defaults
to the current namespace.
type: string
required:
- name
type: object
podTemplate:
description: PodTemplate provides customisation options (labels, annotations,
affinity rules, resource requests, and so on) for the APM Server pods.
type: object
secureSettings:
description: SecureSettings is a list of references to Kubernetes secrets
containing sensitive configuration options for APM Server.
items:
description: SecretSource defines a data source based on a Kubernetes
Secret.
properties:
entries:
description: Entries define how to project each key-value pair
in the secret to filesystem paths. If not defined, all keys
will be projected to similarly named paths in the filesystem.
If defined, only the specified keys will be projected to the
corresponding paths.
items:
description: KeyToPath defines how to map a key in a Secret
object to a filesystem path.
properties:
key:
description: Key is the key contained in the secret.
type: string
path:
description: Path is the relative file path to map the key
to. Path must not be an absolute file path and must not
contain any ".." components.
type: string
required:
- key
type: object
type: array
secretName:
description: SecretName is the name of the secret.
type: string
required:
- secretName
type: object
type: array
serviceAccountName:
description: ServiceAccountName is used to check access from the current
resource to a resource (eg. Elasticsearch) in a different namespace.
Can only be used if ECK is enforcing RBAC on references.
type: string
version:
description: Version of the APM Server.
type: string
required:
- version
type: object
status:
description: ApmServerStatus defines the observed state of ApmServer
properties:
availableNodes:
description: AvailableNodes is the number of available replicas in the
deployment.
format: int32
type: integer
elasticsearchAssociationStatus:
description: ElasticsearchAssociationStatus is the status of any auto-linking
to Elasticsearch clusters.
type: string
health:
description: Health of the deployment.
type: string
aAssociationStatus:
description: KibanaAssociationStatus is the status of any auto-linking
to Kibana.
type: string
secretTokenSecret:
description: SecretTokenSecretName is the name of the Secret that contains
the secret token
type: string
service:
description: ExternalService is the name of the service the agents should
connect to.
type: string
version:
description: 'Version of the stack resource currently running. During
version upgrades, multiple versions may run in parallel: this value
specifies the lowest version currently running.'
type: string
type: object
version: v1
versions:
- name: v1
served: true
storage: true
- name: v1beta1
served: true
storage: false
- name: v1alpha1
served: false
storage: false
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
labels:
app.kubernetes.io/instance: 'elastic-operator'
app.kubernetes.io/name: 'eck-operator-crds'
app.kubernetes.io/version: '1.3.1'
name: beats.beat.k8s.elastic.co
spec:
additionalPrinterColumns:
- JSONPath: .status.health
name: health
type: string
- JSONPath: .status.availableNodes
description: Available nodes
name: available
type: integer
- JSONPath: .status.expectedNodes
description: Expected nodes
name: expected
type: integer
- JSONPath: .spec.type
description: Beat type
name: type
type: string
- JSONPath: .status.version
description: Beat version
name: version
type: string
- JSONPath: .metadata.creationTimestamp
name: age
type: date
group: beat.k8s.elastic.co
names:
categories:
- elastic
kind: Beat
listKind: BeatList
plural: beats
shortNames:
- beat
singular: beat
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: Beat is the Schema for the Beats API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BeatSpec defines the desired state of a Beat.
properties:
config:
description: Config holds the Beat configuration. At most one of [`Config`,
`ConfigRef`] can be specified.
type: object
configRef:
description: ConfigRef contains a reference to an existing Kubernetes
Secret holding the Beat configuration. Beat settings must be specified
as yaml, under a single "beat.yml" entry. At most one of [`Config`,
`ConfigRef`] can be specified.
properties:
secretName:
description: SecretName is the name of the secret.
type: string
type: object
daemonSet:
description: DaemonSet specifies the Beat should be deployed as a DaemonSet,
and allows providing its spec. Cannot be used along with `deployment`.
If both are absent a default for the Type is used.
properties: {}
type: object
deployment:
description: Deployment specifies the Beat should be deployed as a Deployment,
and allows providing its spec. Cannot be used along with `daemonSet`.
If both are absent a default for the Type is used.
properties:
replicas:
format: int32
type: integer
strategy:
description: DeploymentStrategy describes how to replace existing
pods with new ones.
properties:
rollingUpdate:
description: 'Rolling update config params. Present only if
DeploymentStrategyType = RollingUpdate. --- TODO: Update this
to follow our convention for oneOf, whatever we decide it
to be.'
properties:
maxSurge:
anyOf:
- type: integer
- type: string
description: 'The maximum number of pods that can be scheduled
above the desired number of pods. Value can be an absolute
number (ex: 5) or a percentage of desired pods (ex: 10%).
This can not be 0 if MaxUnavailable is 0. Absolute number
is calculated from percentage by rounding up. Defaults
to 25%. Example: when this is set to 30%, the new ReplicaSet
can be scaled up immediately when the rolling update starts,
such that the total number of old and new pods do not
exceed 130% of desired pods. Once old pods have been killed,
new ReplicaSet can be scaled up further, ensuring that
total number of pods running at any time during the update
is at most 130% of desired pods.'
maxUnavailable:
anyOf:
- type: integer
- type: string
description: 'The maximum number of pods that can be unavailable
during the update. Value can be an absolute number (ex:
5) or a percentage of desired pods (ex: 10%). Absolute
number is calculated from percentage by rounding down.
This can not be 0 if MaxSurge is 0. Defaults to 25%. Example:
when this is set to 30%, the old ReplicaSet can be scaled
down to 70% of desired pods immediately when the rolling
update starts. Once new pods are ready, old ReplicaSet
can be scaled down further, followed by scaling up the
new ReplicaSet, ensuring that the total number of pods
available at all times during the update is at least 70%
of desired pods.'
type: object
type:
description: Type of deployment. Can be "Recreate" or "RollingUpdate".
Default is RollingUpdate.
type: string
type: object
type: object
elasticsearchRef:
description: ElasticsearchRef is a reference to an Elasticsearch cluster
running in the same Kubernetes cluster.
properties:
name:
description: Name of the Kubernetes object.
type: string
namespace:
description: Namespace of the Kubernetes object. If empty, defaults
to the current namespace.
type: string
required:
- name
type: object
image:
description: Image is the Beat Docker image to deploy. Version and Type
have to match the Beat in the image.
type: string
kibanaRef:
description: KibanaRef is a reference to a Kibana instance running in
the same Kubernetes cluster. It allows automatic setup of dashboards
and visualizations.
properties:
name:
description: Name of the Kubernetes object.
type: string
namespace:
description: Namespace of the Kubernetes object. If empty, defaults
to the current namespace.
type: string
required:
- name
type: object
secureSettings:
description: SecureSettings is a list of references to Kubernetes Secrets
containing sensitive configuration options for the Beat. Secrets data
can be then referenced in the Beat config using the Secret's keys
or as specified in `Entries` field of each SecureSetting.
items:
description: SecretSource defines a data source based on a Kubernetes
Secret.
properties:
entries:
description: Entries define how to project each key-value pair
in the secret to filesystem paths. If not defined, all keys
will be projected to similarly named paths in the filesystem.
If defined, only the specified keys will be projected to the
corresponding paths.
items:
description: KeyToPath defines how to map a key in a Secret
object to a filesystem path.
properties:
key:
description: Key is the key contained in the secret.
type: string
path:
description: Path is the relative file path to map the key
to. Path must not be an absolute file path and must not
contain any ".." components.
type: string
required:
- key
type: object
type: array
secretName:
description: SecretName is the name of the secret.
type: string
required:
- secretName
type: object
type: array
serviceAccountName:
description: ServiceAccountName is used to check access from the current
resource to Elasticsearch resource in a different namespace. Can only
be used if ECK is enforcing RBAC on references.
type: string
type:
description: Type is the type of the Beat to deploy (filebeat, metricbeat,
heartbeat, auditbeat, journalbeat, packetbeat, etc.). Any string can
be used, but well-known types will have the image field defaulted
and have the appropriate Elasticsearch roles created automatically.
It also allows for dashboard setup when combined with a `KibanaRef`.
maxLength: 20
pattern: '[a-zA-Z0-9-]+'
type: string
version:
description: Version of the Beat.
type: string
required:
- type
- version
type: object
status:
description: BeatStatus defines the observed state of a Beat.
properties:
availableNodes:
format: int32
type: integer
elasticsearchAssociationStatus:
description: AssociationStatus is the status of an association resource.
type: string
expectedNodes:
format: int32
type: integer
health:
type: string
kibanaAssociationStatus:
description: AssociationStatus is the status of an association resource.
type: string
version:
description: 'Version of the stack resource currently running. During
version upgrades, multiple versions may run in parallel: this value
specifies the lowest version currently running.'
type: string
type: object
version: v1beta1
versions:
- name: v1beta1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
labels:
app.kubernetes.io/instance: 'elastic-operator'
app.kubernetes.io/name: 'eck-operator-crds'
app.kubernetes.io/version: '1.3.1'
name: elasticsearches.elasticsearch.k8s.elastic.co
spec:
additionalPrinterColumns:
- JSONPath: .status.health
name: health
type: string
- JSONPath: .status.availableNodes
description: Available nodes
name: nodes
type: integer
- JSONPath: .status.version
description: Elasticsearch version
name: version
type: string
- JSONPath: .status.phase
name: phase
type: string
- JSONPath: .metadata.creationTimestamp
name: age
type: date
group: elasticsearch.k8s.elastic.co
names:
categories:
- elastic
kind: Elasticsearch
listKind: ElasticsearchList
plural: elasticsearches
shortNames:
- es
singular: elasticsearch
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: Elasticsearch represents an Elasticsearch resource in a Kubernetes
cluster.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ElasticsearchSpec holds the specification of an Elasticsearch
cluster.
properties:
auth:
description: Auth contains user authentication and authorization security
settings for Elasticsearch.
properties:
fileRealm:
description: FileRealm to propagate to the Elasticsearch cluster.
items:
description: FileRealmSource references users to create in the
Elasticsearch cluster.
properties:
secretName:
description: SecretName is the name of the secret.
type: string
type: object
type: array
roles:
description: Roles to propagate to the Elasticsearch cluster.
items:
description: RoleSource references roles to create in the Elasticsearch
cluster.
properties:
secretName:
description: SecretName is the name of the secret.
type: string
type: object
type: array
type: object
http:
description: HTTP holds HTTP layer settings for Elasticsearch.
properties:
service:
description: Service defines the template for the associated Kubernetes
Service object.
properties:
metadata:
description: ObjectMeta is the metadata of the service. The
name and namespace provided here are managed by ECK and will
be ignored.
type: object
spec:
description: Spec is the specification of the service.
properties:
clusterIP:
description: 'clusterIP is the IP address of the service
and is usually assigned randomly by the master. If an
address is specified manually and is not in use by others,
it will be allocated to the service; otherwise, creation
of the service will fail. This field can not be changed
through updates. Valid values are "None", empty string
(""), or a valid IP address. "None" can be specified for
headless services when proxying is not required. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
externalIPs:
description: externalIPs is a list of IP addresses for which
nodes in the cluster will also accept traffic for this
service. These IPs are not managed by Kubernetes. The
user is responsible for ensuring that traffic arrives
at a node with this IP. A common example is external
load-balancers that are not part of the Kubernetes system.
items:
type: string
type: array
externalName:
description: externalName is the external reference that
kubedns or equivalent will return as a CNAME record for
this service. No proxying will be involved. Must be a
valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123)
and requires Type to be ExternalName.
type: string
externalTrafficPolicy:
description: externalTrafficPolicy denotes if this Service
desires to route external traffic to node-local or cluster-wide
endpoints. "Local" preserves the client source IP and
avoids a second hop for LoadBalancer and Nodeport type
services, but risks potentially imbalanced traffic spreading.
"Cluster" obscures the client source IP and may cause
a second hop to another node, but should have good overall
load-spreading.
type: string
healthCheckNodePort:
description: healthCheckNodePort specifies the healthcheck
nodePort for the service. If not specified, HealthCheckNodePort
is created by the service api backend with the allocated
nodePort. Will use user-specified nodePort value if specified
by the client. Only effects when Type is set to LoadBalancer
and ExternalTrafficPolicy is set to Local.
format: int32
type: integer
ipFamily:
description: ipFamily specifies whether this Service has
a preference for a particular IP family (e.g. IPv4 vs.
IPv6). If a specific IP family is requested, the clusterIP
field will be allocated from that family, if it is available
in the cluster. If no IP family is requested, the cluster's
primary IP family will be used. Other IP fields (loadBalancerIP,
loadBalancerSourceRanges, externalIPs) and controllers
which allocate external load-balancers should use the
same IP family. Endpoints for this Service will be of
this family. This field is immutable after creation.
Assigning a ServiceIPFamily not available in the cluster
(e.g. IPv6 in IPv4 only cluster) is an error condition
and will fail during clusterIP assignment.
type: string
loadBalancerIP:
description: 'Only applies to Service Type: LoadBalancer
LoadBalancer will get created with the IP specified in
this field. This feature depends on whether the underlying
cloud-provider supports specifying the loadBalancerIP
when a load balancer is created. This field will be ignored
if the cloud-provider does not support the feature.'
type: string
loadBalancerSourceRanges:
description: 'If specified and supported by the platform,
this will restrict traffic through the cloud-provider
load-balancer will be restricted to the specified client
IPs. This field will be ignored if the cloud-provider
does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/'
items:
type: string
type: array
ports:
description: 'The list of ports that are exposed by this
service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
items:
description: ServicePort contains information on service's
port.
properties:
appProtocol:
description: The application protocol for this port.
This field follows standard Kubernetes label syntax.
Un-prefixed names are reserved for IANA standard
service names (as per RFC-6335 and http://www.iana.org/assignments/service-names).
Non-standard protocols should use prefixed names
such as mycompany.com/my-custom-protocol. Field
can be enabled with ServiceAppProtocol feature gate.
type: string
name:
description: The name of this port within the service.
This must be a DNS_LABEL. All ports within a ServiceSpec
must have unique names. When considering the endpoints
for a Service, this must match the 'name' field
in the EndpointPort. Optional if only one ServicePort
is defined on this service.
type: string
nodePort:
description: 'The port on each node on which this
service is exposed when type=NodePort or LoadBalancer.
Usually assigned by the system. If specified, it
will be allocated to the service if unused or else
creation of the service will fail. Default is to
auto-allocate a port if the ServiceType of this
Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport'
format: int32
type: integer
port:
description: The port that will be exposed by this
service.
format: int32
type: integer
protocol:
description: The IP protocol for this port. Supports
"TCP", "UDP", and "SCTP". Default is TCP.
type: string
targetPort:
anyOf:
- type: integer
- type: string
description: 'Number or name of the port to access
on the pods targeted by the service. Number must
be in the range 1 to 65535. Name must be an IANA_SVC_NAME.
If this is a string, it will be looked up as a named
port in the target Pod''s container ports. If this
is not specified, the value of the ''port'' field
is used (an identity map). This field is ignored
for services with clusterIP=None, and should be
omitted or set equal to the ''port'' field. More
info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service'
required:
- port
type: object
type: array
publishNotReadyAddresses:
description: publishNotReadyAddresses, when set to true,
indicates that DNS implementations must publish the notReadyAddresses
of subsets for the Endpoints associated with the Service.
The default value is false. The primary use case for setting
this field is to use a StatefulSet's Headless Service
to propagate SRV records for its Pods without respect
to their readiness for purpose of peer discovery.
type: boolean
selector:
additionalProperties:
type: string
description: 'Route service traffic to pods with label keys
and values matching this selector. If empty or not present,
the service is assumed to have an external process managing
its endpoints, which Kubernetes will not modify. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/'
type: object
sessionAffinity:
description: 'Supports "ClientIP" and "None". Used to maintain
session affinity. Enable client IP based session affinity.
Must be ClientIP or None. Defaults to None. More info:
https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
sessionAffinityConfig:
description: sessionAffinityConfig contains the configurations
of session affinity.
properties:
clientIP:
description: clientIP contains the configurations of
Client IP based session affinity.
properties:
timeoutSeconds:
description: timeoutSeconds specifies the seconds
of ClientIP type session sticky time. The value
must be >0 && <=86400(for 1 day) if ServiceAffinity
== "ClientIP". Default value is 10800(for 3 hours).
format: int32
type: integer
type: object
type: object
topologyKeys:
description: topologyKeys is a preference-order list of
topology keys which implementations of services should
use to preferentially sort endpoints when accessing this
Service, it can not be used at the same time as externalTrafficPolicy=Local.
Topology keys must be valid label keys and at most 16
keys may be specified. Endpoints are chosen based on the
first topology key with available backends. If this field
is specified and all entries have no backends that match
the topology of the client, the service has no backends
for that client and connections should fail. The special
value "*" may be used to mean "any topology". This catch-all
value, if used, only makes sense as the last value in
the list. If this is not specified or empty, no topology
constraints will be applied.
items:
type: string
type: array
type:
description: 'type determines how the Service is exposed.
Defaults to ClusterIP. Valid options are ExternalName,
ClusterIP, NodePort, and LoadBalancer. "ExternalName"
maps to the specified externalName. "ClusterIP" allocates
a cluster-internal IP address for load-balancing to endpoints.
Endpoints are determined by the selector or if that is
not specified, by manual construction of an Endpoints
object. If clusterIP is "None", no virtual IP is allocated
and the endpoints are published as a set of endpoints
rather than a stable IP. "NodePort" builds on ClusterIP
and allocates a port on every node which routes to the
clusterIP. "LoadBalancer" builds on NodePort and creates
an external load-balancer (if supported in the current
cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types'
type: string
type: object
type: object
tls:
description: TLS defines options for configuring TLS for HTTP.
properties:
certificate:
description: "Certificate is a reference to a Kubernetes secret
that contains the certificate and private key for enabling
TLS. The referenced secret should contain the following: \n
- `ca.crt`: The certificate authority (optional). - `tls.crt`:
The certificate (or a chain). - `tls.key`: The private key
to the first certificate in the certificate chain."
properties:
secretName:
description: SecretName is the name of the secret.
type: string
type: object
selfSignedCertificate:
description: SelfSignedCertificate allows configuring the self-signed
certificate generated by the operator.
properties:
disabled:
description: Disabled indicates that the provisioning of
the self-signed certifcate should be disabled.
type: boolean
subjectAltNames:
description: SubjectAlternativeNames is a list of SANs to
include in the generated HTTP TLS certificate.
items:
description: SubjectAlternativeName represents a SAN entry
in a x509 certificate.
properties:
dns:
description: DNS is the DNS name of the subject.
type: string
ip:
description: IP is the IP address of the subject.
type: string
type: object
type: array
type: object
type: object
type: object
image:
description: Image is the Elasticsearch Docker image to deploy.
type: string
nodeSets:
description: NodeSets allow specifying groups of Elasticsearch nodes
sharing the same configuration and Pod templates.
items:
description: NodeSet is the specification for a group of Elasticsearch
nodes sharing the same configuration and a Pod template.
properties:
config:
description: Config holds the Elasticsearch configuration.
type: object
count:
description: Count of Elasticsearch nodes to deploy.
format: int32
minimum: 1
type: integer
name:
description: Name of this set of nodes. Becomes a part of the
Elasticsearch node.name setting.
maxLength: 23
pattern: '[a-zA-Z0-9-]+'
type: string
podTemplate:
description: PodTemplate provides customisation options (labels,
annotations, affinity rules, resource requests, and so on) for
the Pods belonging to this NodeSet.
type: object
volumeClaimTemplates:
description: VolumeClaimTemplates is a list of persistent volume
claims to be used by each Pod in this NodeSet. Every claim in
this list must have a matching volumeMount in one of the containers
defined in the PodTemplate. Items defined here take precedence
over any default claims added by the operator with the same
name.
items:
description: PersistentVolumeClaim is a user's request for and
claim to a persistent volume
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of
this representation of an object. Servers should convert
recognized schemas to the latest internal value, and may
reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST
resource this object represents. Servers may infer this
from the endpoint the client submits requests to. Cannot
be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
description: 'Standard object''s metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata'
type: object
spec:
description: 'Spec defines the desired characteristics of
a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims'
properties:
accessModes:
description: 'AccessModes contains the desired access
modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1'
items:
type: string
type: array
dataSource:
description: 'This field can be used to specify either:
* An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot
- Beta) * An existing PVC (PersistentVolumeClaim)
* An existing custom resource/object that implements
data population (Alpha) In order to use VolumeSnapshot
object types, the appropriate feature gate must be
enabled (VolumeSnapshotDataSource or AnyVolumeDataSource)
If the provisioner or an external controller can support
the specified data source, it will create a new volume
based on the contents of the specified data source.
If the specified data source is not supported, the
volume will not be created and the failure will be
reported as an event. In the future, we plan to support
more data source types and the behavior of the provisioner
may change.'
properties:
apiGroup:
description: APIGroup is the group for the resource
being referenced. If APIGroup is not specified,
the specified Kind must be in the core API group.
For any other third-party types, APIGroup is required.
type: string
kind:
description: Kind is the type of resource being
referenced
type: string
name:
description: Name is the name of resource being
referenced
type: string
required:
- kind
- name
type: object
resources:
description: 'Resources represents the minimum resources
the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources'
properties:
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
description: 'Limits describes the maximum amount
of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
description: 'Requests describes the minimum amount
of compute resources required. If Requests is
omitted for a container, it defaults to Limits
if that is explicitly specified, otherwise to
an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
type: object
type: object
selector:
description: A label query over volumes to consider
for binding.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a
selector that contains values, a key, and an
operator that relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are
In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty. If the
operator is Exists or DoesNotExist, the
values array must be empty. This array is
replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is "In",
and the values array contains only "value". The
requirements are ANDed.
type: object
type: object
storageClassName:
description: 'Name of the StorageClass required by the
claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1'
type: string
volumeMode:
description: volumeMode defines what type of volume
is required by the claim. Value of Filesystem is implied
when not included in claim spec.
type: string
volumeName:
description: VolumeName is the binding reference to
the PersistentVolume backing this claim.
type: string
type: object
status:
description: 'Status represents the current information/status
of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims'
properties:
accessModes:
description: 'AccessModes contains the actual access
modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1'
items:
type: string
type: array
capacity:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
description: Represents the actual resources of the
underlying volume.
type: object
conditions:
description: Current Condition of persistent volume
claim. If underlying persistent volume is being resized
then the Condition will be set to 'ResizeStarted'.
items:
description: PersistentVolumeClaimCondition contails
details about state of pvc
properties:
lastProbeTime:
description: Last time we probed the condition.
format: date-time
type: string
lastTransitionTime:
description: Last time the condition transitioned
from one status to another.
format: date-time
type: string
message:
description: Human-readable message indicating
details about last transition.
type: string
reason:
description: Unique, this should be a short, machine
understandable string that gives the reason
for condition's last transition. If it reports
"ResizeStarted" that means the underlying persistent
volume is being resized.
type: string
status:
type: string
type:
description: PersistentVolumeClaimConditionType
is a valid value of PersistentVolumeClaimCondition.Type
type: string
required:
- status
- type
type: object
type: array
phase:
description: Phase represents the current phase of PersistentVolumeClaim.
type: string
type: object
type: object
type: array
required:
- count
- name
type: object
minItems: 1
type: array
podDisruptionBudget:
description: PodDisruptionBudget provides access to the default pod
disruption budget for the Elasticsearch cluster. The default budget
selects all cluster pods and sets `maxUnavailable` to 1. To disable,
set `PodDisruptionBudget` to the empty value (`{}` in YAML).
properties:
metadata:
description: ObjectMeta is the metadata of the PDB. The name and
namespace provided here are managed by ECK and will be ignored.
type: object
spec:
description: Spec is the specification of the PDB.
properties:
maxUnavailable:
anyOf:
- type: integer
- type: string
description: An eviction is allowed if at most "maxUnavailable"
pods selected by "selector" are unavailable after the eviction,
i.e. even in absence of the evicted pod. For example, one
can prevent all voluntary evictions by specifying 0. This
is a mutually exclusive setting with "minAvailable".
minAvailable:
anyOf:
- type: integer
- type: string
description: An eviction is allowed if at least "minAvailable"
pods selected by "selector" will still be available after
the eviction, i.e. even in the absence of the evicted pod. So
for example you can prevent all voluntary evictions by specifying
"100%".
selector:
description: Label query over pods whose evictions are managed
by the disruption budget.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type: string
values:
description: values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty. This
array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key", the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type: object
type: object
type: object
type: object
remoteClusters:
description: RemoteClusters enables you to establish uni-directional
connections to a remote Elasticsearch cluster.
items:
description: RemoteCluster declares a remote Elasticsearch cluster
connection.
properties:
elasticsearchRef:
description: ElasticsearchRef is a reference to an Elasticsearch
cluster running within the same k8s cluster.
properties:
name:
description: Name of the Kubernetes object.
type: string
namespace:
description: Namespace of the Kubernetes object. If empty,
defaults to the current namespace.
type: string
required:
- name
type: object
name:
description: Name is the name of the remote cluster as it is set
in the Elasticsearch settings. The name is expected to be unique
for each remote clusters.
minLength: 1
type: string
required:
- name
type: object
type: array
secureSettings:
description: SecureSettings is a list of references to Kubernetes secrets
containing sensitive configuration options for Elasticsearch.
items:
description: SecretSource defines a data source based on a Kubernetes
Secret.
properties:
entries:
description: Entries define how to project each key-value pair
in the secret to filesystem paths. If not defined, all keys
will be projected to similarly named paths in the filesystem.
If defined, only the specified keys will be projected to the
corresponding paths.
items:
description: KeyToPath defines how to map a key in a Secret
object to a filesystem path.
properties:
key:
description: Key is the key contained in the secret.
type: string
path:
description: Path is the relative file path to map the key
to. Path must not be an absolute file path and must not
contain any ".." components.
type: string
required:
- key
type: object
type: array
secretName:
description: SecretName is the name of the secret.
type: string
required:
- secretName
type: object
type: array
serviceAccountName:
description: ServiceAccountName is used to check access from the current
resource to a resource (eg. a remote Elasticsearch cluster) in a different
namespace. Can only be used if ECK is enforcing RBAC on references.
type: string
transport:
description: Transport holds transport layer settings for Elasticsearch.
properties:
service:
description: Service defines the template for the associated Kubernetes
Service object.
properties:
metadata:
description: ObjectMeta is the metadata of the service. The
name and namespace provided here are managed by ECK and will
be ignored.
type: object
spec:
description: Spec is the specification of the service.
properties:
clusterIP:
description: 'clusterIP is the IP address of the service
and is usually assigned randomly by the master. If an
address is specified manually and is not in use by others,
it will be allocated to the service; otherwise, creation
of the service will fail. This field can not be changed
through updates. Valid values are "None", empty string
(""), or a valid IP address. "None" can be specified for
headless services when proxying is not required. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
externalIPs:
description: externalIPs is a list of IP addresses for which
nodes in the cluster will also accept traffic for this
service. These IPs are not managed by Kubernetes. The
user is responsible for ensuring that traffic arrives
at a node with this IP. A common example is external
load-balancers that are not part of the Kubernetes system.
items:
type: string
type: array
externalName:
description: externalName is the external reference that
kubedns or equivalent will return as a CNAME record for
this service. No proxying will be involved. Must be a
valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123)
and requires Type to be ExternalName.
type: string
externalTrafficPolicy:
description: externalTrafficPolicy denotes if this Service
desires to route external traffic to node-local or cluster-wide
endpoints. "Local" preserves the client source IP and
avoids a second hop for LoadBalancer and Nodeport type
services, but risks potentially imbalanced traffic spreading.
"Cluster" obscures the client source IP and may cause
a second hop to another node, but should have good overall
load-spreading.
type: string
healthCheckNodePort:
description: healthCheckNodePort specifies the healthcheck
nodePort for the service. If not specified, HealthCheckNodePort
is created by the service api backend with the allocated
nodePort. Will use user-specified nodePort value if specified
by the client. Only effects when Type is set to LoadBalancer
and ExternalTrafficPolicy is set to Local.
format: int32
type: integer
ipFamily:
description: ipFamily specifies whether this Service has
a preference for a particular IP family (e.g. IPv4 vs.
IPv6). If a specific IP family is requested, the clusterIP
field will be allocated from that family, if it is available
in the cluster. If no IP family is requested, the cluster's
primary IP family will be used. Other IP fields (loadBalancerIP,
loadBalancerSourceRanges, externalIPs) and controllers
which allocate external load-balancers should use the
same IP family. Endpoints for this Service will be of
this family. This field is immutable after creation.
Assigning a ServiceIPFamily not available in the cluster
(e.g. IPv6 in IPv4 only cluster) is an error condition
and will fail during clusterIP assignment.
type: string
loadBalancerIP:
description: 'Only applies to Service Type: LoadBalancer
LoadBalancer will get created with the IP specified in
this field. This feature depends on whether the underlying
cloud-provider supports specifying the loadBalancerIP
when a load balancer is created. This field will be ignored
if the cloud-provider does not support the feature.'
type: string
loadBalancerSourceRanges:
description: 'If specified and supported by the platform,
this will restrict traffic through the cloud-provider
load-balancer will be restricted to the specified client
IPs. This field will be ignored if the cloud-provider
does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/'
items:
type: string
type: array
ports:
description: 'The list of ports that are exposed by this
service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
items:
description: ServicePort contains information on service's
port.
properties:
appProtocol:
description: The application protocol for this port.
This field follows standard Kubernetes label syntax.
Un-prefixed names are reserved for IANA standard
service names (as per RFC-6335 and http://www.iana.org/assignments/service-names).
Non-standard protocols should use prefixed names
such as mycompany.com/my-custom-protocol. Field
can be enabled with ServiceAppProtocol feature gate.
type: string
name:
description: The name of this port within the service.
This must be a DNS_LABEL. All ports within a ServiceSpec
must have unique names. When considering the endpoints
for a Service, this must match the 'name' field
in the EndpointPort. Optional if only one ServicePort
is defined on this service.
type: string
nodePort:
description: 'The port on each node on which this
service is exposed when type=NodePort or LoadBalancer.
Usually assigned by the system. If specified, it
will be allocated to the service if unused or else
creation of the service will fail. Default is to
auto-allocate a port if the ServiceType of this
Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport'
format: int32
type: integer
port:
description: The port that will be exposed by this
service.
format: int32
type: integer
protocol:
description: The IP protocol for this port. Supports
"TCP", "UDP", and "SCTP". Default is TCP.
type: string
targetPort:
anyOf:
- type: integer
- type: string
description: 'Number or name of the port to access
on the pods targeted by the service. Number must
be in the range 1 to 65535. Name must be an IANA_SVC_NAME.
If this is a string, it will be looked up as a named
port in the target Pod''s container ports. If this
is not specified, the value of the ''port'' field
is used (an identity map). This field is ignored
for services with clusterIP=None, and should be
omitted or set equal to the ''port'' field. More
info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service'
required:
- port
type: object
type: array
publishNotReadyAddresses:
description: publishNotReadyAddresses, when set to true,
indicates that DNS implementations must publish the notReadyAddresses
of subsets for the Endpoints associated with the Service.
The default value is false. The primary use case for setting
this field is to use a StatefulSet's Headless Service
to propagate SRV records for its Pods without respect
to their readiness for purpose of peer discovery.
type: boolean
selector:
additionalProperties:
type: string
description: 'Route service traffic to pods with label keys
and values matching this selector. If empty or not present,
the service is assumed to have an external process managing
its endpoints, which Kubernetes will not modify. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/'
type: object
sessionAffinity:
description: 'Supports "ClientIP" and "None". Used to maintain
session affinity. Enable client IP based session affinity.
Must be ClientIP or None. Defaults to None. More info:
https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
sessionAffinityConfig:
description: sessionAffinityConfig contains the configurations
of session affinity.
properties:
clientIP:
description: clientIP contains the configurations of
Client IP based session affinity.
properties:
timeoutSeconds:
description: timeoutSeconds specifies the seconds
of ClientIP type session sticky time. The value
must be >0 && <=86400(for 1 day) if ServiceAffinity
== "ClientIP". Default value is 10800(for 3 hours).
format: int32
type: integer
type: object
type: object
topologyKeys:
description: topologyKeys is a preference-order list of
topology keys which implementations of services should
use to preferentially sort endpoints when accessing this
Service, it can not be used at the same time as externalTrafficPolicy=Local.
Topology keys must be valid label keys and at most 16
keys may be specified. Endpoints are chosen based on the
first topology key with available backends. If this field
is specified and all entries have no backends that match
the topology of the client, the service has no backends
for that client and connections should fail. The special
value "*" may be used to mean "any topology". This catch-all
value, if used, only makes sense as the last value in
the list. If this is not specified or empty, no topology
constraints will be applied.
items:
type: string
type: array
type:
description: 'type determines how the Service is exposed.
Defaults to ClusterIP. Valid options are ExternalName,
ClusterIP, NodePort, and LoadBalancer. "ExternalName"
maps to the specified externalName. "ClusterIP" allocates
a cluster-internal IP address for load-balancing to endpoints.
Endpoints are determined by the selector or if that is
not specified, by manual construction of an Endpoints
object. If clusterIP is "None", no virtual IP is allocated
and the endpoints are published as a set of endpoints
rather than a stable IP. "NodePort" builds on ClusterIP
and allocates a port on every node which routes to the
clusterIP. "LoadBalancer" builds on NodePort and creates
an external load-balancer (if supported in the current
cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types'
type: string
type: object
type: object
type: object
updateStrategy:
description: UpdateStrategy specifies how updates to the cluster should
be performed.
properties:
changeBudget:
description: ChangeBudget defines the constraints to consider when
applying changes to the Elasticsearch cluster.
properties:
maxSurge:
description: MaxSurge is the maximum number of new pods that
can be created exceeding the original number of pods defined
in the specification. MaxSurge is only taken into consideration
when scaling up. Setting a negative value will disable the
restriction. Defaults to unbounded if not specified.
format: int32
type: integer
maxUnavailable:
description: MaxUnavailable is the maximum number of pods that
can be unavailable (not ready) during the update due to circumstances
under the control of the operator. Setting a negative value
will disable this restriction. Defaults to 1 if not specified.
format: int32
type: integer
type: object
type: object
version:
description: Version of Elasticsearch.
type: string
required:
- nodeSets
- version
type: object
status:
description: ElasticsearchStatus defines the observed state of Elasticsearch
properties:
availableNodes:
description: AvailableNodes is the number of available instances.
format: int32
type: integer
health:
description: ElasticsearchHealth is the health of the cluster as returned
by the health API.
type: string
phase:
description: ElasticsearchOrchestrationPhase is the phase Elasticsearch
is in from the controller point of view.
type: string
version:
description: 'Version of the stack resource currently running. During
version upgrades, multiple versions may run in parallel: this value
specifies the lowest version currently running.'
type: string
type: object
version: v1
versions:
- name: v1
served: true
storage: true
- name: v1beta1
served: true
storage: false
- name: v1alpha1
served: false
storage: false
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
labels:
app.kubernetes.io/instance: 'elastic-operator'
app.kubernetes.io/name: 'eck-operator-crds'
app.kubernetes.io/version: '1.3.1'
name: enterprisesearches.enterprisesearch.k8s.elastic.co
spec:
additionalPrinterColumns:
- JSONPath: .status.health
name: health
type: string
- JSONPath: .status.availableNodes
description: Available nodes
name: nodes
type: integer
- JSONPath: .status.version
description: Enterprise Search version
name: version
type: string
- JSONPath: .metadata.creationTimestamp
name: age
type: date
group: enterprisesearch.k8s.elastic.co
names:
categories:
- elastic
kind: EnterpriseSearch
listKind: EnterpriseSearchList
plural: enterprisesearches
shortNames:
- ent
singular: enterprisesearch
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: EnterpriseSearch is a Kubernetes CRD to represent Enterprise Search.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: EnterpriseSearchSpec holds the specification of an Enterprise
Search resource.
properties:
config:
description: Config holds the Enterprise Search configuration.
type: object
configRef:
description: ConfigRef contains a reference to an existing Kubernetes
Secret holding the Enterprise Search configuration. Configuration
settings are merged and have precedence over settings specified in
`config`.
properties:
secretName:
description: SecretName is the name of the secret.
type: string
type: object
count:
description: Count of Enterprise Search instances to deploy.
format: int32
type: integer
elasticsearchRef:
description: ElasticsearchRef is a reference to the Elasticsearch cluster
running in the same Kubernetes cluster.
properties:
name:
description: Name of the Kubernetes object.
type: string
namespace:
description: Namespace of the Kubernetes object. If empty, defaults
to the current namespace.
type: string
required:
- name
type: object
http:
description: HTTP holds the HTTP layer configuration for Enterprise
Search resource.
properties:
service:
description: Service defines the template for the associated Kubernetes
Service object.
properties:
metadata:
description: ObjectMeta is the metadata of the service. The
name and namespace provided here are managed by ECK and will
be ignored.
type: object
spec:
description: Spec is the specification of the service.
properties:
clusterIP:
description: 'clusterIP is the IP address of the service
and is usually assigned randomly by the master. If an
address is specified manually and is not in use by others,
it will be allocated to the service; otherwise, creation
of the service will fail. This field can not be changed
through updates. Valid values are "None", empty string
(""), or a valid IP address. "None" can be specified for
headless services when proxying is not required. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
externalIPs:
description: externalIPs is a list of IP addresses for which
nodes in the cluster will also accept traffic for this
service. These IPs are not managed by Kubernetes. The
user is responsible for ensuring that traffic arrives
at a node with this IP. A common example is external
load-balancers that are not part of the Kubernetes system.
items:
type: string
type: array
externalName:
description: externalName is the external reference that
kubedns or equivalent will return as a CNAME record for
this service. No proxying will be involved. Must be a
valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123)
and requires Type to be ExternalName.
type: string
externalTrafficPolicy:
description: externalTrafficPolicy denotes if this Service
desires to route external traffic to node-local or cluster-wide
endpoints. "Local" preserves the client source IP and
avoids a second hop for LoadBalancer and Nodeport type
services, but risks potentially imbalanced traffic spreading.
"Cluster" obscures the client source IP and may cause
a second hop to another node, but should have good overall
load-spreading.
type: string
healthCheckNodePort:
description: healthCheckNodePort specifies the healthcheck
nodePort for the service. If not specified, HealthCheckNodePort
is created by the service api backend with the allocated
nodePort. Will use user-specified nodePort value if specified
by the client. Only effects when Type is set to LoadBalancer
and ExternalTrafficPolicy is set to Local.
format: int32
type: integer
ipFamily:
description: ipFamily specifies whether this Service has
a preference for a particular IP family (e.g. IPv4 vs.
IPv6). If a specific IP family is requested, the clusterIP
field will be allocated from that family, if it is available
in the cluster. If no IP family is requested, the cluster's
primary IP family will be used. Other IP fields (loadBalancerIP,
loadBalancerSourceRanges, externalIPs) and controllers
which allocate external load-balancers should use the
same IP family. Endpoints for this Service will be of
this family. This field is immutable after creation.
Assigning a ServiceIPFamily not available in the cluster
(e.g. IPv6 in IPv4 only cluster) is an error condition
and will fail during clusterIP assignment.
type: string
loadBalancerIP:
description: 'Only applies to Service Type: LoadBalancer
LoadBalancer will get created with the IP specified in
this field. This feature depends on whether the underlying
cloud-provider supports specifying the loadBalancerIP
when a load balancer is created. This field will be ignored
if the cloud-provider does not support the feature.'
type: string
loadBalancerSourceRanges:
description: 'If specified and supported by the platform,
this will restrict traffic through the cloud-provider
load-balancer will be restricted to the specified client
IPs. This field will be ignored if the cloud-provider
does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/'
items:
type: string
type: array
ports:
description: 'The list of ports that are exposed by this
service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
items:
description: ServicePort contains information on service's
port.
properties:
appProtocol:
description: The application protocol for this port.
This field follows standard Kubernetes label syntax.
Un-prefixed names are reserved for IANA standard
service names (as per RFC-6335 and http://www.iana.org/assignments/service-names).
Non-standard protocols should use prefixed names
such as mycompany.com/my-custom-protocol. Field
can be enabled with ServiceAppProtocol feature gate.
type: string
name:
description: The name of this port within the service.
This must be a DNS_LABEL. All ports within a ServiceSpec
must have unique names. When considering the endpoints
for a Service, this must match the 'name' field
in the EndpointPort. Optional if only one ServicePort
is defined on this service.
type: string
nodePort:
description: 'The port on each node on which this
service is exposed when type=NodePort or LoadBalancer.
Usually assigned by the system. If specified, it
will be allocated to the service if unused or else
creation of the service will fail. Default is to
auto-allocate a port if the ServiceType of this
Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport'
format: int32
type: integer
port:
description: The port that will be exposed by this
service.
format: int32
type: integer
protocol:
description: The IP protocol for this port. Supports
"TCP", "UDP", and "SCTP". Default is TCP.
type: string
targetPort:
anyOf:
- type: integer
- type: string
description: 'Number or name of the port to access
on the pods targeted by the service. Number must
be in the range 1 to 65535. Name must be an IANA_SVC_NAME.
If this is a string, it will be looked up as a named
port in the target Pod''s container ports. If this
is not specified, the value of the ''port'' field
is used (an identity map). This field is ignored
for services with clusterIP=None, and should be
omitted or set equal to the ''port'' field. More
info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service'
required:
- port
type: object
type: array
publishNotReadyAddresses:
description: publishNotReadyAddresses, when set to true,
indicates that DNS implementations must publish the notReadyAddresses
of subsets for the Endpoints associated with the Service.
The default value is false. The primary use case for setting
this field is to use a StatefulSet's Headless Service
to propagate SRV records for its Pods without respect
to their readiness for purpose of peer discovery.
type: boolean
selector:
additionalProperties:
type: string
description: 'Route service traffic to pods with label keys
and values matching this selector. If empty or not present,
the service is assumed to have an external process managing
its endpoints, which Kubernetes will not modify. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/'
type: object
sessionAffinity:
description: 'Supports "ClientIP" and "None". Used to maintain
session affinity. Enable client IP based session affinity.
Must be ClientIP or None. Defaults to None. More info:
https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
sessionAffinityConfig:
description: sessionAffinityConfig contains the configurations
of session affinity.
properties:
clientIP:
description: clientIP contains the configurations of
Client IP based session affinity.
properties:
timeoutSeconds:
description: timeoutSeconds specifies the seconds
of ClientIP type session sticky time. The value
must be >0 && <=86400(for 1 day) if ServiceAffinity
== "ClientIP". Default value is 10800(for 3 hours).
format: int32
type: integer
type: object
type: object
topologyKeys:
description: topologyKeys is a preference-order list of
topology keys which implementations of services should
use to preferentially sort endpoints when accessing this
Service, it can not be used at the same time as externalTrafficPolicy=Local.
Topology keys must be valid label keys and at most 16
keys may be specified. Endpoints are chosen based on the
first topology key with available backends. If this field
is specified and all entries have no backends that match
the topology of the client, the service has no backends
for that client and connections should fail. The special
value "*" may be used to mean "any topology". This catch-all
value, if used, only makes sense as the last value in
the list. If this is not specified or empty, no topology
constraints will be applied.
items:
type: string
type: array
type:
description: 'type determines how the Service is exposed.
Defaults to ClusterIP. Valid options are ExternalName,
ClusterIP, NodePort, and LoadBalancer. "ExternalName"
maps to the specified externalName. "ClusterIP" allocates
a cluster-internal IP address for load-balancing to endpoints.
Endpoints are determined by the selector or if that is
not specified, by manual construction of an Endpoints
object. If clusterIP is "None", no virtual IP is allocated
and the endpoints are published as a set of endpoints
rather than a stable IP. "NodePort" builds on ClusterIP
and allocates a port on every node which routes to the
clusterIP. "LoadBalancer" builds on NodePort and creates
an external load-balancer (if supported in the current
cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types'
type: string
type: object
type: object
tls:
description: TLS defines options for configuring TLS for HTTP.
properties:
certificate:
description: "Certificate is a reference to a Kubernetes secret
that contains the certificate and private key for enabling
TLS. The referenced secret should contain the following: \n
- `ca.crt`: The certificate authority (optional). - `tls.crt`:
The certificate (or a chain). - `tls.key`: The private key
to the first certificate in the certificate chain."
properties:
secretName:
description: SecretName is the name of the secret.
type: string
type: object
selfSignedCertificate:
description: SelfSignedCertificate allows configuring the self-signed
certificate generated by the operator.
properties:
disabled:
description: Disabled indicates that the provisioning of
the self-signed certifcate should be disabled.
type: boolean
subjectAltNames:
description: SubjectAlternativeNames is a list of SANs to
include in the generated HTTP TLS certificate.
items:
description: SubjectAlternativeName represents a SAN entry
in a x509 certificate.
properties:
dns:
description: DNS is the DNS name of the subject.
type: string
ip:
description: IP is the IP address of the subject.
type: string
type: object
type: array
type: object
type: object
type: object
image:
description: Image is the Enterprise Search Docker image to deploy.
type: string
podTemplate:
description: PodTemplate provides customisation options (labels, annotations,
affinity rules, resource requests, and so on) for the Enterprise Search
pods.
type: object
serviceAccountName:
description: ServiceAccountName is used to check access from the current
resource to a resource (eg. Elasticsearch) in a different namespace.
Can only be used if ECK is enforcing RBAC on references.
type: string
version:
description: Version of Enterprise Search.
type: string
type: object
status:
description: EnterpriseSearchStatus defines the observed state of EnterpriseSearch
properties:
associationStatus:
description: Association is the status of any auto-linking to Elasticsearch
clusters.
type: string
availableNodes:
description: AvailableNodes is the number of available replicas in the
deployment.
format: int32
type: integer
health:
description: Health of the deployment.
type: string
service:
description: ExternalService is the name of the service associated to
the Enterprise Search Pods.
type: string
version:
description: 'Version of the stack resource currently running. During
version upgrades, multiple versions may run in parallel: this value
specifies the lowest version currently running.'
type: string
type: object
version: v1beta1
versions:
- name: v1beta1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
labels:
app.kubernetes.io/instance: 'elastic-operator'
app.kubernetes.io/name: 'eck-operator-crds'
app.kubernetes.io/version: '1.3.1'
name: kibanas.kibana.k8s.elastic.co
spec:
additionalPrinterColumns:
- JSONPath: .status.health
name: health
type: string
- JSONPath: .status.availableNodes
description: Available nodes
name: nodes
type: integer
- JSONPath: .status.version
description: Kibana version
name: version
type: string
- JSONPath: .metadata.creationTimestamp
name: age
type: date
group: kibana.k8s.elastic.co
names:
categories:
- elastic
kind: Kibana
listKind: KibanaList
plural: kibanas
shortNames:
- kb
singular: kibana
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: Kibana represents a Kibana resource in a Kubernetes cluster.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: KibanaSpec holds the specification of a Kibana instance.
properties:
config:
description: 'Config holds the Kibana configuration. See: https://www.elastic.co/guide/en/kibana/current/settings.html'
type: object
count:
description: Count of Kibana instances to deploy.
format: int32
type: integer
elasticsearchRef:
description: ElasticsearchRef is a reference to an Elasticsearch cluster
running in the same Kubernetes cluster.
properties:
name:
description: Name of the Kubernetes object.
type: string
namespace:
description: Namespace of the Kubernetes object. If empty, defaults
to the current namespace.
type: string
required:
- name
type: object
http:
description: HTTP holds the HTTP layer configuration for Kibana.
properties:
service:
description: Service defines the template for the associated Kubernetes
Service object.
properties:
metadata:
description: ObjectMeta is the metadata of the service. The
name and namespace provided here are managed by ECK and will
be ignored.
type: object
spec:
description: Spec is the specification of the service.
properties:
clusterIP:
description: 'clusterIP is the IP address of the service
and is usually assigned randomly by the master. If an
address is specified manually and is not in use by others,
it will be allocated to the service; otherwise, creation
of the service will fail. This field can not be changed
through updates. Valid values are "None", empty string
(""), or a valid IP address. "None" can be specified for
headless services when proxying is not required. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
externalIPs:
description: externalIPs is a list of IP addresses for which
nodes in the cluster will also accept traffic for this
service. These IPs are not managed by Kubernetes. The
user is responsible for ensuring that traffic arrives
at a node with this IP. A common example is external
load-balancers that are not part of the Kubernetes system.
items:
type: string
type: array
externalName:
description: externalName is the external reference that
kubedns or equivalent will return as a CNAME record for
this service. No proxying will be involved. Must be a
valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123)
and requires Type to be ExternalName.
type: string
externalTrafficPolicy:
description: externalTrafficPolicy denotes if this Service
desires to route external traffic to node-local or cluster-wide
endpoints. "Local" preserves the client source IP and
avoids a second hop for LoadBalancer and Nodeport type
services, but risks potentially imbalanced traffic spreading.
"Cluster" obscures the client source IP and may cause
a second hop to another node, but should have good overall
load-spreading.
type: string
healthCheckNodePort:
description: healthCheckNodePort specifies the healthcheck
nodePort for the service. If not specified, HealthCheckNodePort
is created by the service api backend with the allocated
nodePort. Will use user-specified nodePort value if specified
by the client. Only effects when Type is set to LoadBalancer
and ExternalTrafficPolicy is set to Local.
format: int32
type: integer
ipFamily:
description: ipFamily specifies whether this Service has
a preference for a particular IP family (e.g. IPv4 vs.
IPv6). If a specific IP family is requested, the clusterIP
field will be allocated from that family, if it is available
in the cluster. If no IP family is requested, the cluster's
primary IP family will be used. Other IP fields (loadBalancerIP,
loadBalancerSourceRanges, externalIPs) and controllers
which allocate external load-balancers should use the
same IP family. Endpoints for this Service will be of
this family. This field is immutable after creation.
Assigning a ServiceIPFamily not available in the cluster
(e.g. IPv6 in IPv4 only cluster) is an error condition
and will fail during clusterIP assignment.
type: string
loadBalancerIP:
description: 'Only applies to Service Type: LoadBalancer
LoadBalancer will get created with the IP specified in
this field. This feature depends on whether the underlying
cloud-provider supports specifying the loadBalancerIP
when a load balancer is created. This field will be ignored
if the cloud-provider does not support the feature.'
type: string
loadBalancerSourceRanges:
description: 'If specified and supported by the platform,
this will restrict traffic through the cloud-provider
load-balancer will be restricted to the specified client
IPs. This field will be ignored if the cloud-provider
does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/'
items:
type: string
type: array
ports:
description: 'The list of ports that are exposed by this
service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
items:
description: ServicePort contains information on service's
port.
properties:
appProtocol:
description: The application protocol for this port.
This field follows standard Kubernetes label syntax.
Un-prefixed names are reserved for IANA standard
service names (as per RFC-6335 and http://www.iana.org/assignments/service-names).
Non-standard protocols should use prefixed names
such as mycompany.com/my-custom-protocol. Field
can be enabled with ServiceAppProtocol feature gate.
type: string
name:
description: The name of this port within the service.
This must be a DNS_LABEL. All ports within a ServiceSpec
must have unique names. When considering the endpoints
for a Service, this must match the 'name' field
in the EndpointPort. Optional if only one ServicePort
is defined on this service.
type: string
nodePort:
description: 'The port on each node on which this
service is exposed when type=NodePort or LoadBalancer.
Usually assigned by the system. If specified, it
will be allocated to the service if unused or else
creation of the service will fail. Default is to
auto-allocate a port if the ServiceType of this
Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport'
format: int32
type: integer
port:
description: The port that will be exposed by this
service.
format: int32
type: integer
protocol:
description: The IP protocol for this port. Supports
"TCP", "UDP", and "SCTP". Default is TCP.
type: string
targetPort:
anyOf:
- type: integer
- type: string
description: 'Number or name of the port to access
on the pods targeted by the service. Number must
be in the range 1 to 65535. Name must be an IANA_SVC_NAME.
If this is a string, it will be looked up as a named
port in the target Pod''s container ports. If this
is not specified, the value of the ''port'' field
is used (an identity map). This field is ignored
for services with clusterIP=None, and should be
omitted or set equal to the ''port'' field. More
info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service'
required:
- port
type: object
type: array
publishNotReadyAddresses:
description: publishNotReadyAddresses, when set to true,
indicates that DNS implementations must publish the notReadyAddresses
of subsets for the Endpoints associated with the Service.
The default value is false. The primary use case for setting
this field is to use a StatefulSet's Headless Service
to propagate SRV records for its Pods without respect
to their readiness for purpose of peer discovery.
type: boolean
selector:
additionalProperties:
type: string
description: 'Route service traffic to pods with label keys
and values matching this selector. If empty or not present,
the service is assumed to have an external process managing
its endpoints, which Kubernetes will not modify. Only
applies to types ClusterIP, NodePort, and LoadBalancer.
Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/'
type: object
sessionAffinity:
description: 'Supports "ClientIP" and "None". Used to maintain
session affinity. Enable client IP based session affinity.
Must be ClientIP or None. Defaults to None. More info:
https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies'
type: string
sessionAffinityConfig:
description: sessionAffinityConfig contains the configurations
of session affinity.
properties:
clientIP:
description: clientIP contains the configurations of
Client IP based session affinity.
properties:
timeoutSeconds:
description: timeoutSeconds specifies the seconds
of ClientIP type session sticky time. The value
must be >0 && <=86400(for 1 day) if ServiceAffinity
== "ClientIP". Default value is 10800(for 3 hours).
format: int32
type: integer
type: object
type: object
topologyKeys:
description: topologyKeys is a preference-order list of
topology keys which implementations of services should
use to preferentially sort endpoints when accessing this
Service, it can not be used at the same time as externalTrafficPolicy=Local.
Topology keys must be valid label keys and at most 16
keys may be specified. Endpoints are chosen based on the
first topology key with available backends. If this field
is specified and all entries have no backends that match
the topology of the client, the service has no backends
for that client and connections should fail. The special
value "*" may be used to mean "any topology". This catch-all
value, if used, only makes sense as the last value in
the list. If this is not specified or empty, no topology
constraints will be applied.
items:
type: string
type: array
type:
description: 'type determines how the Service is exposed.
Defaults to ClusterIP. Valid options are ExternalName,
ClusterIP, NodePort, and LoadBalancer. "ExternalName"
maps to the specified externalName. "ClusterIP" allocates
a cluster-internal IP address for load-balancing to endpoints.
Endpoints are determined by the selector or if that is
not specified, by manual construction of an Endpoints
object. If clusterIP is "None", no virtual IP is allocated
and the endpoints are published as a set of endpoints
rather than a stable IP. "NodePort" builds on ClusterIP
and allocates a port on every node which routes to the
clusterIP. "LoadBalancer" builds on NodePort and creates
an external load-balancer (if supported in the current
cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types'
type: string
type: object
type: object
tls:
description: TLS defines options for configuring TLS for HTTP.
properties:
certificate:
description: "Certificate is a reference to a Kubernetes secret
that contains the certificate and private key for enabling
TLS. The referenced secret should contain the following: \n
- `ca.crt`: The certificate authority (optional). - `tls.crt`:
The certificate (or a chain). - `tls.key`: The private key
to the first certificate in the certificate chain."
properties:
secretName:
description: SecretName is the name of the secret.
type: string
type: object
selfSignedCertificate:
description: SelfSignedCertificate allows configuring the self-signed
certificate generated by the operator.
properties:
disabled:
description: Disabled indicates that the provisioning of
the self-signed certifcate should be disabled.
type: boolean
subjectAltNames:
description: SubjectAlternativeNames is a list of SANs to
include in the generated HTTP TLS certificate.
items:
description: SubjectAlternativeName represents a SAN entry
in a x509 certificate.
properties:
dns:
description: DNS is the DNS name of the subject.
type: string
ip:
description: IP is the IP address of the subject.
type: string
type: object
type: array
type: object
type: object
type: object
image:
description: Image is the Kibana Docker image to deploy.
type: string
podTemplate:
description: PodTemplate provides customisation options (labels, annotations,
affinity rules, resource requests, and so on) for the Kibana pods
type: object
secureSettings:
description: SecureSettings is a list of references to Kubernetes secrets
containing sensitive configuration options for Kibana.
items:
description: SecretSource defines a data source based on a Kubernetes
Secret.
properties:
entries:
description: Entries define how to project each key-value pair
in the secret to filesystem paths. If not defined, all keys
will be projected to similarly named paths in the filesystem.
If defined, only the specified keys will be projected to the
corresponding paths.
items:
description: KeyToPath defines how to map a key in a Secret
object to a filesystem path.
properties:
key:
description: Key is the key contained in the secret.
type: string
path:
description: Path is the relative file path to map the key
to. Path must not be an absolute file path and must not
contain any ".." components.
type: string
required:
- key
type: object
type: array
secretName:
description: SecretName is the name of the secret.
type: string
required:
- secretName
type: object
type: array
serviceAccountName:
description: ServiceAccountName is used to check access from the current
resource to a resource (eg. Elasticsearch) in a different namespace.
Can only be used if ECK is enforcing RBAC on references.
type: string
version:
description: Version of Kibana.
type: string
required:
- version
type: object
status:
description: KibanaStatus defines the observed state of Kibana
properties:
associationStatus:
description: AssociationStatus is the status of an association resource.
type: string
availableNodes:
description: AvailableNodes is the number of available replicas in the
deployment.
format: int32
type: integer
health:
description: Health of the deployment.
type: string
version:
description: 'Version of the stack resource currently running. During
version upgrades, multiple versions may run in parallel: this value
specifies the lowest version currently running.'
type: string
type: object
version: v1
versions:
- name: v1
served: true
storage: true
- name: v1beta1
served: true
storage: false
- name: v1alpha1
served: false
storage: false
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: eck-operator/templates/cluster-roles.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: elastic-operator
labels:
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
rules:
- apiGroups:
- "authorization.k8s.io"
resources:
- subjectaccessreviews
verbs:
- create
- apiGroups:
- ""
resources:
- pods
- endpoints
- events
- persistentvolumeclaims
- secrets
- services
- configmaps
- serviceaccounts
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- apps
resources:
- deployments
- statefulsets
- daemonsets
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- policy
resources:
- poddisruptionbudgets
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- elasticsearch.k8s.elastic.co
resources:
- elasticsearches
- elasticsearches/status
- elasticsearches/finalizers
- enterpriselicenses
- enterpriselicenses/status
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- kibana.k8s.elastic.co
resources:
- kibanas
- kibanas/status
- kibanas/finalizers
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- apm.k8s.elastic.co
resources:
- apmservers
- apmservers/status
- apmservers/finalizers
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- enterprisesearch.k8s.elastic.co
resources:
- enterprisesearches
- enterprisesearches/status
- enterprisesearches/finalizers
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- beat.k8s.elastic.co
resources:
- beats
- beats/status
- beats/finalizers
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
- apiGroups:
- admissionregistration.k8s.io
resources:
- mutatingwebhookconfigurations
- validatingwebhookconfigurations
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
---
# Source: eck-operator/templates/cluster-roles.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: "elastic-operator-view"
labels:
rbac.authorization.k8s.io/aggregate-to-view: "true"
rbac.authorization.k8s.io/aggregate-to-edit: "true"
rbac.authorization.k8s.io/aggregate-to-admin: "true"
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
rules:
- apiGroups: ["elasticsearch.k8s.elastic.co"]
resources: ["elasticsearches"]
verbs: ["get", "list", "watch"]
- apiGroups: ["apm.k8s.elastic.co"]
resources: ["apmservers"]
verbs: ["get", "list", "watch"]
- apiGroups: ["kibana.k8s.elastic.co"]
resources: ["kibanas"]
verbs: ["get", "list", "watch"]
- apiGroups: ["enterprisesearch.k8s.elastic.co"]
resources: ["enterprisesearches"]
verbs: ["get", "list", "watch"]
- apiGroups: ["beat.k8s.elastic.co"]
resources: ["beats"]
verbs: ["get", "list", "watch"]
---
# Source: eck-operator/templates/cluster-roles.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: "elastic-operator-edit"
labels:
rbac.authorization.k8s.io/aggregate-to-edit: "true"
rbac.authorization.k8s.io/aggregate-to-admin: "true"
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
rules:
- apiGroups: ["elasticsearch.k8s.elastic.co"]
resources: ["elasticsearches"]
verbs: ["create", "delete", "deletecollection", "patch", "update"]
- apiGroups: ["apm.k8s.elastic.co"]
resources: ["apmservers"]
verbs: ["create", "delete", "deletecollection", "patch", "update"]
- apiGroups: ["kibana.k8s.elastic.co"]
resources: ["kibanas"]
verbs: ["create", "delete", "deletecollection", "patch", "update"]
- apiGroups: ["enterprisesearch.k8s.elastic.co"]
resources: ["enterprisesearches"]
verbs: ["create", "delete", "deletecollection", "patch", "update"]
- apiGroups: ["beat.k8s.elastic.co"]
resources: ["beats"]
verbs: ["create", "delete", "deletecollection", "patch", "update"]
---
# Source: eck-operator/templates/operator-role-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: elastic-operator
labels:
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: elastic-operator
subjects:
- kind: ServiceAccount
name: elastic-operator
namespace: elastic-system
---
# Source: eck-operator/templates/webhook.yaml
apiVersion: v1
kind: Service
metadata:
name: elastic-webhook-server
namespace: elastic-system
labels:
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
spec:
ports:
- name: https
port: 443
targetPort: 9443
selector:
control-plane: elastic-operator
---
# Source: eck-operator/templates/statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: elastic-operator
namespace: elastic-system
labels:
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
spec:
selector:
matchLabels:
control-plane: elastic-operator
serviceName: elastic-operator
replicas: 1
template:
metadata:
annotations:
# Rename the fields "error" to "error.message" and "source" to "event.source"
# This is to avoid a conflict with the ECS "error" and "source" documents.
"co.elastic.logs/raw": "[{\"type\":\"container\",\"json.keys_under_root\":true,\"paths\":[\"/var/log/containers/*${data.kubernetes.container.id}.log\"],\"processors\":[{\"convert\":{\"mode\":\"rename\",\"ignore_missing\":true,\"fields\":[{\"from\":\"error\",\"to\":\"_error\"}]}},{\"convert\":{\"mode\":\"rename\",\"ignore_missing\":true,\"fields\":[{\"from\":\"_error\",\"to\":\"error.message\"}]}},{\"convert\":{\"mode\":\"rename\",\"ignore_missing\":true,\"fields\":[{\"from\":\"source\",\"to\":\"_source\"}]}},{\"convert\":{\"mode\":\"rename\",\"ignore_missing\":true,\"fields\":[{\"from\":\"_source\",\"to\":\"event.source\"}]}}]}]"
"checksum/config": ee68b5877bbac6a3071ec907276f8966d24c9878268247a2b3da09837ba8c5fa
labels:
control-plane: elastic-operator
spec:
terminationGracePeriodSeconds: 10
serviceAccountName: elastic-operator
securityContext:
runAsNonRoot: true
containers:
- image: "docker.elastic.co/eck/eck-operator:1.3.1"
imagePullPolicy: IfNotPresent
name: manager
args:
- "manager"
- "--config=/conf/eck.yaml"
- "--distribution-channel=all-in-one"
env:
- name: OPERATOR_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: WEBHOOK_SECRET
value: "elastic-webhook-server-cert"
resources:
limits:
cpu: 1
memory: 512Mi
requests:
cpu: 100m
memory: 150Mi
ports:
- containerPort: 9443
name: https-webhook
protocol: TCP
volumeMounts:
- mountPath: "/conf"
name: conf
readOnly: true
- mountPath: /tmp/k8s-webhook-server/serving-certs
name: cert
readOnly: true
volumes:
- name: conf
configMap:
name: elastic-operator
- name: cert
secret:
defaultMode: 420
secretName: "elastic-webhook-server-cert"
---
# Source: eck-operator/templates/webhook.yaml
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
name: elastic-webhook.k8s.elastic.co
labels:
control-plane: elastic-operator
app.kubernetes.io/version: "1.3.1"
webhooks:
- clientConfig:
caBundle: Cg==
service:
name: elastic-webhook-server
namespace: elastic-system
path: /validate-apm-k8s-elastic-co-v1-apmserver
failurePolicy: Ignore
name: elastic-apm-validation-v1.k8s.elastic.co
rules:
- apiGroups:
- apm.k8s.elastic.co
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- apmservers
- clientConfig:
caBundle: Cg==
service:
name: elastic-webhook-server
namespace: elastic-system
path: /validate-apm-k8s-elastic-co-v1beta1-apmserver
failurePolicy: Ignore
name: elastic-apm-validation-v1beta1.k8s.elastic.co
rules:
- apiGroups:
- apm.k8s.elastic.co
apiVersions:
- v1beta1
operations:
- CREATE
- UPDATE
resources:
- apmservers
- clientConfig:
caBundle: Cg==
service:
name: elastic-webhook-server
namespace: elastic-system
path: /validate-beat-k8s-elastic-co-v1beta1-beat
failurePolicy: Ignore
name: elastic-beat-validation-v1beta1.k8s.elastic.co
rules:
- apiGroups:
- beat.k8s.elastic.co
apiVersions:
- v1beta1
operations:
- CREATE
- UPDATE
resources:
- beats
- clientConfig:
caBundle: Cg==
service:
name: elastic-webhook-server
namespace: elastic-system
path: /validate-enterprisesearch-k8s-elastic-co-v1beta1-enterprisesearch
failurePolicy: Ignore
name: elastic-ent-validation-v1beta1.k8s.elastic.co
rules:
- apiGroups:
- enterprisesearch.k8s.elastic.co
apiVersions:
- v1beta1
operations:
- CREATE
- UPDATE
resources:
- enterprisesearches
- clientConfig:
caBundle: Cg==
service:
name: elastic-webhook-server
namespace: elastic-system
path: /validate-elasticsearch-k8s-elastic-co-v1-elasticsearch
failurePolicy: Ignore
name: elastic-es-validation-v1.k8s.elastic.co
rules:
- apiGroups:
- elasticsearch.k8s.elastic.co
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- elasticsearches
- clientConfig:
caBundle: Cg==
service:
name: elastic-webhook-server
namespace: elastic-system
path: /validate-elasticsearch-k8s-elastic-co-v1beta1-elasticsearch
failurePolicy: Ignore
name: elastic-es-validation-v1beta1.k8s.elastic.co
rules:
- apiGroups:
- elasticsearch.k8s.elastic.co
apiVersions:
- v1beta1
operations:
- CREATE
- UPDATE
resources:
- elasticsearches
- clientConfig:
caBundle: Cg==
service:
name: elastic-webhook-server
namespace: elastic-system
path: /validate-kibana-k8s-elastic-co-v1-kibana
failurePolicy: Ignore
name: elastic-kb-validation-v1.k8s.elastic.co
rules:
- apiGroups:
- kibana.k8s.elastic.co
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- kibanas
- clientConfig:
caBundle: Cg==
service:
name: elastic-webhook-server
namespace: elastic-system
path: /validate-kibana-k8s-elastic-co-v1beta1-kibana
failurePolicy: Ignore
name: elastic-kb-validation-v1beta1.k8s.elastic.co
rules:
- apiGroups:
- kibana.k8s.elastic.co
apiVersions:
- v1beta1
operations:
- CREATE
- UPDATE
resources:
- kibanas
After deploying the Elasticsearch operator, you can deploy the following custom resource to start an Elasticsearch instance inside the Kubernetes cluster.
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: es-cluster
spec:
version: 6.8.13
nodeSets:
- name: default
count: 1
config:
node.store.allow_mmap: false
You can validate that the Elasticsearch instance is up and running correctly by observing an Elasticsearch pod is in the RUNNING state.
kubectl get pods -l elasticsearch.k8s.elastic.co/cluster-name
NAME READY STATUS RESTARTS AGE es-cluster-es-default-0 1/1 Running 0 5m
After the Elasticsearch instance, you need to get the default user password from the secret created by the operator. This password is needed in the cdap-security.xml file for CDAP to authenticate itself to Elasticsearch.
export ES_USER=elastic
export ES_PASS=$(kubectl get secret es-cluster-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
- No labels