/
Installing CDAP on Kubernetes

Installing CDAP on Kubernetes

Owned by Terence Yim
Last updated: Nov 08, 2022 by Arjan Bal
85 min read

CDAP installation on Kubernetes was introduced in CDAP 6.2.3.

This document describes how to install CDAP on a Kubernetes cluster.

Dependencies

This section describes the infrastructure and software dependencies for operating CDAP in Kubernetes.

Kubernetes cluster

CDAP supports using Kubernetes (k8s) as the distributed resource manager. When CDAP is deployed to a k8s cluster, it spawns multiple Deployments and StatefulSets for running various CDAP services. The following diagram shows each of the CDAP services in the Kubernetes cluster:

The CDAP operator is responsible for deploying and managing all the CDAP services inside the cluster. The CDAP operator also supports managing multiple CDAP instances within the same k8s cluster. If multiple CDAP instances are deployed to the same k8s cluster, It is recommended to deploy them to different namespaces to provide better isolation.

Limitations

Currently CDAP only supports running one replica (pod) per service, except for the Preview Runner. Failure resiliency is handled by k8s to have pod restart upon failure. For pods created by StatefulSets, it relies on the infrastructure to have persistent volumes being re-mountable to the new pod, which potentially could be on a different machine.

Another limitation of operating CDAP in Kubernetes is that it does not support native compute profile. This means all user program executions are external to the Kubernetes cluster, and require a Hadoop cluster for program executions.

PostgreSQL database

CDAP needs a shared storage for its own metadata, such as deployed artifacts and applications, run histories, preferences, lineage information, and many more. Currently, CDAP supports both PostgreSQL and HBase as the metadata store. When running CDAP in Kubernetes, we recommend using PostgreSQL.

Elasticsearch

CDAP has support for metadata search, and it is backed by either Elasticsearch or HBase. In the Kubernetes environment, Elasticsearch is recommended. You can either configure CDAP to use an existing Elasticsearch cluster or run an Elasticsearch in Kubernetes by using the Elasticsearch Operator.

Hadoop Compatible File System (HCFS)

CDAP stores artifacts and runtime information through the HDFS API. Any of the HCFS implementations is supported.

Installation

This section describes the steps to deploy CDAP on Kubernetes.

Prerequisites

  • An operational Kubernetes cluster.

    • Recommended to have 64 GB of memory resources and 20 available virtual CPU for production deployment.

    • For better security, the Kubernetes cluster should have RBAC enabled.

    • Have kubectl set up to connect to the Kubernetes cluster.

  • A PostgreSQL database that is reachable from the Kubernetes cluster.

  • An Elasticsearch instance that is reachable from the Kubernetes cluster.

    • Refer to the Appendix section on how to set up an Elasticsearch instance inside the Kubernetes cluster.

Deploy CDAP Operator

CDAP provides a CDAP operator for easy deployment and management of CDAP in Kubernetes. You can deploy the following YAML to create all the necessary resources to have the operator running in the Kubernetes cluster, inside the cdap-system namespace.

 

# Create operator namespace apiVersion: v1 kind: Namespace metadata: name: cdap-system labels: name: cdap-system control-plane: cdap-operator --- # Create operator service account apiVersion: v1 kind: ServiceAccount metadata: name: cdap-operator namespace: cdap-system labels: control-plane: cdap-operator --- # Source cdap-operator/config/rbac/cdapmaster_editor_role.yaml # permissions to do edit cdapmasters. apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: cdapmaster-editor-role rules: - apiGroups: - cdap.cdap.io resources: - cdapmasters verbs: - create - delete - get - list - patch - update - watch - apiGroups: - cdap.cdap.io resources: - cdapmasters/status verbs: - get - patch - update --- # Source cdap-operator/config/rbac/cdapmaster_viewer_role.yaml # permissions to do viewer cdapmasters. apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: cdapmaster-viewer-role rules: - apiGroups: - cdap.cdap.io resources: - cdapmasters verbs: - get - list - watch - apiGroups: - cdap.cdap.io resources: - cdapmasters/status verbs: - get --- # Source cdap-operator/config/rbac/role.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: creationTimestamp: null name: cdap-operator-role rules: - apiGroups: - "" resources: - configmaps verbs: - create - delete - get - list - patch - update - watch - apiGroups: - "" resources: - services verbs: - create - delete - get - list - patch - update - watch - apiGroups: - apps resources: - deployments verbs: - create - delete - get - list - patch - update - watch - apiGroups: - apps resources: - deployments/status verbs: - get - patch - update - apiGroups: - apps resources: - statefulsets verbs: - create - delete - get - list - patch - update - watch - apiGroups: - batch resources: - jobs verbs: - create - delete - get - list - patch - update - watch - apiGroups: - cdap.cdap.io resources: - cdapmasters verbs: - create - delete - get - list - patch - update - watch - apiGroups: - cdap.cdap.io resources: - cdapmasters/status verbs: - get - patch - update --- # Source cdap-operator/config/rbac/role_binding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: cdap-operator-rolebinding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cdap-operator-role subjects: - kind: ServiceAccount name: cdap-operator namespace: cdap-system --- # Source cdap-operator/config/crd/bases/cdap.cdap.io_cdapmasters.yaml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.2.4 creationTimestamp: null name: cdapmasters.cdap.cdap.io spec: group: cdap.cdap.io names: kind: CDAPMaster listKind: CDAPMasterList plural: cdapmasters singular: cdapmaster scope: Namespaced validation: openAPIV3Schema: description: CDAPMaster is the Schema for the cdapmasters API properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: "CDAPMasterSpec defines the desired state of CDAPMaster \n Important notes: * The field name of each service MUST match the constant values of ServiceName in constants.go as reflection is used to find field value. * For services that are optional (i.e. may or may not be required for CDAP to be operational), their service specification fields are pointers. By default, these optional services are disabled. Set to non-nil to enable them." properties: appFabric: description: AppFabric is specification for the CDAP app-fabric service. properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string storageClassName: description: StorageClassName is the name of the StorageClass for the persistent volume used by the service. type: string storageSize: description: StorageSize is specification for the persistent volume size used by the service. type: string type: object config: additionalProperties: type: string description: Config is a set of configurations that goes into cdap-site.xml. type: object configMapVolumes: additionalProperties: type: string description: ConfigMapVolumes defines a map from ConfigMap names to volume mount path. Key is the configmap object name. Value is the mount path. This adds ConfigMap data to the directory specified by the volume mount path. type: object image: description: Image is the docker image name for the CDAP backend. type: string imagePullPolicy: description: ImagePullPolicy is the policy for pulling docker images on Pod creation. type: string locationURI: description: LocationURI is an URI specifying an object storage for CDAP. type: string logLevels: additionalProperties: type: string description: LogLevels is a set of logger name to log level settings. type: object logs: description: Logs is specification for the CDAP logging service. properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string storageClassName: description: StorageClassName is the name of the StorageClass for the persistent volume used by the service. type: string storageSize: description: StorageSize is specification for the persistent volume size used by the service. type: string type: object messaging: description: Messaging is specification for the CDAP messaging service. properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string storageClassName: description: StorageClassName is the name of the StorageClass for the persistent volume used by the service. type: string storageSize: description: StorageSize is specification for the persistent volume size used by the service. type: string type: object metadata: description: Metadata is specification for the CDAP metadata service. properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string storageClassName: description: StorageClassName is the name of the StorageClass for the persistent volume used by the service. type: string storageSize: description: StorageSize is specification for the persistent volume size used by the service. type: string type: object metrics: description: Metrics is specification for the CDAP metrics service. properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string storageClassName: description: StorageClassName is the name of the StorageClass for the persistent volume used by the service. type: string storageSize: description: StorageSize is specification for the persistent volume size used by the service. type: string type: object preview: description: Preview is specification for the CDAP preview service. properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string storageClassName: description: StorageClassName is the name of the StorageClass for the persistent volume used by the service. type: string storageSize: description: StorageSize is specification for the persistent volume size used by the service. type: string type: object router: description: Router is specification for the CDAP router service. properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string replicas: description: Replicas is number of replicas for the service. format: int32 type: integer resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string servicePort: description: ServicePort is the port number for the service. format: int32 type: integer serviceType: description: ServiceType is the service type in kubernetes, default is NodePort. type: string type: object runtime: description: 'Runtime is specification for the CDAP runtime service. This is an optional service and may not be required for CDAP to be operational. To disable this service: either omit or set the field to nil To enable this service: set it to a pointer to a RuntimeSpec struct (can be an empty struct)' properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string storageClassName: description: StorageClassName is the name of the StorageClass for the persistent volume used by the service. type: string storageSize: description: StorageSize is specification for the persistent volume size used by the service. type: string type: object securitySecret: description: SecuritySecret is secret that contains security related configurations for CDAP. type: string serviceAccountName: description: ServiceAccountName is the service account for all the service pods. type: string systemappconfigs: additionalProperties: type: string description: SystemAppConfigs specifies configs used by CDAP to run system apps dynamically. Each entry is of format <filename, json app config> which will create a separate system config file with entry value as file content. type: object userInterface: description: UserInterface is specification for the CDAP UI service. properties: env: description: Env is a list of environment variables for the master service container. items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: description: Specifies the output format of the exposed resources, defaults to "1" type: string resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array metadata: description: Metadata for the service. type: object nodeSelector: additionalProperties: type: string description: NodeSelector is a selector which must be true for the pod to fit on a node. type: object priorityClassName: description: PriorityClassName is to specify the priority of the pods for this service. type: string replicas: description: Replicas is number of replicas for the service. format: int32 type: integer resources: description: Resources are Compute resources required by the service. properties: limits: additionalProperties: type: string description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: type: string description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object runtimeClassName: description: RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run pods for this service. If no RuntimeClass resource matches the named class, pods will not be running. type: string serviceAccountName: description: ServiceAccountName overrides the service account for the service pods. type: string servicePort: description: ServicePort is the port number for the service. format: int32 type: integer serviceType: description: ServiceType is the service type in kubernetes, default is NodePort. type: string type: object userInterfaceImage: description: UserInterfaceImage is the docker image name for the CDAP UI. type: string required: - locationURI type: object status: description: CDAPMasterStatus defines the observed state of CDAPMaster properties: components: description: Object status array for all matching objects items: description: ObjectStatus is a generic status holder for objects properties: group: description: Object group type: string kind: description: Kind of object type: string link: description: Link to object type: string name: description: Name of object type: string pdb: description: PDB status properties: currenthealthy: description: currentHealthy format: int32 type: integer desiredhealthy: description: desiredHealthy format: int32 type: integer required: - currenthealthy - desiredhealthy type: object status: description: 'Status. Values: InProgress, Ready, Unknown' type: string sts: description: StatefulSet status properties: currentcount: description: CurrentReplicas defines the no of MySQL instances that are created format: int32 type: integer progress: description: 'progress is a fuzzy indicator. Interpret as a percentage (0-100) eg: for statefulsets, progress = 100*readyreplicas/replicas' format: int32 type: integer readycount: description: ReadyReplicas defines the no of MySQL instances that are ready format: int32 type: integer replicas: description: Replicas defines the no of MySQL instances desired format: int32 type: integer required: - currentcount - progress - readycount - replicas type: object type: object type: array conditions: description: Conditions represents the latest state of the object items: description: Condition describes the state of an object at a certain point. properties: lastTransitionTime: description: Last time the condition transitioned from one status to another. format: date-time type: string lastUpdateTime: description: Last time the condition was probed format: date-time type: string message: description: A human readable message indicating details about the transition. type: string reason: description: The reason for the condition's last transition. type: string status: description: Status of the condition, one of True, False, Unknown. type: string type: description: Type of condition. type: string required: - status - type type: object type: array downgradeStartTimeMillis: description: DowngradeStartTimeMillis is the start time in milliseconds of the downgrade process format: int64 type: integer imageToUse: description: ImageToUse is the Docker image of CDAP backend the operator uses to deploy. type: string observedGeneration: description: ObservedGeneration is the most recent generation observed. It corresponds to the Object's generation, which is updated on mutation by the API Server. format: int64 type: integer upgradeStartTimeMillis: description: UpgradeStartTimeMillis is the start time in milliseconds of the upgrade process format: int64 type: integer userInterfaceImageToUse: description: UserInterfaceImageToUse is the Docker image of CDAP UI the operator uses to deploy. type: string type: object type: object version: v1alpha1 versions: - name: v1alpha1 served: true storage: true status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: [] --- # StatefulSet for running the cdap controller apiVersion: apps/v1 kind: StatefulSet metadata: name: cdap-controller namespace: cdap-system labels: control-plane: cdap-operator spec: selector: matchLabels: control-plane: cdap-operator serviceName: cdap-operator-service template: metadata: labels: control-plane: cdap-operator spec: serviceAccountName: cdap-operator containers: - command: - /manager image: gcr.io/cdapio/cdap-controller:latest name: manager resources: limits: cpu: 100m memory: 30Mi requests: cpu: 100m memory: 20Mi terminationGracePeriodSeconds: 10

Create RBAC Roles and RoleBinding

CDAP interacts with Kubernetes for configuration, service discovery, and also workload management. Deploying the following YAML file will create the necessary set of RBAC Roles and RoleBinding to the service account called cdap.

 

# Create cdap service account apiVersion: v1 kind: ServiceAccount metadata: name: cdap --- # Create cdap role apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: cdap-role rules: - apiGroups: - "" resources: - configmaps verbs: - create - delete - get - list - patch - update - watch - apiGroups: - "" resources: - secrets verbs: - create - delete - get - list - patch - update - watch - apiGroups: - "" resources: - pods verbs: - create - get - list - watch - delete - deletecollection - apiGroups: - "" resources: - services verbs: - create - delete - get - list - patch - update - watch - apiGroups: - "" resources: - persistentvolumeclaims verbs: - deletecollection - apiGroups: - apps resources: - deployments verbs: - create - delete - get - list - patch - update - watch - apiGroups: - apps resources: - statefulsets verbs: - create - delete - get - list - patch - update - watch - apiGroups: - apps resources: - replicasets verbs: - get - list - update - watch - apiGroups: - batch resources: - jobs verbs: - create - delete - get - list - patch - update - watch --- # Create cdap RoleBinding apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: cdap-rolebinding roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: cdap-role subjects: - kind: ServiceAccount name: cdap

Prepare the secret token for CDAP

We need to set up a secret in Kubernetes to provide the cdap-security.xml file to CDAP, which will contain the PostgreSQL and Elasticsearch password. The following command assumes the database username and password are in the environment variables DB_USER and DB_PASS respectively. For Elasticsearch authentication, it expects that the username and password comes from the ES_USER and ES_PASS environment variables.

# Create the content of the cdap-security.xml export CDAP_SECURITY=$(cat << EOF | base64 | tr -d '\n' <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>data.storage.sql.jdbc.username</name> <value>${DB_USER}</value> </property> <property> <name>data.storage.sql.jdbc.password</name> <value>${DB_PASS}</value> </property> <property> <name>metadata.elasticsearch.credentials.username</name> <value>${ES_USER}</value> </property> <property> <name>metadata.elasticsearch.credentials.password</name> <value>${ES_PASS}</value> </property> </configuration> EOF ) # Create the secret cat << EOF | kubectl apply -f - apiVersion: v1 kind: Secret metadata: name: cdap-security type: Opaque data: cdap-security.xml: $CDAP_SECURITY EOF

Deploy CDAP

Finally we are ready to deploy CDAP into the Kubernetes cluster. The following YAML provides a simple example. You will need to replace the locationURI with an HCFS compatible file system (e.g. HDFS, Google Cloud Storage, or Amazon AWS). Also, the data.storage.sql.jdbc.connection.url should be configured to point to a PostgreSQL database. Refer to cdap-default.xml for an explanation about the configurations.

apiVersion: cdap.cdap.io/v1alpha1 kind: CDAPMaster metadata: name: cdap spec: appFabric: {} logs: {} messaging: {} metadata: {} metrics: {} preview: {} router: {} runtime: {} userInterface: {} locationURI: <hcfs-uri> serviceAccountName: cdap securitySecret: cdap-security config: enable.preview: "true" data.storage.implementation: postgresql data.storage.sql.jdbc.connection.url: jdbc:postgresql://<postgresql-host>/cdap data.storage.sql.jdbc.driver.name: org.postgresql.Driver metadata.storage.implementation: elastic metadata.elasticsearch.cluster.hosts: https://es-cluster-es-http:9200 metadata.elasticsearch.tls.verify: "false" hdfs.user: root preview.max.runs: "1" preview.poller.count: "1" preview.runner.container.count: "5" preview.waiting.queue.capacity: "100" preview.waiting.queue.timeout.seconds: "3600" provisioner.system.properties.gcp-dataproc.preferExternalIP: "true" logLevels: io.cdap.cdap: DEBUG io.cdap.wrangler: DEBUG org.apache.hadoop.hdfs.DataStreamer: ERROR org.spark_project: WARN

 

You can also configure each of the CDAP services with different cpu, memory, storage, and environments. The following is a simple example that shows how to change the memory and disk size for the appFabric service.

appFabric: env: - name: OPTS value: -XX:+UseG1GC -XX:+ExitOnOutOfMemoryError resources: requests: memory: 8000Mi storageSize: 200Gi

Refer to the Custom Resource Definition (CRD) for all the supported settings.

You can verify CDAP is running correctly by listing out the pods in the Kubernetes cluster.

kubectl get pods -l cdap.instance
NAME READY STATUS RESTARTS AGE cdap-cdap-appfabric-0 1/1 Running 0 114s cdap-cdap-logs-0 1/1 Running 0 2m6s cdap-cdap-messaging-0 1/1 Running 0 2m6s cdap-cdap-metadata-54db5876dc-kplkw 1/1 Running 0 2m6s cdap-cdap-metrics-0 1/1 Running 0 2m6s cdap-cdap-preview-0 1/1 Running 0 119s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-0 1/1 Running 0 79s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-1 1/1 Running 0 79s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-2 1/1 Running 0 79s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-3 1/1 Running 0 79s cdap-cdap-preview-runner-a590df9d-6673-4c-4ead35346f-4 1/1 Running 0 79s cdap-cdap-router-987b785c9-dfcqx 1/1 Running 0 2m6s cdap-cdap-runtime-0 1/1 Running 0 2m6s cdap-cdap-service-system-pipeline-studio-6e8722c4-a064-4b2gx2tc 1/1 Running 0 17s cdap-cdap-userinterface-877f4555d-qvqmw 1/1 Running 0 2m5s

After CDAP is fully up and running, both the UI and REST can be accessed via the user-interface and router services exposed by CDAP.

kubectl get service -l cdap.instance
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE cdap-cdap-router NodePort 10.192.14.102 <none> 11015:31426/TCP 4m41s cdap-cdap-userinterface NodePort 10.192.14.106 <none> 11011:30504/TCP 4m41s

For quick testing, you can use kubectl port-foward to provide access to the CDAP service. For example, you can expose the user interface and then access it through localhost:11011 from the browser.

kubectl port-forward service/cdap-cdap-userinterface 11011

For production use cases, it is better to expose the CDAP services through a load balancer. Consult with your Kubernetes provider for how to deploy a load balancer.

Enable Authentication Service

To enable the Authentication Service in K8s environment to provide Perimeter Security, extra configurations are needed in the CDAP YAML file.

  1. Set the following configurations in the CDAP YAML file "config:" section.

    # Enable perimeter security  security.enabled: "true" # A key file generated by the AuthenticationTool that is mapped into the pod via k8s secret (see below for instructions) security.data.keyfile.path: "/etc/cdap/auth/auth.key" # Disable kerberos (it is defaulted to true when security.enabled is true) kerberos.auth.enabled: "false"

  2. Add configurations for the the authentication handler based on Configuring Managed Authentication under the "config:" section.

  3. Use the CDAP docker image to generate an "auth.key" file.

    docker run -it --rm \ --mount type=bind,source=$(pwd),target=/auth \ gcr.io/cdapio/cdap:latest \ io.cdap.cdap.security.tools.AuthenticationTool -g /auth/auth.key

  4. Create a k8s secret from the "auth.key" file.

    kubectl create secret generic cdap-auth --from-file=auth.key

  5. Add the secret to the CDAP YAML file to map the secret into CDAP pods by adding a "secretVolumes" (same level as other options, like "config").

    config: .... secretVolumes:   cdap-auth: "/etc/cdap/auth"

Now, you can start CDAP with security enabled, without needing Zookeeper.

Running CDAP Programs

Starting in CDAP 6.7.0, you can run CDAP programs on Kubernetes using Spark.

Note: MapReduce and Spark Streaming engines are not supported.

To run CDAP programs on Kubernetes, as a prerequisite the following service account and role binding needs to be created as a requirement from Spark.

  • Create service account

kubectl create serviceaccount spark

  • Create role binding

kubectl create clusterrolebinding spark-role --clusterrole=edit --serviceaccount=default:spark --namespace=default

Verify by running a pipeline

Run a pipeline using CDAP UI:

Limitations

coming soon

Appendix

This section describes how to create the resources required for the CDAP installation using Google Cloud Platform.

Preparation

We will be using the standard bash shell and gcloud command line tool to perform the setup. Install Google Cloud SDK before you proceed.

Set up the following environment variables for using the gcloud command:

export PROJECT=<project-name> export REGION=<region> export NAME=<cluster-name> export PROJECT_NUM=$(gcloud projects describe ${PROJECT} \  --format="value(projectNumber)") export DB_USER=cdap export DB_PASS=$(openssl rand -base64 14) export DB_NAME=cdap

Kubernetes

Create a Google Container Engine (GKE) as the Kubernetes cluster. Make sure the GKE API is enabled before executing the following commands.

# Create a 6 nodes regional GKE cluster in the default network gcloud container clusters create "${NAME}" --region "${REGION}" \  --no-enable-basic-auth --machine-type "e2-standard-4" --num-nodes "2" --scopes \  "https://www.googleapis.com/auth/cloud-platform" --enable-ip-alias --subnetwork \  "projects/${PROJECT}/regions/${REGION}/subnetworks/default" --release-channel \  "regular" --project "${PROJECT}" # Connect to the GKE cluster gcloud container clusters get-credentials "${NAME}" --region "${REGION}" \  --project "${PROJECT}"

Postgresql Database

Create a Google Cloud SQL instance to serve as the PostgreSQL database. Make sure the Cloud SQL API is enabled before executing the following commands:

# Create a private PostgreSQL CloudSQL instance in the default network gcloud beta sql instances create "${NAME}" --database-version=POSTGRES_12 \  --cpu=1 --memory=4GiB --region="${REGION}" --no-assign-ip --network="default" \  --project "${PROJECT}" # Create a PostgreSQL user gcloud sql users create "${DB_USER}" --instance="${NAME}" --password="${DB_PASS}" \  --project "${PROJECT}" # Create a PostgreSQL database gcloud sql databases create "${DB_NAME}" --instance="${NAME}" --project="${PROJECT}" # Export the DB IP address export DB_IP=`gcloud sql instances describe "${NAME}" --project="${PROJECT}" \  --format="value(ipAddresses.ipAddress)"`

Elasticsearch in Kubernetes

We are using the Elasticsearch Operator to operate an Elasticsearch instance inside the Kubernetes cluster. You can deploy the following YAML to create all the necessary resources to have the operator running in the Kubernetes cluster, inside the elastic-system namespace:

 

# Source: eck-operator/templates/operator-namespace.yaml apiVersion: v1 kind: Namespace metadata: name: elastic-system labels: name: elastic-system control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" --- # Source: eck-operator/templates/service-account.yaml apiVersion: v1 kind: ServiceAccount metadata: name: elastic-operator namespace: elastic-system labels: control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" --- # Source: eck-operator/templates/webhook.yaml apiVersion: v1 kind: Secret metadata: name: "elastic-webhook-server-cert" namespace: elastic-system labels: control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" --- # Source: eck-operator/templates/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: elastic-operator namespace: elastic-system labels: control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" data: eck.yaml: |- log-verbosity: 0 metrics-port: 0 container-registry: docker.elastic.co max-concurrent-reconciles: 3 ca-cert-validity: 8760h ca-cert-rotate-before: 24h cert-validity: 8760h cert-rotate-before: 24h set-default-security-context: true kube-client-timeout: 60s elasticsearch-client-timeout: 180s disable-telemetry: false validate-storage-class: true enable-webhook: true webhook-name: elastic-webhook.k8s.elastic.co --- # Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.4.0 creationTimestamp: null labels: app.kubernetes.io/instance: 'elastic-operator' app.kubernetes.io/name: 'eck-operator-crds' app.kubernetes.io/version: '1.3.1' name: apmservers.apm.k8s.elastic.co spec: additionalPrinterColumns: - JSONPath: .status.health name: health type: string - JSONPath: .status.availableNodes description: Available nodes name: nodes type: integer - JSONPath: .status.version description: APM version name: version type: string - JSONPath: .metadata.creationTimestamp name: age type: date group: apm.k8s.elastic.co names: categories: - elastic kind: ApmServer listKind: ApmServerList plural: apmservers shortNames: - apm singular: apmserver scope: Namespaced subresources: status: {} validation: openAPIV3Schema: description: ApmServer represents an APM Server resource in a Kubernetes cluster. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: ApmServerSpec holds the specification of an APM Server. properties: config: description: 'Config holds the APM Server configuration. See: https://www.elastic.co/guide/en/apm/server/current/configuring-howto-apm-server.html' type: object count: description: Count of APM Server instances to deploy. format: int32 type: integer elasticsearchRef: description: ElasticsearchRef is a reference to the output Elasticsearch cluster running in the same Kubernetes cluster. properties: name: description: Name of the Kubernetes object. type: string namespace: description: Namespace of the Kubernetes object. If empty, defaults to the current namespace. type: string required: - name type: object http: description: HTTP holds the HTTP layer configuration for the APM Server resource. properties: service: description: Service defines the template for the associated Kubernetes Service object. properties: metadata: description: ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored. type: object spec: description: Spec is the specification of the service. properties: clusterIP: description: 'clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are "None", empty string (""), or a valid IP address. "None" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string externalIPs: description: externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system. items: type: string type: array externalName: description: externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName. type: string externalTrafficPolicy: description: externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. "Local" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. "Cluster" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. type: string healthCheckNodePort: description: healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local. format: int32 type: integer ipFamily: description: ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment. type: string loadBalancerIP: description: 'Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.' type: string loadBalancerSourceRanges: description: 'If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/' items: type: string type: array ports: description: 'The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' items: description: ServicePort contains information on service's port. properties: appProtocol: description: The application protocol for this port. This field follows standard Kubernetes label syntax. Un-prefixed names are reserved for IANA standard service names (as per RFC-6335 and http://www.iana.org/assignments/service-names). Non-standard protocols should use prefixed names such as mycompany.com/my-custom-protocol. Field can be enabled with ServiceAppProtocol feature gate. type: string name: description: The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service. type: string nodePort: description: 'The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport' format: int32 type: integer port: description: The port that will be exposed by this service. format: int32 type: integer protocol: description: The IP protocol for this port. Supports "TCP", "UDP", and "SCTP". Default is TCP. type: string targetPort: anyOf: - type: integer - type: string description: 'Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod''s container ports. If this is not specified, the value of the ''port'' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the ''port'' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service' required: - port type: object type: array publishNotReadyAddresses: description: publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery. type: boolean selector: additionalProperties: type: string description: 'Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/' type: object sessionAffinity: description: 'Supports "ClientIP" and "None". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string sessionAffinityConfig: description: sessionAffinityConfig contains the configurations of session affinity. properties: clientIP: description: clientIP contains the configurations of Client IP based session affinity. properties: timeoutSeconds: description: timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == "ClientIP". Default value is 10800(for 3 hours). format: int32 type: integer type: object type: object topologyKeys: description: topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value "*" may be used to mean "any topology". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied. items: type: string type: array type: description: 'type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. "ExternalName" maps to the specified externalName. "ClusterIP" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is "None", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. "NodePort" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. "LoadBalancer" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types' type: string type: object type: object tls: description: TLS defines options for configuring TLS for HTTP. properties: certificate: description: "Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain." properties: secretName: description: SecretName is the name of the secret. type: string type: object selfSignedCertificate: description: SelfSignedCertificate allows configuring the self-signed certificate generated by the operator. properties: disabled: description: Disabled indicates that the provisioning of the self-signed certifcate should be disabled. type: boolean subjectAltNames: description: SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate. items: description: SubjectAlternativeName represents a SAN entry in a x509 certificate. properties: dns: description: DNS is the DNS name of the subject. type: string ip: description: IP is the IP address of the subject. type: string type: object type: array type: object type: object type: object image: description: Image is the APM Server Docker image to deploy. type: string kibanaRef: description: KibanaRef is a reference to a Kibana instance running in the same Kubernetes cluster. It allows APM agent central configuration management in Kibana. properties: name: description: Name of the Kubernetes object. type: string namespace: description: Namespace of the Kubernetes object. If empty, defaults to the current namespace. type: string required: - name type: object podTemplate: description: PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the APM Server pods. type: object secureSettings: description: SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for APM Server. items: description: SecretSource defines a data source based on a Kubernetes Secret. properties: entries: description: Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths. items: description: KeyToPath defines how to map a key in a Secret object to a filesystem path. properties: key: description: Key is the key contained in the secret. type: string path: description: Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any ".." components. type: string required: - key type: object type: array secretName: description: SecretName is the name of the secret. type: string required: - secretName type: object type: array serviceAccountName: description: ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references. type: string version: description: Version of the APM Server. type: string required: - version type: object status: description: ApmServerStatus defines the observed state of ApmServer properties: availableNodes: description: AvailableNodes is the number of available replicas in the deployment. format: int32 type: integer elasticsearchAssociationStatus: description: ElasticsearchAssociationStatus is the status of any auto-linking to Elasticsearch clusters. type: string health: description: Health of the deployment. type: string aAssociationStatus: description: KibanaAssociationStatus is the status of any auto-linking to Kibana. type: string secretTokenSecret: description: SecretTokenSecretName is the name of the Secret that contains the secret token type: string service: description: ExternalService is the name of the service the agents should connect to. type: string version: description: 'Version of the stack resource currently running. During version upgrades, multiple versions may run in parallel: this value specifies the lowest version currently running.' type: string type: object version: v1 versions: - name: v1 served: true storage: true - name: v1beta1 served: true storage: false - name: v1alpha1 served: false storage: false status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: [] --- # Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.4.0 creationTimestamp: null labels: app.kubernetes.io/instance: 'elastic-operator' app.kubernetes.io/name: 'eck-operator-crds' app.kubernetes.io/version: '1.3.1' name: beats.beat.k8s.elastic.co spec: additionalPrinterColumns: - JSONPath: .status.health name: health type: string - JSONPath: .status.availableNodes description: Available nodes name: available type: integer - JSONPath: .status.expectedNodes description: Expected nodes name: expected type: integer - JSONPath: .spec.type description: Beat type name: type type: string - JSONPath: .status.version description: Beat version name: version type: string - JSONPath: .metadata.creationTimestamp name: age type: date group: beat.k8s.elastic.co names: categories: - elastic kind: Beat listKind: BeatList plural: beats shortNames: - beat singular: beat scope: Namespaced subresources: status: {} validation: openAPIV3Schema: description: Beat is the Schema for the Beats API. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: BeatSpec defines the desired state of a Beat. properties: config: description: Config holds the Beat configuration. At most one of [`Config`, `ConfigRef`] can be specified. type: object configRef: description: ConfigRef contains a reference to an existing Kubernetes Secret holding the Beat configuration. Beat settings must be specified as yaml, under a single "beat.yml" entry. At most one of [`Config`, `ConfigRef`] can be specified. properties: secretName: description: SecretName is the name of the secret. type: string type: object daemonSet: description: DaemonSet specifies the Beat should be deployed as a DaemonSet, and allows providing its spec. Cannot be used along with `deployment`. If both are absent a default for the Type is used. properties: {} type: object deployment: description: Deployment specifies the Beat should be deployed as a Deployment, and allows providing its spec. Cannot be used along with `daemonSet`. If both are absent a default for the Type is used. properties: replicas: format: int32 type: integer strategy: description: DeploymentStrategy describes how to replace existing pods with new ones. properties: rollingUpdate: description: 'Rolling update config params. Present only if DeploymentStrategyType = RollingUpdate. --- TODO: Update this to follow our convention for oneOf, whatever we decide it to be.' properties: maxSurge: anyOf: - type: integer - type: string description: 'The maximum number of pods that can be scheduled above the desired number of pods. Value can be an absolute number (ex: 5) or a percentage of desired pods (ex: 10%). This can not be 0 if MaxUnavailable is 0. Absolute number is calculated from percentage by rounding up. Defaults to 25%. Example: when this is set to 30%, the new ReplicaSet can be scaled up immediately when the rolling update starts, such that the total number of old and new pods do not exceed 130% of desired pods. Once old pods have been killed, new ReplicaSet can be scaled up further, ensuring that total number of pods running at any time during the update is at most 130% of desired pods.' maxUnavailable: anyOf: - type: integer - type: string description: 'The maximum number of pods that can be unavailable during the update. Value can be an absolute number (ex: 5) or a percentage of desired pods (ex: 10%). Absolute number is calculated from percentage by rounding down. This can not be 0 if MaxSurge is 0. Defaults to 25%. Example: when this is set to 30%, the old ReplicaSet can be scaled down to 70% of desired pods immediately when the rolling update starts. Once new pods are ready, old ReplicaSet can be scaled down further, followed by scaling up the new ReplicaSet, ensuring that the total number of pods available at all times during the update is at least 70% of desired pods.' type: object type: description: Type of deployment. Can be "Recreate" or "RollingUpdate". Default is RollingUpdate. type: string type: object type: object elasticsearchRef: description: ElasticsearchRef is a reference to an Elasticsearch cluster running in the same Kubernetes cluster. properties: name: description: Name of the Kubernetes object. type: string namespace: description: Namespace of the Kubernetes object. If empty, defaults to the current namespace. type: string required: - name type: object image: description: Image is the Beat Docker image to deploy. Version and Type have to match the Beat in the image. type: string kibanaRef: description: KibanaRef is a reference to a Kibana instance running in the same Kubernetes cluster. It allows automatic setup of dashboards and visualizations. properties: name: description: Name of the Kubernetes object. type: string namespace: description: Namespace of the Kubernetes object. If empty, defaults to the current namespace. type: string required: - name type: object secureSettings: description: SecureSettings is a list of references to Kubernetes Secrets containing sensitive configuration options for the Beat. Secrets data can be then referenced in the Beat config using the Secret's keys or as specified in `Entries` field of each SecureSetting. items: description: SecretSource defines a data source based on a Kubernetes Secret. properties: entries: description: Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths. items: description: KeyToPath defines how to map a key in a Secret object to a filesystem path. properties: key: description: Key is the key contained in the secret. type: string path: description: Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any ".." components. type: string required: - key type: object type: array secretName: description: SecretName is the name of the secret. type: string required: - secretName type: object type: array serviceAccountName: description: ServiceAccountName is used to check access from the current resource to Elasticsearch resource in a different namespace. Can only be used if ECK is enforcing RBAC on references. type: string type: description: Type is the type of the Beat to deploy (filebeat, metricbeat, heartbeat, auditbeat, journalbeat, packetbeat, etc.). Any string can be used, but well-known types will have the image field defaulted and have the appropriate Elasticsearch roles created automatically. It also allows for dashboard setup when combined with a `KibanaRef`. maxLength: 20 pattern: '[a-zA-Z0-9-]+' type: string version: description: Version of the Beat. type: string required: - type - version type: object status: description: BeatStatus defines the observed state of a Beat. properties: availableNodes: format: int32 type: integer elasticsearchAssociationStatus: description: AssociationStatus is the status of an association resource. type: string expectedNodes: format: int32 type: integer health: type: string kibanaAssociationStatus: description: AssociationStatus is the status of an association resource. type: string version: description: 'Version of the stack resource currently running. During version upgrades, multiple versions may run in parallel: this value specifies the lowest version currently running.' type: string type: object version: v1beta1 versions: - name: v1beta1 served: true storage: true status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: [] --- # Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.4.0 creationTimestamp: null labels: app.kubernetes.io/instance: 'elastic-operator' app.kubernetes.io/name: 'eck-operator-crds' app.kubernetes.io/version: '1.3.1' name: elasticsearches.elasticsearch.k8s.elastic.co spec: additionalPrinterColumns: - JSONPath: .status.health name: health type: string - JSONPath: .status.availableNodes description: Available nodes name: nodes type: integer - JSONPath: .status.version description: Elasticsearch version name: version type: string - JSONPath: .status.phase name: phase type: string - JSONPath: .metadata.creationTimestamp name: age type: date group: elasticsearch.k8s.elastic.co names: categories: - elastic kind: Elasticsearch listKind: ElasticsearchList plural: elasticsearches shortNames: - es singular: elasticsearch scope: Namespaced subresources: status: {} validation: openAPIV3Schema: description: Elasticsearch represents an Elasticsearch resource in a Kubernetes cluster. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: ElasticsearchSpec holds the specification of an Elasticsearch cluster. properties: auth: description: Auth contains user authentication and authorization security settings for Elasticsearch. properties: fileRealm: description: FileRealm to propagate to the Elasticsearch cluster. items: description: FileRealmSource references users to create in the Elasticsearch cluster. properties: secretName: description: SecretName is the name of the secret. type: string type: object type: array roles: description: Roles to propagate to the Elasticsearch cluster. items: description: RoleSource references roles to create in the Elasticsearch cluster. properties: secretName: description: SecretName is the name of the secret. type: string type: object type: array type: object http: description: HTTP holds HTTP layer settings for Elasticsearch. properties: service: description: Service defines the template for the associated Kubernetes Service object. properties: metadata: description: ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored. type: object spec: description: Spec is the specification of the service. properties: clusterIP: description: 'clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are "None", empty string (""), or a valid IP address. "None" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string externalIPs: description: externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system. items: type: string type: array externalName: description: externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName. type: string externalTrafficPolicy: description: externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. "Local" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. "Cluster" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. type: string healthCheckNodePort: description: healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local. format: int32 type: integer ipFamily: description: ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment. type: string loadBalancerIP: description: 'Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.' type: string loadBalancerSourceRanges: description: 'If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/' items: type: string type: array ports: description: 'The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' items: description: ServicePort contains information on service's port. properties: appProtocol: description: The application protocol for this port. This field follows standard Kubernetes label syntax. Un-prefixed names are reserved for IANA standard service names (as per RFC-6335 and http://www.iana.org/assignments/service-names). Non-standard protocols should use prefixed names such as mycompany.com/my-custom-protocol. Field can be enabled with ServiceAppProtocol feature gate. type: string name: description: The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service. type: string nodePort: description: 'The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport' format: int32 type: integer port: description: The port that will be exposed by this service. format: int32 type: integer protocol: description: The IP protocol for this port. Supports "TCP", "UDP", and "SCTP". Default is TCP. type: string targetPort: anyOf: - type: integer - type: string description: 'Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod''s container ports. If this is not specified, the value of the ''port'' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the ''port'' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service' required: - port type: object type: array publishNotReadyAddresses: description: publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery. type: boolean selector: additionalProperties: type: string description: 'Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/' type: object sessionAffinity: description: 'Supports "ClientIP" and "None". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string sessionAffinityConfig: description: sessionAffinityConfig contains the configurations of session affinity. properties: clientIP: description: clientIP contains the configurations of Client IP based session affinity. properties: timeoutSeconds: description: timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == "ClientIP". Default value is 10800(for 3 hours). format: int32 type: integer type: object type: object topologyKeys: description: topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value "*" may be used to mean "any topology". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied. items: type: string type: array type: description: 'type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. "ExternalName" maps to the specified externalName. "ClusterIP" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is "None", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. "NodePort" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. "LoadBalancer" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types' type: string type: object type: object tls: description: TLS defines options for configuring TLS for HTTP. properties: certificate: description: "Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain." properties: secretName: description: SecretName is the name of the secret. type: string type: object selfSignedCertificate: description: SelfSignedCertificate allows configuring the self-signed certificate generated by the operator. properties: disabled: description: Disabled indicates that the provisioning of the self-signed certifcate should be disabled. type: boolean subjectAltNames: description: SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate. items: description: SubjectAlternativeName represents a SAN entry in a x509 certificate. properties: dns: description: DNS is the DNS name of the subject. type: string ip: description: IP is the IP address of the subject. type: string type: object type: array type: object type: object type: object image: description: Image is the Elasticsearch Docker image to deploy. type: string nodeSets: description: NodeSets allow specifying groups of Elasticsearch nodes sharing the same configuration and Pod templates. items: description: NodeSet is the specification for a group of Elasticsearch nodes sharing the same configuration and a Pod template. properties: config: description: Config holds the Elasticsearch configuration. type: object count: description: Count of Elasticsearch nodes to deploy. format: int32 minimum: 1 type: integer name: description: Name of this set of nodes. Becomes a part of the Elasticsearch node.name setting. maxLength: 23 pattern: '[a-zA-Z0-9-]+' type: string podTemplate: description: PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Pods belonging to this NodeSet. type: object volumeClaimTemplates: description: VolumeClaimTemplates is a list of persistent volume claims to be used by each Pod in this NodeSet. Every claim in this list must have a matching volumeMount in one of the containers defined in the PodTemplate. Items defined here take precedence over any default claims added by the operator with the same name. items: description: PersistentVolumeClaim is a user's request for and claim to a persistent volume properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: description: 'Standard object''s metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata' type: object spec: description: 'Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims' properties: accessModes: description: 'AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1' items: type: string type: array dataSource: description: 'This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.' properties: apiGroup: description: APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required. type: string kind: description: Kind is the type of resource being referenced type: string name: description: Name is the name of resource being referenced type: string required: - kind - name type: object resources: description: 'Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources' properties: limits: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object requests: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object selector: description: A label query over volumes to consider for binding. properties: matchExpressions: description: matchExpressions is a list of label selector requirements. The requirements are ANDed. items: description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values. properties: key: description: key is the label key that the selector applies to. type: string operator: description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist. type: string values: description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch. items: type: string type: array required: - key - operator type: object type: array matchLabels: additionalProperties: type: string description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed. type: object type: object storageClassName: description: 'Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1' type: string volumeMode: description: volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. type: string volumeName: description: VolumeName is the binding reference to the PersistentVolume backing this claim. type: string type: object status: description: 'Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims' properties: accessModes: description: 'AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1' items: type: string type: array capacity: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ description: Represents the actual resources of the underlying volume. type: object conditions: description: Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'. items: description: PersistentVolumeClaimCondition contails details about state of pvc properties: lastProbeTime: description: Last time we probed the condition. format: date-time type: string lastTransitionTime: description: Last time the condition transitioned from one status to another. format: date-time type: string message: description: Human-readable message indicating details about last transition. type: string reason: description: Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports "ResizeStarted" that means the underlying persistent volume is being resized. type: string status: type: string type: description: PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type type: string required: - status - type type: object type: array phase: description: Phase represents the current phase of PersistentVolumeClaim. type: string type: object type: object type: array required: - count - name type: object minItems: 1 type: array podDisruptionBudget: description: PodDisruptionBudget provides access to the default pod disruption budget for the Elasticsearch cluster. The default budget selects all cluster pods and sets `maxUnavailable` to 1. To disable, set `PodDisruptionBudget` to the empty value (`{}` in YAML). properties: metadata: description: ObjectMeta is the metadata of the PDB. The name and namespace provided here are managed by ECK and will be ignored. type: object spec: description: Spec is the specification of the PDB. properties: maxUnavailable: anyOf: - type: integer - type: string description: An eviction is allowed if at most "maxUnavailable" pods selected by "selector" are unavailable after the eviction, i.e. even in absence of the evicted pod. For example, one can prevent all voluntary evictions by specifying 0. This is a mutually exclusive setting with "minAvailable". minAvailable: anyOf: - type: integer - type: string description: An eviction is allowed if at least "minAvailable" pods selected by "selector" will still be available after the eviction, i.e. even in the absence of the evicted pod. So for example you can prevent all voluntary evictions by specifying "100%". selector: description: Label query over pods whose evictions are managed by the disruption budget. properties: matchExpressions: description: matchExpressions is a list of label selector requirements. The requirements are ANDed. items: description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values. properties: key: description: key is the label key that the selector applies to. type: string operator: description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist. type: string values: description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch. items: type: string type: array required: - key - operator type: object type: array matchLabels: additionalProperties: type: string description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed. type: object type: object type: object type: object remoteClusters: description: RemoteClusters enables you to establish uni-directional connections to a remote Elasticsearch cluster. items: description: RemoteCluster declares a remote Elasticsearch cluster connection. properties: elasticsearchRef: description: ElasticsearchRef is a reference to an Elasticsearch cluster running within the same k8s cluster. properties: name: description: Name of the Kubernetes object. type: string namespace: description: Namespace of the Kubernetes object. If empty, defaults to the current namespace. type: string required: - name type: object name: description: Name is the name of the remote cluster as it is set in the Elasticsearch settings. The name is expected to be unique for each remote clusters. minLength: 1 type: string required: - name type: object type: array secureSettings: description: SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Elasticsearch. items: description: SecretSource defines a data source based on a Kubernetes Secret. properties: entries: description: Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths. items: description: KeyToPath defines how to map a key in a Secret object to a filesystem path. properties: key: description: Key is the key contained in the secret. type: string path: description: Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any ".." components. type: string required: - key type: object type: array secretName: description: SecretName is the name of the secret. type: string required: - secretName type: object type: array serviceAccountName: description: ServiceAccountName is used to check access from the current resource to a resource (eg. a remote Elasticsearch cluster) in a different namespace. Can only be used if ECK is enforcing RBAC on references. type: string transport: description: Transport holds transport layer settings for Elasticsearch. properties: service: description: Service defines the template for the associated Kubernetes Service object. properties: metadata: description: ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored. type: object spec: description: Spec is the specification of the service. properties: clusterIP: description: 'clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are "None", empty string (""), or a valid IP address. "None" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string externalIPs: description: externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system. items: type: string type: array externalName: description: externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName. type: string externalTrafficPolicy: description: externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. "Local" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. "Cluster" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. type: string healthCheckNodePort: description: healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local. format: int32 type: integer ipFamily: description: ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment. type: string loadBalancerIP: description: 'Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.' type: string loadBalancerSourceRanges: description: 'If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/' items: type: string type: array ports: description: 'The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' items: description: ServicePort contains information on service's port. properties: appProtocol: description: The application protocol for this port. This field follows standard Kubernetes label syntax. Un-prefixed names are reserved for IANA standard service names (as per RFC-6335 and http://www.iana.org/assignments/service-names). Non-standard protocols should use prefixed names such as mycompany.com/my-custom-protocol. Field can be enabled with ServiceAppProtocol feature gate. type: string name: description: The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service. type: string nodePort: description: 'The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport' format: int32 type: integer port: description: The port that will be exposed by this service. format: int32 type: integer protocol: description: The IP protocol for this port. Supports "TCP", "UDP", and "SCTP". Default is TCP. type: string targetPort: anyOf: - type: integer - type: string description: 'Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod''s container ports. If this is not specified, the value of the ''port'' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the ''port'' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service' required: - port type: object type: array publishNotReadyAddresses: description: publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery. type: boolean selector: additionalProperties: type: string description: 'Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/' type: object sessionAffinity: description: 'Supports "ClientIP" and "None". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string sessionAffinityConfig: description: sessionAffinityConfig contains the configurations of session affinity. properties: clientIP: description: clientIP contains the configurations of Client IP based session affinity. properties: timeoutSeconds: description: timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == "ClientIP". Default value is 10800(for 3 hours). format: int32 type: integer type: object type: object topologyKeys: description: topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value "*" may be used to mean "any topology". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied. items: type: string type: array type: description: 'type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. "ExternalName" maps to the specified externalName. "ClusterIP" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is "None", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. "NodePort" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. "LoadBalancer" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types' type: string type: object type: object type: object updateStrategy: description: UpdateStrategy specifies how updates to the cluster should be performed. properties: changeBudget: description: ChangeBudget defines the constraints to consider when applying changes to the Elasticsearch cluster. properties: maxSurge: description: MaxSurge is the maximum number of new pods that can be created exceeding the original number of pods defined in the specification. MaxSurge is only taken into consideration when scaling up. Setting a negative value will disable the restriction. Defaults to unbounded if not specified. format: int32 type: integer maxUnavailable: description: MaxUnavailable is the maximum number of pods that can be unavailable (not ready) during the update due to circumstances under the control of the operator. Setting a negative value will disable this restriction. Defaults to 1 if not specified. format: int32 type: integer type: object type: object version: description: Version of Elasticsearch. type: string required: - nodeSets - version type: object status: description: ElasticsearchStatus defines the observed state of Elasticsearch properties: availableNodes: description: AvailableNodes is the number of available instances. format: int32 type: integer health: description: ElasticsearchHealth is the health of the cluster as returned by the health API. type: string phase: description: ElasticsearchOrchestrationPhase is the phase Elasticsearch is in from the controller point of view. type: string version: description: 'Version of the stack resource currently running. During version upgrades, multiple versions may run in parallel: this value specifies the lowest version currently running.' type: string type: object version: v1 versions: - name: v1 served: true storage: true - name: v1beta1 served: true storage: false - name: v1alpha1 served: false storage: false status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: [] --- # Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.4.0 creationTimestamp: null labels: app.kubernetes.io/instance: 'elastic-operator' app.kubernetes.io/name: 'eck-operator-crds' app.kubernetes.io/version: '1.3.1' name: enterprisesearches.enterprisesearch.k8s.elastic.co spec: additionalPrinterColumns: - JSONPath: .status.health name: health type: string - JSONPath: .status.availableNodes description: Available nodes name: nodes type: integer - JSONPath: .status.version description: Enterprise Search version name: version type: string - JSONPath: .metadata.creationTimestamp name: age type: date group: enterprisesearch.k8s.elastic.co names: categories: - elastic kind: EnterpriseSearch listKind: EnterpriseSearchList plural: enterprisesearches shortNames: - ent singular: enterprisesearch scope: Namespaced subresources: status: {} validation: openAPIV3Schema: description: EnterpriseSearch is a Kubernetes CRD to represent Enterprise Search. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: EnterpriseSearchSpec holds the specification of an Enterprise Search resource. properties: config: description: Config holds the Enterprise Search configuration. type: object configRef: description: ConfigRef contains a reference to an existing Kubernetes Secret holding the Enterprise Search configuration. Configuration settings are merged and have precedence over settings specified in `config`. properties: secretName: description: SecretName is the name of the secret. type: string type: object count: description: Count of Enterprise Search instances to deploy. format: int32 type: integer elasticsearchRef: description: ElasticsearchRef is a reference to the Elasticsearch cluster running in the same Kubernetes cluster. properties: name: description: Name of the Kubernetes object. type: string namespace: description: Namespace of the Kubernetes object. If empty, defaults to the current namespace. type: string required: - name type: object http: description: HTTP holds the HTTP layer configuration for Enterprise Search resource. properties: service: description: Service defines the template for the associated Kubernetes Service object. properties: metadata: description: ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored. type: object spec: description: Spec is the specification of the service. properties: clusterIP: description: 'clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are "None", empty string (""), or a valid IP address. "None" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string externalIPs: description: externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system. items: type: string type: array externalName: description: externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName. type: string externalTrafficPolicy: description: externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. "Local" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. "Cluster" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. type: string healthCheckNodePort: description: healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local. format: int32 type: integer ipFamily: description: ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment. type: string loadBalancerIP: description: 'Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.' type: string loadBalancerSourceRanges: description: 'If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/' items: type: string type: array ports: description: 'The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' items: description: ServicePort contains information on service's port. properties: appProtocol: description: The application protocol for this port. This field follows standard Kubernetes label syntax. Un-prefixed names are reserved for IANA standard service names (as per RFC-6335 and http://www.iana.org/assignments/service-names). Non-standard protocols should use prefixed names such as mycompany.com/my-custom-protocol. Field can be enabled with ServiceAppProtocol feature gate. type: string name: description: The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service. type: string nodePort: description: 'The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport' format: int32 type: integer port: description: The port that will be exposed by this service. format: int32 type: integer protocol: description: The IP protocol for this port. Supports "TCP", "UDP", and "SCTP". Default is TCP. type: string targetPort: anyOf: - type: integer - type: string description: 'Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod''s container ports. If this is not specified, the value of the ''port'' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the ''port'' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service' required: - port type: object type: array publishNotReadyAddresses: description: publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery. type: boolean selector: additionalProperties: type: string description: 'Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/' type: object sessionAffinity: description: 'Supports "ClientIP" and "None". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string sessionAffinityConfig: description: sessionAffinityConfig contains the configurations of session affinity. properties: clientIP: description: clientIP contains the configurations of Client IP based session affinity. properties: timeoutSeconds: description: timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == "ClientIP". Default value is 10800(for 3 hours). format: int32 type: integer type: object type: object topologyKeys: description: topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value "*" may be used to mean "any topology". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied. items: type: string type: array type: description: 'type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. "ExternalName" maps to the specified externalName. "ClusterIP" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is "None", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. "NodePort" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. "LoadBalancer" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types' type: string type: object type: object tls: description: TLS defines options for configuring TLS for HTTP. properties: certificate: description: "Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain." properties: secretName: description: SecretName is the name of the secret. type: string type: object selfSignedCertificate: description: SelfSignedCertificate allows configuring the self-signed certificate generated by the operator. properties: disabled: description: Disabled indicates that the provisioning of the self-signed certifcate should be disabled. type: boolean subjectAltNames: description: SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate. items: description: SubjectAlternativeName represents a SAN entry in a x509 certificate. properties: dns: description: DNS is the DNS name of the subject. type: string ip: description: IP is the IP address of the subject. type: string type: object type: array type: object type: object type: object image: description: Image is the Enterprise Search Docker image to deploy. type: string podTemplate: description: PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Enterprise Search pods. type: object serviceAccountName: description: ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references. type: string version: description: Version of Enterprise Search. type: string type: object status: description: EnterpriseSearchStatus defines the observed state of EnterpriseSearch properties: associationStatus: description: Association is the status of any auto-linking to Elasticsearch clusters. type: string availableNodes: description: AvailableNodes is the number of available replicas in the deployment. format: int32 type: integer health: description: Health of the deployment. type: string service: description: ExternalService is the name of the service associated to the Enterprise Search Pods. type: string version: description: 'Version of the stack resource currently running. During version upgrades, multiple versions may run in parallel: this value specifies the lowest version currently running.' type: string type: object version: v1beta1 versions: - name: v1beta1 served: true storage: true status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: [] --- # Source: eck-operator/charts/eck-operator-crds/templates/all-crds.yaml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.4.0 creationTimestamp: null labels: app.kubernetes.io/instance: 'elastic-operator' app.kubernetes.io/name: 'eck-operator-crds' app.kubernetes.io/version: '1.3.1' name: kibanas.kibana.k8s.elastic.co spec: additionalPrinterColumns: - JSONPath: .status.health name: health type: string - JSONPath: .status.availableNodes description: Available nodes name: nodes type: integer - JSONPath: .status.version description: Kibana version name: version type: string - JSONPath: .metadata.creationTimestamp name: age type: date group: kibana.k8s.elastic.co names: categories: - elastic kind: Kibana listKind: KibanaList plural: kibanas shortNames: - kb singular: kibana scope: Namespaced subresources: status: {} validation: openAPIV3Schema: description: Kibana represents a Kibana resource in a Kubernetes cluster. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: KibanaSpec holds the specification of a Kibana instance. properties: config: description: 'Config holds the Kibana configuration. See: https://www.elastic.co/guide/en/kibana/current/settings.html' type: object count: description: Count of Kibana instances to deploy. format: int32 type: integer elasticsearchRef: description: ElasticsearchRef is a reference to an Elasticsearch cluster running in the same Kubernetes cluster. properties: name: description: Name of the Kubernetes object. type: string namespace: description: Namespace of the Kubernetes object. If empty, defaults to the current namespace. type: string required: - name type: object http: description: HTTP holds the HTTP layer configuration for Kibana. properties: service: description: Service defines the template for the associated Kubernetes Service object. properties: metadata: description: ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored. type: object spec: description: Spec is the specification of the service. properties: clusterIP: description: 'clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are "None", empty string (""), or a valid IP address. "None" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string externalIPs: description: externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system. items: type: string type: array externalName: description: externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName. type: string externalTrafficPolicy: description: externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. "Local" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. "Cluster" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. type: string healthCheckNodePort: description: healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local. format: int32 type: integer ipFamily: description: ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment. type: string loadBalancerIP: description: 'Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.' type: string loadBalancerSourceRanges: description: 'If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature." More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/' items: type: string type: array ports: description: 'The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' items: description: ServicePort contains information on service's port. properties: appProtocol: description: The application protocol for this port. This field follows standard Kubernetes label syntax. Un-prefixed names are reserved for IANA standard service names (as per RFC-6335 and http://www.iana.org/assignments/service-names). Non-standard protocols should use prefixed names such as mycompany.com/my-custom-protocol. Field can be enabled with ServiceAppProtocol feature gate. type: string name: description: The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service. type: string nodePort: description: 'The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport' format: int32 type: integer port: description: The port that will be exposed by this service. format: int32 type: integer protocol: description: The IP protocol for this port. Supports "TCP", "UDP", and "SCTP". Default is TCP. type: string targetPort: anyOf: - type: integer - type: string description: 'Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod''s container ports. If this is not specified, the value of the ''port'' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the ''port'' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service' required: - port type: object type: array publishNotReadyAddresses: description: publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery. type: boolean selector: additionalProperties: type: string description: 'Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/' type: object sessionAffinity: description: 'Supports "ClientIP" and "None". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies' type: string sessionAffinityConfig: description: sessionAffinityConfig contains the configurations of session affinity. properties: clientIP: description: clientIP contains the configurations of Client IP based session affinity. properties: timeoutSeconds: description: timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == "ClientIP". Default value is 10800(for 3 hours). format: int32 type: integer type: object type: object topologyKeys: description: topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value "*" may be used to mean "any topology". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied. items: type: string type: array type: description: 'type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. "ExternalName" maps to the specified externalName. "ClusterIP" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is "None", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. "NodePort" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. "LoadBalancer" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types' type: string type: object type: object tls: description: TLS defines options for configuring TLS for HTTP. properties: certificate: description: "Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain." properties: secretName: description: SecretName is the name of the secret. type: string type: object selfSignedCertificate: description: SelfSignedCertificate allows configuring the self-signed certificate generated by the operator. properties: disabled: description: Disabled indicates that the provisioning of the self-signed certifcate should be disabled. type: boolean subjectAltNames: description: SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate. items: description: SubjectAlternativeName represents a SAN entry in a x509 certificate. properties: dns: description: DNS is the DNS name of the subject. type: string ip: description: IP is the IP address of the subject. type: string type: object type: array type: object type: object type: object image: description: Image is the Kibana Docker image to deploy. type: string podTemplate: description: PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Kibana pods type: object secureSettings: description: SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Kibana. items: description: SecretSource defines a data source based on a Kubernetes Secret. properties: entries: description: Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths. items: description: KeyToPath defines how to map a key in a Secret object to a filesystem path. properties: key: description: Key is the key contained in the secret. type: string path: description: Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any ".." components. type: string required: - key type: object type: array secretName: description: SecretName is the name of the secret. type: string required: - secretName type: object type: array serviceAccountName: description: ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references. type: string version: description: Version of Kibana. type: string required: - version type: object status: description: KibanaStatus defines the observed state of Kibana properties: associationStatus: description: AssociationStatus is the status of an association resource. type: string availableNodes: description: AvailableNodes is the number of available replicas in the deployment. format: int32 type: integer health: description: Health of the deployment. type: string version: description: 'Version of the stack resource currently running. During version upgrades, multiple versions may run in parallel: this value specifies the lowest version currently running.' type: string type: object version: v1 versions: - name: v1 served: true storage: true - name: v1beta1 served: true storage: false - name: v1alpha1 served: false storage: false status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: [] --- # Source: eck-operator/templates/cluster-roles.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: elastic-operator labels: control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" rules: - apiGroups: - "authorization.k8s.io" resources: - subjectaccessreviews verbs: - create - apiGroups: - "" resources: - pods - endpoints - events - persistentvolumeclaims - secrets - services - configmaps - serviceaccounts verbs: - get - list - watch - create - update - patch - delete - apiGroups: - apps resources: - deployments - statefulsets - daemonsets verbs: - get - list - watch - create - update - patch - delete - apiGroups: - policy resources: - poddisruptionbudgets verbs: - get - list - watch - create - update - patch - delete - apiGroups: - elasticsearch.k8s.elastic.co resources: - elasticsearches - elasticsearches/status - elasticsearches/finalizers - enterpriselicenses - enterpriselicenses/status verbs: - get - list - watch - create - update - patch - delete - apiGroups: - kibana.k8s.elastic.co resources: - kibanas - kibanas/status - kibanas/finalizers verbs: - get - list - watch - create - update - patch - delete - apiGroups: - apm.k8s.elastic.co resources: - apmservers - apmservers/status - apmservers/finalizers verbs: - get - list - watch - create - update - patch - delete - apiGroups: - enterprisesearch.k8s.elastic.co resources: - enterprisesearches - enterprisesearches/status - enterprisesearches/finalizers verbs: - get - list - watch - create - update - patch - delete - apiGroups: - beat.k8s.elastic.co resources: - beats - beats/status - beats/finalizers verbs: - get - list - watch - create - update - patch - delete - apiGroups: - storage.k8s.io resources: - storageclasses verbs: - get - list - watch - apiGroups: - admissionregistration.k8s.io resources: - mutatingwebhookconfigurations - validatingwebhookconfigurations verbs: - get - list - watch - create - update - patch - delete --- # Source: eck-operator/templates/cluster-roles.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: "elastic-operator-view" labels: rbac.authorization.k8s.io/aggregate-to-view: "true" rbac.authorization.k8s.io/aggregate-to-edit: "true" rbac.authorization.k8s.io/aggregate-to-admin: "true" control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" rules: - apiGroups: ["elasticsearch.k8s.elastic.co"] resources: ["elasticsearches"] verbs: ["get", "list", "watch"] - apiGroups: ["apm.k8s.elastic.co"] resources: ["apmservers"] verbs: ["get", "list", "watch"] - apiGroups: ["kibana.k8s.elastic.co"] resources: ["kibanas"] verbs: ["get", "list", "watch"] - apiGroups: ["enterprisesearch.k8s.elastic.co"] resources: ["enterprisesearches"] verbs: ["get", "list", "watch"] - apiGroups: ["beat.k8s.elastic.co"] resources: ["beats"] verbs: ["get", "list", "watch"] --- # Source: eck-operator/templates/cluster-roles.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: "elastic-operator-edit" labels: rbac.authorization.k8s.io/aggregate-to-edit: "true" rbac.authorization.k8s.io/aggregate-to-admin: "true" control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" rules: - apiGroups: ["elasticsearch.k8s.elastic.co"] resources: ["elasticsearches"] verbs: ["create", "delete", "deletecollection", "patch", "update"] - apiGroups: ["apm.k8s.elastic.co"] resources: ["apmservers"] verbs: ["create", "delete", "deletecollection", "patch", "update"] - apiGroups: ["kibana.k8s.elastic.co"] resources: ["kibanas"] verbs: ["create", "delete", "deletecollection", "patch", "update"] - apiGroups: ["enterprisesearch.k8s.elastic.co"] resources: ["enterprisesearches"] verbs: ["create", "delete", "deletecollection", "patch", "update"] - apiGroups: ["beat.k8s.elastic.co"] resources: ["beats"] verbs: ["create", "delete", "deletecollection", "patch", "update"] --- # Source: eck-operator/templates/operator-role-binding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: elastic-operator labels: control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: elastic-operator subjects: - kind: ServiceAccount name: elastic-operator namespace: elastic-system --- # Source: eck-operator/templates/webhook.yaml apiVersion: v1 kind: Service metadata: name: elastic-webhook-server namespace: elastic-system labels: control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" spec: ports: - name: https port: 443 targetPort: 9443 selector: control-plane: elastic-operator --- # Source: eck-operator/templates/statefulset.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: elastic-operator namespace: elastic-system labels: control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" spec: selector: matchLabels: control-plane: elastic-operator serviceName: elastic-operator replicas: 1 template: metadata: annotations: # Rename the fields "error" to "error.message" and "source" to "event.source" # This is to avoid a conflict with the ECS "error" and "source" documents. "co.elastic.logs/raw": "[{\"type\":\"container\",\"json.keys_under_root\":true,\"paths\":[\"/var/log/containers/*${data.kubernetes.container.id}.log\"],\"processors\":[{\"convert\":{\"mode\":\"rename\",\"ignore_missing\":true,\"fields\":[{\"from\":\"error\",\"to\":\"_error\"}]}},{\"convert\":{\"mode\":\"rename\",\"ignore_missing\":true,\"fields\":[{\"from\":\"_error\",\"to\":\"error.message\"}]}},{\"convert\":{\"mode\":\"rename\",\"ignore_missing\":true,\"fields\":[{\"from\":\"source\",\"to\":\"_source\"}]}},{\"convert\":{\"mode\":\"rename\",\"ignore_missing\":true,\"fields\":[{\"from\":\"_source\",\"to\":\"event.source\"}]}}]}]" "checksum/config": ee68b5877bbac6a3071ec907276f8966d24c9878268247a2b3da09837ba8c5fa labels: control-plane: elastic-operator spec: terminationGracePeriodSeconds: 10 serviceAccountName: elastic-operator securityContext: runAsNonRoot: true containers: - image: "docker.elastic.co/eck/eck-operator:1.3.1" imagePullPolicy: IfNotPresent name: manager args: - "manager" - "--config=/conf/eck.yaml" - "--distribution-channel=all-in-one" env: - name: OPERATOR_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: WEBHOOK_SECRET value: "elastic-webhook-server-cert" resources: limits: cpu: 1 memory: 512Mi requests: cpu: 100m memory: 150Mi ports: - containerPort: 9443 name: https-webhook protocol: TCP volumeMounts: - mountPath: "/conf" name: conf readOnly: true - mountPath: /tmp/k8s-webhook-server/serving-certs name: cert readOnly: true volumes: - name: conf configMap: name: elastic-operator - name: cert secret: defaultMode: 420 secretName: "elastic-webhook-server-cert" --- # Source: eck-operator/templates/webhook.yaml apiVersion: admissionregistration.k8s.io/v1beta1 kind: ValidatingWebhookConfiguration metadata: name: elastic-webhook.k8s.elastic.co labels: control-plane: elastic-operator app.kubernetes.io/version: "1.3.1" webhooks: - clientConfig: caBundle: Cg== service: name: elastic-webhook-server namespace: elastic-system path: /validate-apm-k8s-elastic-co-v1-apmserver failurePolicy: Ignore name: elastic-apm-validation-v1.k8s.elastic.co rules: - apiGroups: - apm.k8s.elastic.co apiVersions: - v1 operations: - CREATE - UPDATE resources: - apmservers - clientConfig: caBundle: Cg== service: name: elastic-webhook-server namespace: elastic-system path: /validate-apm-k8s-elastic-co-v1beta1-apmserver failurePolicy: Ignore name: elastic-apm-validation-v1beta1.k8s.elastic.co rules: - apiGroups: - apm.k8s.elastic.co apiVersions: - v1beta1 operations: - CREATE - UPDATE resources: - apmservers - clientConfig: caBundle: Cg== service: name: elastic-webhook-server namespace: elastic-system path: /validate-beat-k8s-elastic-co-v1beta1-beat failurePolicy: Ignore name: elastic-beat-validation-v1beta1.k8s.elastic.co rules: - apiGroups: - beat.k8s.elastic.co apiVersions: - v1beta1 operations: - CREATE - UPDATE resources: - beats - clientConfig: caBundle: Cg== service: name: elastic-webhook-server namespace: elastic-system path: /validate-enterprisesearch-k8s-elastic-co-v1beta1-enterprisesearch failurePolicy: Ignore name: elastic-ent-validation-v1beta1.k8s.elastic.co rules: - apiGroups: - enterprisesearch.k8s.elastic.co apiVersions: - v1beta1 operations: - CREATE - UPDATE resources: - enterprisesearches - clientConfig: caBundle: Cg== service: name: elastic-webhook-server namespace: elastic-system path: /validate-elasticsearch-k8s-elastic-co-v1-elasticsearch failurePolicy: Ignore name: elastic-es-validation-v1.k8s.elastic.co rules: - apiGroups: - elasticsearch.k8s.elastic.co apiVersions: - v1 operations: - CREATE - UPDATE resources: - elasticsearches - clientConfig: caBundle: Cg== service: name: elastic-webhook-server namespace: elastic-system path: /validate-elasticsearch-k8s-elastic-co-v1beta1-elasticsearch failurePolicy: Ignore name: elastic-es-validation-v1beta1.k8s.elastic.co rules: - apiGroups: - elasticsearch.k8s.elastic.co apiVersions: - v1beta1 operations: - CREATE - UPDATE resources: - elasticsearches - clientConfig: caBundle: Cg== service: name: elastic-webhook-server namespace: elastic-system path: /validate-kibana-k8s-elastic-co-v1-kibana failurePolicy: Ignore name: elastic-kb-validation-v1.k8s.elastic.co rules: - apiGroups: - kibana.k8s.elastic.co apiVersions: - v1 operations: - CREATE - UPDATE resources: - kibanas - clientConfig: caBundle: Cg== service: name: elastic-webhook-server namespace: elastic-system path: /validate-kibana-k8s-elastic-co-v1beta1-kibana failurePolicy: Ignore name: elastic-kb-validation-v1beta1.k8s.elastic.co rules: - apiGroups: - kibana.k8s.elastic.co apiVersions: - v1beta1 operations: - CREATE - UPDATE resources: - kibanas

After deploying the Elasticsearch operator, you can deploy the following custom resource to start an Elasticsearch instance inside the Kubernetes cluster.

apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: es-cluster spec: version: 6.8.13 nodeSets: - name: default count: 1 config: node.store.allow_mmap: false

You can validate that the Elasticsearch instance is up and running correctly by observing an Elasticsearch pod is in the RUNNING state.

kubectl get pods -l elasticsearch.k8s.elastic.co/cluster-name
NAME READY STATUS RESTARTS AGE es-cluster-es-default-0 1/1 Running 0 5m

After the Elasticsearch instance, you need to get the default user password from the secret created by the operator. This password is needed in the cdap-security.xml file for CDAP to authenticate itself to Elasticsearch.

export ES_USER=elastic export ES_PASS=$(kubectl get secret es-cluster-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')

 

Related content

Created in 2020 by Google Inc.