Wikipedia

Kubernetes

Kubernetes (/ˌk(j)uːbərˈnɛtɪs, -ˈneɪtɪs, -ˈneɪtiːz, -ˈnɛtiːz/, K8s)[3] is an open-source container orchestration system for automating software deployment, scaling, and management.[4][5] Originally designed by Google, the project is now maintained by a worldwide community of contributors, and the trademark is held by the Cloud Native Computing Foundation.

Kubernetes
Original author(s)Google
Developer(s)Cloud Native Computing Foundation
Initial release0.2[1] / 9 September 2014; 10 years ago (2014-09-09)
Stable release
1.31.1[2]Edit this on Wikidata / 12 September 2024; 2 months ago (12 September 2024)
Repository
Written inGo
TypeCluster management software
LicenseApache License 2.0
Websitekubernetes.io

The name Kubernetes originates from the Greek κυβερνήτης (kubernḗtēs), meaning governor, 'helmsman' or 'pilot'. Kubernetes is often abbreviated as K8s, counting the eight letters between the K and the s (a numeronym).[6]

Kubernetes assembles one or more computers, either virtual machines or bare metal, into a cluster which can run workloads in containers. It works with various container runtimes, such as containerd and CRI-O.[7] Its suitability for running and managing workloads of all sizes and styles has led to its widespread adoption in clouds and data centers. There are multiple distributions of this platform – from independent software vendors (ISVs) as well as hosted-on-cloud offerings from all the major public cloud vendors.[8]

Kubernetes is one of the most widely deployed software systems in the world[9] being used across companies including Google, Microsoft, Amazon, Apple, Meta, Nvidia, Reddit and Pinterest.

History

edit
 
Google Kubernetes Engine talk at 2017 Google Cloud Summit

Kubernetes (Ancient Greek: κυβερνήτης, romanized: kubernḗtēs, 'steersman, navigator' or 'guide', and the etymological root of cybernetics)[5] was announced by Google on June 6, 2014.[10] The project was conceived and created by Google employees Joe Beda, Brendan Burns, and Craig McLuckie. Others at Google soon joined to help build the project including Ville Aikas, Dawn Chen, Brian Grant, Tim Hockin, and Daniel Smith.[11][12] Other companies such as Red Hat and CoreOS joined the effort soon after, with notable contributors such as Clayton Coleman and Kelsey Hightower.[10]

The design and development of Kubernetes was inspired by Google's Borg cluster manager and based on Promise Theory.[13][14] Many of its top contributors had previously worked on Borg;[15][16] they codenamed Kubernetes "Project 7" after the Star Trek ex-Borg character Seven of Nine[17] and gave its logo a seven-spoked ship's wheel (designed by Tim Hockin). Unlike Borg, which was written in C++,[15] Kubernetes is written in the Go language.

Kubernetes was announced in June, 2014 and version 1.0 was released on July 21, 2015.[18] Google worked with the Linux Foundation to form the Cloud Native Computing Foundation (CNCF)[19] and offered Kubernetes as the seed technology.

Google was already offering a managed Kubernetes service, GKE, and Red Hat was supporting Kubernetes as part of OpenShift since the inception of the Kubernetes project in 2014.[20] In 2017, the principal competitors rallied around Kubernetes and announced adding native support for it:

On March 6, 2018, Kubernetes Project reached ninth place in the list of GitHub projects by the number of commits, and second place in authors and issues, after the Linux kernel.[26]

Until version 1.18, Kubernetes followed an N-2 support policy, meaning that the three most recent minor versions receive security updates and bug fixes.[27] Starting with version 1.19, Kubernetes follows an N-3 support policy.[28]

Concepts

edit
 
Kubernetes architecture diagram

Kubernetes defines a set of building blocks ("primitives") that collectively provide mechanisms that deploy, maintain, and scale applications based on CPU, memory[29] or custom metrics.[30] Kubernetes is loosely coupled and extensible to meet the needs of different workloads. The internal components as well as extensions and containers that run on Kubernetes rely on the Kubernetes API.[31] The platform exerts its control over compute and storage resources by defining resources as objects, which can then be managed as such.

Kubernetes follows the primary/replica architecture. The components of Kubernetes can be divided into those that manage an individual node and those that are part of the control plane.[31][32]

Control plane

edit

The Kubernetes master node handles the Kubernetes control plane of the cluster, managing its workload and directing communication across the system. The Kubernetes control plane consists of various components such as TLS encryption, RBAC, and a strong authentication method, network separation, each its own process, that can run both on a single master node or on multiple masters supporting high-availability clusters.[32] The various components of the Kubernetes control plane are as follows.[33]

Etcd

edit

Etcd[34] is a persistent, lightweight, distributed, key-value data store (originally developed for Container Linux). It reliably stores the configuration data of the cluster, representing the overall state of the cluster at any given point of time. etcd favors consistency over availability in the event of a network partition (see CAP theorem). The consistency is crucial for correctly scheduling and operating services.

API server

edit

The API server serves the Kubernetes API using JSON over HTTP, which provides both the internal and external interface to Kubernetes.[31][35] The API server processes, validates REST requests, and updates the state of the API objects in etcd, thereby allowing clients to configure workloads and containers across worker nodes.[36] The API server uses etcd's watch API to monitor the cluster, roll out critical configuration changes, or restore any divergences of the state of the cluster back to the desired state as declared in etcd.

As an example, a human operator may specify that three instances of a particular "pod" (see below) need to be running, and etcd stores this fact. If the Deployment controller finds that only two instances are running (conflicting with the etcd declaration),[37] it schedules the creation of an additional instance of that pod.[32]

Scheduler

edit

The scheduler is an extensible component that selects the node that an unscheduled pod (the basic unit of workloads to be scheduled) runs, based on resource availability and other constraints. The scheduler tracks resource allocation on each node to ensure that workload is not scheduled in excess of available resources. For this purpose, the scheduler must know the resource requirements, resource availability, and other user-provided constraints or policy directives such as quality-of-service, affinity/anti-affinity requirements, and data locality. The scheduler's role is to match resource "supply" to workload "demand".[38]

Kubernetes allows running multiple schedulers within a single cluster. As such, scheduler plug-ins may be developed and installed as in-process extensions to the native vanilla scheduler by running it as a separate scheduler, as long as they conform to the Kubernetes scheduling framework.[39] This allows cluster administrators to extend or modify the behavior of the default Kubernetes scheduler according to their needs.

Controllers

edit

A controller is a reconciliation loop that drives the actual cluster state toward the desired state, communicating with the API server to create, update, and delete the resources it manages (e.g., pods or service endpoints).[40][35]

An example controller is a ReplicaSet controller, which handles replication and scaling by running a specified number of copies of a pod across the cluster. The controller also handles creating replacement pods if the underlying node fails.[40] Other controllers that are part of the core Kubernetes system include a DaemonSet controller for running exactly one pod on every machine (or some subset of machines), and a Job controller for running pods that run to completion (e.g. as part of a batch job).[41] Labels selectors often form part of the controller's definition that specify the set of pods that a controller manages.[42]

The controller manager is a single process that manages several core Kubernetes controllers (including the examples described above), is distributed as part of the standard Kubernetes installation and responding to the loss of nodes.[33]

Custom controllers may also be installed in the cluster, further allowing the behavior and API of Kubernetes to be extended when used in conjunction with custom resources (see custom resources, controllers and operators below).

Nodes

edit

A node, also known as a worker or a minion, is a machine where containers (workloads) are deployed. Every node in the cluster must run a container runtime, as well as the below-mentioned components, for communication with the primary network configuration of these containers.

kubelet

edit

kubelet is responsible for the running state of each node, ensuring that all containers on the node are healthy. It takes care of starting, stopping, and maintaining application containers organized into pods as directed by the control plane.[31][43] kubelet monitors the state of a pod, and if not in the desired state, the pod re-deploys to the same node. Node status is relayed every few seconds via heartbeat messages to the API server. Once the control plane detects a node failure, a higher-level controller is expected to observe this state change and launch pods on another healthy node.[44]

Container runtime

edit

A container runtime is responsible for the lifecycle of containers, including launching, reconciling and killing of containers. kubelet interacts with container runtimes via the Container Runtime Interface (CRI),[45][46] which decouples the maintenance of core Kubernetes from the actual CRI implementation.

Originally, kubelet interfaced exclusively with the Docker runtime[47] through a "dockershim". However, from November 2020[48] up to April 2022, Kubernetes has deprecated the shim in favor of directly interfacing with the container through containerd, or replacing Docker with a runtime that is compliant with the Container Runtime Interface (CRI).[49][45][50] With the release of v1.24 in May 2022, the "dockershim" has been removed entirely.[51]

Examples of popular container runtimes that are compatible with kubelet include containerd (initially supported via Docker), rkt[52] and CRI-O.

kube-proxy

edit

kube-proxy is an implementation of a network proxy and a load balancer, and it supports the service abstraction along with the other networking operations.[31] It is responsible for routing traffic to the appropriate container based on IP and port number of the incoming request.

Namespaces

edit

In Kubernetes, namespaces are utilized to segregate the resources it handles into distinct and non-intersecting collections.[53] They are intended for use in environments with many users spread across multiple teams, or projects, or even separating environments like development, test, and production.

Pods

edit

The basic scheduling unit in Kubernetes is a pod,[54] which consists of one or more containers that are guaranteed to be co-located on the same node.[31] Each pod in Kubernetes is assigned a unique IP address within the cluster, allowing applications to use ports without the risk of conflict.[55] Within the pod, all containers can reference each other.

A container resides inside a pod. The container is the lowest level of a micro-service, which holds the running application, libraries, and their dependencies.

Workloads

edit

Kubernetes supports several abstractions of workloads that are at a higher level over simple pods. This allows users to declaratively define and manage these high-level abstractions, instead of having to manage individual pods by themselves. Several of these abstractions, supported by a standard installation of Kubernetes, are described below.

ReplicaSets, ReplicationControllers and Deployments

edit

A ReplicaSet's purpose is to maintain a stable set of replica pods running at any given time. As such, it is often used to guarantee the availability of a specified number of identical Pods.[56] The ReplicaSet can also be said to be a grouping mechanism that lets Kubernetes maintain the number of instances that have been declared for a given pod. The definition of a ReplicaSet uses a selector, whose evaluation will result in identifying all pods that are associated with it.

A ReplicationController, similar to a ReplicaSet, serves the same purpose and behaves similarly to a ReplicaSet, which is to ensure that there will always be a specified number of pod replicas as desired. The ReplicationController workload was the predecessor of a ReplicaSet, but was eventually deprecated in favor of ReplicaSet to make use of set-based label selectors.[56]

Deployments are a higher-level management mechanism for ReplicaSets. While the ReplicaSet controller manages the scale of the ReplicaSet, the Deployment controller manages what happens to the ReplicaSet – whether an update has to be rolled out, or rolled back, etc. When Deployments are scaled up or down, this results in the declaration of the ReplicaSet changing, and this change in the declared state is managed by the ReplicaSet controller.[37]

StatefulSets

edit

StatefulSets are controllers that enforce the properties of uniqueness and ordering amongst instances of a pod, and can be used to run stateful applications.[57] While scaling stateless applications is only a matter of adding more running pods, doing so for stateful workloads is harder, because the state needs to be preserved if a pod is restarted. If the application is scaled up or down, the state may need to be redistributed.

Databases are an example of stateful workloads. When run in high-availability mode, many databases come with the notion of a primary instance and secondary instances. In this case, the notion of ordering of instances is important. Other applications like Apache Kafka distribute the data amongst their brokers; hence, one broker is not the same as another. In this case, the notion of instance uniqueness is important.

DaemonSets

edit

DaemonSets are responsible for ensuring that a pod is created on every single node in the cluster.[58] Generally, most workloads scale in response to a desired replica count, depending on the availability and performance requirements as needed by the application. However, in other scenarios it may be necessary to deploy a pod to every single node in the cluster, scaling up the number of total pods as nodes are added and garbage collecting them as they are removed. This is particularly helpful for use cases where the workload has some dependency on the actual node or host machine, such as log collection, ingress controllers, and storage services.

Services

edit
 
Simplified view showing how Services interact with Pod networking in a Kubernetes cluster

A Kubernetes service is a set of pods that work together, such as one tier of a multi-tier application. The set of pods that constitute a service are defined by a label selector.[31] Kubernetes provides two modes of service discovery, using environment variables or using Kubernetes DNS.[59] Service discovery assigns a stable IP address and DNS name to the service, and load balances traffic in a round-robin manner to network connections of that IP address among the pods matching the selector (even as failures cause the pods to move from machine to machine).[55] By default a service is exposed inside a cluster (e.g., back end pods might be grouped into a service, with requests from the front-end pods load-balanced among them), but a service can also be exposed outside a cluster (e.g., for clients to reach front-end pods).[60]

Volumes

edit

Filesystems in the Kubernetes container provide ephemeral storage, by default. This means that a restart of the pod will wipe out any data on such containers, and therefore, this form of storage is quite limiting in anything but trivial applications. A Kubernetes volume[61] provides persistent storage that exists for the lifetime of the pod itself. This storage can also be used as shared disk space for containers within the pod. Volumes are mounted at specific mount points within the container, which are defined by the pod configuration, and cannot mount onto other volumes or link to other volumes. The same volume can be mounted at different points in the file system tree by different containers.

ConfigMaps and Secrets

edit

A common application challenge is deciding where to store and manage configuration information, some of which may contain sensitive data. Configuration data can be anything as fine-grained as individual properties, or coarse-grained information like entire configuration files such as JSON or XML documents. Kubernetes provides two closely related mechanisms to deal with this need, known as ConfigMaps and Secrets, both of which allow for configuration changes to be made without requiring an application rebuild.

The data from ConfigMaps and Secrets will be made available to every single instance of the application to which these objects have been bound via the Deployment. A Secret and/or a ConfigMap is sent to a node only if a pod on that node requires it, which will only be stored in memory on the node. Once the pod that depends on the Secret or ConfigMap is deleted, the in-memory copy of all bound Secrets and ConfigMaps are deleted as well.

The data from a ConfigMap or Secret is accessible to the pod through one of the following ways:[62]

  1. As environment variables, which will be consumed by kubelet from the ConfigMap when the container is launched;
  2. Mounted within a volume accessible within the container's filesystem, which supports automatic reloading without restarting the container.

The biggest difference between a Secret and a ConfigMap is that Secrets are specifically designed for containing secure and confidential data, although they are not encrypted at rest by default, and requires additional setup in order to fully secure the use of Secrets within the cluster.[63] Secrets are often used to store confidential or sensitive data like certificates, credentials to work with image registries, passwords, and ssh keys.

Labels and selectors

edit

Kubernetes enables clients (users or internal components) to attach keys called labels to any API object in the system, such as pods and nodes. Correspondingly, label selectors are queries against labels that resolve to matching objects.[31] When a service is defined, one can define the label selectors that will be used by the service router/load balancer to select the pod instances that the traffic will be routed to. Thus, simply changing the labels of the pods or changing the label selectors on the service can be used to control which pods get traffic and which don't, which can be used to support various deployment patterns like blue–green deployments or A/B testing. This capability to dynamically control how services utilize implementing resources provides a loose coupling within the infrastructure.

For example, if an application's pods have labels for a system tier (with values such as frontend, backend, for example) and a release_track (with values such as canary, production, for example), then an operation on all of backend and canary nodes can use a label selector, such as:[42]

tier=backend AND release_track=canary

Just like labels, field selectors also let one select Kubernetes resources. Unlike labels, the selection is based on the attribute values inherent to the resource being selected, rather than user-defined categorization. metadata.name and metadata.namespace are field selectors that will be present on all Kubernetes objects. Other selectors that can be used depend on the object/resource type.

Add-ons

edit

Add-ons are additional features of the Kubernetes cluster implemented as applications running within it. The pods may be managed by Deployments, ReplicationControllers, and so on. There are many add-ons. Some of the more important are:

DNS
Cluster DNS is a DNS server, in addition to the other DNS server(s) in the environment, which serves DNS records for Kubernetes services. Containers started by Kubernetes automatically include this DNS server in their DNS searches.
Web UI
This is a general purpose, web-based UI for Kubernetes clusters. It allows administrators to manage and troubleshoot applications running in the cluster, as well as the cluster itself.
Resource monitoring
Container Resource Monitoring records metrics about containers in a central database, and provides a UI for browsing that data.
Cost monitoring
Kubernetes cost monitoring applications allow breakdown of costs by pods, nodes, namespaces, and labels.
Cluster-level logging
To prevent the loss of event data in the event of node or pod failures, container logs can be saved to a central log store with a search/browsing interface. Kubernetes provides no native storage for log data, but one can integrate many existing logging solutions into the Kubernetes cluster.

Storage

edit

Containers emerged as a way to make software portable. The container contains all the packages needed to run a service. The provided file system makes containers extremely portable and easy to use in development. A container can be moved from development to test or production with no or relatively few configuration changes.

Historically Kubernetes was suitable only for stateless services. However, many applications have a database, which requires persistence, which leads to the creation of persistent storage for Kubernetes. Implementing persistent storage for containers is one of the top challenges of Kubernetes administrators, DevOps and cloud engineers. Containers may be ephemeral, but more and more of their data is not, so one needs to ensure the data's survival in case of container termination or hardware failure. When deploying containers with Kubernetes or containerized applications, organizations often realize that they need persistent storage. They need to provide fast and reliable storage for databases, root images and other data used by the containers.

In addition to the landscape, the Cloud Native Computing Foundation (CNCF), has published other information about Kubernetes Persistent Storage including a blog helping to define the container attached storage pattern. This pattern can be thought of as one that uses Kubernetes itself as a component of the storage system or service.[64]

More information about the relative popularity of these and other approaches can be found on the CNCF's landscape survey as well, which showed that OpenEBS – a Stateful Persistent Storage platform from Datacore Software,[65] and Rook – a storage orchestration project – were the two projects most likely to be in evaluation as of the Fall of 2019.[66]

Container Attached Storage is a type of data storage that emerged as Kubernetes gained prominence. The Container Attached Storage approach or pattern relies on Kubernetes itself for certain capabilities while delivering primarily block, file, object and interfaces to workloads running on Kubernetes.[67]

Common attributes of Container Attached Storage include the use of extensions to Kubernetes, such as custom resource definitions, and the use of Kubernetes itself for functions that otherwise would be separately developed and deployed for storage or data management. Examples of functionality delivered by custom resource definitions or by Kubernetes itself include retry logic, delivered by Kubernetes itself, and the creation and maintenance of an inventory of available storage media and volumes, typically delivered via a custom resource definition.[68][69]

Container Storage Interface (CSI)

edit

In Kubernetes version 1.9, the initial Alpha release of Container Storage Interface (CSI) was introduced.[70] Previously, storage volume plug-ins were included in the Kubernetes distribution. By creating a standardized CSI, the code required to interface with external storage systems was separated from the core Kubernetes code base. Just one year later, the CSI feature was made Generally Available (GA) in Kubernetes.[71]

API

edit

A key component of the Kubernetes control plane is the API Server, which exposes an HTTP API that can be invoked by other parts of the cluster as well as end users and external components. This API is a REST API and is declarative in nature, and is the same API exposed to the control plane.[72] The API server is backed by etcd to store all records persistently.[73]

API objects

edit

In Kubernetes, all objects serve as the "record of intent" of the cluster's state, and are able to define the desired state that the writer of the object wishes for the cluster to be in.[74] As such, most Kubernetes objects have the same set of nested fields, as follows:

  • spec: Describes the desired state of the resource, which can be controlled by end users, or other higher-level controllers;
  • status: Describes the current state of the resource, which is actively updated by the controller of the resource.

All objects in Kubernetes are subject to the same API conventions. Some of these include:

  • Must have the following metadata under the nested object field metadata:[75]
    • namespace: a label that objects are subdivided into;
    • name: a string that uniquely identifies the object within the defined namespace;
    • uid: a unique string that is able to distinguish between objects with the same name across space and time (even across deletions and recreations with the same name).
  • May be managed by another controller, which is defined in the metadata.ownerReferences field:[76]
    • At most one other object shall be the managing controller of the controllee object, which is defined by the controller field.
  • May be garbage collected if the owner is deleted:[77]
    • When an object is deleted, all dependent objects may also be deleted in a cascading fashion.

Custom resources, controllers and operators

edit

The Kubernetes API can be extended using Custom Resources, which represent objects that are not part of the standard Kubernetes installation. These custom resources are declared using Custom Resource Definitions (CRDs), which is a kind of resource that can be dynamically registered and unregistered without shutting down or restarting a cluster that is currently running.[78]

Custom controllers are another extension mechanism that interact with the Kubernetes API, similar to the default controllers in the standard pre-installed Kubernetes controller manager. These controllers may interact with custom resources to allow for a declarative API: users may declare the desired state of the system via the custom resources, and it is the responsibility of the custom controller to observe the change and reconcile it.

The combination of custom resources and custom controllers are often referred to as a Kubernetes Operator. The key use case for operators are to capture the aim of a human operator who is managing a service or set of services and to implement them using automation, and with a declarative API supporting this automation. Human operators who look after specific applications and services have deep knowledge of how the system ought to behave, how to deploy it, and how to react if there are problems.

Examples of problems solved by operators include taking and restoring backups of that application's state, and handling upgrades of the application code alongside related changes such as database schemas or extra configuration settings. Several notable projects under the Cloud Native Computing Foundation's incubation program follow the operator pattern to extend Kubernetes, including Argo, Open Policy Agent and Istio.[79]

API security

edit

Kubernetes defines the following strategies for controlling access to its API.[80]

Transport security

edit

The Kubernetes API server listens on a TCP port that serves HTTPS traffic, in order to enforce transport layer security (TLS) using CA certificates.[33]

In older versions of Kubernetes, the API server supported listening on both HTTP and HTTPS ports (with the HTTP port number having no transport security whatsoever). This was deprecated in v1.10 and eventually dropped support in v1.20 of Kubernetes.[81]

Authentication

edit

All requests made to the Kubernetes API server are expected to be authenticated, and supports several authentication strategies, some of which are listed below:[82]

  1. X.509 client certificates
  2. Bearer tokens
  3. Service account tokens, intended for programmatic API access

Users are typically expected to indicate and define cluster URL details along with the necessary credentials in a kubeconfig file, which are natively supported by other Kubernetes tools like kubectl and the official Kubernetes client libraries.[83]

Authorization

edit

The Kubernetes API supports the following authorization modes:[84]

  1. Node authorization mode: Grants a fixed list of operations of API requests that kubelets are allowed to perform, in order to function properly.
  2. Attribute-based access control (ABAC) mode: Grants access rights to users through the use of defined access control policies which combine attributes together.
  3. Role-based access control (RBAC) mode: Grants access rights to users based on roles that are granted to the user, where each role defines a list of actions that are allowed.
  4. Webhook mode: Queries a REST API service to determine if a user is authorized to perform a given action.[33]

API clients

edit

Kubernetes supports several official API clients:

Cluster API

edit

The same API design principles have been used to define an API to harness a program in order to create, configure, and manage Kubernetes clusters. This is called the Cluster API.[87] A key concept embodied in the API is using Infrastructure as Software, or the notion that the Kubernetes cluster infrastructure is itself a resource / object that can be managed just like any other Kubernetes resources. Similarly, machines that make up the cluster are also treated as a Kubernetes resource. The API has two pieces – the core API, and a provider implementation. The provider implementation consists of cloud-provider specific functions that let Kubernetes provide the cluster API in a fashion that is well-integrated with the cloud-provider's services and resources.[33]

Uses

edit

Kubernetes is commonly used as a way to host a microservice-based implementation, because it and its associated ecosystem of tools provide all the capabilities needed to address key concerns of any microservice architecture.

Distributions

edit

Various vendors offer Kubernetes-based platforms or infrastructure as a service (IaaS) that deploy Kubernetes.[88][89]

These are typically categorized according to open-source, commercial or managed distributions. Several notable distributions are listed below:[90]

Open-source distributions

edit
  • Amazon EKS-D
  • k0s
  • k3s
  • SUSE Rancher Kubernetes Engine (RKE)
  • OKD.IO The Community Distribution of Kubernetes that powers Red Hat OpenShift

Commercial distributions

edit

Managed distributions

edit
  • Alibaba Cloud ACK (Alibaba Cloud Container Service for Kubernetes)
  • Amazon EKS (Elastic Kubernetes Service)
  • Canonical MicroK8s and Charmed Kubernetes
  • DigitalOcean managed Kubernetes Service
  • Google GKE (Google Kubernetes Engine)
  • Huawei CCE (Huawei Cloud Container Engine)
  • IBM Cloud Kubernetes Services
  • Microsoft AKS (Azure Kubernetes Services)
  • Mirantis Kubernetes Engine with OpsCare Plus managed services
  • Oracle Container Engine for Kubernetes
  • Platform9 Managed Kubernetes
  • Wind River Systems Wind River Studio

Release timeline

edit
Release timeline
Version Release date End of Life date[91] Notes
Old version, no longer maintained: 1.0 10 July 2015 Original Release
Old version, no longer maintained: 1.1 9 November 2015 [92]
Old version, no longer maintained: 1.2 16 March 2016 23 October 2016 [93]
Old version, no longer maintained: 1.3 1 July 2016 1 November 2016 [94]
Old version, no longer maintained: 1.4 26 September 2016 21 April 2017 [95]
Old version, no longer maintained: 1.5 12 December 2016 1 October 2017 [96]
Old version, no longer maintained: 1.6 28 March 2017 23 November 2017 [97]
Old version, no longer maintained: 1.7 30 June 2017 4 April 2018 [98]
Old version, no longer maintained: 1.8 28 August 2017 12 July 2018 [99]
Old version, no longer maintained: 1.9 15 December 2017 29 September 2018 [100]
Old version, no longer maintained: 1.10 28 March 2018 13 February 2019 [101]
Old version, no longer maintained: 1.11 3 July 2018 1 May 2019 [102]
Old version, no longer maintained: 1.12 27 September 2018 8 July 2019 [103]
Old version, no longer maintained: 1.13 3 December 2018 15 October 2019 [104]
Old version, no longer maintained: 1.14 25 March 2019 11 December 2019 [105]
Old version, no longer maintained: 1.15 20 June 2019 6 May 2020 [106]
Old version, no longer maintained: 1.16 22 October 2019 2 September 2020 [107]
Old version, no longer maintained: 1.17 9 December 2019 13 January 2021 [108]
Old version, no longer maintained: 1.18 25 March 2020 18 June 2021 [109]
Old version, no longer maintained: 1.19 26 August 2020[110] 28 October 2021 From Kubernetes version 1.19 on, the support window has been extended to one year of full support plus two months of maintenance mode period.[28]
[111]
Old version, no longer maintained: 1.20 8 December 2020 28 February 2022 [112]
Old version, no longer maintained: 1.21 8 April 2021 28 June 2022 [113]
Old version, no longer maintained: 1.22 4 August 2021 28 October 2022 [114]
Old version, no longer maintained: 1.23 7 December 2021 28 February 2023 [115]
Old version, no longer maintained: 1.24 3 May 2022 28 July 2023 [116]
Old version, no longer maintained: 1.25 23 August 2022 27 October 2023 [117]
Old version, no longer maintained: 1.26 9 December 2022 24 February 2024 [118]
Old version, no longer maintained: 1.27 11 April 2023 28 June 2024 [119]
Old version, yet still maintained: 1.28 15 August 2023 28 October 2024 [120]
Old version, yet still maintained: 1.29 13 December 2023 28 February 2025 [121]
Old version, yet still maintained: 1.30 17 April 2024 28 June 2025 [122]
Current stable version: 1.31 13 August 2024 28 October 2025 [123]
Legend:
Old version, not maintained
Old version, still maintained
Latest version
Latest preview version
Future release

Support windows

edit

The chart below visualizes the period for which each release is/was supported[91]

See also

edit

References

edit
  1. ^ "v0.2". github.com. 2014-09-09.
  2. ^ "Release 1.31.1". 12 September 2024. Retrieved 22 September 2024.
  3. ^ "Kubernetes GitHub Repository". GitHub. January 22, 2021.
  4. ^ "kubernetes/kubernetes". GitHub. Archived from the original on 2017-04-21. Retrieved 2017-03-28.
  5. ^ a b "What is Kubernetes?". Kubernetes. Retrieved 2017-03-31.
  6. ^ "Overview Kubernetes". Kubernetes. Archived from the original on 2023-01-08. Retrieved 2022-01-04.
  7. ^ "Container runtimes". Kubernetes. Retrieved 2021-11-14.
  8. ^ "Turnkey Cloud Solutions". Kubernetes Documentation. Retrieved July 25, 2023.
  9. ^ "CNCF Annual Survey 2022". CNCF. January 31, 2023.
  10. ^ a b Metz, Cade. "Google Open Sources Its Secret Weapon in Cloud Computing". Wired. Archived from the original on 10 September 2015. Retrieved 24 September 2015.
  11. ^ Metz, Cade. "Google Made Its Secret Blueprint Public to Boost Its Cloud". Wired. Archived from the original on 2016-07-01. Retrieved 2016-06-27.
  12. ^ Burns, Brendan (July 20, 2018), The History of Kubernetes & the Community Behind It, archived from the original on 2022-02-27
  13. ^ Kubernetes: The Documentary [PART 1], retrieved 2023-12-14
  14. ^ "https://twitter.com/kelseyhightower/status/1527333243845873664". X (formerly Twitter). Retrieved 2023-12-14. {{cite web}}: External link in |title= (help)
  15. ^ a b Abhishek Verma; Luis Pedrosa; Madhukar R. Korupolu; David Oppenheimer; Eric Tune; John Wilkes (April 21–24, 2015). "Large-scale cluster management at Google with Borg". Proceedings of the European Conference on Computer Systems (EuroSys). Archived from the original on 2017-07-27.
  16. ^ "Borg, Omega, and Kubernetes - ACM Queue". queue.acm.org. Archived from the original on 2016-07-09. Retrieved 2016-06-27.
  17. ^ "Early Stage Startup Heptio Aims to Make Kubernetes Friendly". Retrieved 2016-12-06.
  18. ^ "As Kubernetes Hits 1.0, Google Donates Technology To Newly Formed Cloud Native Computing Foundation". TechCrunch. 21 July 2015. Archived from the original on 23 September 2015. Retrieved 24 September 2015.
  19. ^ "Cloud Native Computing Foundation". Archived from the original on 2017-07-03.
  20. ^ "Red Hat and Google collaborate on Kubernetes to manage Docker containers at scale". 2014-07-10. Retrieved 2022-08-06.
  21. ^ "VMware and Pivotal Launch Pivotal Container Service (PKS) and Collaborate with Google Cloud to Bring Kubernetes to Enterprise Customers". 2017-08-29. Retrieved 2022-08-06.
  22. ^ Lardinois, Frederic (2017-09-06). "Mesosphere adds Kubernetes support to its data center operating system". Retrieved 2022-08-06.
  23. ^ "Docker Announces Enhancements to the Docker Platform to Simplify and Advance the Management of Kubernetes for Enterprise IT". 2017-10-17. Archived from the original on 2020-09-26.
  24. ^ Monroy, Gabe (2017-10-24). "Introducing AKS (managed Kubernetes) and Azure Container Registry improvements". Retrieved 2022-08-06.
  25. ^ "Introducing Amazon Elastic Container Service for Kubernetes (Preview)". 2017-11-29. Retrieved 2022-08-06.
  26. ^ Conway, Sarah (6 March 2018). "Kubernetes Is First CNCF Project To Graduate". Cloud Native Computing Foundation. Archived from the original on 29 October 2018. Retrieved 3 December 2018. Compared to the 1.5 million projects on GitHub, Kubernetes is No. 9 for commits and No. 2 for authors/issues, second only to Linux.
  27. ^ "Kubernetes version and version skew support policy". Kubernetes. Retrieved 2020-03-03.
  28. ^ a b "Kubernetes 1.19 Release Announcement > Increase Kubernetes support window to one year". Kubernetes. 26 August 2020. Retrieved 2020-08-28.
  29. ^ Sharma, Priyanka (13 April 2017). "Autoscaling based on CPU/Memory in Kubernetes—Part II". Powerupcloud Tech Blog. Medium. Archived from the original on 17 October 2019. Retrieved 27 December 2018.
  30. ^ "Configure Kubernetes Autoscaling With Custom Metrics". Bitnami. BitRock. 15 November 2018. Archived from the original on 27 March 2019. Retrieved 27 December 2018.
  31. ^ a b c d e f g h "An Introduction to Kubernetes". DigitalOcean. Archived from the original on 1 October 2015. Retrieved 24 September 2015.
  32. ^ a b c "Kubernetes Infrastructure". OpenShift Community Documentation. OpenShift. Archived from the original on 6 July 2015. Retrieved 24 September 2015.
  33. ^ a b c d e "Kubernetes Hardening Guide" (PDF). United States Department of Defense. Retrieved January 26, 2024.
  34. ^ Container Linux by CoreOS: Cluster infrastructure
  35. ^ a b Marhubi, Kamal (2015-09-26). "Kubernetes from the ground up: API server". kamalmarhubi.com. Archived from the original on 2015-10-29. Retrieved 2015-11-02.
  36. ^ Ellingwood, Justin (2 May 2018). "An Introduction to Kubernetes". DigitalOcean. Archived from the original on 5 July 2018. Retrieved 20 July 2018. One of the most important primary services is an API server. This is the main management point of the entire cluster as it allows a user to configure Kubernetes' workloads and organizational units. It is also responsible for making sure that the etcd store and the service details of deployed containers are in agreement. It acts as the bridge between various components to maintain cluster health and disseminate information and commands.
  37. ^ a b "Deployments". Kubernetes. Retrieved 2022-02-27.
  38. ^ "The Three Pillars of Kubernetes Container Orchestration - Rancher Labs". rancher.com. 18 May 2017. Archived from the original on 24 June 2017. Retrieved 22 May 2017.
  39. ^ "Scheduling Framework". Kubernetes Documentation. Retrieved July 24, 2023.
  40. ^ a b "Overview of a Replication Controller". Documentation. CoreOS. Archived from the original on 2015-09-22. Retrieved 2015-11-02.
  41. ^ Sanders, Jake (2015-10-02). "Kubernetes: Exciting Experimental Features". Livewyer. Archived from the original on 2015-10-20. Retrieved 2015-11-02.
  42. ^ a b "Intro: Docker and Kubernetes training - Day 2". Red Hat. 2015-10-20. Archived from the original on 2015-10-29. Retrieved 2015-11-02.
  43. ^ Marhubi, Kamal (2015-08-27). "What [..] is a Kubelet?". kamalmarhubi.com. Archived from the original on 2015-11-13. Retrieved 2015-11-02.
  44. ^ "Kubernetes Security | Issues and Best Practices | Snyk". snyk.io. 26 July 2020. Retrieved 2021-05-16.
  45. ^ a b "Introducing Container Runtime Interface (CRI) in Kubernetes". Kubernetes. 2016-12-19. Retrieved 2021-05-16.
  46. ^ "Container Runtime Interface (CRI)". Kubernetes Documentation. Retrieved July 24, 2023.
  47. ^ "Kubernetes v1.12: Introducing RuntimeClass". kubernetes.io. 10 October 2018.
  48. ^ "Deprecate Dockershim - Kubernetes Github repository - PR 94624". Github.com.
  49. ^ "Don't Panic: Kubernetes and Docker". Kubernetes Blog. 2 December 2020. Retrieved 2020-12-22.
  50. ^ "kubernetes/community". GitHub. Retrieved 2021-05-16.
  51. ^ "Updated: Dockershim Removal FAQ". Kubernetes Blog. 17 February 2022.
  52. ^ "rktnetes brings rkt container engine to Kubernetes". kubernetes.io. 11 July 2016.
  53. ^ "Namespaces". kubernetes.io.
  54. ^ "Pods". kubernetes.io.
  55. ^ a b Langemak, Jon (2015-02-11). "Kubernetes 101 – Networking". Das Blinken Lichten. Archived from the original on 2015-10-25. Retrieved 2015-11-02.
  56. ^ a b "ReplicaSet". kubernetes.io. Retrieved 2020-03-03.
  57. ^ "StatefulSets". kubernetes.io.
  58. ^ "DaemonSet". kubernetes.io.
  59. ^ "Service". kubernetes.io.
  60. ^ Langemak, Jon (2015-02-15). "Kubernetes 101 – External Access Into The Cluster". Das Blinken Lichten. Archived from the original on 2015-10-26. Retrieved 2015-11-02.
  61. ^ "Volumes". kubernetes.io.
  62. ^ "ConfigMaps". Kubernetes Documentation. Retrieved July 24, 2023.
  63. ^ "Secrets". Kubernetes. Retrieved 2023-07-23.
  64. ^ "Container Attached Storage: A primer". Cloud Native Computing Foundation. 2018-04-19. Retrieved 2021-05-16.
  65. ^ "Dataore Acquired MayaData original developer of OpenEBS". datacore.com. 18 November 2021.
  66. ^ "CNCF SURVEY 2019" (PDF). cncf.io.
  67. ^ "Container Attached Storage: A primer". Cloud Native Computing Foundation. 2018-04-19. Retrieved 2020-10-09.
  68. ^ "Container Attached Storage | SNIA". www.snia.org. Retrieved 2020-10-09.
  69. ^ "Cloud Native Application Checklist: Cloud Native Storage". www.replex.io. Retrieved 2020-10-09.
  70. ^ "Introducing Container Storage Interface (CSI) Alpha for Kubernetes". kubernetes.io. 10 January 2018.
  71. ^ "Container Storage Interface (CSI) for Kubernetes GA". kubernetes.io. 15 January 2019.
  72. ^ Marinescu, Dan C. (2018-01-01), Marinescu, Dan C. (ed.), "Chapter 8 - Cloud Hardware and Software", Cloud Computing (Second Edition), Morgan Kaufmann, pp. 281–319, doi:10.1016/b978-0-12-812810-7.00011-x, ISBN 978-0-12-812810-7, retrieved 2023-11-08
  73. ^ "Operating etcd clusters for Kubernetes". Kubernetes Documentation. Retrieved July 24, 2023.
  74. ^ "Objects In Kubernetes". Kubernetes Documentation. Retrieved July 24, 2023.
  75. ^ "API Conventions". GitHub. Kubernetes. Retrieved July 24, 2023.
  76. ^ "Owners and Dependents". Kubernetes Documentation. Retrieved July 24, 2023.
  77. ^ "Garbage Collection". Kubernetes Documentation. Retrieved July 24, 2023.
  78. ^ "Custom Resources". Kubernetes Documentation. Retrieved July 24, 2023.
  79. ^ "Cloud Native Landscape". Cloud Native Computing Foundation. Retrieved July 24, 2023.
  80. ^ "Controlling Access to the Kubernetes API". Kubernetes Documentation. Retrieved July 24, 2023.
  81. ^ "Remove the apiserver insecure port · Issue #91506 · kubernetes/kubernetes". GitHub.
  82. ^ "Authorization". Kubernetes Documentation. Retrieved July 24, 2023.
  83. ^ "Organizing Cluster Access Using kubeconfig Files". Kubernetes Documentation. Retrieved July 24, 2023.
  84. ^ "Authorization Overview". Kubernetes Documentation. Retrieved July 24, 2023.
  85. ^ "Command line tool (kubectl)". Kubernetes Documentation. Retrieved July 24, 2023.
  86. ^ "Client Libraries". Kubernetes Documentation. Retrieved July 24, 2023.
  87. ^ "Introduction - The Cluster API Book". cluster-api.sigs.k8s.io.
  88. ^ "The 7 Most Popular Kubernetes Distributions". Retrieved 2021-12-28.
  89. ^ MSV, Janakiram. "Why Kubernetes Developer Ecosystem Needs A PaaS". Forbes. Retrieved 2021-05-16.
  90. ^ "5 Cloud Native Trends to Watch out for in 2022". The New Stack. 2022-01-03. Retrieved 2022-02-03.
  91. ^ a b "Kubernetes Patch Releases". GitHub. 4 May 2022. Retrieved 2022-05-09.
  92. ^ "Kubernetes 1.1 Performance upgrades, improved tooling and a growing community". kubernetes.io. November 9, 2015.
  93. ^ "Kubernetes 1.2: Even more performance upgrades, plus easier application deployment and management". kubernetes.io. March 17, 2016.
  94. ^ "Kubernetes 1.3: Bridging Cloud Native and Enterprise Workloads". kubernetes.io. July 6, 2016.
  95. ^ "Kubernetes 1.4: Making it easy to run on Kubernetes anywhere". kubernetes.io. September 26, 2016.
  96. ^ "Kubernetes 1.5: Supporting Production Workloads". kubernetes.io. December 13, 2016.
  97. ^ "Kubernetes 1.6: Multi-user, Multi-workloads at Scale". kubernetes.io. March 28, 2017.
  98. ^ "Kubernetes 1.7: Security Hardening, Stateful Application Updates and Extensibility". kubernetes.io. June 30, 2017.
  99. ^ "Kubernetes 1.8: Security, Workloads and Feature Depth". kubernetes.io. September 29, 2017.
  100. ^ "Kubernetes 1.9: Apps Workloads GA and Expanded Ecosystem". kubernetes.io. December 15, 2017.
  101. ^ "Kubernetes 1.10: Stabilizing Storage, Security, and Networking". kubernetes.io. March 26, 2018.
  102. ^ "Kubernetes 1.11: In-Cluster Load Balancing and CoreDNS Plugin Graduate to General Availability". kubernetes.io. June 27, 2018.
  103. ^ "Kubernetes 1.12: Kubelet TLS Bootstrap and Azure Virtual Machine Scale Sets (VMSS) Move to General Availability". kubernetes.io. September 27, 2018.
  104. ^ "Kubernetes 1.13: Simplified Cluster Management with Kubeadm, Container Storage Interface (CSI), and CoreDNS as Default DNS are Now Generally Available". kubernetes.io. December 3, 2018.
  105. ^ "Kubernetes 1.14: Production-level support for Windows Nodes, Kubectl Updates, Persistent Local Volumes GA". kubernetes.io. March 25, 2019.
  106. ^ "Kubernetes 1.15: Extensibility and Continuous Improvement". kubernetes.io. June 19, 2019.
  107. ^ "Kubernetes 1.16: Custom Resources, Overhauled Metrics, and Volume Extensions". kubernetes.io. September 18, 2019.
  108. ^ "Kubernetes 1.17: Stability". kubernetes.io. December 9, 2019.
  109. ^ "Kubernetes 1.18: Fit & Finish". kubernetes.io. March 25, 2020.
  110. ^ "Kubernetes 1.19 Release Announcement". Kubernetes. 26 August 2020. Retrieved 2020-08-28.
  111. ^ "Kubernetes 1.19: Accentuate the Paw-sitive". Kubernetes. 2020-08-26. Retrieved 2024-01-13.
  112. ^ "Kubernetes 1.20: The Raddest Release". kubernetes.io. December 8, 2020.
  113. ^ "Kubernetes 1.21: Power to the Community". kubernetes.io. April 8, 2021.
  114. ^ "Kubernetes 1.22: Reaching New Peaks". kubernetes.io. August 4, 2021.
  115. ^ "Kubernetes 1.23: The Next Frontier". kubernetes.io. December 7, 2021.
  116. ^ "Kubernetes 1.24: Stargazer". kubernetes.io. May 3, 2022.
  117. ^ "Kubernetes v1.25: Combiner". kubernetes.io. August 23, 2022.
  118. ^ "Kubernetes v1.26: Electrifying". kubernetes.io. December 9, 2022.
  119. ^ "Kubernetes v1.27: Chill Vibes". kubernetes.io. April 11, 2023.
  120. ^ "Kubernetes v1.28: Planternetes". kubernetes.io. August 15, 2023.
  121. ^ "Kubernetes v1.29: Mandala". kubernetes.io. December 13, 2023.
  122. ^ "Kubernetes v1.30: Uwubernetes". kubernetes.io. April 17, 2024.
  123. ^ "Kubernetes v1.31: Elli". kubernetes.io. August 13, 2024.
edit
Retrieved from "https://en.wiki.x.io/w/index.php?title=Kubernetes&oldid=1252762000"