Introduction

Managing Your Clusters (including public, private, hybrid, edge, etc) as easily as Visiting the Internet.

Clusternet (Cluster Internet) is an open source add-on that helps you manage thousands of millions of Kubernetes clusters as easily as visiting the Internet. No matter the clusters are running on public cloud, private cloud, hybrid cloud, or at the edge, Clusternet lets you manage/visit them all as if they were running locally. This also help eliminate the need to juggle different management tools for each cluster.

Clusternet can also help deploy and coordinate applications to multiple clusters from a single set of APIs in a hosting cluster.

Clusternet will help setup network tunnels in a configurable way, when your clusters are running in a VPC network, at the edge, or behind a firewall.

Clusternet also provides a Kubernetes-styled API, where you can continue using the Kubernetes way, such as KubeConfig, to visit a certain Managed Kubernetes cluster, or a Kubernetes service.

Clusternet is multiple platforms supported now, including linux/amd64, linux/arm64, linux/ppc64le, linux/s390x , linux/386 and linux/arm;

Core Features

• Kubernetes Multi-Cluster Management and Governance
  • managing Kubernetes clusters running in cloud providers, such as AWS, Google Cloud, Tencent Cloud, Alibaba Cloud, etc
  • managing on-premise Kubernetes clusters
  • managing any Certified Kubernetes Distributions, such as k3s
  • managing Kubernetes clusters running at the edge
  • automatically discovering and registering clusters created by cluster-api
  • parent cluster can also register itself as a child cluster to run workloads
  • managing Kubernetes upper than v1.17.x (Learn more about Kubernetes Version Skew)
  • visiting any managed clusters with dynamic RBAC rules (Learn more from this tuorial)
  • cluster auto-labelling based on Node Feature Discovery
• Application Coordinations
  • Scheduling Framework (in-tree plugins, out-of-tree plugins)
  • Cross-Cluster Scheduling
    • replication scheduling
    • static dividing scheduling by weight
    • dynamic dividing scheduling by capacity
      • cluster resource predictor framework for in-tree and out-of-tree implementations
      • various deployment topologies for cluster resource predictors
    • subgroup cluster scheduling
    • multi-cluster failover migration
  • Various Resource Types
    • Kubernetes native objects, such as Deployment, StatefulSet, etc
    • CRDs
    • helm charts, including OCI-based Helm charts
  • Resource interpretation with in-tree or out-of-tree controller
  • Setting Overrides
    • two-stage priority based override strategies
    • easy to rollback overrides
    • cross-cluster canary rollout
  • Multi-Cluster Services
    • multi-cluster services discovery with mcs-api
• CLI
  • providing a kubectl plugin, which can be installed with kubectl krew install clusternet
  • consistent user experience with kubectl
  • create/update/watch/delete multi-cluster resources
  • interacting with any child clusters the same as local cluster
• Client-go
  • easy to integrate via a client-go wrapper

Architecture

Clusternet is a lightweight addon that consists of four components, clusternet-agent, clusternet-scheduler, clusternet-controller-manager (added since v0.15.0) and clusternet-hub.

clusternet-agent is responsible for

• auto-registering current cluster to a parent cluster as a child cluster, which is also been called ManagedCluster;
• reporting heartbeats of current cluster, including Kubernetes version, running platform, healthz/readyz/livez status, etc;
• setting up a websocket connection that provides full-duplex communication channels over a single TCP connection to parent cluster;

clusternet-scheduler is responsible for

• scheduling resources/feeds to matched child clusters based on SchedulingStrategy;

clusternet-controller (added since v0.15.0) is responsible for

• approving cluster registration requests and creating dedicated resources, such as namespaces, serviceaccounts and RBAC rules, for each child cluster;
• coordinating and deploying applications to multiple clusters from a single set of APIs;

clusternet-hub is responsible for

• serving as an aggregated apiserver (AA), which is used to provide shadow APIs and serve as a websocket server that maintain multiple active websocket connections from child clusters;
• providing Kubernetes-styled API to redirect/proxy/upgrade requests to each child cluster;

Kubernetes Version Skew

Clusternet is compatible with multiple Kubernetes versions. For example, you could run clusternet-hub with Kubernetes v1.20.8, while the versions of child Kubernetes clusters could range from v1.18.x to v1.23.x.

For clusters running Kubernetes with version upper than v1.24.0, please upgrade Clusternet to no less than v0.13.0.

Note:

• ✓ Clusternet is compatible with this Kubernetes version.
• * Clusternet has no guarantees to support this Kubernetes version. More compatible tests will be needed.