Cloud SDKs and Kubernetes client-go

Most production Go code in the cloud world spends a lot of time calling cloud APIs and the Kubernetes API. This lesson surveys the SDKs, the authentication models, and the controller pattern that drives every operator in the ecosystem.

AWS SDK for Go v2

The v2 SDK (GA since 2021) is what new projects use. It is context-aware, paginated APIs are first-class, and credential resolution is unified.

import (
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

ctx := context.Background()
cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("eu-west-1"))
if err != nil { return err }

client := s3.NewFromConfig(cfg)
out, err := client.ListBuckets(ctx, &s3.ListBucketsInput{})
if err != nil { return err }
for _, b := range out.Buckets {
    fmt.Println(*b.Name)
}

The credential chain

config.LoadDefaultConfig tries credentials in order:

AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env vars
Shared config: ~/.aws/credentials, ~/.aws/config
IMDS (EC2 instance role)
ECS container credentials
IRSA / EKS Pod Identity (the modern Kubernetes path)

You should rarely pass static credentials. In Kubernetes use IRSA or EKS Pod Identity; in Lambda, the execution role; on EC2, an instance profile.

Pagination

paginator := s3.NewListObjectsV2Paginator(client, &s3.ListObjectsV2Input{
    Bucket: aws.String("my-bucket"),
})
for paginator.HasMorePages() {
    page, err := paginator.NextPage(ctx)
    if err != nil { return err }
    for _, obj := range page.Contents {
        fmt.Println(*obj.Key)
    }
}

Always use the paginator helpers; do not roll your own token loop.

Google Cloud Go SDK

Google has two generations: the older cloud.google.com/go/* hand-written clients and the auto-generated cloud.google.com/go/... v2 clients (newer, mostly via gRPC). For new code prefer the v2 clients.

import "cloud.google.com/go/storage"

ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil { return err }
defer client.Close()

bucket := client.Bucket("my-bucket")
it := bucket.Objects(ctx, nil)
for {
    attrs, err := it.Next()
    if errors.Is(err, iterator.Done) { break }
    if err != nil { return err }
    fmt.Println(attrs.Name)
}

Authentication: GOOGLE_APPLICATION_CREDENTIALS pointing at a service account JSON, or Workload Identity (the Kubernetes path), or your gcloud user creds for local dev.

Azure SDK for Go

import (
    "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
    "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
)

cred, err := azidentity.NewDefaultAzureCredential(nil)
if err != nil { return err }

client, err := azblob.NewClient("https://acct.blob.core.windows.net/", cred, nil)
if err != nil { return err }

pager := client.NewListContainersPager(nil)
for pager.More() {
    page, err := pager.NextPage(ctx)
    if err != nil { return err }
    for _, c := range page.ContainerItems {
        fmt.Println(*c.Name)
    }
}

The credential chain (DefaultAzureCredential) tries environment variables, Managed Identity, Azure CLI, etc. — same pattern as AWS and GCP.

The Kubernetes API: client-go

client-go is the official Go client for the Kubernetes API. It is what kubectl, every controller, and every operator uses under the hood.

import (
    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/rest"
    "k8s.io/client-go/tools/clientcmd"
    "k8s.io/client-go/util/homedir"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// In-cluster (when running inside a Pod with a ServiceAccount)
config, err := rest.InClusterConfig()
if err != nil {
    // Fall back to kubeconfig for local dev
    kubeconfig := filepath.Join(homedir.HomeDir(), ".kube", "config")
    config, err = clientcmd.BuildConfigFromFlags("", kubeconfig)
    if err != nil { return err }
}

clientset, err := kubernetes.NewForConfig(config)
if err != nil { return err }

pods, err := clientset.CoreV1().Pods("default").List(ctx, metav1.ListOptions{})
if err != nil { return err }
for _, p := range pods.Items {
    fmt.Println(p.Namespace, p.Name, p.Status.Phase)
}

Watching for Changes

Polling is wrong. The Kubernetes API streams changes via Watch. client-go provides informers that handle the watch loop, cache, and resync logic for you:

factory := informers.NewSharedInformerFactory(clientset, time.Minute*10)
podInformer := factory.Core().V1().Pods().Informer()

podInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
    AddFunc:    func(obj interface{}) { /* ... */ },
    UpdateFunc: func(old, new interface{}) { /* ... */ },
    DeleteFunc: func(obj interface{}) { /* ... */ },
})

stop := make(chan struct{})
defer close(stop)
factory.Start(stop)
factory.WaitForCacheSync(stop)
<-stop

Informers are the foundation of every Kubernetes controller. They cache state in-memory; reads against the cache are local and free.

controller-runtime: The Operator Abstraction

Writing an operator with raw client-go is a lot of boilerplate. controller-runtime wraps it in a much smaller API and is the foundation of kubebuilder and operator-sdk.

The core abstraction is the Reconciler:

type DatabaseReconciler struct {
    client.Client
    Scheme *runtime.Scheme
}

func (r *DatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    // 1. Fetch the requested object
    db := &platformv1.Database{}
    if err := r.Get(ctx, req.NamespacedName, db); err != nil {
        return ctrl.Result{}, client.IgnoreNotFound(err)
    }

    // 2. Reconcile actual state toward desired state
    if err := r.ensureSecret(ctx, db); err != nil {
        return ctrl.Result{}, err
    }
    if err := r.ensureRDS(ctx, db); err != nil {
        return ctrl.Result{}, err
    }

    // 3. Update status, request requeue if needed
    db.Status.Phase = "Ready"
    if err := r.Status().Update(ctx, db); err != nil {
        return ctrl.Result{}, err
    }
    return ctrl.Result{RequeueAfter: time.Minute * 5}, nil
}

func (r *DatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&platformv1.Database{}).
        Complete(r)
}

The reconciler runs whenever the watched resource changes (informer event), and re-runs on a periodic resync. Reconcile must be idempotent — it can run many times and should converge to the same state.

Scaffolding an Operator

# kubebuilder
kubebuilder init --domain acme.io --repo github.com/acme/db-operator
kubebuilder create api --group platform --version v1 --kind Database

# operator-sdk
operator-sdk init --domain acme.io --repo github.com/acme/db-operator
operator-sdk create api --group platform --version v1 --kind Database --resource --controller

Either tool generates the CRD, the API types, the controller skeleton, the RBAC manifests, the Dockerfile, and the deployment YAML.

Where to Go from Here

Build something small. A CLI that talks to S3. A controller that watches a CRD and writes to a database. The shape of the language doesn't fully click until you ship.
Read the source of one CNCF project. Cluster API, Argo CD, and cert-manager are excellent reading.
Pick a cert. CKAD if you want to validate Kubernetes development; AWS Developer Associate if your work is cloud-heavy.
Contribute upstream. Most CNCF projects have "good first issue" labels and welcoming maintainers.

You now have enough Go to be productive in any cloud-native codebase. The language's small surface area means the gap between "can read it" and "can contribute" is smaller than in most ecosystems — invest the next month writing real code and you will be there.