Install on Kubernetes

The Authwise operator runs an install as a set of Kubernetes workloads and keeps them in step. You describe the install in one custom resource, an AuthwiseIdentityServer; the operator deploys the identity server, the Admin API, the admin console, the hosted login pages, the platform service, and a cache, bootstraps the database, seeds the first administrator, and mints a credential for automation. This page is the operator’s surface. The CLI and the Terraform provider take over once the install is Ready.

On a managed cloud, the same operator runs the install with the cloud’s database, storage, and identity in place of the generic prerequisites: Install on Google Cloud and Install on AWS cover what changes.

Before you begin

You need the following before the first apply. The operator provisions none of them.

• A Kubernetes cluster and, once, cluster-admin on it. The operator install creates a custom resource definition, a ClusterRole, and a ClusterRoleBinding.
• cert-manager. The operator’s admission webhook serves TLS from a cert-manager Certificate, and each install uses cert-manager for the mutual TLS between its own components. Without it the operator never becomes ready.
• An ingress controller with an IngressClass, and DNS records for three hostnames pointed at it: the admin issuer host where people sign in, the Admin API host, and the console host. For TLS, either reference a kubernetes.io/tls Secret per hostname from the install, or terminate TLS at an edge in front of the ingress and leave the install on plain HTTP behind it.
• A PostgreSQL or YugabyteDB server you can create databases on, with two owner credentials: one for the identity server’s database, which needs CREATEDB, and one for the platform service’s database, which needs CREATEDB and CREATEROLE. Neither needs superuser. The platform service creates its own application role from the password you give it.
• Object storage for uploaded assets such as logos: an S3-compatible bucket that already exists, or a PersistentVolumeClaim. The platform service never creates a bucket.
• An SMTP relay the platform service can send through: host, port, and a sender address.
• Registry credentials. The component images are pulled from registry.authwise.com, whose projects are private. Get a read credential for it from Authwise; the operator image itself is public.
• Tools: kubectl, kustomize, git, jq, and Docker on the machine you prepare the install from. Docker runs two one-shot commands from the product images: enrolling the administrator’s password and generating the keyset.

Install the operator

The operator is deployed from its repository’s kustomize bases at a release tag. Pick a release from the operator’s releases page and use its version in both places below; never deploy a moving tag.

1. Clone the repository at the release:

   git clone https://git.authwise.com/authwise/authwise-operator.git
   cd authwise-operator
   git checkout vOPERATOR_VERSION

   OPERATOR_VERSION is the release, such as 0.22.0.

2. Apply the custom resource definitions, server-side. The definition is too large for client-side apply’s annotation:

   kustomize build config/crd | kubectl apply --server-side -f -

3. Deploy the operator at the same version:

   make deploy IMG=registry.authwise.com/authwise/authwise-operator:OPERATOR_VERSION

   make deploy renders config/default, which carries the definitions, RBAC, the manager, the webhook configurations, and the cert-manager Issuer and Certificate. It writes the image into config/default/kustomization.yaml, so the working tree is dirty afterwards; that is expected.

4. Verify the rollout and the version in the startup log:

   kubectl -n authwise-operator-system rollout status deploy/authwise-operator-manager
   kubectl -n authwise-operator-system logs deploy/authwise-operator-manager | head -5

   The log’s version field must match the tag. 0.0.0-dev means an unstamped build.

Everything lands in the authwise-operator-system namespace. If the manager stays in ContainerCreating, cert-manager is missing or its serving-cert has not been issued.

Prepare the namespace

Each install lives in its own namespace and references Secrets you create there. The names below are the ones the sample install uses; change them together if you change them at all.

1. Create the namespace and the registry credential:

   kubectl create namespace NAMESPACE
   kubectl -n NAMESPACE create secret docker-registry authwise-registry \
     --docker-server=registry.authwise.com \
     --docker-username=REGISTRY_USERNAME \
     --docker-password=REGISTRY_PASSWORD

   REGISTRY_USERNAME and REGISTRY_PASSWORD are the read credential for registry.authwise.com.

2. Create the server secret, 32 random bytes as hex. The identity server derives its session and encryption keys from it, so it is unique per install and must not change afterwards:

   kubectl -n NAMESPACE create secret generic authwise-server \
     --from-literal=secret="$(openssl rand -hex 32)"

3. Store the two database owner passwords:

   kubectl -n NAMESPACE create secret generic authwise-db \
     --from-literal=owner-password=IDENTITY_DB_PASSWORD
   kubectl -n NAMESPACE create secret generic authwise-platform-db \
     --from-literal=owner-password=PLATFORM_DB_PASSWORD

   The sample install uses the owner credential for both the owner and the application connection. Splitting a least-privilege application role out later is a hardening step, not a prerequisite.

4. Generate the platform keyset and store it. The keyset wraps the issuer signing keys, so it must contain a key with the name the install’s issuerKeys.keyName uses; the sample uses primary. The operator never generates key material, and losing the keyset loses everything encrypted under it, so keep a copy in your backups.

   docker login registry.authwise.com
   docker run --rm --entrypoint /usr/bin/keysetgen \
     -v "$PWD:/work" \
     registry.authwise.com/authwise/platform-hosted:PLATFORM_VERSION \
     -path /work/keyset.yaml -name primary
   kubectl -n NAMESPACE create secret generic authwise-platform-keyset \
     --from-file=keyset.yaml

   PLATFORM_VERSION is the platform service release the operator pins; the operator release notes name it. Running keysetgen again on the same file adds a key version and keeps the old ones, which is how rotation works.

5. If assets go on a volume rather than S3, create the claim. One replica of the platform service can use ReadWriteOnce; more need ReadWriteMany.

   kubectl -n NAMESPACE apply -f - <<'EOF'
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: authwise-platform-objects
   spec:
     accessModes: [ReadWriteOnce]
     resources:
       requests:
         storage: 10Gi
   EOF

Enrol the administrator’s password

The operator does not generate or hash the first administrator’s password. Choose one, hash it with the identity server image, and put the hash in the install. The output is a PHC string: it is safe to commit, because it lets a holder verify a guess, not recover the password.

printf '%s' "$PASSWORD" | docker run --rm -i --entrypoint /usr/bin/engine \
  registry.authwise.com/authwise/kit/profiles/authwise-full-engine:KIT_VERSION \
  enroll-password | jq -r .credential

KIT_VERSION is the identity server release the operator pins. Use -i and not -t, because a TTY breaks the pipe, and printf rather than echo so no newline becomes part of the password.

Set the password you intend to use before the first apply. The bootstrap runs once per database; editing the hash on a running install does nothing, and afterwards the console’s password-recovery flow is the only way to change it.

Write the install

The resource below is a complete install of the Authwise product on PostgreSQL, with assets on the claim from the previous section and email over SMTP. Save it as authwise.yaml and replace every placeholder.

apiVersion: identity.authwise.com/v1alpha1
kind: AuthwiseIdentityServer
metadata:
  name: authwise
  namespace: NAMESPACE
spec:
  # The product. Immutable: a preset is a binary and database boundary.
  preset: authwise-full

  imagePullSecrets:
    - name: authwise-registry

  bootstrap:
    admin:
      # The host people sign in on. The identity server resolves the tenant
      # from the request's Host header, so this must be what clients reach.
      issuerHost: ADMIN_HOST
      username: ADMIN_EMAIL
      audienceURI: https://API_HOST
      credential: "PHC_STRING"
    # Mint an OAuth 2.0 client-credentials secret for automation into the
    # Secret authwise-admin-credential.
    machineCredential: {}

  platform:
    hosted:
      key:
        keysetSecret:
          name: authwise-platform-keyset
          key: keyset.yaml
      bucketObject:
        buckets: [assets]
        volumeClaimName: authwise-platform-objects
      repository:
        mode: gorm
        tls:
          sslMode: require
        gorm:
          app: &platformdb
            host: DB_HOST
            port: 5432
            name: authwise_platform
            username: PLATFORM_DB_USER
            passwordSecret:
              name: authwise-platform-db
              key: owner-password
          owner: *platformdb
      messaging:
        store: sql
        email:
          provider: smtp
          from: no-reply@EXAMPLE_DOMAIN
          smtp:
            host: SMTP_HOST
            port: 587
            startTLS: true

  console:
    # The console's OAuth 2.0 client, seeded by the bootstrap.
    clientId: c-01

  networking:
    ingress:
      className: nginx
    issuer:
      hosts: [ADMIN_HOST]
      tlsSecretName: ADMIN_HOST_TLS_SECRET
    api:
      hosts: [API_HOST]
      tlsSecretName: API_HOST_TLS_SECRET
    console:
      hosts: [CONSOLE_HOST]
      tlsSecretName: CONSOLE_HOST_TLS_SECRET

  config:
    # Immutable.
    baseURL: https://ADMIN_HOST
    serverSecret:
      name: authwise-server
      key: secret
    repository:
      mode: gorm
      gorm:
        app: &db
          host: DB_HOST
          port: 5432
          name: authwise_identity
          username: IDENTITY_DB_USER
          passwordSecret:
            name: authwise-db
            key: owner-password
        owner: *db
    issuerKeys:
      keyName: primary
      activeDays: 120
      visibleAfterActiveDays: 60
    asset:
      mode: ObjectStorage
      bucketName: assets

Placeholder	Value
NAMESPACE	The namespace you prepared.
ADMIN_HOST	The admin issuer hostname, such as admin.example.com. People sign in here, and the hosted login and account pages are served here.
API_HOST	The Admin API hostname, such as api.example.com. The console, awctl, and Terraform call it.
CONSOLE_HOST	The admin console hostname, such as console.example.com.
ADMIN_EMAIL	The first administrator’s email address. It is the only account that can sign in until it creates others.
PHC_STRING	The output of enroll-password.
DB_HOST, IDENTITY_DB_USER, PLATFORM_DB_USER	The database server and the two owner roles. The database names are created by the install.
SMTP_HOST, EXAMPLE_DOMAIN	The relay and the sender domain. A relay that needs authentication takes username and a passwordSecret under smtp.
*_TLS_SECRET	kubernetes.io/tls Secrets for each hostname. Omit the tlsSecretName lines to serve plain HTTP behind an edge that terminates TLS.

What the sample leaves at its default: the login layouts, the account page, the console workload, and the cache are all deployed; the registry is registry.authwise.com/authwise; the component versions are the ones the operator release pins. Two things to know about the shape:

• YugabyteDB uses mode: yb with a yb block whose connections take hosts as a list and port 5433, in both repository blocks. A server that does not speak TLS needs sslMode: disable under platform.repository.tls.
• S3 replaces volumeClaimName with provider: s3 and an s3 block that names the endpoint, region, and a credentials Secret. The bucket must exist.

Apply and watch

kubectl apply -f authwise.yaml
kubectl -n NAMESPACE get authwiseidentityservers -w

The operator first runs two Jobs: setup, which creates the database, and migrate, which applies the schema and seeds the admin tenant, its issuer, and the administrator. The workloads are held back until migrate succeeds, then the machine credential is minted. READY turns True when every workload has its replicas:

NAME       PRESET          READY   REPLICAS   URL                        AGE
authwise   authwise-full   True    10         https://admin.example.com  6m

If it stays False, the conditions say why:

kubectl -n NAMESPACE describe authwiseidentityserver authwise

Degraded with a failed Job means setup or migrate failed; read the Job’s last pod with kubectl -n NAMESPACE describe pod -l job-name=JOB_NAME, where JOB_NAME is the name in the condition. A workload in ImagePullBackOff means the registry credential is missing or is for the wrong registry host.

After the install

Sign in to the console

Open https://CONSOLE_HOST and sign in with ADMIN_EMAIL and the password you enrolled. The install starts with one tenant, the admin tenant, which owns the console and the Admin API. Everything the console, awctl, and Terraform manage lives inside a tenant.

Read the machine credential

The operator minted an OAuth 2.0 client_credentials client in the admin tenant for automation and stored it in a Secret named after the install:

kubectl -n NAMESPACE get secret authwise-admin-credential \
  -o jsonpath='{.data.credential\.json}' | base64 -d

{
  "token_url": "https://admin.example.com/oauth/token",
  "client_id": "c-02",
  "client_secret": "cs-...",
  "audience": "a-01",
  "endpoint": "authwise-api.NAMESPACE.svc:8444"
}

audience is the admin audience’s AWID. It is an identifier, not a URL; the token grant rejects anything else, and this is the most common way a client-credentials request fails. endpoint is the Admin API’s in-cluster address, which suits automation running in the cluster.

Store the credential somewhere durable. It is minted once per install; deleting the Secret does not mint another, and the secret cannot be read back from the server, only rotated.

Reach the Admin API

The Admin API serves gRPC and its HTTP gateway on one port, 8444 on the Service authwise-api. awctl and the Terraform provider speak gRPC to it.

• From inside the cluster, use the endpoint from the credential file.

• From a workstation, forward the Service port and use a plaintext endpoint:

  kubectl -n NAMESPACE port-forward svc/authwise-api 8444:8444

  Then AWCTL_ENDPOINT=localhost:8444 AWCTL_INSECURE=true for awctl, or endpoint = "localhost:8444" with insecure = true for the provider.

• Through the ingress at API_HOST:443, if your ingress controller proxies HTTP/2 gRPC to the api backend. The install renders one Ingress for all of its hostnames, so a controller annotation set through networking.ingress.annotations applies to the login and console hostnames too. Verify it before relying on it.

What to keep

Item	Why
The platform keyset	Everything encrypted under it is lost without it. Rotation is additive; never drop a key name while ciphertext references it.
The machine credential	Minted once; not recoverable from the server.
The administrator’s password	The operator holds only the hash.
The server secret and database passwords	Referenced by the install, never readable from it.
The registry credential	Every image pull depends on it.

Upgrade

Upgrade the operator first, then let the installs follow it. Each operator release pins the component versions it was built and tested against; an install with no spec.version takes the operator’s pin, and one that sets it holds the identity server at that release.

1. Check out the new release and re-apply the definitions, which may have gained fields, then deploy:

   git checkout vOPERATOR_VERSION
   kustomize build config/crd | kubectl apply --server-side -f -
   make deploy IMG=registry.authwise.com/authwise/authwise-operator:OPERATOR_VERSION

2. Read the release notes. Each release states whether the identity server moves in place or needs a fresh database, and which component versions move. Roll the identity server with or before the console, never after.

3. Watch READY on each install. On an in-place move the old pods keep serving until the new migration succeeds.

Existing installs are untouched by the operator upgrade itself; the operator reconciles their workloads in place.

Troubleshooting

Symptom	Cause and fix
Manager pod stuck in ContainerCreating	cert-manager is missing or serving-cert is not issued.
Every write to an install is rejected while the operator is down	The webhook’s failure policy is Fail, deliberately. Restore the manager.
Workload pods in ImagePullBackOff	No imagePullSecrets, or a credential for a different registry host than spec.registry.
annotations: Too long applying the definitions	Use kubectl apply --server-side.
Every endpoint answers issuer not found for this url	bootstrap.admin.issuerHost is not the host clients reach.
migrate fails with invalid key: "primary" is not in the keyset	The keyset has no key named by issuerKeys.keyName. Generate it and re-create the keyset Secret.
Platform pod crash-loops with a TLS error	The database does not speak TLS; set platform.repository.tls.sslMode: disable.
Admission refuses the install	Contradictory routing, a component and its external reference both set, a console routed with no API host, or an edit to preset or config.baseURL, which are immutable. The message names the field.
StaticDataDrifted=True	A supplied static-data ConfigMap changed after the overlay was written. Delete the Secret named in status.bootstrap.adminSecret; the operator rewrites it and re-runs migrate.

Deleting an AuthwiseIdentityServer deletes its workloads. The database and the Secrets you created stay. To rebuild against an empty database, also delete the authwise-admin-credential Secret first, or the rebuild’s mint fails to store its result.

Workplace and Guard

Authwise Workplace is the same resource with preset: workplace. The engine, API, and console images change to the Workplace product; the workforce lockdown is compiled into them, not configured. The preset is immutable because it is a product boundary.

Authwise Guard is a second resource, AuthwiseGuardServer, running guard-control and its console. It issues no tokens: spec.auth.issuer names an identity install’s issuer, and spec.access.endpoint names that install’s Admin API Service, to which it delegates every permission decision. It needs a database and a registry credential, and nothing the operator writes.