Install on AWS
This page is the AWS variant of Install on Kubernetes. The Authwise operator deploys the same workloads on Amazon Elastic Kubernetes Service (EKS) as on any other cluster. What changes is where the database, the keys, and the uploaded assets live, and how pods get an AWS identity. Read the Kubernetes page first; the sections here link to it rather than repeat it.
An install on AWS runs its platform service in one of two ways:
- platform-hosted, deployed by the operator. Keys live in a keyset Secret you generate, assets live in an S3 bucket, and email goes out through Amazon SES. The service reaches S3 and SES through IAM Roles for Service Accounts (IRSA), so no AWS credential is stored in the cluster. This is the recommended starting point.
- platform-aws, deployed by you. AWS KMS wraps the keys and an S3 bucket
holds the assets, with the same IRSA identity. The operator has no
awsblock: it does not deploy platform-aws. You run it as a Deployment in the install’s namespace and point the install at it withspec.platform.enabled: falseandspec.config.platform.
| platform-hosted | platform-aws | |
|---|---|---|
| Deployed by | The operator | You |
| Issuer and secret keys | A keyset file in a Secret | An AWS KMS key |
| Uploaded assets | An S3 bucket | An S3 bucket |
| Pod identity | IRSA on the install’s ServiceAccount | IRSA on the service’s ServiceAccount |
| SES, configured in the install | SES, configured in the service’s own file |
Both run the identity server’s database on Amazon RDS for PostgreSQL.
Before you begin
Section titled “Before you begin”The Kubernetes prerequisites apply, with the database, object storage, and email items replaced by the following.
- An EKS cluster with an IAM OpenID Connect (OIDC) provider. IRSA needs it. If the cluster has none, create one as described in Create an IAM OIDC provider for your cluster.
- An RDS for PostgreSQL instance whose security group accepts
connections on port
5432from the cluster’s nodes. The workloads connect to it directly. - An SES sender identity, verified for the domain or address the install sends from, and an account that is out of the SES sandbox if the recipients are not all verified addresses.
- An ingress controller. This page uses ingress-nginx, which provisions
a load balancer through its
LoadBalancerService. Point the three DNS records at that load balancer. - Tools: the
awsCLI, authenticated to an account where you can administer RDS, S3, KMS, SES, and IAM, in addition tokubectl,kustomize,git,jq,curl, and Docker.
The platform-aws image is published on Authwise’s internal registry. Get the
image reference and a read credential for it from Authwise; the steps below
call it PLATFORM_AWS_IMAGE.
Create the AWS resources
Section titled “Create the AWS resources”The placeholders are ACCOUNT_ID, the account; REGION, the region such
as us-east-2; CLUSTER, the EKS cluster name; INSTANCE, the RDS
instance identifier; NAMESPACE, the Kubernetes namespace the install lives
in; and BUCKET, a globally unique bucket name.
-
Find the database endpoint and download the RDS certificate bundle for the region:
Terminal window aws rds describe-db-instances --db-instance-identifier INSTANCE \--query 'DBInstances[0].Endpoint.Address' --output textcurl -fsSLo rds-ca.pem https://truststore.pki.rds.amazonaws.com/REGION/REGION-bundle.pemThe endpoint is
DB_HOST. The instance’s master user holdsrds_superuser, which carriesCREATEDBandCREATEROLE, so it serves as the owner of both the identity server’s and the platform service’s databases. Splitting them is a hardening step, not a prerequisite. RDS server certificates name the endpoint, soverify-fullworks. -
Create the bucket, with public access blocked:
Terminal window aws s3api create-bucket --bucket BUCKET --region REGION \--create-bucket-configuration LocationConstraint=REGIONaws s3api put-public-access-block --bucket BUCKET \--public-access-block-configuration \BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=trueIn
us-east-1, omit--create-bucket-configuration. The platform service never creates a bucket; it probes every declared bucket at startup and fails if one is missing. -
platform-aws only. Create the key that wraps the issuer keys, and an alias to name it by:
Terminal window aws kms create-key --description "Authwise issuer keys" \--query KeyMetadata.KeyId --output textaws kms create-alias --alias-name alias/authwise-primary --target-key-id KEY_IDKEY_IDis the first command’s output. The install’sissuerKeys.keyNamecarriesalias/authwise-primary; the service accepts a key ID, a key ARN, or an alias there. -
Write the IAM policy the install’s pods need. Save it as
policy.json. Omit theKmsstatement for platform-hosted, and theSesstatement if the install does not send email.{"Version": "2012-10-17","Statement": [{"Sid": "Objects","Effect": "Allow","Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"],"Resource": "arn:aws:s3:::BUCKET/*"},{"Sid": "Bucket","Effect": "Allow","Action": ["s3:ListBucket"],"Resource": "arn:aws:s3:::BUCKET"},{"Sid": "Ses","Effect": "Allow","Action": ["ses:SendEmail", "ses:SendRawEmail"],"Resource": "*"},{"Sid": "Kms","Effect": "Allow","Action": ["kms:Encrypt", "kms:Decrypt", "kms:DescribeKey"],"Resource": "arn:aws:kms:REGION:ACCOUNT_ID:key/KEY_ID"}]}s3:ListBucketis not for listing. Without it, S3 answers a read of a missing object with403instead of404, and the service cannot tell “not found” from “forbidden”. -
Create the IAM role that the Kubernetes ServiceAccount assumes. Which ServiceAccount depends on the platform choice: platform-hosted runs as the install’s ServiceAccount,
authwise-identity-serverfor an install namedauthwise; platform-aws runs asauthwise-platform, created in a later step. Save the trust policy astrust.jsonwith the one you use:Terminal window OIDC_PROVIDER=$(aws eks describe-cluster --name CLUSTER \--query 'cluster.identity.oidc.issuer' --output text | sed 's|https://||')cat > trust.json <<EOF{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Principal": {"Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/${OIDC_PROVIDER}"},"Action": "sts:AssumeRoleWithWebIdentity","Condition": {"StringEquals": {"${OIDC_PROVIDER}:aud": "sts.amazonaws.com","${OIDC_PROVIDER}:sub": "system:serviceaccount:NAMESPACE:SERVICE_ACCOUNT"}}}]}EOFaws iam create-role --role-name authwise-install \--assume-role-policy-document file://trust.jsonaws iam put-role-policy --role-name authwise-install \--policy-name authwise-install --policy-document file://policy.jsonSERVICE_ACCOUNTisauthwise-identity-serverorauthwise-platform. The role’s ARN,arn:aws:iam::ACCOUNT_ID:role/authwise-install, goes on the ServiceAccount as theeks.amazonaws.com/role-arnannotation; EKS then injects a web identity token into every pod that runs as it, and the AWS SDK’s default credential chain picks it up with no configuration.
Install the operator
Section titled “Install the operator”Nothing differs on EKS. Follow Install the operator.
Prepare the namespace
Section titled “Prepare the namespace”Follow Prepare the namespace with these changes:
-
One database Secret serves both repository blocks, because the master user owns both databases:
Terminal window kubectl -n NAMESPACE create secret generic authwise-db \--from-literal=owner-password=DB_PASSWORD -
Store the RDS certificate bundle for the platform service:
Terminal window kubectl -n NAMESPACE create secret generic rds-ca \--from-file=ca.pem=rds-ca.pem -
For platform-hosted, generate the keyset as the Kubernetes page describes. Skip the
PersistentVolumeClaim: assets go to S3. -
For platform-aws, skip the keyset too.
Enrol the administrator’s password as on the Kubernetes page.
Run platform-aws
Section titled “Run platform-aws”Skip this section if the operator deploys platform-hosted.
The identity server reaches the platform service over mutual TLS: the service presents a certificate for its Service name and requires a client certificate from every caller. The operator issues that material itself for the components it deploys. For a service you deploy, you issue it, with cert-manager, from a CA you create for the purpose.
-
Create the ServiceAccount, annotated with the role from the previous section:
Terminal window kubectl -n NAMESPACE create serviceaccount authwise-platformkubectl -n NAMESPACE annotate serviceaccount authwise-platform \eks.amazonaws.com/role-arn=arn:aws:iam::ACCOUNT_ID:role/authwise-install -
Create the CA, the server certificate, and the client certificate the identity server presents:
Terminal window kubectl -n NAMESPACE apply -f - <<'EOF'apiVersion: cert-manager.io/v1kind: Issuermetadata:name: authwise-platform-selfsignedspec:selfSigned: {}---apiVersion: cert-manager.io/v1kind: Certificatemetadata:name: authwise-platform-caspec:isCA: truecommonName: authwise-platform-casecretName: authwise-platform-caprivateKey:algorithm: ECDSAsize: 256issuerRef:name: authwise-platform-selfsignedkind: Issuer---apiVersion: cert-manager.io/v1kind: Issuermetadata:name: authwise-platform-caspec:ca:secretName: authwise-platform-ca---apiVersion: cert-manager.io/v1kind: Certificatemetadata:name: authwise-platform-serverspec:secretName: authwise-platform-server-tlsdnsNames:- authwise-platform- authwise-platform.NAMESPACE.svc- authwise-platform.NAMESPACE.svc.cluster.localusages: [digital signature, key encipherment, server auth]issuerRef:name: authwise-platform-cakind: Issuer---apiVersion: cert-manager.io/v1kind: Certificatemetadata:name: authwise-platform-clientspec:secretName: authwise-platform-client-tlscommonName: authwise-identity-serverusages: [digital signature, key encipherment, client auth]issuerRef:name: authwise-platform-cakind: IssuerEOFcert-manager writes
ca.crt,tls.crt, andtls.keyinto both-tlsSecrets, which is the layout the service and the identity server read. -
Write the service’s configuration. It reads the file named by
CONFIG_PATHS; the server block is required, and every path points at a mount from the next step:Terminal window kubectl -n NAMESPACE create configmap authwise-platform-config \--from-literal=config.yaml='platform:server:host: 0.0.0.0port: 9989tlsCertPath: /etc/platform/tls/tls.crttlsKeyPath: /etc/platform/tls/tls.keytlsCaPath: /etc/platform/tls/ca.crt' -
Deploy the service and its Service:
Terminal window kubectl -n NAMESPACE apply -f - <<'EOF'apiVersion: apps/v1kind: Deploymentmetadata:name: authwise-platformspec:replicas: 1selector:matchLabels:app: authwise-platformtemplate:metadata:labels:app: authwise-platformspec:serviceAccountName: authwise-platformimagePullSecrets:- name: authwise-registrycontainers:- name: platformimage: PLATFORM_AWS_IMAGEenv:- name: CONFIG_PATHSvalue: /etc/platform/config/config.yaml- name: AWS_REGIONvalue: REGIONports:- name: grpccontainerPort: 9989readinessProbe:tcpSocket:port: grpcperiodSeconds: 10livenessProbe:tcpSocket:port: grpcperiodSeconds: 20volumeMounts:- name: configmountPath: /etc/platform/configreadOnly: true- name: tlsmountPath: /etc/platform/tlsreadOnly: truevolumes:- name: configconfigMap:name: authwise-platform-config- name: tlssecret:secretName: authwise-platform-server-tls---apiVersion: v1kind: Servicemetadata:name: authwise-platformspec:selector:app: authwise-platformports:- name: grpcport: 9989targetPort: grpcEOFAWS_REGIONis the region of the key and the bucket; the SDK takes it from the environment. The role and token come from the ServiceAccount annotation, so there is nothing else to mount. The service’s gRPC health check answers only on the mutual TLS listener, which a kubelet probe cannot present a certificate to, so readiness is a TCP probe.The service also publishes to an SQS queue when
DATAFLOW_QUEUE_URLis set. An Authwise install never publishes, so leave it unset.
To send email through platform-aws, add the same platform.messaging block
and top-level repository block to its configuration that the operator
renders for platform-hosted; messaging and the datastore are shared by every
platform implementation and read the same keys. With SES and no
credentials, the role’s ses:SendEmail permission is all it needs. The
database passwords arrive as environment variables named after the key,
REPOSITORY_GORM_OWNER_PASSWORD and REPOSITORY_GORM_APP_PASSWORD, never
in the file.
Write the install
Section titled “Write the install”The resource below is a complete install on RDS with the operator deploying
platform-hosted, assets in S3, and email through SES, all under the IRSA
role. Save it as authwise.yaml and replace every placeholder. It differs
from the Kubernetes sample
in the ServiceAccount annotation, the database connections, the object
store, and the email provider.
apiVersion: identity.authwise.com/v1alpha1kind: AuthwiseIdentityServermetadata: name: authwise namespace: NAMESPACEspec: preset: authwise-full
imagePullSecrets: - name: authwise-registry
# Every workload of the install runs as this ServiceAccount, so the role is # reachable from the engine, api, and console pods as well as the platform. # Keep its policy to the bucket, the sender identity, and the key. serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/authwise-install
bootstrap: admin: issuerHost: ADMIN_HOST username: ADMIN_EMAIL audienceURI: https://API_HOST credential: "PHC_STRING" machineCredential: {}
platform: hosted: key: keysetSecret: name: authwise-platform-keyset key: keyset.yaml bucketObject: provider: s3 # The declared bucket is the S3 bucket itself, and asset.bucketName # below names the same one. buckets: [BUCKET] s3: region: REGION # Virtual-hosted addressing, which is what AWS S3 serves. No # credentials: the SDK uses the role from the ServiceAccount. usePathStyle: false repository: mode: gorm tls: sslMode: verify-full caSecret: name: rds-ca key: ca.pem gorm: app: &platformdb host: DB_HOST port: 5432 name: authwise_platform username: DB_USER passwordSecret: name: authwise-db key: owner-password owner: *platformdb messaging: store: sql email: provider: ses from: no-reply@EXAMPLE_DOMAIN ses: region: REGION
console: clientId: c-01
networking: ingress: className: nginx issuer: hosts: [ADMIN_HOST] tlsSecretName: ADMIN_HOST_TLS_SECRET api: hosts: [API_HOST] tlsSecretName: API_HOST_TLS_SECRET console: hosts: [CONSOLE_HOST] tlsSecretName: CONSOLE_HOST_TLS_SECRET
config: baseURL: https://ADMIN_HOST serverSecret: name: authwise-server key: secret repository: mode: gorm gorm: app: &db host: DB_HOST port: 5432 name: authwise_identity username: DB_USER passwordSecret: name: authwise-db key: owner-password owner: *db issuerKeys: keyName: primary activeDays: 120 visibleAfterActiveDays: 60 asset: mode: ObjectStorage bucketName: BUCKETThe placeholders not listed here are the Kubernetes sample’s.
| Placeholder | Value |
|---|---|
DB_HOST | The RDS instance endpoint. |
DB_USER, DB_PASSWORD | The master user and its password, stored in the authwise-db Secret. |
EXAMPLE_DOMAIN | A domain or address verified in SES. provider: sns under sms, with a region, adds SMS the same way. |
The identity server’s own database connection has no TLS fields in the
resource. Its driver encrypts the connection when the server offers TLS,
which RDS does, but it does not verify the server certificate. Set the
instance’s rds.force_ssl parameter to 1 to refuse unencrypted
connections at the server.
Use platform-aws instead
Section titled “Use platform-aws instead”To point the install at the platform-aws Deployment from
Run platform-aws, drop the serviceAccount block, and
replace the whole platform block and the three config blocks shown
here. Everything else stays as above.
spec: platform: enabled: false
config: platform: host: authwise-platform.NAMESPACE.svc port: 9989 tls: secretName: authwise-platform-client-tls issuerKeys: keyName: alias/authwise-primary activeDays: 120 visibleAfterActiveDays: 60 asset: mode: ObjectStorage bucketName: BUCKETconfig.platform.host must be one of the server certificate’s dnsNames,
because the identity server verifies the name it dials. keyName is passed
to AWS KMS as the key to encrypt with; the ciphertext records which key
produced it, so decryption needs no name. The schema migration fails with
invalid key if the alias does not exist in AWS_REGION. bucketName is
the S3 bucket.
Apply and watch
Section titled “Apply and watch”Follow Apply and watch and
After the install. With
platform-aws, the REPLICAS count in the install’s status is one lower,
because the platform workload is not the operator’s.
Troubleshooting
Section titled “Troubleshooting”The Kubernetes troubleshooting table applies. These are the AWS additions.
| Symptom | Cause and fix |
|---|---|
Platform pod logs NoCredentialProviders or AccessDenied from S3 or KMS | The ServiceAccount is not annotated with the role, the role’s trust policy names a different namespace or ServiceAccount, or the cluster has no IAM OIDC provider. Check kubectl describe pod for the AWS_ROLE_ARN environment variable EKS injects. |
| platform-hosted fails at startup with a bucket probe error | buckets names a bucket that does not exist in s3.region, or the role lacks s3:ListBucket on it. |
S3 calls fail with PermanentRedirect or a DNS error | usePathStyle was left at its default against AWS S3. Set it false, and set s3.region to the bucket’s region. |
migrate fails with invalid key | The alias or key ID in issuerKeys.keyName does not exist in the region AWS_REGION names. |
Invitations fail with MessageRejected: Email address is not verified | The from address or its domain is not verified in SES, or the account is in the SES sandbox and the recipient is not verified. |
Engine or api pods log x509: certificate signed by unknown authority dialing the platform | The client Secret in config.platform.tls.secretName was not issued by the CA that signed the server certificate. |
Engine or api pods log certificate is valid for ... not ... dialing the platform | config.platform.host is not in the server certificate’s dnsNames. |
| platform-hosted crash-loops with a database TLS error | The bundle in rds-ca is for a different region than the instance, or the instance is not TLS-enabled. |