Deploying Quilt via Ravion: a step-by-step tutorial

How to turn a high-friction, multi-step Quilt install into a form a customer fills out once — using a quilt-catalog Ravion module definition wired around the quiltdata/iac modules/quilt Terraform (proposed in quiltdata/iac#127).

The problem: onboarding is the hard part

Installing Quilt into a customer’s AWS account has never been the interesting part of the work — but it has always been the slow part. Our infrastructure is solid, open Terraform (the quiltdata/iac modules/quilt module wrapping a CloudFormation stack), and a competent cloud team can deploy it. But “competent cloud team, several careful hours, a dozen things to get exactly right” is precisely the tax that slows every new customer down:

  • Request and DNS-validate an ACM certificate — in the right account, covering the right hostnames.
  • Create the DNS records — the catalog host and the registry and s3-proxy hosts Quilt derives from it.
  • Wire the certificate ARN, the domain, and the network into the stack parameters.
  • Run terraform apply, watch a CloudFormation stack build for half an hour, and hope you got the parameters right.

This tutorial walks through wrapping modules/quilt (and modules/cnames) unmodified behind a typed, form-fillable Ravion module — so onboarding becomes “pick a domain” instead of “read the runbook.”

What Ravion is

Ravion is a control plane for infrastructure-as-code. It connects to your AWS account over OIDC and runs Terraform/OpenTofu on temporary EC2 runners inside your own account — your code, state, and secrets never leave it. Platform teams wrap Terraform into typed, form-fillable modules; developers (and agents) consume them without touching HCL.

The primitive this tutorial relies on: a module definition can declare a typed $ref:<module-type> input, and Ravion auto-chains it — resolving the referenced module instance’s outputs and feeding them in before running this module’s Terraform. That chaining is declared at the definition layer (a module.yaml-equivalent registered with Ravion), not something a generic stack gets for free.

What’s in examples/ravion/

Three files, from PR #127:

FileRole
module.yamlA sketch of the typed inputs and ref fields a Ravion module definition needs.
main.tfRoot config Ravion drives — calls modules/quilt and modules/cnames verbatim.
README.mdHow the pieces fit together, hostname/certificate coverage, cross-account composition.

No CloudFormation template is committed anywhere in this — main.tf fetches it at plan/apply time from a published per-release S3 artifact via template_url, per issue #126’s explicit non-goal.

Note: modules/quilt‘s template_file input requires a local path (it’s filemd5()‘d and uploaded to S3), so main.tf can’t pass template_url straight through as a URL string. Instead it fetches the template via data "http" + local_file before handing it to module "quilt".

Prerequisites this tutorial assumes

Some Ravion scaffolding has to exist before any of this runs. You need, in order:

  • an organization (exists once you sign up),
  • a project (ravion project create),
  • an environment within that project (ravion environment create), and — unless you want the runner on the account’s default VPC —
  • an execution environment bound to a specific AWS account/VPC/subnet/security group (ravion environment execution create).

This tutorial assumes a project quilt-iac and environment dev already exist; substitute your own IDs throughout.

Step 1: pin the source and check out the branch

git clone https://github.com/quiltdata/iac.git
cd iac
git checkout 53c2578e62b2c5d7bb5eb9f7424b7ab44726011e   # add-ravion-support-126

Step 2: connect the target AWS account(s)

One-time per AWS account Ravion will operate in. If your deploying account and your domain-owning account differ (the common case — see the callout at the end of this tutorial), connect both:

ravion aws account create --given-id customer-prod --name "Customer prod"
ravion aws account create --given-id customer-dns  --name "Customer DNS zone"

Each returns an awsact_… id (and a CloudFormation template URL/role to attach in that account to reach CONNECTED). If the account has no default VPC, or you’d rather pin the Terraform runner to a specific network, register an execution environment against it:

ravion environment execution create \
  --type AWS \
  --given-id customer-prod-ee \
  --name "Customer prod execution env" \
  --aws-account-id <awsact_… from above> \
  --region us-east-1 \
  --vpc-id vpc-… --subnet-id subnet-… --security-group-id sg-…

Step 3: register the quilt-catalog module definition

quilt-catalog is not a built-in Ravion type (ravion module definition list shows ~20 built-ins; quilt-catalog isn’t one of them), so it has to be registered once per organization before anyone can create an instance of it.

First, create the definition shell:

ravion module definition create \
  --name "Quilt Catalog" \
  --type quilt-catalog \
  --description "Deploy the Quilt data platform via modules/quilt, with auto-chained cert + DNS."

This returns a mdef_… id in DRAFT status. Then publish a version with the actual schema + wiring as --config (JSON, not YAML — despite the module.yaml name, the wire format at this layer is JSON):

ravion module version create \
  --module-definition-id <mdef_… from above> \
  --version "1.0.0" \
  --description "Initial release" \
  --config "$(cat quilt-catalog.json)"

quilt-catalog.json:

{
  "inputs": [
    {"id": "name", "type": "string", "label": "Stack name",
     "description": "Stack name (<=20 chars, lowercase alphanumeric + hyphens).",
     "required": true},
    {"id": "template_url", "type": "string", "label": "CloudFormation template URL",
     "description": "URL of the published Quilt CloudFormation template for the release you're deploying.",
     "required": true},
    {"id": "catalog_domain", "type": "string", "label": "Catalog domain",
     "description": "Public hostname for the Quilt catalog, e.g. quilt.customer.com. Registry and s3-proxy hostnames are derived from this value.",
     "required": true},
    {"id": "admin_email", "type": "string", "label": "Admin email",
     "description": "Initial admin account email address.", "required": true},
    {"id": "sizing", "type": "string", "label": "Sizing", "default": "medium",
     "values": [
       {"label": "Small", "value": "small"},
       {"label": "Medium", "value": "medium"},
       {"label": "Large", "value": "large"},
       {"label": "X-Large", "value": "xlarge"}
     ]},
    {"id": "internal", "type": "boolean", "label": "Internal (VPN-only) load balancer", "default": false},
    {"id": "create_new_vpc", "type": "boolean", "label": "Create new VPC", "default": true},
    {"id": "aws_account_id", "type": "string", "label": "AWS Account", "required": true,
     "values": "$values:ravion/aws_accounts"},
    {"id": "aws_region", "type": "string", "label": "Region", "required": true,
     "values": "$values:aws/regions"},
    {"id": "execution_environment_id", "type": "string", "label": "Execution environment",
     "values": "$values:ravion/execution_environments"},
    {"id": "certificate", "type": "$ref:rvn-acm-certificate", "label": "TLS certificate",
     "required": true,
     "mapped_inputs": [
       {"id": "certificate_arn", "label": "Certificate ARN", "type": "string",
        "default": "<< ref.stack.output.certificate_arn >>"}
     ]},
    {"id": "dns", "type": "$ref:rvn-route53", "label": "DNS hosted zone",
     "required": true,
     "mapped_inputs": [
       {"id": "zone_id", "label": "Zone ID", "type": "string",
        "default": "<< ref.stack.output.zone_id >>"}
     ]}
  ],
  "stack": {
    "type": "opentofu",
    "pipelines": {
      "defaults": {
        "variant": "standard",
        "input": {
          "repo": "https://github.com/quiltdata/iac.git",
          "branch": "add-ravion-support-126",
          "base_path": "examples/ravion",
          "aws_account_id": "<< module.input.aws_account_id >>",
          "aws_region": "<< module.input.aws_region >>",
          "execution_environment_id": "<< module.input.execution_environment_id || defaults.execution_environment_id >>",
          "tool": "opentofu",
          "terraform_variables": {
            "name": "<< module.input.name >>",
            "template_url": "<< module.input.template_url >>",
            "catalog_domain": "<< module.input.catalog_domain >>",
            "admin_email": "<< module.input.admin_email >>",
            "sizing": "<< module.input.sizing >>",
            "internal": "<< module.input.internal >>",
            "create_new_vpc": "<< module.input.create_new_vpc >>",
            "certificate_arn": "<< module.input.certificate_arn >>",
            "zone_id": "<< module.input.zone_id >>"
          }
        }
      },
      "change": {"pipeline_id": "<< defaults.change_pipeline_id >>"},
      "destroy": {"pipeline_id": "<< defaults.destroy_pipeline_id >>"}
    }
  }
}

Two things worth knowing about this shape:

  • A referenced module’s output can only be pulled in through mapped_inputs on the ref field itself (<< ref.stack.output.x >> there), then re-referenced elsewhere as a normal << module.input.x >>. Using << ref.stack.output.x >> directly inside terraform_variables is rejected outright.
  • mapped_inputs entries require id, label, and type — a value key (as opposed to default) is rejected with field is not allowed.

Once published, confirm it rendered correctly:

ravion module schema quilt-catalog

Step 4: provision the certificate and dns dependencies

Create these as their own Ravion module instances before creating the Quilt module — in whichever connected account owns the domain. Input schemas are available via ravion module schema rvn-acm-certificate / rvn-route53:

ravion module create \
  --environment-id <env_… for the domain-owning environment> \
  --given-id quilt-dns --name "Quilt DNS zone" \
  --type rvn-route53 \
  --input '{
    "aws_account_id": "<awsact_… for customer-dns>",
    "aws_region": "us-east-1",
    "zone_creation_enabled": false,
    "zone_id": "<existing Route53 hosted zone ID, e.g. Z1234567890ABC>"
  }' \
  --initial-stack-run APPLY
ravion module create \
  --environment-id <env_… for the domain-owning environment> \
  --given-id quilt-cert --name "Quilt ACM certificate" \
  --type rvn-acm-certificate \
  --input '{
    "aws_account_id": "<awsact_… for customer-prod, i.e. the ALB account>",
    "aws_region": "us-east-1",
    "domains": ["quilt.customer.com", "quilt-registry.customer.com", "quilt-s3-proxy.customer.com"],
    "route53_validation_records_creation_enabled": true,
    "route53_zone_id": "<same Route53 hosted zone ID as above>",
    "certificate_validation_wait_enabled": true
  }' \
  --initial-stack-run APPLY

Note domains is a list, and it must cover all three Quilt hostnames — catalog, -registry, and -s3-proxy — not just the catalog host (see the DNS/TLS coverage callout below for why). Ravion does not derive these three hostnames for you across module boundaries — a $ref:rvn-acm-certificate input only consumes whatever domains list the referenced instance was created with, so compute the three hostnames yourself before this step (the same derivation main.tf uses internally: <sub>-registry.<rest> / <sub>-s3-proxy.<rest>).

Each module create --json call returns an "id": "minst_…" — capture both; they’re what Step 5 references.

Step 5: create the quilt-catalog module instance

ravion module create \
  --environment-id <env_… for the deploying environment> \
  --given-id quilt --name "Customer Quilt deployment" \
  --type quilt-catalog \
  --input '{
    "name":            "customer",
    "template_url":    "https://quilt-releases.s3.amazonaws.com/1.8.0/quilt-template.yaml",
    "catalog_domain":  "quilt.customer.com",
    "admin_email":     "admin@customer.com",
    "sizing":          "medium",
    "aws_account_id":  "<awsact_… for customer-prod>",
    "aws_region":      "us-east-1",
    "certificate":     "<minst_… from quilt-cert above>",
    "dns":             "<minst_… from quilt-dns above>"
  }' \
  --initial-stack-run APPLY

A few things to note about this call:

  • There is no --account flag on module create — --environment-id and --name are the actual required flags.
  • certificate / dns are bare module-instance-id strings ("minst_…"), not the given-id, and not a {"moduleGivenIdRef": …} object — that object form only appears in the config-as-code / rendered schema notation, not the CLI --input payload.
  • quilt-catalog must exist as a registered module definition (Step 3) before this call can succeed — it is not a built-in type.

Ravion resolves the certificate/dns refs’ stack outputs into certificate_arn/zone_id, plans the stack (Quilt + modules/cnames), and applies it on a runner inside the deploying account — even when the certificate and DNS live in a different one.

Cross-account: the common case

Very few customers are single-account: the domain usually lives in a central networking/shared-services account, the workload lives elsewhere. That’s exactly the split modeled above if customer-dns and customer-prod are different connected accounts — certificate/dns refs resolve across Ravion-connected accounts the same way. This is the norm, not an edge case.

DNS/TLS: a cert covering one hostname is not enough

Quilt’s public hostname contract is the catalog host plus two derived siblings — <sub>-registry.<rest> and <sub>-s3-proxy.<rest> — all pointing at the same load balancer. A certificate that only covers the catalog host (easy mistake — it’s the only hostname the customer actually typed in) will deploy a stack whose registry and s3-proxy endpoints have no valid TLS. If this happens, fixing it requires republishing the module definition with the certificate ref’s immutability relaxed, then ravion module update --module-version-id <new-version> --input '{"certificate": "<new minst_…>"}' --autoapprove to swap the cert in place without recreating the stack. Get the domains list right in Step 4 and this doesn’t come up.

Step 6: verify

Once APPLY completes, the stack’s Terraform outputs are exposed automatically (there is no outputs: key to declare — module outputs are surfaced as stack.output.*) and give you everything needed to confirm the deploy:

  • quilt_url — https://<catalog_domain>, should respond HTTP 200 with valid TLS.
  • load_balancer_dns — the ALB DNS name modules/cnames pointed the three hostnames at.
  • admin_password / db_password — sensitive, retrieve via Ravion’s secrets access rather than plan/apply logs.
ravion module get <minst_… for quilt> --json   # stack.output.quilt_url, etc.

Step 7: day-2 updates

Changing a typed input (sizing, catalog_domain, admin_email, …) and re-running the module re-plans and re-applies the same module "quilt" / module "cnames" resources in place — no destroy/recreate:

ravion module update <minst_… for quilt> \
  --input '{"admin_email": "new@customer.com"}' \
  --autoapprove
Plan: 0 to add, 1 to change, 0 to destroy
  ~ module.quilt.aws_cloudformation_stack.stack will be updated in-place
      ~ "AdminEmail" = "old@quilt.bio" -> "new@quilt.bio"

Source: examples/ravion/ on add-ravion-support-126 @ 53c2578, PR quiltdata/iac#127, closing quiltdata/iac#126.

Leave a comment

Blog at WordPress.com.

Up ↑