quyennv.com

Senior DevOps Engineer · Healthcare, Singapore

Pulumi: Architecture, How It Works, and Implementation

#pulumi#iac#infrastructure-as-code#devops#cloud

Pulumi is an Infrastructure as Code (IaC) platform that lets you define cloud and on-prem resources using general-purpose languages (TypeScript, JavaScript, Python, Go, C#, Java) or YAML. Unlike DSL-only tools, you get loops, conditionals, types, and your usual IDE. Under the hood, a language host runs your program to produce a desired state, and a deployment engine compares it to the current state and drives resource providers (AWS, Azure, Kubernetes, etc.) to create, update, or delete resources. This post covers Pulumi’s architecture, how it works, and how to implement a project.


Why Pulumi?

AspectBenefit
Real languagesUse TypeScript, Python, Go, etc. Reuse existing skills, IDE support, and testing.
Declarative outcomeYou declare desired state; Pulumi figures out create/update/delete (same idea as Terraform).
AbstractionLoops, functions, and components let you avoid copy-paste and build reusable modules.
Multi-cloudOne tool and one language for AWS, Azure, GCP, Kubernetes, and 150+ providers via the Pulumi Registry.
StateEngine stores state (local file or Pulumi Cloud / S3 / Azure Blob) for drift detection and safe updates.

Pulumi architecture

Pulumi has four main parts: the CLI, the language host, the deployment engine, and resource providers. Communication between engine, language host, and providers uses gRPC.

+----------------------------------- Pulumi CLI -----------------------------------+
|  pulumi up / preview / destroy / stack ...                                       |
+----------------------------------------+-----------------------------------------+
                                         |
         +-------------------------------+-------------------------------+
         |                               |                               |
         v                               v                               v
+----------------+              +----------------+              +----------------+
| Language Host  |   gRPC       |    Engine      |   gRPC       |   Resource     |
| (runs your     |<------------>| (orchestrates, |<------------>|   Providers    |
|  program)      |              |  state, diff)  |              | (AWS, Azure,   |
|                |              |                |              |  K8s, ...)     |
| - SDK          |              | - Desired vs   |              | - Create       |
| - Executor     |              |   current      |              | - Update       |
|   (pulumi-     |              | - Calls        |              | - Delete       |
|    language-*) |              |   providers    |              | - Diff, Check  |
+----------------+              +----------------+              +----------------+
         |                               |                               |
         |                               v                               v
         |                      +----------------+              +----------------+
         |                      | State backend  |              | Cloud / K8s /  |
         |                      | (file, Cloud,  |              | API (actual    |
         |                      |  S3, Azure)    |              |  resources)    |
         |                      +----------------+              +----------------+
         v
  Your program
  (index.ts, __main__.py, main.go, ...)

Core components

ComponentRole
CLIEntry point. Runs pulumi up, preview, destroy, stack, config, etc. Starts the language host and engine and passes commands to the engine.
Language hostRuns your Pulumi program and registers resources with the engine. Two parts: (1) Language SDK — library in your runtime (e.g. @pulumi/pulumi, pulumi for Python) that intercepts resource constructors and sends registration requests to the engine; (2) Language executor — binary (e.g. pulumi-language-nodejs, pulumi-language-python) that the CLI launches to execute the program and talk to the engine over gRPC.
Deployment engineOrchestrates deployment. Loads current state from the backend; receives desired state from the language host (resource registrations); builds a dependency graph; asks providers to Create, Update, or Delete resources; persists new state at the end. Runs as a process started by the CLI; communicates with language host and providers via gRPC.
Resource providersPlugins (gRPC servers) that implement the resource protocol: Check (validate inputs), Diff (desired vs current), Create, Update, Delete. Each provider (e.g. pulumi-resource-aws) knows how to talk to one cloud or API. The engine downloads and starts them as needed.
State backendWhere the engine stores the current state (resource IDs and properties). Options: local (file in project directory), Pulumi Cloud (SaaS), or self-managed (e.g. S3, Azure Blob, GCS) for locking and team use.

How Pulumi works

When you run pulumi up

  1. CLI reads the project (e.g. Pulumi.yaml) and the active stack (e.g. dev, prod). It starts the engine and the language host (e.g. Node for a TypeScript project).
  2. Language host runs your program (e.g. index.ts). Every time the program constructs a resource (e.g. new aws.s3.Bucket(...)), the SDK sends a RegisterResource request to the engine with the resource type, name, and inputs.
  3. Engine receives the stream of desired resources and builds a resource graph (with dependencies derived from references between resources). It loads the current state for this stack from the backend.
  4. Engine compares desired vs current: for each resource it decides Create, Update, Delete, or no-op. For Create/Update it calls the appropriate provider (e.g. AWS) via gRPC: Create, Update, or Delete. Providers call the real cloud APIs.
  5. Engine waits for all operations to complete (respecting dependencies), then writes the new state (resource urns, IDs, outputs) to the backend.
  6. CLI prints a summary (created, updated, deleted) and any outputs you exported.

Preview vs apply

  • pulumi preview (or pulumi up --preview): Engine computes the diff and shows what would be created/updated/deleted, but does not apply. No state change.
  • pulumi up: Performs the above flow and applies changes. State is updated after a successful run.

State and backends

  • State is a snapshot of: which resources exist, their provider-assigned IDs, and stored outputs. The engine uses it to compute the diff on the next run.
  • Backend can be local (state file in the project), Pulumi Cloud (hosted state, secrets, CI integration), or a self-hosted HTTP/S3/Azure/GCS backend. Remote backends support state locking so two pulumi up runs do not conflict.

Implementation: from zero to a running stack

1. Install Pulumi

  • macOS (Homebrew): brew install pulumi
  • Windows (winget): winget install Pulumi.Pulumi
  • Linux: Install the CLI from pulumi.com/docs/install or your package manager.

Confirm:

pulumi version

2. Configure cloud credentials

Pulumi does not store cloud credentials; it uses the same credentials as the cloud CLI or environment variables.

  • AWS: aws configure or AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (or IAM role if on EC2/ECS).
  • Azure: az login (or service principal env vars).
  • GCP: gcloud auth application-default login (or service account key).

3. Create a new project

From an empty directory:

pulumi new

Choose a template (e.g. aws-typescript). The CLI will create:

  • Pulumi.yaml — project name and runtime.
  • Pulumi.dev.yaml (or similar) — stack-specific config (created when you create a stack).
  • index.ts (or main.go, __main__.py) — your program.
  • package.json (for Node) or requirements.txt (for Python), etc.

Example Pulumi.yaml:

name: my-infra
runtime: nodejs
description: My Pulumi project

4. Project structure (TypeScript example)

my-infra/
  Pulumi.yaml           # Project metadata and runtime
  Pulumi.dev.yaml       # Stack config (e.g. dev)
  Pulumi.prod.yaml      # Stack config (e.g. prod)
  package.json
  tsconfig.json
  index.ts              # Main program

5. Write the program

Example: two S3 buckets and an export (TypeScript with AWS):

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket1 = new aws.s3.Bucket("app-bucket", {
    tags: { Environment: pulumi.getStack() },
});

const bucket2 = new aws.s3.Bucket("data-bucket");

export const bucket1Name = bucket1.id;
export const bucket2Name = bucket2.id;
  • Resource constructor (new aws.s3.Bucket(...)) registers the resource with the engine. The first argument is the logical name (unique in the program); the second is inputs (props).
  • References (e.g. bucket1.id) create implicit dependencies so the engine orders create/update correctly.
  • export exposes values as stack outputs (shown after pulumi up and available via pulumi stack output).

6. Stack and config

  • Stack = a target environment (e.g. dev, staging, prod). Each stack has its own state and config.
  • Create a stack: pulumi stack init prod
  • Switch stack: pulumi stack select dev
  • Config (stack-specific):
pulumi config set aws:region ap-southeast-1
pulumi config set myapp:environment production

In code (TypeScript):

const config = new pulumi.Config();
const env = config.get("environment") ?? "dev";
const region = config.get("aws:region");
  • Secrets: pulumi config set --secret dbPassword xyz — stored encrypted in Pulumi Cloud (or in backend-supported encryption). Read with config.requireSecret("dbPassword").

7. Preview and apply

# Install dependencies (Node)
npm install

# See what would change (no state change)
pulumi preview

# Apply changes (create/update/delete resources, update state)
pulumi up
  • pulumi up will show a diff and ask for confirmation (use --yes in CI to skip the prompt).
  • Outputs are printed at the end; query with pulumi stack output bucket1Name.

8. Destroy and backend

  • Destroy all resources for the current stack: pulumi destroy (then confirm).
  • Backend: By default, state is stored in Pulumi Cloud (you get a free account). To use a local file instead:
pulumi login file://.

State is then stored under the project directory. For teams, use Pulumi Cloud or a self-managed backend (S3, Azure Blob, GCS) for locking and history.


Summary table

TopicSummary
ArchitectureCLI → language host (runs program, SDK registers resources) + engine (diff, state, orchestration) + providers (gRPC, call cloud APIs). Communication via gRPC.
How it worksProgram runs → resources registered with engine → engine loads current state → computes diff → calls providers Create/Update/Delete → writes new state to backend.
ImplementationInstall CLI and cloud credentials → pulumi new → edit program (e.g. index.ts) → configure stack and config → pulumi preview / pulumi up / pulumi destroy. State in Pulumi Cloud or self-managed backend.

Pulumi gives you real programming languages for IaC with the same declarative, state-driven model as tools like Terraform: you declare desired state, and the engine and providers make the cloud match it. Use the Pulumi Registry and language docs to explore providers and APIs for your stack.

← All posts

Comments