AWS Step Functions vs Apache Airflow 2026 Comparison

July 28, 2026

23 min read

Every team running production data pipelines or multi-step application logic on AWS eventually asks the same question: build it as a state machine in AWS Step Functions, or run it as a DAG in Apache Airflow? The two tools solve overlapping problems but come from opposite philosophies. Step Functions is a proprietary, fully serverless orchestrator that AWS bills by the state transition, with a true $0 floor when nothing is running. Airflow is free, open-source, Python-native software that had grown to more than 43,800 GitHub stars and over 3,600 contributors by August 2026, per Astronomer-sourced figures reported by Refonte Learning, and it still requires you to run a scheduler and metadata database around the clock, whether managed through Amazon MWAA, Google Cloud Composer, or self-hosted on EC2 or Kubernetes.

This comparison works through the pricing models, execution limits, architecture, and 2026 feature set of both tools using figures pulled directly from AWS’s and Apache’s own documentation, plus independent pricing analyses. By the end, you’ll know exactly which one fits an AWS-native serverless app, which one fits a data engineering team’s nightly ETL runs, and where the two genuinely overlap.

Google · Preferred Sources

Don't miss new tech stories on Google

Add Tech Insider once in the Google app and our stories appear in your news suggestions.

Add Now

What Is AWS Step Functions?

AWS Step Functions is a fully managed, serverless orchestration service that coordinates AWS Lambda functions and other AWS services into visual workflows called state machines. You define the workflow using Amazon States Language (ASL), a JSON-based format that newer authoring tools also let you write in YAML, and Step Functions handles execution, retries, error handling, and state tracking without you provisioning a single server.

Workflows are built from states: Task states invoke Lambda functions or other AWS services, Choice states branch based on conditions, Parallel states run branches concurrently, and Map states iterate over collections. According to AWS’s own documentation, Step Functions supports two workflow types with meaningfully different performance and pricing characteristics: Standard and Express. The service integrates directly with what AWS describes as thousands of API actions across its service catalog, using three integration patterns: Request Response, Run a Job (the .sync pattern for waiting on a job to complete), and Wait for a Callback with a task token (the .waitForTaskToken pattern for human-in-the-loop approvals). That catalog keeps growing: AWS added 28 new service integrations for Step Functions on March 26, 2026, and followed up on June 3, 2026 with seven more agent-related integrations, including the AWS DevOps Agent Service, per AWS’s own service integration documentation.

Because Step Functions is a managed AWS service rather than open-source software, there’s no version number to track and no server to patch. AWS also ships a visual canvas called Workflow Studio that lets you drag and drop states into a working state machine, which lowers the barrier for teams without deep infrastructure-as-code experience. AWS pushed that managed-service model further on June 3, 2026, with an AgentCore-powered agentic reasoning step that plugs directly into state machines at no extra Step Functions charge beyond standard pricing, initially available in four regions (us-east-1, us-west-2, eu-central-1, and ap-southeast-2) and confirmed again in AWS’s Q2 2026 serverless roundup published July 20, 2026. That AI Agent Workflow capability ships with its own usage allowance, too: a Markaicode pricing analysis dated March 2025 notes that Step Functions bundles 10 GB of free data transfer per month for AI Agent Workflow traffic before standard data-transfer rates apply.

What Is Apache Airflow?

Apache Airflow is an open-source workflow orchestration platform originally built at Airbnb and now maintained under the Apache Software Foundation, released under the Apache License 2.0. Workflows are authored as Directed Acyclic Graphs, or DAGs, written in Python, which gives engineers the full power of a general-purpose programming language to define dependencies, branching logic, and dynamic task generation rather than a fixed workflow-description syntax.

Airflow’s GitHub repository had climbed past 43,800 stars and more than 3,600 contributors by August 2026, according to figures Refonte Learning attributed to Astronomer, including over 300 contributors to the Airflow 3.0 rewrite alone, reflecting one of the largest communities in the data engineering tooling space. Astronomer put a real-world number on that reach too, reporting in February 2025 that more than 77,000 organizations were already running Airflow as of November 2024. That community shows up in survey data too: Apache Airflow’s own January 2026 survey drew exactly 5,818 responses from practitioners across 122 countries, while a separate 2025 Community Survey, published in April 2026, gathered nearly 6,000 practitioners answering 168 questions. The project ships operators and provider packages for essentially every major cloud and database platform, so a single Airflow deployment can orchestrate jobs across AWS, Azure, GCP, Snowflake, and on-premises systems from one control plane, something Step Functions cannot do since it’s scoped to AWS.

Airflow is free to download and run, but it is not free to operate. Someone has to run the scheduler, the metadata database, the webserver, and the workers, whether that’s your own infrastructure team, Amazon MWAA, Google Cloud Composer, or the commercial Astro platform from Astronomer, the company founded by core Airflow contributors and backed by roughly $378 million across eight funding rounds as of May 2025, including a $93 million Series D that closed that same month. Astronomer’s own State of Airflow 2026 report, published in January 2026, found that 48% of Astro customers were already running Airflow 3, a figure that climbed to 60% among its largest enterprise accounts. That operational overhead is the single biggest philosophical difference between Airflow and Step Functions, and it shapes almost every other point in this comparison.

Architecture: Serverless State Machines vs Python-Native DAGs

Step Functions’ architecture is intentionally narrow. A state machine is a declarative description of states and transitions. AWS’s control plane executes it, tracks state, and bills per transition. There’s no persistent scheduler process to manage and no metadata database to size or back up, because AWS owns that layer entirely. The tradeoff is that all business logic beyond simple conditionals has to live in the services you’re calling, typically Lambda functions, which means a complex workflow can end up spread across dozens of small functions.

Airflow’s architecture, especially since the version 3.0 rewrite, is a service-oriented system: a scheduler that parses DAGs and queues tasks, an API server that mediates all access instead of giving workers direct database connections, one or more executors (Celery, Kubernetes, or the new Edge Executor) that actually run tasks, and a metadata database (typically PostgreSQL) that tracks every DAG run and task instance. Because DAGs are Python, a single task can contain arbitrarily complex logic without needing to be broken into separate managed services. That flexibility is also Airflow’s biggest operational cost: every one of those components needs to be sized, monitored, and kept available.

Where Kubernetes Fits

Teams running Airflow at scale frequently deploy it on Kubernetes using the KubernetesExecutor, which launches each task in its own pod for isolation and elastic scaling. If your organization already runs workloads on Amazon EKS, adding Airflow as another workload is a natural fit, and autoscalers like the ones compared in our Karpenter vs Cluster Autoscaler vs KEDA breakdown apply directly to sizing Airflow’s worker fleet. Step Functions has no equivalent consideration, since AWS runs the execution layer for you.

Authoring Experience in Practice

Workflow Studio’s drag-and-drop canvas means a Step Functions state machine can be assembled by someone who has never written a line of ASL by hand, then exported to JSON for version control once the logic is settled. Airflow’s 3.0 UI, rebuilt on React and FastAPI, is not an authoring tool in the same sense, it’s a monitoring and debugging surface for DAGs that are still written first as Python code in a repository. That split matters operationally: Step Functions changes can originate in a visual tool and get committed afterward, while Airflow changes always start in code and get reviewed through a normal pull-request workflow before they ever reach the UI.

AWS Step Functions vs Apache Airflow: Full Specs Comparison

The table below lines up the two platforms across the specs that actually drive a build-vs-adopt decision, sourced from AWS’s Step Functions documentation and Apache’s Airflow release notes and GitHub repository.

Spec	AWS Step Functions	Apache Airflow
License / cost model	Proprietary AWS service, pay-per-use	Apache License 2.0, free and open source
Authoring language	Amazon States Language (JSON, newer YAML support)	Python
Hosting	Fully managed by AWS only	Self-hosted, Amazon MWAA, Google Cloud Composer, or Astronomer
Cloud portability	AWS only	Any cloud or on-prem, via provider packages
Max workflow duration	Up to 1 year (Standard)	No hard cap, governed by scheduling and SLAs you define
Execution rate ceiling	2,000/sec (Standard), 100,000/sec (Express)	Scales with worker/executor capacity you provision
State transition rate	4,000/sec (Standard), “nearly unlimited” (Express)	Not applicable (task-based, not transition-based)
Execution semantics	Exactly-once (Standard), at-least-once or at-most-once (Express)	Configurable retries per task, executor-dependent
Pricing unit	Per state transition (Standard) or per GB-second (Express)	Free software, you pay only for compute/managed-service hosting
Free tier	4,000 state transitions/month	Entire codebase is free, infra costs still apply
GitHub stars	Not applicable (closed source)	46,300
Latest major version	Continuously updated by AWS, no version number	3.3.0 (Airflow 3.0 shipped April 22, 2025)
Native integrations	Thousands of API actions across AWS services	Provider packages for AWS, Azure, GCP, Snowflake, and more
Visual authoring	Workflow Studio drag-and-drop designer	React and FastAPI-based UI (rebuilt in Airflow 3.0)
Human-in-the-loop support	Native via `.waitForTaskToken`	Possible via sensors and custom operators, not native

Two rows are worth sitting with. First, the execution-rate gap between Step Functions’ own Standard and Express modes: Express handles 50 times the executions per second that Standard does, because AWS trades exactly-once guarantees for raw throughput. Second, Airflow has no equivalent ceiling published anywhere, because its scaling limit is whatever you provision, not a number AWS or Apache commits to in writing.

Pricing Breakdown: Step Functions, Self-Hosted Airflow, MWAA, and Cloud Composer

Pricing is where the two tools diverge most sharply, and it’s the part buyers get wrong most often because the units aren’t comparable on the surface. Step Functions Standard Workflows cost $0.025 per 1,000 state transitions, or $25 per million, confirmed directly on AWS’s Step Functions pricing page and independently verified by Markaicode’s pricing analysis as of March 2025, with the first 4,000 transitions each month still free as of August 2026, per AWS’s own Step Functions pricing documentation. A five-state workflow run 100,000 times in a month would rack up 500,000 transitions, or roughly $12.50. Express Workflows bill differently: instead of counting transitions, AWS charges by execution count plus duration and memory consumed, at roughly $0.00001667 per GB-second, which is why Express is dramatically cheaper for short, high-volume jobs like IoT event processing.

Airflow itself has no license fee, but somebody pays for the infrastructure that keeps its scheduler and metadata database running 24 hours a day. Amazon MWAA, AWS’s managed Airflow offering, prices this directly: a Small MWAA environment costs $0.49 per hour, which works out to roughly $357 a month just for the base environment, and a Large environment runs $0.99 per hour, around $723 a month, per AWS’s MWAA pricing page. On top of that base cost, MWAA bills additional workers separately (roughly $0.055/hour for a small worker, $0.22/hour for a large one) and metadata storage at $0.10 per GB-month. Critically, that base environment cost accrues whether you run one DAG a month or ten thousand. Version support has kept pace with the open-source project, too: AWS added MWAA support for Apache Airflow 2.11 beginning in January 2026, then followed with Apache Airflow 3.2 support in May 2026, per AWS’s own MWAA documentation.

Google Cloud Composer, GCP’s managed Airflow service, moved away from fixed Small/Medium/Large environment tiers in Composer 3 to a consumption-based model built around what Google calls Composer Compute Units, which meter actual vCPU, memory, and storage usage rather than a flat environment fee, according to Google Cloud’s official pricing page. Astronomer’s commercial Astro platform adds a third managed option with its own pricing tiers on top of open-source Airflow, aimed at enterprise teams that want managed infrastructure without being tied to a single cloud.

Option	Base / Idle Cost	Usage Cost
Step Functions Standard	$0 (true pay-per-use)	$25 per million state transitions
Step Functions Express	$0 (true pay-per-use)	~$0.00001667/GB-second + request cost
Self-hosted Airflow	Cost of the compute you provision (EC2, EKS, etc.)	Free software, no per-transition charge
Amazon MWAA (Small)	~$357/month base environment	+ per-worker and metadata storage costs
Amazon MWAA (Large)	~$723/month base environment	+ per-worker and metadata storage costs
Google Cloud Composer 3	Consumption-based (Composer Compute Units)	Meters actual vCPU/memory/storage used

The practical takeaway: Step Functions is cheaper at low and irregular volume because there’s no floor, while a self-hosted or MWAA-managed Airflow deployment starts paying rent the moment the environment exists, regardless of whether it’s orchestrating anything. At very high, steady transition volumes, that math can flip, since Step Functions Standard’s per-transition charge scales linearly while Airflow’s infrastructure cost is comparatively fixed. Teams already tracking cloud waste and FinOps overhead should model both curves against their actual workflow volume before committing.

Standard vs Express Workflows: Step Functions’ Own Fork in the Road

Before comparing Step Functions to Airflow at all, it’s worth understanding that Step Functions itself forks into two genuinely different products. Standard Workflows guarantee exactly-once execution, can run for up to a year, and are designed for long-running, auditable processes like order fulfillment or infrastructure provisioning where you need a complete execution history. AWS caps Standard at 2,000 executions per second and, as of August 2026, its developer guide still lists that same 4,000 state transitions per second and one-year execution ceiling, per its Step Functions developer guide.

Express Workflows flip almost every one of those tradeoffs. They’re capped at five minutes of execution time, use at-least-once (asynchronous) or at-most-once (synchronous) semantics instead of exactly-once, and in exchange support 100,000 executions per second with a state transition rate AWS describes as nearly unlimited. That’s a 50 times higher execution ceiling than Standard, and it’s the mode AWS recommends for high-volume, short-lived event processing, the closest Step Functions gets to competing with Airflow on raw task throughput.

The practical rule of thumb: pick Standard when you need an audit trail and exactly-once guarantees for a workflow that might run for hours or days, like a Step Functions-orchestrated data pipeline coordinating SQS queues and Lambda functions. Pick Express when you’re processing a high-volume stream of short-lived events, like transforming IoT telemetry or API request logs, where losing exactly-once guarantees is an acceptable tradeoff for throughput.

Apache Airflow 3.0: What Changed and Why It Matters

Airflow 3.0 reached general availability on April 22, 2025, and the Apache Airflow project calls it the most significant release in the project’s history, according to the official Airflow 3.0 announcement. The rewrite moved Airflow to a service-oriented architecture built around a new Task Execution API and an independent airflow api-server, so workers no longer need direct database connections, a change that materially improves security in multi-tenant deployments.

The release bundled four major changes worth calling out individually.

DAG versioning, so teams can see exactly which version of a DAG produced a given run.
A new Asset syntax that enables event-driven, data-aware scheduling instead of relying purely on time-based cron triggers.
A completely rebuilt UI, built on React and FastAPI in place of the older Flask-based interface.
A new Edge Executor for running tasks in distributed or intermittently connected environments.

Airflow 3.1 followed in the fall of 2025, and by mid-2026 the project had reached version 3.3.0, which added a Language Task SDK letting engineers write task logic in Java and Go in addition to Python, plus a stateful task store, according to Apache’s release notes. Adoption has followed quickly: Astronomer’s State of Airflow 2026 report, published in January 2026, found that 26% of users had already migrated to Airflow 3, and 84% of those still on the 2.x branch said they planned to upgrade. The legacy 2.x branch has kept getting maintenance patches in parallel, reaching version 2.9.3 by August 2026 per Apache Airflow’s own announcements page, for teams that haven’t cut over to 3.x yet.

The timing matters for anyone still running Airflow 2: Astronomer, the company founded by core Airflow maintainers, has confirmed that Airflow 2 reaches end of life in April 2026, which means teams evaluating Step Functions against Airflow right now should be pricing in a 3.0 migration regardless of which platform they choose to stay on.

Performance, Scale, and Reliability Benchmarks

Step Functions publishes hard numbers because AWS controls the entire execution environment: 2,000 executions per second and 4,000 state transitions per second for Standard, 100,000 executions per second for Express, and a maximum Standard workflow duration of one year, all directly from AWS’s documentation. Those numbers are guarantees, not aspirational targets, because AWS is the one running the infrastructure behind them.

Airflow’s performance ceiling is a function of what you provision, not a number Apache publishes. A minimally-sized MWAA Small environment or a self-hosted deployment on a handful of small workers will bottleneck well before Step Functions does. A large Airflow deployment on Kubernetes with dozens of worker pods and a well-tuned scheduler, by contrast, can comfortably run tens of thousands of concurrent tasks. The honest framing is that Step Functions gives you a fixed, AWS-guaranteed ceiling with zero tuning required, while Airflow gives you a scaling curve that goes as high as your budget and operational maturity allow, with the KubernetesExecutor and Celery Executor both handling horizontal worker scaling in different ways.

Reliability follows a similar split. Step Functions Standard’s exactly-once semantics and up-to-one-year duration make it well suited to workflows where losing state or double-processing a step is unacceptable, like financial transactions or infrastructure orchestration. Airflow’s reliability depends on how you’ve configured retries, SLAs, and your executor. The 3.0 rewrite’s independent API server was specifically designed to make the scheduler and workers more resilient to database connectivity issues, a known pain point in Airflow 2 deployments at scale.

Observability, Local Testing, and Debugging

How easy a workflow is to debug at 2 a.m. often matters more than its throughput ceiling, and the two platforms approach this differently. Step Functions’ console shows a real-time execution graph for every run, with per-state input and output payloads, error traces, and timing available without leaving the browser. A “Test State” feature lets you isolate and re-run a single state with sample input before wiring it into the full state machine, and executions integrate natively with CloudWatch Logs and AWS X-Ray for distributed tracing across the Lambda functions a workflow calls.

Airflow’s debugging story is built around the same Grid and Graph views used for monitoring: click into any task instance in a DAG run to see its logs, retry history, and duration. What Airflow adds that Step Functions structurally can’t is offline testing, since the airflow dags test CLI command runs a full DAG locally without touching the production scheduler or metadata database, which makes catching logic errors before deployment considerably cheaper. Because DAGs are Python, standard unit-testing frameworks like pytest apply directly to individual task functions, something that isn’t possible with ASL’s declarative JSON.

Real-World Use Cases: Where Each Tool Actually Gets Used

Both platforms show up in production in fairly predictable patterns, shaped directly by their architecture and pricing model.

Serverless order-fulfillment pipelines. A Step Functions Standard workflow coordinating Lambda functions, DynamoDB writes, and SNS notifications is a textbook e-commerce checkout flow, where exactly-once execution and a full audit trail matter more than raw throughput.
Nightly ETL and data warehouse loading. This is the original use case Airbnb built Airflow to solve, and it remains Airflow’s home turf: DAGs that extract from source systems, transform with Python or Spark, and load into a warehouse on a schedule, with dependency management between hundreds of tasks.
High-volume IoT and event processing. Step Functions Express Workflows, with their 100,000-executions-per-second ceiling, fit streaming telemetry transformation where losing exactly-once guarantees is an acceptable tradeoff for throughput and cost.
Machine learning training and evaluation pipelines. Airflow 3.0’s expanded event-driven and ML-workflow support, combined with Python-native task authoring, makes it a common backbone for orchestrating data prep, model training, evaluation, and deployment steps that need arbitrary custom logic between stages.
Human-in-the-loop approval workflows. Step Functions’ native .waitForTaskToken pattern lets a workflow pause indefinitely until a human approves a step, common in infrastructure change management or content moderation, without polling or custom sensor logic.
Cross-cloud and hybrid data platforms. Enterprises running workloads across AWS, Azure, GCP, and on-prem systems lean on Airflow specifically because its provider-package model lets one orchestration layer coordinate all of them, something Step Functions structurally cannot do outside AWS.

Sample Workflow Definitions Side by Side

The authoring difference between the two tools is easiest to see in code. Here’s a minimal two-step Step Functions state machine defined in Amazon States Language:

{
  "Comment": "Simple sequential workflow",
  "StartAt": "ValidateOrder",
  "States": {
    "ValidateOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ValidateOrder",
      "Next": "ChargePayment"
    },
    "ChargePayment": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ChargePayment",
      "End": true
    }
  }
}

And the equivalent logic as an Airflow 3.0 DAG in Python:

from airflow.sdk import dag, task

@dag(schedule=None, catchup=False)
def order_pipeline():
    @task
    def validate_order():
        ...

    @task
    def charge_payment():
        ...

    validate_order() >> charge_payment()

order_pipeline()

The ASL version is declarative and has no room for arbitrary logic outside what the Lambda functions themselves contain. The Airflow version is regular Python, meaning loops, conditionals, dynamic task generation, and third-party libraries are all available directly inside the DAG definition, which is exactly the flexibility-versus-simplicity tradeoff that defines this whole comparison.

Pros and Cons of AWS Step Functions

Step Functions earns its place in an AWS-native stack, but it comes with real constraints worth weighing before committing a team to it.

Pro: zero idle cost. True pay-per-use billing with a real $0 floor means a workflow that never runs costs nothing beyond the free tier.
Pro: no infrastructure to manage. No scheduler, no metadata database, no patching, no capacity planning.
Pro: deep native AWS integration. Thousands of API actions across AWS services are callable directly from state definitions without custom connectors.
Pro: built-in human-in-the-loop support. The .waitForTaskToken pattern handles approval workflows natively.
Con: total AWS lock-in. State machines are not portable to any other cloud.
Con: JSON/ASL authoring is less expressive than Python. Complex branching or dynamic logic pushes work out into more Lambda functions rather than staying in the workflow definition itself.
Con: Express Workflows cap out at five minutes. Long-running Express-style high-throughput jobs aren’t possible, so you’re forced back to Standard’s lower throughput ceiling.
Con: cost can scale unpredictably with state count. A workflow with many states run at high volume on Standard racks up transitions fast, since every state entered counts.

Pros and Cons of Apache Airflow

Airflow’s flexibility is its main selling point, but that flexibility comes bundled with operational responsibility Step Functions simply doesn’t have.

Pro: free and open source. Apache License 2.0 means no licensing cost and no vendor contract required to start.
Pro: Python-native authoring. Full programming language expressiveness, including dynamic task generation, custom operators, and the entire Python package ecosystem.
Pro: massive community and ecosystem. 46,300 GitHub stars and provider packages for essentially every cloud and data platform in production use today.
Pro: cloud-portable. The same DAGs can orchestrate AWS, Azure, GCP, and on-prem systems from a single control plane.
Con: always-on infrastructure cost. The scheduler and metadata database run continuously whether or not any DAG is executing, and MWAA’s Small environment alone runs roughly $357 a month before a single worker is added.
Con: operational complexity. Even managed options require decisions about executor type, worker sizing, and metadata database maintenance.
Con: Airflow 2 end-of-life pressure. With Airflow 2 reaching end of life in April 2026, teams on older versions face a mandatory 3.0 migration on top of any other roadmap work.
Con: no published hard throughput ceiling. Unlike Step Functions’ documented numbers, Airflow’s real-world capacity is only as good as the infrastructure and executor tuning behind it.

Migration Guide: Moving Between Step Functions and Airflow

Teams migrate in both directions, usually for opposite reasons: Airflow-to-Step-Functions moves happen when a team wants to shed infrastructure overhead for a mostly-AWS-native workload, while Step-Functions-to-Airflow moves happen when workflow logic outgrows what’s comfortable to express in ASL, or when multi-cloud requirements appear.

Step 1: Inventory workflow complexity. Linear and simple branching DAGs with a handful of tasks translate cleanly in either direction. DAGs with dynamic task generation, heavy Python logic, or complex cross-task data sharing are harder to move to Step Functions and are better candidates for staying on or moving to Airflow.
Step 2: Map operators to equivalents. Airflow’s PythonOperator or TaskFlow-decorated functions typically become Lambda-backed Task states in ASL. BranchPythonOperator logic becomes a Choice state. Airflow’s TriggerDagRunOperator or subDAGs become nested Step Functions workflows invoked via the .sync integration pattern.
Step 3: Rebuild scheduling. Airflow’s cron-based or Asset-driven scheduling has no direct Step Functions equivalent. Use Amazon EventBridge Scheduler instead to trigger state machine executions on a schedule or in response to events.
Step 4: Decide what stays in Lambda vs what becomes workflow logic. Moving from Airflow to Step Functions usually means extracting Python logic that lived inside a DAG task into standalone Lambda functions, since ASL itself can’t run arbitrary code.
Step 5: Run in parallel before cutover. Execute both versions against production data (or a shadow copy) for at least one full cycle, comparing outputs and execution history before decommissioning the old pipeline.
Step 6: Migrate monitoring and alerting. Airflow’s UI-based DAG run history and Step Functions’ execution history in the console serve the same purpose but aren’t interchangeable. Rebuild alerting rules in CloudWatch (for Step Functions) or your existing observability stack (for Airflow) rather than assuming dashboards carry over.

Teams already running container workloads on ECS, EKS, or Fargate often find the Airflow side of a migration easier, since Airflow’s KubernetesExecutor or ECS-based deployments slot into existing container tooling rather than requiring a new operational model.

Security and Access Control

Step Functions inherits AWS’s IAM model directly, which is either its biggest security advantage or its steepest learning curve depending on how comfortable a team already is with IAM. Two separate policy layers matter: who is allowed to start, stop, or view state machine executions, and what the state machine’s own execution role is permitted to call. Get the execution role wrong and a workflow either fails on a permissions error or, worse, has broader access than it needs. Encryption at rest is handled through AWS KMS, and every execution is automatically logged to CloudTrail for audit purposes with no extra configuration.

Airflow secures access through role-based access control in its webserver, with built-in Admin, Op, User, Viewer, and Public roles that can be extended with custom roles in Airflow 3.0’s updated security model. Secrets, database passwords, API keys, and connection strings, are stored as Airflow Connections or Variables, which can be backed by AWS Secrets Manager, HashiCorp Vault, or GCP Secret Manager instead of Airflow’s own metadata database for production deployments. Network security is entirely your responsibility on self-hosted Airflow. Amazon MWAA runs inside your own VPC, which narrows that gap somewhat but still leaves security-group and subnet configuration in your hands rather than fully managed away, the way it is with Step Functions.

Which Should You Choose? Use-Case Recommendations

The honest answer for most teams isn’t binary. Here’s how to think about it by scenario.

Choose Step Functions if you’re building AWS-native serverless applications where workflow steps are mostly Lambda invocations or calls to other AWS services, and you want zero infrastructure to operate.
Choose Step Functions Express specifically if you’re processing high-volume, short-duration events like IoT telemetry or clickstream data, where the 100,000-executions-per-second ceiling and lower per-unit cost outweigh losing exactly-once guarantees.
Choose Airflow if your workflows are primarily data engineering pipelines with complex Python logic, dynamic DAG generation, or heavy dependency chains across dozens or hundreds of tasks.
Choose Airflow if you need to orchestrate across multiple clouds or on-prem systems from a single control plane, since Step Functions structurally cannot leave AWS.
Choose Amazon MWAA over self-hosting if you want Airflow’s flexibility but don’t want to operate the scheduler and metadata database yourselves, and the roughly $357-plus monthly floor is acceptable against your team’s time cost of self-managing.
Choose self-hosted Airflow on Kubernetes if you’re already running EKS at scale and have the platform team to operate it, since you avoid MWAA’s markup and gain full control over executor tuning.
Run both if your organization has both AWS-native application glue and heavy data engineering pipelines. Plenty of teams use Step Functions for service orchestration and Airflow for data pipelines, invoking each other via API calls or EventBridge where the two need to hand off work.

As a quick reference, here’s how the decision typically shakes out by team profile:

Team Profile	Recommended Tool	Primary Reason
Small team, AWS-only, mostly Lambda glue code	Step Functions Standard	Zero infrastructure, pay only for what runs
High-volume event/IoT processing on AWS	Step Functions Express	100,000 executions/sec ceiling at low per-unit cost
Data engineering team, complex Python ETL	Apache Airflow (self-hosted or MWAA)	Python-native DAGs handle dependency complexity
Multi-cloud or hybrid on-prem environment	Apache Airflow	Provider packages orchestrate any platform, not just AWS
Platform team wants managed Airflow, no ops burden	Amazon MWAA or Google Cloud Composer	AWS/GCP run the scheduler and metadata database for you
ML pipeline: training, evaluation, deployment	Apache Airflow	3.0’s event-driven, ML-oriented scheduling and Python flexibility

The Verdict: Step Functions vs Airflow in 2026

These aren’t really competitors so much as tools that solve different halves of the orchestration problem. Step Functions wins decisively on operational simplicity and cost predictability for AWS-native application workflows: zero idle cost, no servers, and documented throughput ceilings of 2,000 to 100,000 executions per second depending on workflow type. Airflow wins decisively on expressiveness and portability: Python-native authoring, an open-source community that had grown past 43,800 GitHub stars and 3,600-plus contributors by August 2026, and the ability to orchestrate any cloud or system from one place, at the cost of an infrastructure bill that starts around $357 a month on Amazon MWAA’s smallest tier before a single worker is added.

If the workload is fundamentally “call these AWS services in this order, sometimes with a human approval step,” Step Functions is the lower-cost, lower-maintenance choice. If the workload is fundamentally “run complex, interdependent data or ML jobs across systems that may not all live in AWS,” Airflow, especially post-3.0 with its new API server architecture and Java/Go task support in 3.3.0, remains the more capable platform. The Airflow 2 end-of-life deadline in April 2026 makes this a good moment for any team still on the older version to make that call deliberately rather than by default.

Frequently Asked Questions

Is AWS Step Functions free?
Step Functions has a genuine free tier of 4,000 state transitions per month. Beyond that, Standard Workflows cost $0.025 per 1,000 state transitions ($25 per million), and Express Workflows bill by execution count, duration, and memory instead. There’s no charge at all if no workflow executes, since Step Functions has no idle infrastructure cost.

Is Apache Airflow really free to use?
The software itself is free under the Apache License 2.0 with no licensing fees. What isn’t free is the infrastructure to run it: a scheduler, metadata database, and webserver need to run continuously, whether that’s your own servers, Amazon MWAA (from roughly $357/month for the smallest environment), or Google Cloud Composer’s consumption-based pricing.

Can I run Apache Airflow on AWS?
Yes, in three ways: self-hosted on EC2 or EKS, or through Amazon MWAA, AWS’s managed Airflow service that handles the scheduler, metadata database, and webserver for you at an hourly rate based on environment size.

What is the difference between Step Functions Standard and Express Workflows?
Standard Workflows guarantee exactly-once execution, can run up to one year, and are billed per state transition, capped at 2,000 executions per second. Express Workflows trade exactly-once guarantees for much higher throughput, up to 100,000 executions per second, are capped at five minutes of runtime, and bill by duration and memory instead of transitions.

Does Airflow 3.0 change how it compares to Step Functions?
Yes, meaningfully. Airflow 3.0’s service-oriented architecture, independent API server, DAG versioning, and event-driven Asset-based scheduling close some of the operational-maturity gap with fully managed services like Step Functions, even though Airflow still requires you to run infrastructure that Step Functions doesn’t.

Can I migrate an Airflow DAG to Step Functions, or the reverse?
Yes, though the difficulty depends on workflow complexity. Simple linear or branching DAGs map reasonably cleanly to Step Functions state machines, with Python task logic extracted into Lambda functions. Complex DAGs with dynamic task generation or heavy interdependencies are harder to migrate and are often better left on Airflow, or migrated to Step Functions in pieces.

Which is better for machine learning pipelines?
Airflow is generally the stronger fit for end-to-end ML pipelines involving data preparation, training, and evaluation, particularly since Airflow 3.0 expanded native support for event-driven and ML-oriented workflows. That focus shows up in usage data: Astronomer’s State of Airflow 2026 report, updated in August 2026, found that 32% of Airflow users now run GenAI or MLOps workloads on the platform, a share that rises to 62% among Astro customers specifically. Step Functions works well for the serving and inference side, orchestrating Lambda-based inference endpoints, but is a less natural fit for the heavier Python-based training logic.

Is Amazon MWAA the same thing as Apache Airflow?
Amazon MWAA runs unmodified open-source Apache Airflow, it’s a managed hosting layer rather than a fork, so DAGs written for MWAA are portable to self-hosted Airflow or other managed options like Google Cloud Composer with minimal changes.

Related Coverage

Nadia Dubois

AI & Innovation Editor

Nadia Dubois is the AI & Innovation Editor at Tech Insider, where she tracks the rapid evolution of artificial intelligence, from foundation models to real-world enterprise deployment. She previously covered AI and startups for La Tribune and contributed to MIT Technology Review's European coverage. Nadia specializes in generative AI, AI regulation, and the intersection of technology and European industrial policy. She holds a dual degree in Computational Linguistics and Journalism from Sciences Po Paris.

View all articles

Step Functions vs Airflow: $0 vs $357/mo Floor [2026]