Running Apache Airflow + Docker for Free Using GitHub Codespaces

While building my ETL pipeline project, I ran into a common beginner problem:

Running Apache Airflow locally on Windows with Docker was painful.

Problems included:

Low disk space
Docker setup issues
Linux compatibility problems
Environment debuggin…


This content originally appeared on DEV Community and was authored by Tanmay

While building my ETL pipeline project, I ran into a common beginner problem:

Running Apache Airflow locally on Windows with Docker was painful.

Problems included:

  • Low disk space
  • Docker setup issues
  • Linux compatibility problems
  • Environment debugging overhead

With only ~17GB free on my laptop, running multiple Airflow containers locally became difficult.

So I moved the entire setup to GitHub Codespaces.

What Codespaces Provided

Out of the box:

  • Ubuntu Linux environment
  • Docker pre-installed
  • VS Code in browser
  • Auto-cloned GitHub repo
  • Port forwarding for Airflow UI

Workflow 

docker-compose up

Then:

  • Open Airflow UI
  • Trigger ETL DAG
  • Verify successful execution ✔️

Airflow was running in ~90 seconds.

Important Security Lesson

I accidentally committed my GCP service account key once.

GitHub Secret Scanning blocked the push automatically.

Immediately added:

*.json
.env

Never commit cloud credentials.

Why This Setup Helps Beginners

Codespaces removes a huge amount of local environment friction and lets you focus more on:

  • Airflow orchestration
  • ETL pipelines
  • Docker workflows
  • Cloud integrations

If you'd like:

  • Beginner-friendly walkthrough → check Medium
  • Engineering-focused breakdown → check Hashnode

Medium: Link
Hashnode: Link

Project Repo:ETL Pipeline

DataEngineering #Docker #ApacheAirflow #GitHubCodespaces


This content originally appeared on DEV Community and was authored by Tanmay


Print Share Comment Cite Upload Translate Updates
APA

Tanmay | Sciencx (2026-06-08T06:03:39+00:00) Running Apache Airflow + Docker for Free Using GitHub Codespaces. Retrieved from https://www.scien.cx/2026/06/08/running-apache-airflow-docker-for-free-using-github-codespaces/

MLA
" » Running Apache Airflow + Docker for Free Using GitHub Codespaces." Tanmay | Sciencx - Monday June 8, 2026, https://www.scien.cx/2026/06/08/running-apache-airflow-docker-for-free-using-github-codespaces/
HARVARD
Tanmay | Sciencx Monday June 8, 2026 » Running Apache Airflow + Docker for Free Using GitHub Codespaces., viewed ,<https://www.scien.cx/2026/06/08/running-apache-airflow-docker-for-free-using-github-codespaces/>
VANCOUVER
Tanmay | Sciencx - » Running Apache Airflow + Docker for Free Using GitHub Codespaces. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2026/06/08/running-apache-airflow-docker-for-free-using-github-codespaces/
CHICAGO
" » Running Apache Airflow + Docker for Free Using GitHub Codespaces." Tanmay | Sciencx - Accessed . https://www.scien.cx/2026/06/08/running-apache-airflow-docker-for-free-using-github-codespaces/
IEEE
" » Running Apache Airflow + Docker for Free Using GitHub Codespaces." Tanmay | Sciencx [Online]. Available: https://www.scien.cx/2026/06/08/running-apache-airflow-docker-for-free-using-github-codespaces/. [Accessed: ]
rf:citation
» Running Apache Airflow + Docker for Free Using GitHub Codespaces | Tanmay | Sciencx | https://www.scien.cx/2026/06/08/running-apache-airflow-docker-for-free-using-github-codespaces/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts