AWS Cashback AWS EC2 continuous integration setup

Introduction: Why EC2 CI Can Be a Bit Like Herding Cats

Continuous integration on AWS EC2 sounds like it should be straightforward: you push code, something builds, tests run, everyone claps, and the pipeline confidently hands back a green checkmark. In reality, setting up CI is often less “clap-clap” and more “okay, who changed the PATH again?”

This guide shows you how to create a reliable CI setup using EC2. We’ll focus on a build-and-test pipeline that runs automatically whenever you update your code. You’ll learn how to set up an EC2 instance (or use one as a build worker), install the tools you need, define a build script, and wire it into a continuous integration workflow. Along the way, we’ll address security concerns, caching, artifact storage, and troubleshooting.

Nothing here requires you to memorize obscure AWS incantations. But you will need to be willing to do the occasional sanity check—like confirming the build user can actually access the repository, and that your tests aren’t secretly relying on your local machine’s “special” environment variables. Spoiler: they are.

What “Continuous Integration on EC2” Usually Means

When people say “CI on EC2,” they usually mean one of these patterns:

• EC2 as a dedicated build runner: You create an EC2 instance that continuously (or on-demand) pulls tasks from your CI system, executes builds/tests, and reports results.
• EC2 as an on-demand worker: Your pipeline spins up an EC2 instance (or uses an existing one), runs the build, then shuts it down to save cost.
• Hybrid: The pipeline logic lives elsewhere (like GitHub Actions, GitLab CI, Jenkins, etc.), while EC2 provides the execution environment for builds.

In this article, we’ll keep it broadly applicable and show a practical approach that you can adapt. We’ll assume you want an EC2-based environment capable of running:

• Dependency installation (e.g., npm, pip, Maven, Gradle)
• Build steps (compile, bundle, package)
• Automated tests (unit tests, integration tests, linting)
• Artifact upload (so builds aren’t just vibes)

The “how” depends on your preferred CI orchestrator. But the core fundamentals stay the same: environment, credentials, scripts, automation, observability, and repeatability.

High-Level Architecture (No, We Won’t Build a Castle)

Let’s describe a simple architecture that works well for many teams:

1. Source control (e.g., GitHub, GitLab, Bitbucket).
2. CI trigger runs when you push commits or open pull requests.
3. CI job sends build tasks to EC2 (either through a runner, SSH-based command, SSM, or a job agent).
4. Build script runs on EC2: installs dependencies, runs tests, builds artifacts.
5. Results & logs are collected and displayed in your CI system.
6. Artifacts get uploaded to S3 (or stored elsewhere) for later use.

At the end of the day, you want: “Push code → build/test output appears → artifacts available → pipeline is trustworthy.” Reliability is the real product here.

Prerequisites: Decide Your Tooling Before You Click “Launch”

Before you configure EC2, take a moment to decide what your CI runner will be. If you already use a CI service, you might integrate EC2 as a build environment. If you’re starting fresh, you might adopt a CI server like Jenkins, or a managed workflow system.

To keep this guide consistent and adaptable, we’ll assume you can run scripts on EC2 triggered by your CI system. Your CI orchestrator could be:

• Jenkins (with an EC2 plugin or agent)
• GitHub Actions (self-hosted runner)
• GitLab CI runner on EC2
• A custom pipeline that SSHs or uses AWS Systems Manager (SSM) to execute commands

If you want the simplest “conceptual” approach, consider using:

• Self-hosted runner (if your CI platform supports it), or
• SSM Run Command (often safer than SSH with key pairs)

We’ll also talk about security later, because credentials mishaps are the #1 way to turn CI into a horror story.

Step 1: Provisioning EC2 the Sensible Way

Creating an EC2 instance is easy. Creating one that behaves predictably is the part where you earn your coffee.

Choose an AMI and Instance Type

Select an AMI that matches your runtime needs:

• Ubuntu 22.04 LTS is a common baseline
• Amazon Linux 2023 works too, but package commands differ slightly

Pick an instance type that can handle typical builds. For many projects, t3.small or t3.medium is enough. If you run heavy integration tests or compile large codebases, scale up or add parallelism.

Also consider whether you need GPU instances (spoiler: if you do, you’re probably already aware).

Networking and Security Group Basics

Ideally, you won’t expose the EC2 instance to the world. Your CI pipeline doesn’t need inbound internet access to run builds.

For SSH-based approaches, you’d restrict port 22 to your IP addresses. For SSM-based approaches, you may not need inbound access at all.

Here’s the mindset: “Minimum open ports, maximum sanity.”

Use an IAM Role Rather Than Wandering Around With Access Keys

Instead of hardcoding AWS credentials in scripts, attach an IAM role to your EC2 instance. That role can allow access to:

• Read/write S3 for artifacts
• CloudWatch for logs
• Optionally pull from private package registries (with proper configuration)

We’ll cover the exact permissions later, but the key rule is: do not store AWS credentials in your repo like they’re loose change in the couch.

Step 2: Prepare the EC2 Build Environment

Your EC2 instance needs to be a clean, repeatable environment. The goal is: when CI runs, it should not depend on who last logged into the machine.

Update Packages and Install System Dependencies

After you launch EC2, connect to it (via SSH or SSM). Then update and install essentials:

sudo apt-get update
sudo apt-get -y upgrade
sudo apt-get -y install git curl ca-certificates build-essential unzip

If your project needs additional libraries (e.g., for image processing, database clients, headless browsers), install them now or in your build script.

But don’t do everything randomly. Prefer a documented setup step, and ideally use an automated provisioning method later (like a bootstrap script or AMI baking).

Install Language Runtimes and Package Managers

Examples:

• Node.js: install via NodeSource or nvm (nvm can be fine, but keep your CI PATH consistent)
• Python: use pyenv or install directly via apt + virtualenv
• Java: install OpenJDK and ensure you’re consistent with build tools

For Node.js, a common approach might be:

curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs
node -v
npm -v

If your project has strict version requirements, install exactly what the project specifies. CI should be faithful, not improvisational.

AWS Cashback Create a Dedicated Build User

Don’t run CI jobs as root if you can avoid it. Create a user like ci-runner:

sudo adduser --disabled-password --gecos "" ci-runner
sudo usermod -aG sudo ci-runner

You may choose not to grant sudo at all and instead ensure all needed dependencies are installed beforehand. Either way, isolate CI filesystem writes to that user’s home directory.

AWS Cashback If your builds write to /tmp or to a project workspace, ensure permissions are correct so you don’t get a “permission denied” surprise at 2 a.m.

Step 3: Make a Build Script That’s Actually Useful

Instead of embedding build logic directly in your CI configuration (which can get messy fast), create a build script in your repo. This makes local runs and CI runs consistent.

For example, add a script at ./ci/build.sh. It might:

• Install dependencies
• Run linting
• Run tests
• Build artifacts
• Export artifact metadata

Example build script for a Node.js project

Here’s a generic example. Adjust commands for your project structure:

#!/usr/bin/env bash
set -euo pipefail

echo "== CI build started =="

echo "Node version: $(node -v)"
echo "NPM version: $(npm -v)"

WORKDIR="${WORKDIR:-$PWD}"
cd "$WORKDIR"

# Install dependencies cleanly
npm ci

# Lint and test
if npm run -s lint; then
  echo "Lint passed"
else
  echo "Lint failed"
  exit 1
fi

npm test

# Build artifact
npm run build

echo "Build completed successfully"

# Optionally list artifact output
if [ -d "dist" ]; then
  echo "Artifact contents (dist):"
  ls -lah dist
fi

Then in your repo:

chmod +x ci/build.sh

Notice the script uses:

• set -euo pipefail to fail fast
• WORKDIR for flexibility
• Clear echo statements (because logs are your future self’s best friend)

Example build script for Python

#!/usr/bin/env bash
set -euo pipefail

echo "== CI build started =="

python --version
pip --version

# Create virtual environment
python -m venv .venv
source .venv/bin/activate

pip install --upgrade pip
pip install -r requirements.txt

# Lint (optional)
if [ -f "requirements-dev.txt" ]; then
  pip install -r requirements-dev.txt
fi

# Run tests
pytest -q

# Build artifacts if applicable
# python setup.py sdist bdist_wheel

echo "Build completed successfully"

Yes, you could do more sophisticated dependency caching, but first you want correctness.

Step 4: Wire CI to the EC2 Instance

Now the fun part: making your CI system trigger EC2 builds. There are multiple ways. I’ll outline two common approaches: self-hosted runners and SSM-based remote execution.

Approach A: Self-hosted runner on EC2 (Recommended If Your CI Platform Supports It)

If you’re using a platform like GitHub Actions, you can install a self-hosted runner on EC2. The platform sends jobs to the runner automatically, and the runner executes commands locally.

The steps typically look like:

• Launch EC2
• Install the runner software
• Configure it with a registration token
• Start the runner as a service
• Create workflow jobs that target the runner

Conceptually, your workflow config might include something like:

name: CI

on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  build:
    runs-on: [self-hosted, linux]
    steps:
      - uses: actions/checkout@v4
      - name: Run build
        run: ./ci/build.sh

The exact syntax depends on your CI service, but the principle is: the CI job runs on EC2 automatically.

Also, consider labeling runners by environment. For example, one EC2 instance might be labeled “linux-small” and another “linux-large” for different workload sizes.

Approach B: SSM Run Command (Often Safer Than SSH)

With AWS Systems Manager (SSM), you can execute commands on EC2 without exposing SSH to the internet. This is great for CI because:

• Less network exposure
• Centralized audit logs
• IAM-based access control

Typical requirements:

• EC2 must have SSM agent installed (usually already present on modern AMIs)
• EC2 instance needs an IAM role with SSM permissions
• You run SSM commands from your CI system using AWS credentials or a trusted integration

A script execution flow could be:

1. AWS Cashback Your CI job calls AWS API to send an SSM command.
2. SSM command checks out code (or you sync it) and runs the build script.
3. CI polls for command completion and collects output.

This can be powerful, but it also adds orchestration complexity. So if you’re early in your setup, self-hosted runners may be simpler.

Step 5: Dependency Caching (Because You Want to Feel Like a Wizard)

CI can be slow if each run installs dependencies from scratch. Caching improves build times dramatically.

Caching can happen at multiple levels:

• CI system cache (e.g., GitHub Actions cache)
• Persistent directories on the EC2 instance
• Package manager cache (npm cache, pip cache)

If your EC2 instance is long-lived (not replaced often), caching on the instance is easiest.

Example: npm cache directory

npm has a cache directory. You can point it to a path you reuse:

NPM_CACHE_DIR="/home/ci-runner/.npm-cache"
mkdir -p "$NPM_CACHE_DIR"
export npm_config_cache="$NPM_CACHE_DIR"

npm ci

However, be careful: caching doesn’t fix dependency drift. If your lockfile changes, you still need clean installs. npm ci helps with this because it’s based on the lockfile.

Example: pip cache

export PIP_CACHE_DIR="/home/ci-runner/.cache/pip"
mkdir -p "$PIP_CACHE_DIR"
pip install -r requirements.txt

Step 6: Artifacts and Test Reports (So Builds Leave Receipts)

CI isn’t just about passing or failing. It’s also about producing outputs you can use later: packages, binaries, bundles, and reports that explain what happened.

Upload build artifacts to S3

AWS Cashback If your build creates a dist folder or similar, upload it to S3. You’ll want:

• A bucket
• IAM permissions for the EC2 role or CI role
• A consistent naming scheme

Example shell commands:

ARTIFACT_NAME="myapp-${GITHUB_SHA:-local}.tar.gz"

# Create tarball
tar -czf "$ARTIFACT_NAME" dist/

# Upload
aws s3 cp "$ARTIFACT_NAME" "s3://your-bucket/ci-artifacts/${ARTIFACT_NAME}"

Even if your CI system already uploads artifacts, S3 can be useful as a durable store or for downstream deployment jobs.

Upload test results (JUnit XML)

Many tools produce test reports in JUnit XML format. Capture them so your CI UI can display nice structured results.

For example, with pytest you can do:

pytest --junitxml=reports/junit.xml

Then upload that file as CI artifacts, or store in S3.

Step 7: Environment Variables and Secrets (The Part Where Things Go Wrong)

CI pipelines often need secrets: API keys, database credentials for integration tests, or tokens to publish artifacts. Do not store secrets in your repo.

Use your CI platform’s secrets management and/or AWS Systems Manager Parameter Store / Secrets Manager.

AWS Cashback Common best practices

• Never echo secrets in logs (yes, it happens)
• Use least privilege for IAM roles
• Rotate credentials periodically
• Separate dev and prod accounts if possible

AWS Cashback Example: Using environment variables in your build script

Suppose your application needs an API base URL during integration tests:

export API_BASE_URL="${API_BASE_URL:-http://localhost:3000}"

# then run tests
npm test

Your CI config supplies API_BASE_URL and secrets securely.

Step 8: Observability and Logging (How to Catch Failures Before They Grow Teeth)

When CI fails, you want to know why quickly. Logging matters. Output verbosity should be enough to diagnose issues, not enough to fill your log storage like a squirrel hoarding acorns.

Use consistent log formatting

In your build scripts, print:

• versions of runtime tools
• dependency install steps
• paths where artifacts are created
• test commands

Store logs somewhere searchable

If your CI system shows logs, that may be enough. But also consider sending system-level logs to CloudWatch or ensuring your SSM/runner logs are retained.

Step 9: Reliability Improvements (Because CI Shouldn’t Flake Like a Croissant)

CI pipelines become unreliable due to:

• Temporary network issues fetching dependencies
• Race conditions in tests
• Services not ready when integration tests start
• Non-deterministic builds

Here are some strategies:

• Make tests deterministic: avoid relying on timeouts without good reason.
• Add health checks before running integration tests.
• AWS Cashback Use retries sparingly for flaky network-dependent steps (like fetching dependencies), not for failing tests.
• Ensure clean workspaces: don’t let old build files contaminate new runs.

AWS Cashback Clean workspace example

rm -rf build dist .venv node_modules

If that feels heavy-handed, note you can combine it with caching for speed. The idea is: when something changes, builds should reflect it—not leftovers from last Tuesday.

Step 10: CI Workflow Design (Stages, Fail Fast, and Don’t Reward Bad Code)

Design your pipeline with stages. Typical stages:

• Lint / formatting checks
• Unit tests
• Integration tests
• Build/package
• Artifact upload

Fail fast: if lint fails, there’s no point running 10 minutes of tests. Your pipeline should stop early when possible.

Also, handle pull requests separately from main branch merges. Pull requests might run faster checks, while main branch merges run full integration and packaging.

Suggested Repository Structure

A tidy repo makes CI easier to maintain. Here’s a sample layout:

project-root/
  ci/
    build.sh
    test.sh
    package.sh
  src/
  tests/
  package.json (or pyproject.toml / build.gradle / etc.)
  dist/ (generated)
  reports/

Scripts in ci/ help keep CI configuration readable. They also help you reuse the same logic locally when you’re debugging.

Putting It All Together: An Example End-to-End Flow

Let’s walk through a practical example using a hypothetical Node.js project, with artifacts uploaded to S3, and tests producing output.

1) EC2 instance setup (baseline)

• Install Node.js 20
• Create build user
• Install git and build essentials
• Attach IAM role with S3 write permissions

2) Build script in repo

• Run npm ci
• Run lint
• Run unit tests
• Run build
• Create artifact archive

3) CI workflow triggers on push and pull request

• Checkout code
• Run build script on EC2 runner
• Upload artifacts and test reports

4) S3 upload (optional)

• Store artifact archive in bucket
• AWS Cashback Provide URL to downstream steps (deployment, etc.)

If you implement this flow and you’re still getting failures, congratulations: you’ve reached the stage where debugging skills become part of the job description.

Troubleshooting: The Usual Suspects

Here are common problems and how to approach them. No magic, just detective work.

Problem: “Command not found” during CI

Likely causes:

• AWS Cashback PATH isn’t set for the build user
• Runtime version not installed
• Using nvm but CI doesn’t load shell initialization

Fixes:

• Print versions and PATH at the start of your build script
• Use absolute paths where possible
• Prefer system-wide installs for CI stability

Problem: Permission denied when writing files

• Build script runs as a different user than expected
• Workspace directory ownership is wrong

Fix: ensure the build script uses the workspace owned by the runner user. Consider cleaning and resetting ownership for persistent directories.

Problem: Integration tests fail because services aren’t ready

Fix:

• Add a retry loop with a short timeout to check service health
• Wait for TCP port availability or HTTP endpoint readiness

Example idea:

for i in {1..30}; do
  if curl -fsS "http://localhost:8080/health" > /dev/null; then
    echo "Service is ready"
    break
  fi
  echo "Waiting for service... ($i)"
  sleep 2
done

Problem: Build is slow every time

Fix:

• Enable dependency caching
• Use lockfiles (npm ci, pip tools with locked dependencies)
• Consider baking an AMI with preinstalled dependencies

Slow builds feel like watching paint dry. Caching is how you stop staring at the wall.

Cost Considerations (Because EC2 Bills Are Real and They Don’t Laugh)

CI costs can creep up if you keep instances running 24/7. Depending on your workload:

• Use smaller instances for steady workloads
• Use autoscaling for runner capacity
• Shut down instances when idle if your setup supports it
• Consider spot instances for non-critical builds

Even if you keep an instance running, caching and reducing unnecessary runs help keep costs down.

Security Checklist (A Quick, Practical One)

If you only read one part of this article, read this one. Security should not be an afterthought.

• Use IAM roles for EC2 (avoid hardcoded keys)
• Restrict inbound traffic (or avoid SSH entirely with SSM)
• Restrict S3 permissions to only what CI needs
• Do not log secrets
• Patch regularly (update packages and OS)

When to Consider a Different Approach

EC2-based CI is great when you need a custom environment, special tooling, or consistent control. But if you want minimal infrastructure management, you might consider:

• Managed CI runners (where available)
• Containerized builds (Docker) with ephemeral runners
• Other AWS services depending on your pipeline

That said, EC2 CI is still a powerful, widely used approach. It’s not old-fashioned; it’s just “adult supervision for builds.”

Conclusion: Your CI Pipeline Should Be Boring (In a Good Way)

Once you set up an EC2 continuous integration setup properly, the best compliment you can receive is: “It works.” Boring reliability is the dream because CI is supposed to catch issues early, not create new ones.

AWS Cashback To recap, a good EC2 CI setup includes:

• A properly provisioned EC2 instance with the right runtime tools
• A repeatable build script in your repo
• A CI workflow that triggers builds on commits and pull requests
• Secure credentials handling via IAM roles and secrets management
• Caching to speed up builds
• Artifact upload and useful logs for debugging
• Reliability practices to avoid flakiness

If you implement the steps above, you’ll end up with a CI pipeline that behaves like a well-trained dog: it follows instructions, doesn’t bite, and occasionally fetches artifacts on command.

Now go forth and make your EC2 builds consistent—preferably before your next production incident tries to teach you a lesson.