AWS CodeBuild Managed Self-Hosted GitHub Action Runners

AWS CodeBuild now supports managed self-hosted GitHub Action runners, allowing you to build powerful CI/CD capabilities right beside your code and quickly implement a build, test and deploy pipeline. Last year AWS announced that customers can define their GitHub Actions steps within any phase of a CodeBuild buildspec file but with a self-hosted runner, jobs execute from GitHub Actions on GitHub.com to a system you deploy and manage.

With the recent announcement that AWS CodeBuild now supports managed GitHub Action runners, AWS can take care of managing the hosting of GitHub Action self-hosted runners within CodeBuild allowing teams to run their GitHub Actions workflow jobs natively within AWS.

For customers managing their self-hosted runners on their own infrastructure, CodeBuild can now provide a secure, scalable and lower latency solution. In addition, CodeBuild managed self-hosted GitHub Action runners bring features, such as:

• Reserved capacity to compile your software within your Amazon VPC and access resources such as Amazon Relational Database Service, Amazon ElastiCache, or any service endpoints that are only reachable from within a specific VPC.
• Available compute platforms including AWS Lambda, Windows, Linux, Linux GPU-enhanced and AWS Graviton Processors (Arm-based instances).

With the compute options available, customers can now run tests on hardware and operating system combinations that closely match production and reduce manual operational tasks by shifting the management of the runners to AWS.

In this blog, I will explore how AWS managed GitHub Action self-hosted runners work by building and deploying an application to AWS using GitHub Actions.

Architecture overview

The architecture of what I’ll be building can be seen below:

The architecture above shows how a developer pushes code changes to GitHub. This triggers CodeBuild to detect the update. CodeBuild then runs the defined GitHub Action Workflow, which builds and deploys it to AWS Lambda.

Step 1. Build a AWS Lambda Function

I’ll start with a simple application to demonstrate how to build and deploy an application on AWS via a Managed Self-Hosted GitHub Actions runner. We’ve written before about why AWS is the best place to run Rust, Amazon CTO Werner Vogels has been an outspoken advocate for exploring energy-efficient programming languages like Rust and AWS have great guides on using Rust to build on AWS such as:

• Building Rust Lambda functions with Cargo Lambda
• Get started with the AWS SDK for Rust

Cargo Lambda is one of the simplest ways to run, build and deploy Rust lambda functions on AWS, I’ll start with the Getting Started guide:

1. Navigate to GitHub.com and create a new GitHub repository
   

   Figure 2 Create a new GitHub repository

2. Clone the repository locally:

   git clone https://github.com/{{user-name}}/rust-api-demo.git

3. From the above cloned repository, install Cargo Lambda:For macOS & Linux:

   brew tap cargo-lambda/cargo-lambda
   brew install cargo-lambda

   Windows users can follow the guide to see all the ways that you can install Cargo Lambda in your system.

4. Use Cargo lambda to create a new project

   cargo lambda new new-lambda-project && cd new-lambda-project

It’s now possible to explore the project, in this case I am using JetBrains RustRover with Amazon Q Developer installed to increase my productivity while working on the application:

Amazon Q Developer is available on a free tier and provides real-time code suggestions as well as advanced suggestions such as in-built chat to reason with the code we’re working on.

5. Add, Commit & Push the code to your GitHub account:

   git add .
   git commit -m “Initial Commit”
   git push origin

Step 2. Create AWS CodeBuild Project

The AWS Documentation outlines how to set up self-hosted GitHub Actions runners in AWS CodeBuild, the key here is to setup a GitHub webhook event of event type WORKFLOW_JOB_QUEUED so that CodeBuild will only process GitHub Actions workflow jobs.

I will create a new CodeBuild project as per the documentation to connect CodeBuild to our GitHub repository and correctly configure a webhook to trigger the GitHub Actions.

1. Open the AWS CodeBuild console
2. Create a build project.
   • In Source:
     • For Source provider, choose GitHub.
     • For Repository, choose Repository in my GitHub account.
     • For Repository URL, enter https://github.com/user-name/repository-name.
   • In Primary source webhook events:
     • For Webhook – optional, select Rebuild every time a code change is pushed to this repository.
     • For Event type, select WORKFLOW_JOB_QUEUED. Once this is enabled, builds will only be triggered by GitHub Actions workflow jobs events.
       

       Figure 4. WORKFLOW_JOB_QUEUED Event Type

   • In Environment:
     • Choose a supported Environment image and Compute. Note that you have the option to override the image and instance settings by using a label in your GitHub Actions workflow YAML.
   • In Buildspec:
     • Note that your Buildspec will be ignored. Instead, CodeBuild will override it to use commands that will setup the self-hosted runner. This project’s primary responsibility is to set up a self-hosted runner in CodeBuild to run GitHub Actions workflow jobs.
3. Continue with the remaining default options and select Create build project.

CodeBuild Service Role Permissions

In order for the CodeBuild service role to be able to successfully create and deploy a Lambda function, the service role will require the necessary permissions. The Service role can be seen when editing the CodeBuild project:

The required Lambda permissions are documented in the Cargo Lambda documentation:

• lambda:GetFunction
• lambda:CreateFunction
• lambda:UpdateFunctionCode

In addition, there are also IAM permissions required:

• iam:CreateRole
• iam:AttachRolePolicy
• iam:UpdateAssumeRolePolicy
• iam:PassRole

Add the required permissions to the service role for the CodeBuild project:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "iam:CreateRole", "iam:AttachRolePolicy", "iam:UpdateAssumeRolePolicy", "iam:PassRole" ], "Resource": [ "arn:aws:iam::{AWS:Account}:role/AWSLambdaBasicExecutionRole", "arn:aws:iam::{AWS:Account}:role/cargo-lambda-role*" ] }, { "Effect": "Allow", "Action": [ "lambda:CreateFunction", "lambda:UpdateFunctionCode", "lambda:GetFunction" ], "Resource": "arn:aws:lambda::{AWS:Account}:function:{function-name}" } ]
} 

Note that I do not need to manage IAM permissions outside of our AWS Account, for example GitHub does not need to know about our AWS permissions.

Step 3. Create a GitHub Action Workflow

GitHub Actions is a continuous integration and continuous deliver (CI/CD) platform that provides automation through building, testing and deploying applications. In this section we will create a GitHub Action Workflow to build and deploy our Lambda.

1. Navigate back to our GitHub project create a workflow within the .github/workflows directory, the Simple Workflow is a good starting point:
   

   Figure 6. Create a Simple Workflow

2. Update the Job to include the tooling required to build our Rust Lambda function, the details can be found in the GitHub Actions section. Our workflow file should now look like this:

name: rust-api-demo-cicd on: push: branches: [ "main" ] pull_request: branches: [ "main" ] env: CARGO_TERM_COLOR: always jobs: build: runs-on: ubuntu-latest steps: - name: Install Rust toolchain uses: dtolnay/rust-toolchain@stable - name: Install Zig toolchain uses: korandoru/setup-zig@v1 with: zig-version: 0.10.0 - name: Install Cargo Lambda uses: jaxxstorm/[email protected] with: repo: cargo-lambda/cargo-lambda tag: v0.14.0 platform: linux arch: x86_64 # Add your build steps below

The above GitHub Actions Workflow currently runs on GitHub; However, I now want to make two further changes:

• Define an AWS CodeBuild runner
• Define Build and Deploy Lambda steps

Define an AWS CodeBuild runner

A GitHub Actions workflow is made up of one or more jobs, each job runs in a runner environment specified by runs-on. The value for runs-on to specify CodeBuild as a runner takes the format:

runs-on: codebuild-<CodeBuildProjectName>-${{ github.run_id }}-${{ github.run_attempt }}

I will update the <CodeBuildProjectName> to the CodeBuild project name that was entered in Step2, e.g. “GitHubActionsDemo”.

When configuring CodeBuild as a runner environment, BuildSpecs are ignored. In order to define the specification of our build environments it is possible to pass in variables including:

• EC2 compute builds: Image, image version, instance size
• Lambda compute builds: environment type, runtime version, instance size

For further details, see the action runner guide.

Define Build and Deploy Lambda steps

The last change is to add steps to check out our code onto the runner and then build and deploy using cargo lambda:

 - name: Build Rust API uses: actions/checkout@v4 - run: cargo lambda build --release - run: cargo lambda deploy

The final workflow looks like this:

name: rust-api-demo-cicd on: push: branches: [ "main" ] pull_request: branches: [ "main" ] env: CARGO_TERM_COLOR: always jobs: build: runs-on: codebuild-GitHubActionsDemo-${{ github.run_id }}-${{ github.run_attempt }} steps: - name: Install Rust toolchain uses: dtolnay/rust-toolchain@stable - name: Install Zig toolchain uses: korandoru/setup-zig@v1 with: zig-version: 0.10.0 - name: Install Cargo Lambda uses: jaxxstorm/[email protected] with: repo: cargo-lambda/cargo-lambda tag: v0.14.0 platform: linux arch: x86_64 # Add your build steps below - name: Build Rust API uses: actions/checkout@v4 - run: cargo lambda build --release - run: cargo lambda deploy

When I commit changes to the workflow to the main branch it will trigger the GitHub Action.

Step 4. Testing our GitHub Action Workflow.

The GitHub Action is currently triggered on all push and pull requests to main branch:

Note that GitHub is where the CI/CD process is being driven, the build logs are available in GitHub as the job is running:

As the build progresses through the deployment step, the details of the Lambda function deployed are shown:

Navigating back to the AWS Console, the deployed Lambda Function can be seen:

And finally, opening the CodeBuild console, it’s possible to observe the status of the Managed GitHub Actions Runner, the build number and also the duration:

Clean Up

To avoid incurring future charges:

1. Delete the Lambda created via the deployment in Step 4.
2. Delete the CodeBuild Project created in Step 2.

Conclusion

As I’ve shown in this blog, setting up GitHub Actions Workflows that run on AWS is now even easier to allow CodeBuild projects to receive GitHub Actions workflow job events and run them on CodeBuild ephemeral hosts. AWS customers can take advantage of natively integrating with AWS and providing security and convenience through features such as defining service role permissions with AWS IAM or passing credentials as environment variables to build jobs with AWS Secrets Manager.

Being able to use CodeBuild’s reserved capacity allows you to provision a fleet of CodeBuild hosts that persist your build environment. These hosts remain available to receive subsequent build requests, which reduces build start-up latencies but also make it possible to compile your software within your VPC and access resources such as Amazon Relational Database Service, Amazon ElastiCache, or any service endpoints that are only reachable from within a specific VPC.

CodeBuild-hosted GitHub Actions runners are supported in all CodeBuild regions and customers managing their CI/CD processes via GitHub Actions can use the compute platforms CodeBuild offers, including Lambda, Windows, Linux, Linux GPU-enhanced and Arm-based instances powered by AWS Graviton Processors.

Read more in our documentation for GitHub Action runner in AWS CodeBuild.

About the author: