In the ever-evolving landscape of software development, the need for robust and efficient continuous integration (CI) and continuous delivery (CD) pipelines has become paramount. Self-hosted runners and agents play a crucial role in this context, particularly when using platforms like GitHub and Azure DevOps. These self-hosted solutions offer a unique blend of flexibility and control, allowing organizations to tailor their CI/CD workflows to meet specific requirements. Whether it’s handling specialized build environments, integrating with proprietary tools, or managing complex dependencies, self-hosted runners provide the customization needed to streamline development processes and enhance productivity.

Security and compliance are other critical aspects where self-hosted runners and agents shine. By hosting runners on their own infrastructure, organizations can enforce stringent security measures and ensure compliance with industry standards and regulations. This level of control is especially important for enterprises dealing with sensitive data or operating in highly regulated sectors. Self-hosted runners allow for the implementation of advanced security protocols, such as network isolation, access controls, and encryption, thereby mitigating risks associated with third-party hosted solutions.

A new Azure verified pattern module has been published which helps you to easily deploy self-hosted runners and agents using infrastructure-as-code. Capabilities of this module are:

• Supports Azure Container Apps with auto scaling from zero
• Supports Azure Container Instances
• Supports Public or Private Networking
• Uses managed identity authentication
• Deploys all Azure resources required for the end to end solution or optionally supply your own resources

Module usage

Let’s explore how using this module will look like. In this scenario, I will use this module to deploy GitHub self-hosted runners on Azure Container Apps using public networking.

GitHub Actions self-hosted runners

Bicep code

module githubRunner 'br/public:avm/ptn/dev-ops/cicd-agents-and-runners:0.1.0' = {
  name: 'githubRunner'
  params: {
    computeTypes: [
      'azure-container-app'
    ]
    namingPrefix: 'gh'
    networkingConfiguration: {
      addressSpace: '10.0.0.0/16'
      networkType: 'createNew'
      virtualNetworkName: 'vnet001'
    }
    selfHostedConfig: {
      githubOrganization: 'sebassem'
      githubRepository: 'private-runners'
      personalAccessToken: '*************************'
      selfHostedType: 'github'
    }
    privateNetworking: false
  }
}

After the deployment is complete, this is what gets deployed.

Now, let’s configure a sample GitHub action to utilize the self-hosted runners we deployed.

When running it, we can see the action using our self-hosted runners on top of Azure Container apps.

Looking at the Azure Container App, we can see that its configured to scale down to zero to save costs when there are no actions triggered.

Azure DevOps self-hosted runners

Now let’s try to deploy Azure DevOps on Azure Container Instances with private networking this time.

Bicep code

module azureDevOpsAgent 'br/public:avm/ptn/dev-ops/cicd-agents-and-runners:0.1.0' = {
  name: 'azureDevOps'
  params: {
    computeTypes: [
      'azure-container-instance'
    ]
    namingPrefix: 'dev'
    networkingConfiguration:  {
      addressSpace: '10.0.0.0/16'
      networkType: 'createNew'
      virtualNetworkName: 'vnet002'
    }
    selfHostedConfig: {
      agentsPoolName: 'aci-pool'
      devOpsOrganization: 'sebassem0787'
      personalAccessToken: '*********************'
      selfHostedType: 'azuredevops'
    }
    privateNetworking: true
  }
}

After the deployment is complete, this is what gets deployed. This time we can see additional resources getting deployed as we chose to deploy with private networking like private endpoints, private dns zones and a nat gateway.

Now, let’s configure a sample Azure DevOps pipeline to utilize the self-hosted runners we deployed. First, we need to create an agent pool named aci-pool.

The, we need to edit the pipeline yaml to utilize that pool.

When running it, we can see the pipeline using our self-hosted agent on top of Azure Container Instances this time.

Looking at the Azure Container Instance deployed, we can see that indeed a container was spun up, ran and then got terminated.

And finally looking at the agent pool settings, we can see the self-hosted agent in our configuration and the pipeline we ran as a job on that agent.

Resources

• Bicep Azure Verified module for self-hosted runners and agents
• Terraform Azure Verified module for self-hosted runners and agents