Your CI/CD agent can’t reach the internet. It sits in a locked down subnet, an air gap, or a private lab. Artifacts need to be pulled from a remote registry. Dependencies must be fetched from a public repository. You have a few options. You can punch holes in the firewall and hope for the best. You can mirror every single package manually. Or you can build a custom proxy server for your DevOps pipeline that gives you control, caching, and security.
That third option is what we are going to build today. No fluff. No corporate jargon. Just a practical, working setup that you can deploy in an afternoon.
A custom proxy server for DevOps pipeline gives you three things: a single egress point for audit and security, a local cache that speeds up builds, and the ability to enforce access rules for every external dependency. This guide walks you through building one with Squid, configuring access controls, and integrating it with your CI/CD runners in air gapped or private network environments.
Why Your Pipeline Needs a Dedicated Proxy
When your build agents live inside a private network, every request to the outside world becomes a security decision. A standard corporate proxy might work, but it often introduces latency, authentication headaches, or rate limits that kill your build times. A custom proxy server built specifically for your DevOps pipeline solves these problems.
Here is what you gain:
- Centralized access control. You decide exactly which domains and repositories your agents can reach.
- Caching. Reusable artifacts are stored locally. Your next build does not need to download the same npm package or Docker image again.
- Audit trail. Every outbound request is logged. You can see exactly what your pipeline fetched and when.
- Resilience. If the external registry goes down, your cached copy keeps your pipeline green.
This setup is especially useful for teams working in regulated industries, defense contractors, or any organization that runs air gapped environments.
What You Will Need
Before we start, gather these ingredients:
- A Linux server (Ubuntu 22.04 or 24.04 LTS works well). This can be a VM or a lightweight container.
- Root or sudo access on that server.
- A static IP address for the proxy server that your build agents can reach.
- Access to the external network (even if limited) so the proxy can fetch resources.
If your network is completely air gapped, you will need a separate mechanism to seed the cache. We will cover that later.
Step 1: Install and Configure Squid
Squid is the workhorse for this project. It is mature, well documented, and handles HTTP and HTTPS traffic with ease.
SSH into your proxy server and run the following commands:
sudo apt update
sudo apt install squid -y
Once installed, the main configuration file lives at /etc/squid/squid.conf. We will edit it to match our pipeline needs.
First, backup the original file:
sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.bak
Now open the file with your editor of choice:
sudo nano /etc/squid/squid.conf
We need to define an ACL (Access Control List) for your internal subnet. Add these lines near the top of the file, right after the default ACLs:
acl pipeline_network src 10.0.0.0/8
acl pipeline_network src 172.16.0.0/12
acl pipeline_network src 192.168.0.0/16
Adjust these ranges to match your actual network. If your build agents are on a specific subnet like 10.20.30.0/24, use that instead.
Next, find the line that says http_access allow localnet and add this line right after it:
http_access allow pipeline_network
This tells Squid to allow traffic from your internal pipeline network.
Now configure the cache. Find the cache_dir line. It might look like this:
#cache_dir ufs /var/spool/squid 100 16 256
Uncomment it and adjust the size. For a pipeline cache, 10 GB is a good starting point:
cache_dir ufs /var/spool/squid 10000 16 256
Save the file and restart Squid:
sudo systemctl restart squid
Make sure it is running:
sudo systemctl status squid
Step 2: Configure Your Build Agents
Your CI/CD agents need to know about the proxy. This is typically done through environment variables.
For most tools, you set two variables:
export http_proxy=http://your-proxy-ip:3128
export https_proxy=http://your-proxy-ip:3128
Replace your-proxy-ip with the actual IP of your proxy server. Port 3128 is the default for Squid.
For permanent configuration, add these lines to the agent’s shell profile or to the CI/CD runner’s environment settings. In GitHub Actions, you would use the env context. In Jenkins, you set them in the node configuration or in a pipeline step.
Here is an example for a Docker based build agent:
ENV http_proxy=http://your-proxy-ip:3128
ENV https_proxy=http://your-proxy-ip:3128
If your tools need to bypass the proxy for internal resources, set the no_proxy variable:
export no_proxy="localhost,127.0.0.1,.your-internal-domain.com"
Step 3: Fine Tune Access Rules
Not every external domain should be reachable. You can lock down which destinations your pipeline can talk to.
In the Squid config, add ACLs for allowed destinations:
acl allowed_domains dstdomain .docker.io .pypi.org .npmjs.org .github.com
http_access allow pipeline_network allowed_domains
http_access deny pipeline_network all
This setup allows your agents to reach only the registries and repositories they need. Everything else is blocked.
Be careful with https. Squid can cache HTTPS traffic if you use SSL bumping, but that requires installing a custom CA certificate on your agents. For many teams, the simpler approach is to cache only HTTP traffic and let HTTPS pass through without caching. That is the default behavior.
Step 4: Monitor and Troubleshoot
Your proxy is now live. Watch the logs to see what your pipelines are doing:
sudo tail -f /var/log/squid/access.log
Each line shows the source IP, the destination, and the result code. A TCP_HIT means the content was served from cache. A TCP_MISS means it was fetched fresh.
If you see TCP_DENIED, it means a request was blocked. Check your ACL rules.
Common issues:
- Agents cannot connect. Verify the firewall on the proxy server allows inbound traffic on port 3128.
- HTTPS errors. If a tool fails to validate certificates, you may need to configure SSL bumping or add the proxy’s CA to the agent’s trust store.
- Slow first builds. That is expected. The cache is empty. Over time, it will fill up and speeds will improve.
Common Mistakes and How to Avoid Them
Here is a table that summarizes the most frequent pitfalls and their fixes:
| Mistake | Symptom | Fix |
|---|---|---|
| Wrong ACL order | All traffic gets denied | Place http_access allow lines before http_access deny all |
Missing no_proxy |
Internal services fail | Add your internal domains to the no_proxy variable |
| Firewall closed | Agents time out | Open port 3128 on the proxy server’s firewall |
| No cache directory | Squid fails to start | Ensure the cache directory exists and is writable by the Squid user |
| SSL certificate errors | Builds fail with TLS errors | Install Squid’s CA certificate on agents or disable SSL bumping |
Advanced: Caching Docker Images
Docker pulls are a major source of bandwidth. You can cache Docker images through your proxy, but it requires a bit more work.
Docker uses HTTPS exclusively. To cache Docker layers, you need to use a registry mirror instead of a generic proxy. Tools like docker-registry-mirror or harbor can sit behind your Squid proxy and cache image layers locally.
Alternatively, you can configure Squid with SSL bumping and install the generated CA on all Docker hosts. This approach works but adds complexity. For most pipeline setups, the simpler path is to run a local Docker registry mirror and point your agents to it.
For a step by step on that, check out our guide on optimizing proxy server performance for enterprise networks.
Security Considerations
Your custom proxy server is now a critical piece of infrastructure. Treat it like one.
- Keep the server patched. Run
sudo apt update && sudo apt upgraderegularly. - Restrict SSH access to the proxy server. Use key based authentication only.
- Log everything. Logs are your best friend when something goes wrong. Learn more about why your proxy server logs are a goldmine for network security analytics in 2026.
- Do not expose the proxy to the public internet. It should only be reachable from your internal network.
A proxy that is accessible from the outside is an open relay. Attackers will find it and abuse it. Always bind Squid to your internal interface only.
Testing Your Setup
Before you let your entire pipeline run through the proxy, test it with a single agent.
On the agent, run:
curl -x http://your-proxy-ip:3128 https://google.com
If you get a response, the proxy is working. Next, test a package download:
curl -x http://your-proxy-ip:3128 https://registry.npmjs.org
Check the Squid access log to confirm the request was proxied.
Once you are confident, update your CI/CD runner configuration to use the proxy for all jobs.
Scaling for Larger Teams
If you have dozens or hundreds of build agents, a single Squid instance might become a bottleneck. You can scale horizontally by running multiple proxy instances behind a load balancer. Each instance can share the same cache directory using a network filesystem, but that adds latency. A simpler approach is to run each proxy with its own local cache and accept that cache misses will be more frequent.
For high availability, consider setting up a pair of proxies with a floating VIP. Our article on how to build a secure proxy cluster for high traffic enterprise networks covers that pattern in detail.
Your Next Steps
Building a custom proxy server for your DevOps pipeline is one of those tasks that pays for itself within the first week. Builds get faster. Security teams get visibility. Developers stop hitting rate limits.
Here is your action plan:
- Spin up a Linux VM and install Squid.
- Configure the ACLs for your network.
- Point a single build agent at the proxy and test.
- Roll out the configuration to all agents.
- Monitor the logs and adjust the allowed domains as needed.
You now have a dedicated, secure, and fast proxy that is built for your pipeline. Not a generic corporate proxy. Not a hacky workaround. A real, purpose built solution.
If you run into issues, the Squid documentation is excellent. And if you want to harden your setup further, take a look at 9 proven strategies to harden your proxy server against zero day exploits in 2026.
Go ahead and set it up. Your future self, waiting for a build to finish, will thank you.