The Missing Half - Network Security and Squid

Have you ever needed to block connections to certain URLs, but wanted to allow all others? Or possibly the reverse, where only certain traffic should be allowed? This is generally referred to as URL filtering. While it’s universal for networks to have a firewall protecting and managing various incoming connection attempts, far fewer networks have a way to control traffic going out. This is often the overlooked half of network security. Too often an outbound connection attempt is allowed by default - that’s clear when you think of phishing links in emails, or similarly spelled domains posing as the real thing. But there are barriers to filtering too, as this kind of security is usually handled by dedicated or expensive appliances. So, did you know there’s a feature rich alternative you may already be familiar with? Did I also mention it’s free? Let’s check out Squid! (aka Squid caching proxy)

My goal here is to give you:

• An initial understanding of Squid’s features and capabilities

• A configuration for URL filtering, including no-cert SSL inspection

• Some ideas around automated deployment in the cloud

With this knowledge you can filter traffic from the smallest of home networks to the largest enterprise environment.

Old Squid, New Tricks

Squid’s filtering capabilities are only as complicated – or as simple – as you want them to be. It can easily filter based on a list of URL addresses you provide. That could be a list of “only allowed”, or the reverse with a list of specific denials. It can also handle SSL traffic, regex URL matching, ordered rule evaluation, filtering based on port or protocol, and it can even apply specific filtering lists based on the source traffic’s MAC address or IP. With all of this you could imagine specific machines or even whole network segments falling under a set of filtering rules, and a different set of machines or segments guarded by others. Really though, the only limit is how fine-grained you feel like getting with your rules.

Avoiding Certifiable Behavior

I mentioned filtering encrypted SSL traffic, and this usually means generating a self-signed SSL certificate to “man-in-the-middle” your own traffic. Not so with Squid though. Traditionally, the idea is to decrypt the traffic “in the middle” between a source and the destination site for inspection. What that really means in practice is someone now has to create, manage, and replace certificates all over the place. There will be insecure connection warnings in consoles and browsers. An entire ongoing workstream dedicated to loading and updating certificates on every machine new and old. Even with all of that time and effort, it’s still common to have issues updating or installing various tools and software packages. Countless wasted hours and costs.

Well, now for the good news, Squid can filter SSL traffic without all that certificate mess. It does this through a series of options it calls “peek”, “splice”, and “bump”. Long story short, Squid can peek in at the traffic’s destination URL during the first few steps in the SSL handshake process. From that little peek at the URL, it can then follow rules to either shut the connection down and send a “forbidden” error code, or to leave it alone and allow the handshake to complete naturally and without issue. No break in the encryption, no exposed information beyond URL, no ongoing certificate management. Frictionless security is great security.

Squid Anatomy 101

Now that you know a little about Squid’s filtering abilities, let’s look at some code. This script would run during the startup process on a Google GCE or AWS EC2 instance. It sets up allowed ports on the OS firewall, installs Squid, and puts down a general Squid configuration that you can adapt for your needs.

#!/bin/bash 
# Update yum or apt-get right off the bat.
yum update || apt-get update

## These rules open ports 80 and 443 for transparent proxying/filtering.
firewall-cmd --add-port=80/tcp --permanent
firewall-cmd --add-port=443/tcp --permanent

## Install squid and ensure some file permissions are correct
yum install -y squid || apt-get -y install squid
chmod 644 /etc/squid/squid.conf
chmod 644 /var/log/squid/cache.log

## Create a placeholder certificate pem file. Alternatively add a line to echo in
## contents for a real file or move one into place if needed.
mkdir /etc/squid/ssl/
touch /etc/squid/ssl/squid.pem

## Checking for content - if something was imported we'll keep it.
## It should be blank otherwise and we'll generate a self-sign.

if [[ $(< /etc/squid/ssl/squid.pem) == "" ]]; then
yum install -y openssl || apt-get -y install openssl
## The ssl_db directory is used during squid's check of outbound SSL SNI addresses.
/usr/lib64/squid/security_file_certgen -c -s /var/spool/squid/ssl_db -M 20MB
cd /etc/squid/ssl
openssl genrsa -out squid.key 2048
openssl req -new -key squid.key -out squid.csr -subj "/C=XX/ST=XX/L=squid/O=squid/CN=squid"
openssl x509 -req -days 365 -in squid.csr -signkey squid.key -out squid.crt
cat squid.key squid.crt | tee squid.pem
fi

## Some basic source and destination lists. Giving 2 of each in this example.
## These are of course blank right now. Plenty of ways to create these and even
## populate them depending on your needs - consider using Terraform variables
## e.g. %{ for each in filter[group].source_ips ~}
## echo "${each}" >> /etc/squid/${group}.srcip
## %{ endfor ~}

## lists of source machines, IP addresses or MAC addresses.
touch /etc/squid/sources1.srcip
touch /etc/squid/sources2.srcip
## lists of destination URLs, can be IP addresses, URLs, or regular expressions
## depends on which ACL element you define. Look up "Squid ACL" for more information.
touch /etc/squid/sources1.dest
touch /etc/squid/sources2.dest

############
## Squid configuration sequence
############

cat > /etc/squid/squid.conf <<EOF  acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network  acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines  acl Safe_ports port 80 # http acl Safe_ports port 443 # https acl Safe_ports port 1025-65535 # unregistered ports  acl SSL_port port 443 # https http_access allow SSL_port  acl CONNECT method CONNECT  ## Example configuration for a simple default whitelist of URLs by domain # acl whitelist dstdomain "/etc/whitelist.acl" # or to use regular expressions, consider: # acl whitelist url_regex "/etc/whitelist.acl"  ## Example configuration for a simple default blacklist of URLs by domain # acl blacklist dstdomain "/etc/blacklist.acl" # or to use regular expressions, consider: # acl blacklist url_regex "/etc/blacklist.acl"  ## Blacklist counter rule when url not ALSO on whitelist. ## e.g. When URL is present on both lists, whitelist overrules blacklist. # http_access deny blacklist !whitelist # Read as "deny http access when an address is on the blacklist and not on the whitelist"  ## Deny requests to certain unsafe ports http_access deny !Safe_ports  ## Only allow cachemgr access from localhost http_access allow localhost manager http_access deny manager  ## Rule allowing access from localhost. http_access allow localhost  ## HTTP filtering port (normally port 80). http_port 80  ## Setting up the squid ACLs and giving them names acl sources1src src "/etc/squid/sources1.srcip" acl sources1dest dstdomain "/etc/squid/sources1.dest" acl sources2src src "/etc/squid/sources2.srcip" acl sources2dest dstdomain "/etc/squid/sources2.dest"  ## The access rule to filter from source file to destination file for each set http_access allow sources1dest sources1src http_access allow sources2dest sources2src  ## The following configures for SSL filtering at 443 http_port 443 cert=/etc/squid/ssl/squid.pem ssl-bump  ## Defining ACLs pointing to the source & destination files for ${set} acl allowed_https_sources1 ssl::server_name "/etc/squid/sources1.dest" acl allowed_https_sources2 ssl::server_name "/etc/squid/sources2.dest"  acl step1 at_step SslBump1 acl step2 at_step SslBump2 acl step3 at_step SslBump3 ssl_bump peek step1 all  ssl_bump peek step2 allowed_https_sources1 ssl_bump peek step2 allowed_https_sources2  ssl_bump splice step3 allowed_https_sources1 ssl_bump splice step3 allowed_https_sources2  ssl_bump terminate step2 all  ## Lastly, deny everything that has not yet been covered by the previous rules. http_access deny all  coredump_dir /var/spool/squid ## # Add any of your own refresh_pattern entries above these. ## refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 EOF

# Reload the squid configuration OR restart squid if that fails OR try another restart method if that fails.

/usr/sbin/squid -k reconfigure || systemctl restart squid || service squid restart

Faster, Squidier, More Productive

I’ve greatly reduced the above example, removing lots of for loops that would be used to programmatically build out the Squid configuration based on inputs from Terraform. The example is much more of a one-off style than the automated deployments that can be made. For example, I have built managed groups of these Squid filters, with auto-scaling and health checks, fed by load-balancers with sticky sessions, and all deployed from a single repository of code with only the TFVars or other inputs varying. This allows for the deployment of highly customized and automated transparent URL filter clusters and in as many VPCs as necessary. This also reduces the number of standing instances that are necessary at any given time to reduce costs, and allows you to rotate instances frequently for image patching. Likewise, if you need to update an allow list? Re-run the build with modified source or destination variables and away you go! Code is always able to be reviewed in this way, and there are no questions about exactly what the state of the lists are at any given time. Simple and repeatable, awesome.

Conclusion

Squid is impressive. It’s free, it’s feature rich, and it’s certainly capable of decreasing your risk for next to no cost. It can make management of filtering rules easier when automated, and give insight into what was allowed, by whom, and when. If you’re not already filtering your outbound traffic, strongly consider this as a solution.

ScaleSec can help guide your team with implementation, or simply make the whole process turn-key. It’s always your choice. We specialize in helping everyone secure their cloud from small businesses to global enterprises. And as always, if you have any questions feel free to email me and I’ll do my best to help you out.

Cheers,
-John