Secure Docker networking in AWS/AZURE

Secure Docker networking in AWS/AZURE

Today I had to add another security brick to the project I am working on – limit access from Docker containers to sensitive resources:

  • block access to AWS and Azure metadata service (block this IP 169.254.169.254)
  • do rate limiting on the number of outgoing emails from Docker containers

Docker networking can be blocked using iptables rules on Linux. Docker community build a special iptables chain to be used for user rules. It is called DOCKER-USER. So I am going to add rules it this chain.

Block access to metadata service

Metadata service has a lot of sensitive information. It contains machine name, real IP address and can contain security certificates. For example, you can get certificate details running the following command Docker on AWS cloud:

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/

token-name

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/token-name

{
"Code" : "Success",
"LastUpdated" : "2019-06-17T07:19:39Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "ASIA3KRQKBXXXXXXX",
"SecretAccessKey" : "p4JrGOtnHXXXXXXXXXXXXXXXXX",
"Token" : "AgoJb3JpZ2XXXXXXXXXXXXXXXXXXX",
"Expiration" : "2019-06-17T13:54:22Z"
}

This information is very sensitive. In case you do not fully trust code running in Dockers you need to block access from Docker to this information. I am using iptables rules for that. I use the following command:

iptables --insert DOCKER-USER 1 --destination 169.254.169.254 --jump REJECT

Rate limiting on outgoing emails

It is possible, that not-trusted code will try to send out spam emails from Docker containers. I do not want to block emails completely as some emails can be fully legitimate. I imply the rate-limiting mechanism to limit the number of outgoing emails per minute. I set it to 5 per minute, but it can be any number. I am blocking access to the following TCP ports used by popular email software (25,465,587).

At the beginning I used the following rule:

iptables --insert DOCKER-USER 1 -p tcp --syn --match multiport --dports 25,465,587 -m limit --limit 5/minute --jump RETURN

But, what about the email requests that fall above the rate limit? One rule in iptables can only have one action: –jump ACCEPT. In iptables, we can not do something –on-failue REJECT. For that, we have to add another rule coming afterwards. Rules in iptables are sequential, on match in a list, the rule does what the action says, RETURN – returns back to the previous chain, REJECT – packet reject, etc… I use the following rule to block TCP syn packets that are above the rate limit.

iptables --insert DOCKER-USER 2 -p tcp --syn --match multiport --dports 25,465,587 --jump DROP

Final commands

So, to sum up, I use the following commands:

iptables --insert DOCKER-USER 1 -p tcp --syn --match multiport --dports 25,465,587 -m limit --limit 5/minute --jump RETURN
iptables --insert DOCKER-USER 2 -p tcp --syn --match multiport --dports 25,465,587 --jump DROP
iptables --insert DOCKER-USER 3 --destination 169.254.169.254 --jump REJECT

Troubleshooting

You can use the following command that is very handly:

These are my rules in the chain:

Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:0x17/0x02 multiport dports 25,465,587 limit: avg 5/min burst 5
0 0 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:0x17/0x02 multiport dports 25,465,587
0 0 REJECT all -- * * 0.0.0.0/0 169.254.169.254 reject-with icmp-port-unreachable
13881 15M RETURN all -- * * 0.0.0.0/0 0.0.0.0/0

About the author

Yuli Stremovsky
Yuli StremovskyParanoid Security Guy
For the past 15 years I’ve been leading the evolution of startups and enterprises to achieve the highest level of security and compliance. Throughout my career I’ve been a Cyber Security expert and advanced solutions architect with many years of hands on experience both on offensive and defensive side. Knowledgeable at the highest level in application development, networking, data and databases, web applications, large scale Software as a Service solutions, cloud security and blockchain technologies.

I’ve been working with CISO’s of international enterprises, helping them set Information Security strategy, and overseeing the implementation of these recommendations. As part of these projects, I’ve been assisting companies to achieve compliance in GDPR, PCI, HIPAA and SOX.

Among my credits, I was a founder of a database security company GreenSQL/Hexatier which was acquired by Huawei and I’ve co-founded Kesem.io, Secure multi-signature Crypto wallet.

Specialties: Software and cloud architecture, Compliance (GDPR, HIPAA, PCI, SOX), blockchain technologies, software development, secure architectures, project management and low level research.