Hack to set up SSL certificate in AWS Marketplace CloudFormation template
No comments devops

Hack to set up SSL certificate in AWS Marketplace CloudFormation template

Posted by

In my spare time, I am working on my own product – it is a special GDPR compliant storage service to save Personal Identification Information (in short PII) in a compliant way. The standard version of the product is available as open-source: https://databunker.org/

As most of the guys are building products for AWS these days, I want my commercial version to be available for the AWS marketplace.

In this article, I am going to show you a way to deploy a standard (not a self-signed) certificate for your product web management or for your API service deployed using CloudFormation template in AWS. This is going to be useful for most of the companies building products for AWS marketplace.

Before going into technical details, I will tell the whole story.

Originally, I planned, that my potential client will launch my product from AWS marketplace. The product and all required infrastructure will be installed using the CloudFormation template. This was my original plan and I started to work on it. Finally, after probably 50 iterations and few sleepless nights, I was able to deploy:

  1. Network Load balancer
  2. Autoscaling group
  3. Instance profile
  4. IAM Role for the VM
  5. Launch configuration and UserData script for the VM
  6. Aurora PSQL Database cluster
  7. Multiple security groups

One thing was missing from my plan. As I am developing a security product, SSL certificate is a MUST for my service.

So, I started to research the way deploy SSL certificates on my product deployed at the customer’ AWS cloud.

I expect an easy solution. After running my CloudFormation template, Amazon AWS created a network load balancer object with the following name: databunker-123456.elb.eu-west.amazonaws.com . I expect, that I will call the AWS Certificate Manager and it will simply give me a certificate for this subdomain.

It turned out it is not supported. Probably because of some security precautions. Amazon AWS will not generate an SSL certificate for my sub-domain. AWS CertificateManager can generate SSL certificates for the domain that my customer has configured or setup self-signed certificates.

But what will happen, if my customer has no custom domain enabled or his domain is configured in his another AWS account?

In my CloudFormation template, I can not expect that my customer has enabled his domain and it is available in his Certificate Manager.

I had to find an alternative solution.

I thought I got to the dead end. I even contacted AWS Marketplace support to get their opinion.

Finally, I figured what to do it by myself. In short, I am using one of my domains as a top domain; using an API provided by the domain registrar, I create a subdomain (under my top domain) for my customer; I configure DNS alias (CNAME) for this subdomain to point to the DNS domain generated by AWS; the setup script (on my client VM) generates SSL certificates for the new domain, without sending certificate signing request to my server; it got verified and the certificate is installed.

The whole process is automatic, I am using the API service provided by the domain registry.

In my solution, I am using the Let’s Encrypt free SSL certificate generation service: https://letsencrypt.org/

It works as following:

  1. I install acme.sh script at Launch Configuration UserData script
  2. Out of the load balancer fqdn domain provided by Amazon, the script generated a new fqdn domain name (using my domain as top domain).
  3. It runs the acme.sh script with the new fqdn domain and generates a certificate signing request and domain validation token (I am using DNS validation method).
  4. The script calls my backend server with the AWS network load balancer domain and DNS verification token received from acme.sh script.
  5. My backend service creates a new subdomain DNS record with the CNAME record pointing to the AWS network load balancer domain and creates _acme-challenge subdomain TXT record with the validation token.
  6. Meantime, on the UserData scripts waits for the TXT record to be available.
  7. It calls again ascme.sh to finish certificate generation.

In this method, I am not sending the client certificate to my backend server. It is only available in my client’s host.

Here is a flow diagram:

Hack to set up SSL certificate in AWS Marketplace CloudFormation template

My UserData setup.sh script

HOMEDIR="/home/ubuntu"
curl -s https://raw.githubusercontent.com/acmesh-official/acme.sh/master/acme.sh -o $HOMEDIR/acme.sh
chmod 755 $HOMEDIR/acme.sh
FINALDOMAIN='databunker-'$(echo -n $AWSDOMAIN | md5sum | tr -d " -")'.n0pe.com'
sudo -u ubuntu $HOMEDIR/acme.sh --issue --log --no-color -d $FINALDOMAIN --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please --force
SSLTOKEN=`awk '/TXT value:/{print $9}' $HOMEDIR/.acme.sh/acme.sh.log | tr -d "'"`
curl -X POST -d '{"clientdomain":"'$AWSDOMAIN'","ssltoken":"'$SSLTOKEN'"}' https://MY-BACKEND-SERVER

echo "validate for DNS challenge records to be ready"
DNSRECORD=`dig -t txt +short _acme-challenge.$FINALDOMAIN | tr -d '"'`
while [[ "$DNSRECORD" != "$SSLTOKEN" ]]; do
  sleep 5
  echo "validate for DNS challenge records to be ready"
  DNSRECORD=`dig -t txt +short _acme-challenge.$FINALDOMAIN | tr -d '"'`
done

echo "finish certificate"
sudo -u ubuntu $HOMEDIR/acme.sh --renew -d $FINALDOMAIN --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please --force

My Lunch Template UserData has the following code:

"export AWSDOMAIN='", { "Fn::GetAtt" : [ "DatabunkerLoadBalancer", "DNSName" ]}, "'\n",
"/bin/bash /home/ubuntu/setup.sh\n",

My DNS backend (partial code only here, I am using name.com DNS service). Most of the DNS registrars provide an API service.

def _generate_subdomain(clientdomain):
        return 'databunker-' + hashlib.md5(clientdomain.encode("utf-8")).hexdigest()

def _create_cname_record(clientdomain):
        subdomain = _generate_subdomain(clientdomain)
        headers = {'Content-Type': 'application/json'}
        data = '{"host":"'+subdomain+'","type":"CNAME","ttl":300,"answer":"'+clientdomain+'"}'
        print("sending: ", data)
        r = requests.post(url=DNS_API_HOST, headers=headers, data=data,
                auth=HTTPBasicAuth(DNS_API_USER, DNS_API_CODE))
        print(r.text)

def _create_txt_record(clientdomain, rndstring):
        txtdomain = '_acme-challenge.' + _generate_subdomain(clientdomain)
        headers = {'Content-Type': 'application/json'}
        data = '{"host":"'+txtdomain+'","type":"TXT","ttl":300,"answer":"'+rndstring+'"}'
        print("sending: ", data)
        r = requests.post(url=DNS_API_HOST, headers=headers, data=data,
                auth=HTTPBasicAuth(DNS_API_USER, DNS_API_CODE))
        print(r.text)

Out of scope here:

  1. Client validation on the backend server is not covered here, to prevent malicious actor to generate hundreds of fake DNS records.
  2. SSL certificate renewal is not covered here.

Final words

With all those great tools build by AWS Amazon, you still need a good DevOps and cloud architect that know to build an optimal solution.