AWS CloudFront Setup

These instructions are only supported by packagecloud:enterprise 2.0.7 or later.

After June 24, 2019, you MUST use CloudFront in order for packagecloud:enterprise to continue working properly. See this page for details.

Overview

To better understand how CloudFront, S3 and Lambda work together to serve packages and indexes, let’s examine how packagecloud:enterprise works with and without CloudFront enabled.

Without CloudFront enabled:

Package Manager makes request to packagecloud, after request is authenticated, a redirect is issued using a SigV2 Signed URL to requested object on S3.
Package Manager follows redirect to S3 and retrieves requested object.

With CloudFront enabled:

Package Manager makes request to packagecloud, after request is authenticated, a redirect is issued using a custom signed URL to requested object on CloudFront.
Package Manager follows redirect to CloudFront.
CloudFront internally triggers a Lambda@Edge function that validates the custom signature in the URL.
If the signature is valid and not expired, then CloudFront requests/caches object from S3 using a SigV4 Signed URL and returns it to the Package Manager.

Required AWS Permissions

The automated setup tasks use the packagecloud_rails['aws_access_key'] and packagecloud_rails['aws_secret_key'] to perform several actions on your AWS account. Make sure that these AWS credentials have the appropriate permissions to do the following:

Create an IAM role and policy.
Create a KMS key and policy.
Create/Modify Lambda Functions.
Create CloudFront Origin Access Identity.
Attach Bucket Policy to S3 bucket.
Create/Modify CloudFront Distributions.

Instructions for enabling CloudFront:

If you are installing packagecloud:enterprise for the first time (and not performing an upgrade of an existing installation) you can skip to the next step. If you are upgrading an existing installation, begin by running packagecloud-ctl reindex-everything in a screen session. This will re-index all repositories and may take some time. This step is required because new attributes will be set on all objects written to S3 so that CloudFront will cache objects correctly. Skipping this step will result in caching issues once CloudFront is enabled.
Run packagecloud-ctl generate-cloudfront-secret and add generated secret to /etc/packagecloud/packagecloud.rb as instructed.
Run packagecloud-ctl reconfigure
Run packagecloud-ctl bootstrap-cloudfront and then run the generated command as instructed (see next step for an example).
Run the generated command from step 4, which looks like packagecloud-ctl reconfigure-cloudfront-distribution arn:aws:lambda:us-east-1:999999999999:function:packagecloud_cloudfront_auth:1' and add configuration values to /etc/packagecloud/packagecloud.rb as instructed.
Run packagecloud-ctl reconfigure
Run packagecloud-ctl restart

CloudFront should now be enabled!

If you run into any issues with the above commands or package downloads do not seem to work, please disable CloudFront (see instructions below) and email us at support@packagecloud.io.

Instructions for disabling CloudFront

Set packagecloud_rails['cloudfront_enabled'] = true to false
Run packagecloud-ctl reconfigure
Run packagecloud-ctl restart

Configuration Reference

Run the commands detailed in the instructions above to get the values for these fields.

        packagecloud_rails['cloudfront_enabled'] = true
        packagecloud_rails['cloudfront_secret'] = 'MDI2ZDhhYmMxYthiMjVmMTBlODM4OWNmZWZiZmI2NWI'
        packagecloud_rails['cloudfront_distribution_id'] = 'F1CA7GXW3ED1U0'
        packagecloud_rails['cloudfront_packages_domain'] = 'f3rj8yxjhizdxw.cloudfront.net'

Cost Allocation Tags

You can monitor the costs associated with CloudFront by using the packagecloud:enterprise AWS tag.

FAQ

Why can’t packagecloud:enterprise just use SigV4 URLs directly, without CloudFront/Lambda?

Due to the way the S3 API works, SigV4 URLs are not suitable for direct use with Package Managers.

I’m not comfortable letting packagecloud:enterprise setup the required AWS resources for me, could I set this up manually or use something like Terraform instead?

The code that creates and configures the required AWS resources is very careful to only make changes to packagecloud-managed resources and never destroys anything, under any circumstances. Also, due to the number of interdependent resources that need be configured, manual setup would be error-prone and cumbersome to document. If you’d like to use Terraform or CloudFormation, send us an email at support@packagecloud.io and we’ll consider creating special instructions for these cases.

Are packages cached inside of CloudFront? What about repository metadata (indexes)?

When CloudFront is enabled, all requests for package files or repository metadata will go through CloudFront. Package files are cached, but repository metadata is written to AWS S3 with a special Cache-Control header so that CloudFront will never cache it. Repository metadata is never cached when it passes through CloudFront to ensure package managers always see the latest generated indexes.