09/14: cdn: Add a basic deployment plan to the README.org

[Chris Marusich]

marusich pushed a commit to branch master
in repository maintenance.

commit b4eee7c69bf25c8bac722ae98d58de740ba6ef1e
Author: Chris Marusich <address@hidden>
Date:   Fri Dec 28 17:38:37 2018 -0800

    cdn: Add a basic deployment plan to the README.org
    
    * cdn/README.org: Discuss the deployment plan.
---
 cdn/README.org | 128 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 127 insertions(+), 1 deletion(-)

diff --git a/cdn/README.org b/cdn/README.org
index e030202..da5a438 100644
--- a/cdn/README.org
+++ b/cdn/README.org
@@ -966,7 +966,7 @@ signatures.
 Currently, we have all the IAM configuration in Terraform config.  That's 
great!
 
 - Integrate the CloudFront distribution with ACM.
-- Figure out how to share the state.  Maybe use the S3 backend?
+- Set up locking with the S3 backend.
 - Package Terraform
 - Package the AWS Provider plugin for Terraform
 - Simplify variable definitions by using .tfvars file?
@@ -1208,3 +1208,129 @@ See the following for details:
 
 
https://docs.aws.amazon.com/cli/latest/reference/cloudfront/create-invalidation.html
 
https://docs.aws.amazon.com/cli/latest/reference/cloudfront/get-invalidation.html
+* Deployment Plan
+** DNS
+For information about how Guix has configured its DNS, please contact
+one of the system administrators or address@hidden
+
+For now, we plan to use ci.guix.info as the entrypoint into the CDN.
+Currently, ci.guix.info points to the berlin build farm.  To integrate
+ci.guix.info with the CDN, we must change ci.guix.info to be a CNAME
+that points to the CloudFront distribution.
+
+** HTTPS (TLS)
+Because ci.guix.info and berlin.guixsd.org currently allow both HTTP
+and HTTPS (i.e., HTTP over TLS), we will do the same for the
+CloudFront distribution.  However, in the future, to protect the
+confidentiality of connections between clients and our build farm (or
+the CDN), we should stop serving requests over HTTP.
+
+In addition, because we want to support HTTPS, we will need to arrange
+for a TLS certificate for ci.guix.info to be used with the CloudFront
+distribution.  For details on how this is done with CloudFront, see:
+
+https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-alternate-domain-names.html
+
+Currently, the server behind berlin.guixsd.org (which currently, like
+ci.guix.info, is a single A record pointing to 141.80.181.40) returns
+a Let's Encrypt certificate with the following two Subject Alternative
+Names:
+
+- berlin.guixsd.org
+- ci.guix.info
+
+You can see this by running:
+
+  echo -n | openssl s_client ci.guix.info:443 | openssl x509 -text
+
+We are not going to re-use this certificate.  Instead, we are going to
+provision a new certificate using Amazon Certificate Manager (ACM).
+We will do this because if we use an ACM certificate, we gain the
+following benefits:
+
+- ACM will automatically rotate our certificate.
+- The ACM CA, like many CAs, is already trusted by Guix's client
+  software and so requires no additional action for clients to use.
+- By using ACM, we do not have to trust Amazon with the
+  berlin.guixsd.org server's private key.
+- Using ACM is a purely additive operation, so rollback is easy.
+- This use of ACM incurs zero additional cost.
+
+On the contrary, if we re-use berlin.guixsd.org's certificate, we must
+accept the following downsides:
+
+- Certificate rotation must be done manually, or via a cron job of
+  some kind that we would have to implement and maintain.  Even if
+  Let's Encrypt makes it easy to rotate the certificate, importing
+  that certificate into AWS and using it with the CloudFront
+  distribution will require additional steps.
+- We must share the berlin.guixsd.org server's private key with AWS.
+  If rolling back also means eliminating the risk introduced by
+  sharing the private key, then rollback will also become more
+  complicated.
+
+We could provision a new Let's Encrypt certificate to make rollback
+easier, but we would still have to import it into Amazon ourselves and
+rotate it manually.  It just makes more sense to use Amazon
+Certificate Manger for this test.  Perhaps in the future, if we decide
+to stick with CloudFront for a long time, we will implement our own
+automatic rotation mechanism so that we can provision and manage our
+own certificate.
+
+Finally, note that to provision a certificate with ACM will require
+access to both the AWS account and also the DNS provider account.
+This is because, as part of the certificate provisioning process, ACM
+requires us to demonstrate domain ownership.  This can be done via
+email verification (in which ACM emails a bunch of admin emails
+associated with the domain) or DNS record verification (in which we
+create a nonce DNS record, specified by ACM, to demonstrate
+ownership).
+
+** Deployment, Validation, and Rollback
+
+Summary of deployment, validation, and rollback plan.
+
+*** Pre-Deployment
+
+Before deploying, make sure the following has been done:
+
+- Fully prepare a working CloudFront distribution using Terraform.
+
+*** Deployment
+
+Deploy as follows:
+
+- Send an email to address@hidden and address@hidden, and notify the
+  #guix chat room on Freenode, to let people know you are beginning.
+- Update DNS so ci.guix.info is a CNAME pointing to the CloudFront
+  distribution's DNS name.
+
+*** Validation
+
+Validate ci.guix.info as follows:
+
+- Using "guix download", download a substitute.  Confirm it succeeds.
+- Using "guix weather", check the weather of ci.guix.info.  Confirm it
+  succeeds and has more than 0% substitutes available.
+- Using "guix build", build something using substitutes.  Confirm that
+  Guix successfully updates substitute information and downloads
+  substitutes.
+- Using IceCat, view the Cuirass web interface.  Confirm it loads and
+  behaves as expected.
+- After 24 hours, check the cache hit rate using the AWS Management
+  Console and confirm that it is greater than 0%.
+
+*** Rollback
+
+Rollback as follows:
+
+- Restore the original DNS record for ci.guix.info.
+- Disable the CloudFront distribution when the sooner of the following
+  two conditions occurs:
+    - Time passes equal to 2x the TTL of ci.guix.info.
+    - Request rate to the CloudFront distribution decreases by at
+      least 90% compared to before the rollback.
+- Repeat validation activities for ci.guix.info.
+- Send an email to address@hidden and address@hidden, and notify the
+  #guix chat room on Freenode, to let people know you have rolled
+  back.