Auto Releasing Sous-Chefs Cookbooks
Sous-Chefs is a community funded organisation and we like to think of ourselves as a home for unloved great cookbooks. We are also a place where these cookbooks will always get maintained moving forward.
While these cookbooks are great, when we typically take over management of the cookbook repos they are not in the best of states, and there is significant work to bring them up to our standards.
We also have all our existing cookbooks (of which there are over 70) and as our standards change, we need to deploy these changes across all of our repositories.
Earlier in the year we built the Xorimabot backend suite
We now have cookbooks which are constantly getting updated but moved onto the next problem, releasing these is a manual process which takes time from our various contributors (Which normally is Ramereth).
Here we truly believe all members of the community should be able to release cookbooks as long as the normal checks (CI, Review) are passed so we did what any SRE team would do, we built automation.
We started to partition the problem into smaller parts, below you can see the architecture we came up with in the end. These applications currently run on Kubernetes, the aim is to migrate them to AWS Lambda simply due to the costs of running this solution. If you would like to aid us with covering the costs of Sous-Chefs donations would be welcome in our Open Collective.
In this architecture we have applications that cover the various problems we foresaw, these are covered in more detail below. These applications uses GitHub's webhooks and APIs to run the applications automatically, we also enforce HMAC secrets for additional security.
Label Validator is designed to ensure that we have a single
Release: label on the pull request. The allowed labels are:
- Release: Major
- Release: Minor
- Release: Patch
These labels are important as other applications use them later on to determine the level of change we should be doing on the release. The results of this check are published back as a status check against the PR, allowing us to enforce this check must pass in order to proceed.
Changelog Validator is designed to ensure that we have an
## Unreleased heading in the changelog. This is where contributors should be putting their change notes. The changes to this file are enforced by Danger.
## Unreleased section is used in later compontents (Cookbook Release Creator) to identify where in the changelog we should be updating. The results of this check are published back as a status check against the pull request, allowing us to enforce this check must pass in order to proceed.
aka: Cookbook Release Validator
Metadata Validator is designed to ensure that the version number in the changelog is the same as the default branch. We manage this version with the Cookbook Release Creator application. The reason we check the individual line is because other items in the metadata may be changed during a pull request.
The results of this check are published back as a status check against the pull request, allowing us to enforce this check must pass in order to proceed.
Cookbook Release Creator
Cookbook Release Creator runs when a pull request is merged into the default branch. It then does a number of automated items for us:
- Set the new
metadata.rbversion based on the
- Updates the changelog with a release version heading based on the same label.
- Creates a Github Release on the repository, this will allow users to follow releases in Github for cookbooks they care about
- A link to the release is also put as a comment on the merged PR
- Creates a Github Deployment request via the GitHub API
- This deployment holds the release notes we will need for the Slack notifier later on in the custom payload attribute
Cookbook Supermarket Uploader
Cookbook Supermarket Uploader runs when a deployment is requested. It does the following:
- Checks the Deployment was created by the same user it is running as
- Clones the git code based on the tag which is pushed when a release is created
- Runs a
knife supermarket shareto upload this to the chef supermarket
- Posts back a sucessful or failed deployment to GitHub's API
Deployment Status Slack Notifier runs when a Deployment Status is updated to Success of Failure. It then route the message to the Chef Community Slack. It gets the release notes from the payload of the deployment and posts them to
Future of the Automation and Sous-Chefs
These applications are already critical to our management infrastructure, and we have many plans for their future.
- Migrate these applications to AWS Lambda to reduce costs
- Add a notification to the slack discorse
- Create a dashboard, because metrics
As always we are a group of volunteers and our doors are always open. If you would like to help with any of our projects or want to get involved in some of the cookbooks we help to maintain, you can find us on the Chef Community Slack the the