View Source

Introduction

Chef is a configuration management tool (like Puppet, Ansible, SaltStack, CFEngine). It is a tool which manages the configuration of centrally managed Linux servers, compute clusters, and desktops at SLAC. Examples of configuration items Chef manages include: sudo privileges, login access privileges, logging, software repositories, cronjobs, baseline security configuration. Chef is the configuration management tool for CentOS 7 and later, Red Hat Enterprise Linux (RHEL) 7 and later, and Ubuntu 16.04 and later. Operating systems earlier than those (RHEL 5 and 6, Solaris) are centrally managed using Taylor (a locally written configuration management tool).

Chef Architecture

Chef Server - the Chef Server has the cookbook code which is run on each client, and information about each client manged by Chef which is updated after every chef-client run. Each host which runs chef contacts the Chef Server before the client run to get any updated code, and after each client run, to report success or failure, and provide a full report of the updated status based on the most recent run.
Chef Client - there is a chef-client cronjob on each host managed by Chef which runs the chef-client code on a routine basis.
Chef Automate Server - the Chef Automate server is an Operations Dashboard which provides details about everything managed by Chef, including status and error messages. The chef-client run on each host sends information to the Chef Automate Server, so the various dashboards provide OCIO with the current state of all centrally managed hosts.
GitHub Organization - the central repository for Chef code is maintained in a SLAC OCIO GitHub organization: https://github.com/SLAC-CHEF/
Jenkins Server - all Chef code goes through a Continuous Integration / Continuous Delivery Pipeline before being automatically delivered to the Chef Server. Automated tests using Test Kitchen are run on Vagrant VMs which are spun up by Jenkins. If all tests pass, and an OCIO system administrator presses the "Proceed" button, then updated code is delivered to the Chef Server.

Chef log files and directories

Chef-client logs are sent to syslog and a local log file. You can view the logs using these methods on each host which is managed:

sudo less -r /var/log/chef/client.log
sudo journalctl -t chef-client
sudo grep -w chef-client /var/log/everything

The logs are also sent to the central syslog server, and to Splunk. On the central syslog server, the log can be viewed here:

ssh loghost
grep -w chef-client /u2/today/SYSLOG/daemon

/var/chef/cache/cookbooks/	This directory contains the cookbooks downloaded from the chef server.
/var/chef/cache/backup/	This directory contains backup files of any changes made by chef.
/root/knife-node-show	This script will show configuration details for the current host.
/afs/slac/g/scs/systems/report/chef/system.info/	This directory contains information about each host managed by chef