About Environments

[edit on GitHub]

An environment is a way to map an organization’s real-life workflow to what can be configured and managed when using Chef Infra. This mapping is accomplished by setting attributes and pinning cookbooks at the environment level. With environments, you can change cookbook configurations depending on the system’s designation. For example, by designating different staging and production environments, you can then define the correct URL of a database server for each environment. Environments also allow organizations to move new cookbook releases from staging to production with confidence by stepping releases through testing environments before entering production.

The _default Environment

Every Chef Infra Server organization must have at least one environment. Each organization starts out with a single _default environment. The _default environment cannot be modified in any way. Nodes, roles, run-lists, cookbooks (and cookbook versions), and attributes specific to an organization can only be associated with a custom environment. Additional environments can be created to reflect each organization’s patterns and workflow. For example, creating production, staging, testing, and development environments.

Environment Attributes

Attributes can be defined in an environment and then used to override the default attributes in a cookbook. When an environment is applied during a Chef Infra Client run, environment attributes are compared to the attributes that are already present on the node. When the environment attributes take precedence over the default attributes, Chef Infra Client applies those new settings and values during a Chef Infra Client run.

Environment attributes can be set to either default attribute level or an override attribute level.

Note

Attributes can be configured in cookbooks (attribute files and recipes), roles, and environments. In addition, Ohai collects attribute data about each node at the start of a Chef Infra Client run. See Attributes for more information about how all of these attributes fit together.

Environment Attribute Types

There are two types of attributes that can be used with environments:

Attribute Type Description
default A default attribute is automatically reset at the start of every Chef Infra Client run and has the lowest attribute precedence. Use default attributes as often as possible in cookbooks.
override An override attribute is automatically reset at the start of every Chef Infra Client run and has a higher attribute precedence than default, force_default, and normal attributes. An override attribute is most often specified in a recipe, but can be specified in an attribute file, for a role, and/or for an environment. A cookbook should be authored so that it uses override attributes only when required.

Attribute Precedence

Environments are one of several locations where attributes can be applied in Chef Infra. It’s important to understand how the precedence level works in order to understand, which attributes will be applied when Chef Infra Client runs.

Attributes are always applied by Chef Infra Client in the following order:

  1. A default attribute located in a cookbook attribute file
  2. A default attribute located in a recipe
  3. A default attribute located in an environment
  4. A default attribute located in a role
  5. A force_default attribute located in a cookbook attribute file
  6. A force_default attribute located in a recipe
  7. A normal attribute located in a cookbook attribute file
  8. A normal attribute located in a recipe
  9. An override attribute located in a cookbook attribute file
  10. An override attribute located in a recipe
  11. An override attribute located in a role
  12. An override attribute located in an environment
  13. A force_override attribute located in a cookbook attribute file
  14. A force_override attribute located in a recipe
  15. An automatic attribute identified by Ohai at the start of a Chef Infra Client run

where the last attribute in the list is the one that is applied to the node.

Note

The attribute precedence order for roles and environments is reversed for default and override attributes. The precedence order for default attributes is environment, then role. The precedence order for override attributes is role, then environment. Applying environment override attributes after role override attributes allows the same role to be used across multiple environments, yet ensuring that values can be set that are specific to each environment (when required). For example, the role for an application server may exist in all environments, yet one environment may use a database server that is different from other environments.

Attribute precedence, viewed from the same perspective as the overview diagram, where the numbers in the diagram match the order of attribute precedence:

_images/overview_chef_attributes_precedence.png

Attribute precedence, when viewed as a table:

_images/overview_chef_attributes_table.png

Cookbook Pinning

Cookbook versions can be pinned in each environment, which allows you to control the rollout of new cookbook releases through successive testing environments before releasing new cookbook versions into production environments. See the environment format examples below for the cookbook pinning syntax.

Environment Formats

Environments may be stored on disk (any in source control) in two formats:

  • As Ruby (i.e. a file that ends with .rb); this format is not available when running Chef Infra Client in local mode
  • As JSON (i.e. a file that ends with .json)

Ruby DSL

Each environment is defined as a Ruby file (i.e. a file that ends with .rb). Each environment file should contain the following domain-specific attributes:

Setting Description
cookbook

A version constraint for a single cookbook. For example:

cookbook 'couchdb', '< 11.0.0'

or:

cookbook 'my_rails_app', '= 1.2.0'

or:

cookbook 'gems', '~> 1.4'
cookbook_versions

A version constraint for a group of cookbooks. For example:

cookbook_versions(
  'couchdb' => '= 11.0.0',
  'my_rails_app' => '~> 1.2.0'
)
default_attributes

Optional. A set of attributes to be applied to all nodes, assuming the node does not already have a value for the attribute. This is useful for setting global defaults that can then be overridden for specific nodes. If more than one role attempts to set a default value for the same attribute, the last role applied is the role to set the attribute value. When nested attributes are present, they are preserved. For example, to specify that a node that has the attribute apache2 should listen on ports 80 and 443 (unless ports are already specified):

default_attributes 'apache2' => { 'listen_ports' => %w(80 443) }
description

A description of the functionality that is covered. For example:

description 'The development environment'
name

A unique name within the organization. Each name must be made up of letters (upper- and lower-case), numbers, underscores, and hyphens: [A-Z][a-z][0-9] and [_-]. Spaces are not allowed. For example:

name 'dev01-24'
override_attributes

Optional. A set of attributes to be applied to all nodes, even if the node already has a value for an attribute. This is useful for ensuring that certain attributes always have specific values. If more than one role attempts to set an override value for the same attribute, the last role applied wins. When nested attributes are present, they are preserved. For example:

override_attributes 'apache2' => { 'max_children' => '50' }

The parameters in a Ruby file are actually Ruby method calls, so parentheses can be used to provide clarity when specifying numerous or deeply-nested attributes. For example:

override_attributes(
  apache2: {
    prefork: { min_spareservers: '5' },
  }
)

or:

override_attributes(
  apache2: {
    prefork: { min_spareservers: '5' },
  },
  tomcat: {
    worker_threads: '100',
  }
)

A Ruby file for each non-default environment must exist in the environments/ subdirectory of the chef-repo. (If the chef-repo does not have this subdirectory, then it should be created.) The complete environment has the following syntax:

name 'environment_name'
description 'environment_description'
cookbook OR cookbook_versions  'cookbook' OR 'cookbook' => 'cookbook_version'
default_attributes 'node' => { 'attribute' => [ 'value', 'value', 'etc.' ] }
override_attributes 'node' => { 'attribute' => [ 'value', 'value', 'etc.' ] }

where both default and override attributes are optional and either a cookbook or cookbook versions (one or more) are specified. For example, an environment named dev that uses the couchdb cookbook (version 11.0.0 or higher) that listens on ports 80 and 443:

name 'dev'
description 'The development environment'
cookbook_versions  'couchdb' => '= 11.0.0'
default_attributes 'apache2' => { 'listen_ports' => %w(80 443) }

Or (using the same scenario) to specify a version constraint for only one cookbook:

cookbook 'couchdb', '= 11.0.0'

More than one cookbook version can be specified:

cookbook_versions({
  'couchdb' => '= 11.0.0',
  'my_rails_app' => '~> 1.2.0'
})

Attributes are optional and can be set at the default and override levels. These will be processed according to attribute precedence. An environment attribute will be applied to all nodes within the environment, except in places where it is overridden by an attribute with higher precedence. For example:

default_attributes 'apache2' => { 'listen_ports' => %w(80 443) }

will have all nodes in the environment (node[:apache2][:listen_ports]) set to '80' and '443' unless they were overridden by an attribute with higher precedence. For example:

override_attributes 'apache2' => { 'listen_ports' => %w(80 443) }

JSON

The JSON format for environments maps directly to the domain-specific Ruby format: the same settings, attributes, and values, and a similar structure and organization, just formatted as JSON. When an environment is defined as JSON the file that contains that data must be defined as a file that ends with .json. For example:

{
  "name": "dev",
  "default_attributes": {
    "apache2": {
      "listen_ports": [
        "80",
        "443"
      ]
    }
  },
  "json_class": "Chef::Environment",
  "description": "",
  "cookbook_versions": {
    "couchdb": "= 11.0.0"
  },
  "chef_type": "environment"
}

The JSON format has two additional settings:

Setting Description
chef_type Always set this to environment. Use this setting for any custom process that consumes environment objects outside of Ruby.
json_class Always set this to Chef::Environment. Chef Infra Client uses this setting to auto-inflate an environment object. If objects are being rebuilt outside of Ruby, ignore it.

Create Environments

An environment can be created in five different ways:

  • Creating a Ruby file in the environments sub-directory of the chef-repo and then pushing it to the Chef server
  • Creating a JSON file directly in the chef-repo and then pushing it to the Chef server
  • Using knife
  • Using the Chef management console web user interface
  • Using the Chef Infra Server REST API

Once an environment exists on the Chef Infra Server, a node can be associated with that environment using the chef_environment method.

Manage Environments

Once created, an environment can be managed in several ways:

  • By using knife and passing the -E ENVIRONMENT_NAME option with knife cookbook upload
  • By using Ruby or JSON files that are stored in a version source control system. These files are pushed to the Chef Infra Server using the knife environment subcommand and the from file argument. This approach allows environment data to be dynamically generated. This approach will not work unless these files are defined in the proper format—Ruby file end with .rb; JSON files end with .json.

These workflows are mutually exclusive: only the most recent environment changes will be kept on the Chef Infra Server, regardless of the source of those changes. All previous changes are overwritten when environment data is updated.

The settings for environments can be modified and environments can be integrated into the larger infrastructure by associating them with nodes and by using recipes to call specific environment settings.

Find Environment from Recipe

Use the following syntax to find the current environment from a recipe:

node.environment

or:

node.chef_environment

Save in a Data Bag

Values that are stored in a data bag are global to the organization and are available to any environment. There are two main strategies that can be used to store per-environment data within a data bag: by using a top-level key that corresponds to the environment or by using separate items for each environment.

A data bag that is storing a top-level key for an environment might look something like this:

{
  "id": "some_data_bag_item",
  "production" : {
    // Hash with all your data here
  },
  "testing" : {
    // Hash with all your data here
  }
}

When using the data bag in a recipe, that data can be accessed from a recipe using code similar to:

bag_item[node.chef_environment]['some_other_key']

The other approach is to use separate items for each environment. Depending on the amount of data, it may all fit nicely within a single item. If this is the case, then creating different items for each environment may be a simple approach to providing per-environment values within a data bag. However, this approach is more time-consuming and may not scale to very large environments or when the data must be stored in many data bag items.

Override Attributes in Roles

Environment attributes that are used with roles can be overridden. Typically, this is done by using attribute precedence, but sometimes it may be necessary to ensure that specific attributes are used based on the presence of specific environments. This type of scenario is best addressed in using a recipe that relies on a top-level key that is stored in a data bag.

For example, to retrieve a value from a data bag based on a specific environment:

mything = data_bag_item('things', 'mything')
attribute_i_want = mything[node.chef_environment]

Set for a Node

A node is considered to be associated with an environment when the chef_environment attribute is set. The chef_environment attribute cannot be set with normal or override attributes (i.e. in a role) because it is actually a method. An environment may be set explicitly using the following methods:

  • By using the knife edit and knife exec subcommands

  • By editing the chef_environment directly using knife or the Chef management console

  • By editing the environment configuration details in the client.rb file, and then using knife bootstrap -e environment_name to bootstrap the changes to the specified environment

    Note

    After the environment has been set via bootstrap, the environment is set in the client.rb file and may not be modified using the Chef management console or the edit argument of the knife node subcommand.

  • By setting the environment configuration entry in the client.rb file ; when Chef Infra Client runs, it will pick up the value and then set the chef_environment attribute of the node

Move Nodes

Use the knife exec subcommand to move nodes between environments, such as from a “dev” to a “production” environment. For example:

$ knife exec -E 'nodes.transform("chef_environment:dev") { |n| n.chef_environment("production") }'

Search Environments

When searching, an environment is an attribute. This allows search results to be limited to a specified environment by using Boolean operators and extra search terms. For example, to use knife to search for all of the servers running CentOS in an environment named “QA”, enter the following:

knife search node "chef_environment:QA AND platform:centos"

Or, to include the same search in a recipe, use a code block similar to:

qa_nodes = search(:node,"chef_environment:QA")
qa_nodes.each do |qa_node|
    # Do useful work specific to qa nodes only
end

Environments in Chef Solo

An environment is defined using JSON or the Ruby DSL. chef-solo will look for environments in /var/chef/environments, but this location can be modified by changing the setting for environment_path in solo.rb. For example, the following setting in solo.rb:

environment_path '/var/chef-solo/environments'

Environment data looks like the following in JSON:

{
  "name": "dev",
  "default_attributes": {
    "apache2": {
      "listen_ports": [
        "80",
        "443"
      ]
    }
  },
  "json_class": "Chef::Environment",
    "description": "",
    "cookbook_versions": {
    "couchdb": "= 11.0.0"
  },
  "chef_type": "environment"
  }

and like the following in the Ruby DSL:

name 'environment_name'
description 'environment_description'
cookbook OR cookbook_versions  'cookbook' OR 'cookbook' => 'cookbook_version'
default_attributes 'node' => { 'attribute' => %w(value value etc.) }
override_attributes 'node' => { 'attribute' => %w(value value etc.) }