API¶
bang¶
- exception bang.TimeoutError[source]¶
Bases: bang.BangError
bang.attributes¶
Constants for attribute names of the various resources.
This module contains the top-level config file attributes including those that are typically placed in ~/.bangrc.
- bang.attributes.ANSIBLE = 'ansible'¶
A dict containing ansible tuning variables.
- bang.attributes.DEPLOYER_CREDS = 'deployer_credentials'¶
A dict containing credentials for various cloud providers in which the keys can be any valid provider. E.g. aws, hpcloud.
- bang.attributes.LOGGING = 'logging'¶
The top-level key for logging-related configuration options.
- bang.attributes.NAME = 'name'¶
The stack name. Its value is used to tag servers and other cloud resources.
- bang.attributes.NAME_TAG_NAME = 'name_tag_name'¶
Like chicken fried chicken... this is a way to configure the name of the tag in which the combined stack-role (a.k.a. name) will be stored. By default, unless this is specified directly in ~/.bangrc, the name value will be assigned to a tag named “Name” (this is the default tag displayed in the AWS management console). I.e. using Bang defaults, the server named “bar” in the stack named “foo” will have the following tags:
stack: foo role: bar Name: foo-bar
In some cases, admins may have other purposes for the “Name” tag. If ~/.bangrc were to have name_tag_name set to descriptor, then the server described above would have the following tags:
stack: foo role: bar descriptor: foo-bar
To prevent Bang from assigning the name value to a tag, assign an empty string to the name_tag_name attribute in ~/.bangrc.
- bang.attributes.PLAYBOOKS = 'playbooks'¶
The ordered list of playbooks to run after provisioning the cloud resources.
- bang.attributes.PROVIDER = 'provider'¶
The resource provider (e.g. aws, hpcloud). Values for the provider attribute will be used to look up the appropriate Provider subclass to use when instantiating the associated resource.
- bang.attributes.SERVER_CLASS = 'server_class'¶
This is a derived attribute that Bang provides for instance tagging, and for Ansible playbooks to consume. It’s a combination of the NAME and the VERSION.
- bang.attributes.STACK = 'stack'¶
This is a derived attribute that Bang provides for instance tagging, and for Ansible playbooks to consume. It’s a combination of the NAME and the VERSION.
- bang.attributes.VERSION = 'version'¶
The stack version. Often, you need a global version of a stack in a playbook. E.g. when a web client wants to query a web service for API compatibility, the playbooks could configure the web service to report this stack version.
bang.attributes.ansible¶
- bang.attributes.ansible.ASK_VAULT_PASS = 'ask_vault_pass'¶
A boolean controlling whether or not to prompt for the vault password
- bang.attributes.ansible.VAULT_PASS = 'vault_pass'¶
The string used to decrypt any ansible vaults referenced in playbooks
- bang.attributes.ansible.VERBOSITY = 'verbosity'¶
An integer indicating verbosity.
bang.attributes.server¶
- bang.attributes.server.BANG_ATTRS = 'bang_server_attributes'¶
Provides the server definition from the Bang config as a fact available to the playbooks. E.g. in order to get access to the disk_image_id in a playbook:
{{bang_server_attributes.disk_image_id}}
bang.config¶
- class bang.config.Config(*args, **kwargs)[source]¶
Bases: dict
A dict-alike that provides a convenient constructor, stashes the path to the config file as an instance attribute, and performs some validation of the values.
- __init__(*args, **kwargs)[source]¶
Parameters: path_to_yaml (str) – Path to a yaml file to use as the data source for the returned instance.
- autoinc()[source]¶
Conditionally updates the stack version in the file associated with this config.
This handles both official releases (i.e. QA configs), and release candidates. Assumptions about version:
Official release versions are MAJOR.minor, where MAJOR and minor are both non-negative integers. E.g.
2.9 2.10 2.11 3.0 3.1 3.2 etc...
Release candidate versions are MAJOR.minor-rc.N, where MAJOR, minor, and N are all non-negative integers.
3.5-rc.1 3.5-rc.2
- classmethod from_config_specs(config_specs, prepare=True)[source]¶
Alternate constructor that merges config attributes from $HOME/.bangrc and config_specs into a single Config object.
The first (and potentially only spec) in config_specs should be main configuration file for the stack to be deployed. The returned object’s filepath will be set to the absolute path of the first config file.
If multiple config specs are supplied, their values are merged together in the order specified in config_specs - That is, later values override earlier values.
Parameters: - config_specs (list of str) – List of config specs.
- prepare (bool) – Flag to control whether or not prepare() is called automatically before returning the object.
Return type:
- prepare()[source]¶
Reorganizes the data such that the deployment logic can find it all where it expects to be.
The raw configuration file is intended to be as human-friendly as possible partly through the following mechanisms:
- In order to minimize repetition, any attributes that are common to all server configurations can be specified in the server_common_attributes stanza even though the stanza itself does not map directly to a deployable resource.
- For reference locality, each security group stanza contains its list of rules even though rules are actually created in a separate stage from the groups themselves.
In order to make the Config object more useful to the program logic, this method performs the following transformations:
- Distributes the server_common_attributes among all the members of the servers stanza.
- Extracts security group rules to a top-level key, and interpolates all source and target values.
- bang.config.find_component_tarball(bucket, comp_name, comp_config)[source]¶
Returns True if the component tarball is found in the bucket.
Otherwise, returns False.
- bang.config.parse_bangrc()[source]¶
Parses $HOME/.bangrc for global settings and deployer credentials. The .bangrc file is expected to be a YAML file whose outermost structure is a key-value map.
Note that even though .bangrc is just a YAML file in which a user could store any top-level keys, it is not expected to be used as a holder of default values for stack-specific configuration attributes - if present, they will be ignored.
Returns {} if $HOME/.bangrc does not exist.
Return type: dict
- bang.config.resolve_config_spec(config_spec, config_dir='')[source]¶
Resolves config_spec to a path to a config file.
Parameters: - config_spec (str) –
Valid config specs:
- The basename of a YAML config file without the .yml extension. The full path to the config file is resolved by appending .yml to the basename, then by searching for the result in the config_dir.
- The path to a YAML config file. The path may be absolute or may be relative to the current working directory. If config_spec contains a / (forward slash), or if it ends in .yml, it is treated as a path.
- config_dir (str) – The directory in which to search for stack configuration files.
Return type: str
- config_spec (str) –
bang.deployers¶
Base classes and definitions for bang deployers (deployable components)
- bang.deployers.get_stage_deployers(keys, stack)[source]¶
Returns a list of deployer objects that create cloud resources. Each member of the list is responsible for provisioning a single stack resource (e.g. a virtual server, a security group, a bucket, etc...).
Parameters: - keys (Iterable) – A list of top-level configuration keys for which to create deployers.
- config (Stack) – A stack object.
Return type: list of Deployer
bang.deployers.cloud¶
- class bang.deployers.cloud.BaseDeployer(stack, config, consul)[source]¶
Bases: bang.deployers.deployer.Deployer
Base class for all cloud resource deployers
- class bang.deployers.cloud.CloudManagerServerDeployer(*args, **kwargs)[source]¶
Bases: bang.deployers.cloud.ServerDeployer
Server deployer for cloud management services.
Cloud management services like RightScale and Scalr provide constructs like server templates (a.k.a. roles) to bundle together disk image ids with on-server configuration automation (e.g. RightScripts, Scalr scripts). This deployer replaces the low-level provisioning functionality in the base ServerDeployer with a create() method that is more suited to the high-level launching mechanism provided by cloud management services.
- class bang.deployers.cloud.LoadBalancerDeployer(*args, **kwargs)[source]¶
Bases: bang.deployers.cloud.RegionedDeployer
Cloud-managed load balancer deployer. Assumes a consul able to create and discover LB instances, as well as match existing backend ‘nodes’ to a list it’s given. It is assumed only a single ‘instance’ per distinct load balancer needs to be created (i.e. that any elasticity is handled by the cloud service).
Example config:
load_balancers: test_balancer: balance_server_name: server_defined_in_servers_section region: region-1.geo-1 provider: hpcloud backend_port: '8080' protocol: tcp port: '443'
- class bang.deployers.cloud.RegionedDeployer(stack, config, consul)[source]¶
Bases: bang.deployers.cloud.BaseDeployer
Deployer that automatically sets its region
- class bang.deployers.cloud.SSHKeyDeployer(*args, **kwargs)[source]¶
Bases: bang.deployers.cloud.RegionedDeployer
Registers SSH keys with cloud providers so they can be used at server-launch time.
- class bang.deployers.cloud.SecurityGroupRulesetDeployer(*args, **kwargs)[source]¶
- class bang.deployers.cloud.ServerDeployer(*args, **kwargs)[source]¶
bang.deployers.default¶
- class bang.deployers.default.ServerDeployer(*args, **kwargs)[source]¶
Bases: bang.deployers.deployer.Deployer
Default deployer that can be used for any servers that are already deployed and do not need special deployment logic (e.g. traditional server rooms, manually deployed cloud servers).
Example of a minimal configuration for a manually provisioned app server:
my_app_server: hostname: my_hostname_or_ip_address groups: - ansible_inventory_group_1 - ansible_inventory_group_n config_scopes: - config_scope_1 - config_scope_n
bang.deployers.deployer¶
bang.inventory¶
bang.providers¶
- bang.providers.get_provider(name, creds)[source]¶
Generates and memoizes a Provider object for the given name.
Parameters: - name (str) – The provider name, as given in the config stanza. This token is used to find the appropriate Provider.
- creds (dict) – The credentials dictionary that is appropriate for the desired provider. Typically, a sub-dict from the main stack config.
Return type: Provider
bang.providers.bases¶
- class bang.providers.bases.Consul(provider)[source]¶
Bases: object
The base class for all service consuls.
Not really the boss of anything, but conveys intent-from-above to foreign entities (e.g. OpenStack Nova/Swift, AWS EC2/S3/RDS, etc...). Also communicates the state of the world back up to the boss.
- class bang.providers.bases.Provider(creds)[source]¶
Bases: object
The base class for all providers.
- gen_component_name(basename, postfix_length=13)[source]¶
Creates a resource identifier with a random postfix. This is an attempt to minimize name collisions in provider namespaces.
Parameters: - basename (str) – The string that will be prefixed with the stack name, and postfixed with some random string.
- postfix_length (int) – The length of the postfix to be appended.
bang.providers.aws¶
- class bang.providers.aws.AWS(creds)[source]¶
Bases: bang.providers.bases.Provider
- CONSUL_MAP = {'databases': <class 'bang.providers.aws.RDS'>, 'buckets': <class 'bang.providers.aws.S3'>, 'server_security_groups': <class 'bang.providers.aws.EC2'>, 'server_security_group_rules': <class 'bang.providers.aws.EC2'>, 'servers': <class 'bang.providers.aws.EC2'>}¶
- class bang.providers.aws.EC2(*args, **kwargs)[source]¶
Bases: bang.providers.bases.Consul
The consul for the compute service in AWS (EC2).
- create_secgroup(name, description)[source]¶
Creates a new server security group.
Parameters: - name (str) – The name of the security group to create.
- description (str) – A short description of the group.
- create_secgroup_rule(protocol, from_port, to_port, source, target)[source]¶
Creates a new server security group rule.
Parameters: - protocol (str) – E.g. tcp, icmp, etc...
- from_port (int) – E.g. 1
- to_port (int) – E.g. 65535
- source (str) –
- target (str) – The target security group. I.e. the group in which this rule should be created.
- create_server(basename, disk_image_id, instance_type, ssh_key_name, tags=None, availability_zone=None, timeout_s=120, **provider_extras)[source]¶
Creates a new server instance. This call blocks until the server is created and available for normal use, or timeout_s has elapsed.
Parameters: - basename (str) – An identifier for the server. A random postfix will be appended to this basename to work around OpenStack Nova REST API limitations.
- disk_image_id (str) – The identifier of the base disk image to use as the rootfs.
- instance_type (str) – The name of an EC2 instance type.
- ssh_key_name (str) – The name of the ssh key to inject into the target server’s authorized_keys file. The key must already have been registered in the target EC2 region.
- tags (Mapping) – Up to 5 key-value pairs of arbitrary strings to use as tags for the server instance.
- availability_zone (str) – The name of the availability zone in which to place the server.
- timeout_s (float) – The number of seconds to poll for an active server before failing. Defaults to 0 (i.e. Expect server to be active immediately).
Return type: dict
- find_secgroup(name)[source]¶
Find a security group by name.
Returns a EC2SecGroup instance if found, otherwise returns None.
- find_servers(tags, running=True)[source]¶
Returns any servers in the region that have tags that match the key-value pairs in tags.
Parameters: - tags (Mapping) – A mapping object in which the keys are the tag names and the values are the tag values.
- running (bool) – A flag to limit server list to instances that are actually running.
Return type: list of dict objects. Each dict describes a single server instance.
- class bang.providers.aws.EC2SecGroup(ec2sg)[source]¶
Bases: object
Represents an EC2 security group.
The rules attribute is a specialized dict whose keys are the normalized rule definitions, and whose values are EC2 grants which can be kwargs-expanded when passing boto.ec2.securitygroup.SecurityGroup.revoke(). E.g.:
{ ('tcp', 1, 65535, 'group-foo'): { 'ip_protocol': 'tcp', 'from_port': '1', 'to_port': '65535', 'src_group': 'group-foo', 'target': SecurityGroup:group-bar, }, ('tcp', 8080, 8080, '15.183.202.114/32'): { 'ip_protocol': 'tcp', 'from_port': '8080', 'to_port': '8080', 'cidr_ip': '15.183.202.114/32', 'target': SecurityGroup:group-bar, }, }
This also maintains a reference to the original boto.ec2.securitygroup.SecurityGroup instance.
Suitable for returning from EC2.find_secgroup().
- class bang.providers.aws.RDS(provider)[source]¶
Bases: bang.providers.bases.Consul
- class bang.providers.aws.S3(*args, **kwargs)[source]¶
Bases: bang.providers.bases.Consul
The consul for the storage service in AWS (S3).
- bang.providers.aws.server_to_dict(server)[source]¶
Returns the dict representation of a server object.
The returned dict is meant to be consumed by ServerDeployer objects.
bang.providers.hpcloud¶
bang.providers.hpcloud.load_balancer¶
bang.providers.hpcloud.reddwarf¶
bang.providers.openstack¶
bang.providers.rs¶
bang.stack¶
- class bang.stack.Stack(config)[source]¶
Bases: object
Deploys infrastructure/platform resources, then configures any deployed servers using ansible playbooks.
- __init__(config)[source]¶
Parameters: config (bang.config.Config) – A mapping object with configuration keys and values. May be arbitrarily nested.
- add_host(host, group_names=None, host_vars=None)[source]¶
Used by deployers to add hosts to the inventory.
Parameters: - host (str) – The host identifier (e.g. hostname, IP address) to use in the inventory.
- group_names (list) – A list of group names to which the host belongs. Note: This list will be sorted in-place.
- host_vars (dict) – A mapping object of host variables. This can be a nested structure, and is used as the source of all the variables provided to the ansible playbooks. Note: Additional key-value pairs (e.g. dynamic ansible values like ``inventory_hostname``) will be inserted into this mapping object.
- add_lb_secgroup(lb_name, hosts, port)[source]¶
Used by the load balancer deployer to register a hostname for a load balancer, in order that security group rules can be applied later. This is multiprocess-safe, but since keys are accessed only be a single load balancer deployer there should be no conflicts.
Parameters: lb_name (str) – The load balancer name (as per the config file) :param list hosts: The load balancer host[s], once known
Parameters: port – The backend port that the LB will connect on
- configure(*args, **kwargs)[source]¶
Executes the ansible playbooks that configure the servers in the stack.
Assumes that the root playbook directory is ./playbooks/ relative to the stack configuration file. Also sets the ansible module_path to be ./common_modules/ relative to the stack configuration file.
E.g. If the stack configuration file is:
$HOME/bang-stacks/my_web_service.yml
then the root playbook directory is:
$HOME/bang-stacks/playbooks/
and the ansible module path is:
$HOME/bang-stacks/common_modules/
- deploy()[source]¶
Iterates through the deployers returned by self.get_deployers().
Deployers in the same stage are run concurrently. The runner only proceeds to the next stage once all of the deployers in the same stage have completed successfully.
Any failures in a stage cause the run to terminate before proceeding to the next stage.
- find_first(attr_name, resources, extra_prefix='')[source]¶
Returns the boto object for the first resource in resources that belongs to this stack. Uses the attribute specified by attr_name to match the stack name.
E.g. An RDS instance for a stack named foo might be named foo-mydb-fis8932ifs. This call:
find_first('id', conn.get_all_dbinstances())
would return the boto.rds.dbinstance.DBInstance object whose id is foo-mydb-fis8932ifs.
Returns None if a matching resource is not found.
If specified, extra_prefix is appended to the stack name prefix before matching.
- gather_inventory()[source]¶
Gathers existing inventory info.
Does not create any new infrastructure.
- get_deployers()[source]¶
Returns a list of stages, where each stage is a list of Deployer objects. It defines the execution order of the various deployers.
- get_namespace(key)[source]¶
Returns a SharedNamespace for the given key. These are used by Deployer objects of the same deployer_class to coordinate control over multiple deployed instances of like resources. E.g. With 5 clones of an application server, 5 Deployer objects in separate, concurrent processes will use the same shared namespace to ensure that each object/process controls a distinct server.
Parameters: key (str) – Unique ID for the namespace. Deployer objects that call get_namespace() with the same key will receive the same SharedNamespace object.
- have_inventory = None¶
Deployers stash inventory data for any newly-created servers in this mapping object. Note: uses SharedMap because this must be multiprocess-safe.
- show_host(host)[source]¶
Satisfies the --host portion of ansible’s external inventory API.
Allows bang to be used as an external inventory script, for example when running ad-hoc ops tasks. For more details, see: http://ansible.cc/docs/api.html#external-inventory-scripts
- show_inventory(*args, **kwargs)[source]¶
Satisfies the --list portion of ansible’s external inventory API.
Allows bang to be used as an external inventory script, for example when running ad-hoc ops tasks. For more details, see: http://ansible.cc/docs/api.html#external-inventory-scripts
bang.util¶
- class bang.util.NullHandler(level=0)[source]¶
Bases: logging.Handler
This handler does nothing. It’s intended to be used to avoid the “No handlers could be found for logger XXX” one-off warning. This is important for library code, which may contain code to log events. If a user of the library does not configure logging, the one-off warning might be produced; to avoid this, the library developer simply needs to instantiate a NullHandler and add it to the top-level logger of the library module or package.
- class bang.util.S3Handler(bucket, prefix='')[source]¶
Bases: logging.handlers.BufferingHandler
Buffers all logging events, then uploads them all at once “atexit” to a single file in S3.
Bases: object
A multiprocess-safe Mapping object that can be used to return values from child processes.
Appends value to the list named list_name.
Performs deep-merge of values onto the Mapping object named dict_name.
If dict_name does not yet exist, then a deep copy of values is assigned as the initial mapping object for the given name.
Parameters: dict_name (str) – The name of the dict onto which the values should be merged.
Bases: object
A multiprocess-safe namespace that can be used to coordinate naming similar resources uniquely. E.g. when searching for existing nodes in a cassandra cluster, you can use this SharedNamespace to make sure other processes aren’t looking at the same node.
Returns True on success.
Returns False if the name already exists in the namespace.
- class bang.util.StrictAttrBag(**kwargs)[source]¶
Bases: object
Generic attribute container that makes constructor arguments available as object attributes.
Checks __init__() argument names against lists of required and optional attributes.
- bang.util.bump_version_tail(oldver)[source]¶
Takes any dot-separated version string and increments the rightmost field (which it expects to be an integer).
- bang.util.count_by_tag(stack, descriptor)[source]¶
Returns the count of currently running or pending instances that match the given stack and deployer combo
- bang.util.count_to_deploy(stack, descriptor, config_count)[source]¶
takes the max of config_count and number of instances running with this stack/descriptor combo
- bang.util.deep_merge_dicts(base, incoming)[source]¶
Performs an in-place deep-merge of key-values from incoming into base. No attempt is made to preserve the original state of the objects passed in as arguments.
Parameters: - base (Any dict-like object) – The target container for the merged values. This will be modified in-place.
- incoming (Any dict-like object) – The container from which incoming values will be copied. Nested dicts in this will be modified.
Return type: None
- bang.util.fork_exec(cmd_list, input_data=None)[source]¶
Like the subprocess.check_*() helper functions, but tailored to bang.
cmd_list is the command to run, and its arguments as a list of strings.
input_data is the optional data to pass to the command’s stdin.
On success, returns the output (i.e. stdout) of the remote command.
On failure, raises BangError with the command’s stderr.
- bang.util.poll_with_timeout(timeout_s, break_func, wake_every_s=60)[source]¶
Calls break_func every wake_every_s seconds for a total duration of timeout_s seconds, or until break_func returns something other than None.
If break_func returns anything other than None, that value is returned immediately.
Otherwise, continues polling until the timeout is reached, then returns None.