Configuration updates for latest builds and major refactor

Configuration updates for latest builds
- Switch to using create-node-config
- Switch sdn services to use etcd over SSL
- This re-uses the client certificate deployed on each node
- Additional node registration changes
- Do not assume that metadata service is available in openshift_facts module
- Call systemctl daemon-reload after installing openshift-master, openshift-sdn-master, openshift-node, openshift-sdn-node
- Fix bug overriding openshift_hostname and openshift_public_hostname in byo playbooks
- Start moving generated configs to /etc/openshift
- Some custom module cleanup
- Add known issue with ansible-1.9 to README_OSE.md
- Update to genericize the kubernetes_register_node module
  - Default to use kubectl for commands
  - Allow for overriding kubectl_cmd
  - In openshift_register_node role, override kubectl_cmd to openshift_kube
- Set default openshift_registry_url for enterprise when deployment_type is enterprise
- Fix openshift_register_node for client config change
- Ensure that master certs directory is created
- Add roles and filter_plugin symlinks to playbooks/common/openshift-master and node
- Allow non-root user with sudo nopasswd access
- Updates for README_OSE.md
- Update byo inventory for adding additional comments
- Updates for node cert/config sync to work with non-root user using sudo
- Move node config/certs to /etc/openshift/node
- Don't use path for mktemp. addresses: https://github.com/openshift/openshift-ansible/issues/154

Create common playbooks
- create common/openshift-master/config.yml
- create common/openshift-node/config.yml
- update playbooks to use new common playbooks
- update launch playbooks to call update playbooks
- fix openshift_registry and openshift_node_ip usage

Set default deployment type to origin
- openshift_repo updates for enabling origin deployments
  - also separate repo and gpgkey file structure
  - remove kubernetes repo since it isn't currently needed
- full deployment type support for bin/cluster
  - honor OS_DEPLOYMENT_TYPE env variable
  - add --deployment-type option, which will override OS_DEPLOYMENT_TYPE if set
  - if neither OS_DEPLOYMENT_TYPE or --deployment-type is set, defaults to
    origin installs

Additional changes:
- Add separate config action to bin/cluster that runs ansible config but does
  not update packages
- Some more duplication reduction in cluster playbooks.
- Rename task files in playbooks dirs to have tasks in their name for clarity.
- update aws/gce scripts to use a directory for inventory (otherwise when
  there are no hosts returned from dynamic inventory there is an error)

libvirt refactor and update

- add libvirt dynamic inventory
- updates to use dynamic inventory for libvirt

5 files changed, 200 insertions, 4 deletions

diff --git a/inventory/libvirt/group_vars/all b/inventory/libvirt/group_vars/all
deleted file mode 100644
index b22da00de..000000000
--- a/inventory/libvirt/group_vars/all
+++ /dev/null
@@ -1,2 +0,0 @@
----
-ansible_ssh_user: root
diff --git a/inventory/libvirt/hosts b/inventory/libvirt/hosts
deleted file mode 100644
index 6a818f268..000000000
--- a/inventory/libvirt/hosts
+++ /dev/null
@@ -1,2 +0,0 @@
-# Eventually we'll add the GCE, AWS, etc dynamic inventories, but for now...
-localhost ansible_python_interpreter=/usr/bin/python2
diff --git a/inventory/libvirt/hosts/hosts b/inventory/libvirt/hosts/hosts
new file mode 100644
index 000000000..9cdc31449
--- /dev/null
+++ b/inventory/libvirt/hosts/hosts
@@ -0,0 +1 @@
+localhost ansible_sudo=no ansible_python_interpreter=/usr/bin/python2 connection=local
diff --git a/inventory/libvirt/hosts/libvirt.ini b/inventory/libvirt/hosts/libvirt.ini
new file mode 100644
index 000000000..62ff204dd
--- /dev/null
+++ b/inventory/libvirt/hosts/libvirt.ini
@@ -0,0 +1,20 @@
+# Ansible libvirt external inventory script settings
+#
+
+[libvirt]
+
+uri = qemu:///system
+
+# API calls to libvirt can be slow. For this reason, we cache the results of an API
+# call. Set this to the path you want cache files to be written to. Two files
+# will be written to this directory:
+#   - ansible-libvirt.cache
+#   - ansible-libvirt.index
+cache_path = /tmp
+
+# The number of seconds a cache file is considered valid. After this many
+# seconds, a new API call will be made, and the cache file will be updated.
+cache_max_age = 900
+
+
+
diff --git a/inventory/libvirt/hosts/libvirt_generic.py b/inventory/libvirt/hosts/libvirt_generic.py
new file mode 100755
index 000000000..0a98e2af3
--- /dev/null
+++ b/inventory/libvirt/hosts/libvirt_generic.py
@@ -0,0 +1,179 @@
+#!/usr/bin/env python
+
+"""
+libvirt external inventory script
+=================================
+
+Ansible has a feature where instead of reading from /etc/ansible/hosts
+as a text file, it can query external programs to obtain the list
+of hosts, groups the hosts are in, and even variables to assign to each host.
+
+To use this, copy this file over /etc/ansible/hosts and chmod +x the file.
+This, more or less, allows you to keep one central database containing
+info about all of your managed instances.
+
+"""
+
+# (c) 2015, Jason DeTiberus <jdetiber@redhat.com>
+#
+# This file is part of Ansible,
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+######################################################################
+
+import argparse
+import ConfigParser
+import os
+import re
+import sys
+from time import time
+import libvirt
+import xml.etree.ElementTree as ET
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
+
+
+class LibvirtInventory(object):
+
+    def __init__(self):
+        self.inventory = dict()  # A list of groups and the hosts in that group
+        self.cache = dict()  # Details about hosts in the inventory
+
+        # Read settings and parse CLI arguments
+        self.read_settings()
+        self.parse_cli_args()
+
+        if self.args.host:
+            print self.json_format_dict(self.get_host_info(), self.args.pretty)
+        elif self.args.list:
+            print self.json_format_dict(self.get_inventory(), self.args.pretty)
+        else:  # default action with no options
+            print self.json_format_dict(self.get_inventory(), self.args.pretty)
+
+    def read_settings(self):
+        config = ConfigParser.SafeConfigParser()
+        config.read(
+            os.path.dirname(os.path.realpath(__file__)) + '/libvirt.ini'
+        )
+        self.libvirt_uri = config.get('libvirt', 'uri')
+
+    def parse_cli_args(self):
+        parser = argparse.ArgumentParser(
+            description='Produce an Ansible Inventory file based on libvirt'
+        )
+        parser.add_argument(
+            '--list',
+            action='store_true',
+            default=True,
+            help='List instances (default: True)'
+        )
+        parser.add_argument(
+            '--host',
+            action='store',
+            help='Get all the variables about a specific instance'
+        )
+        parser.add_argument(
+            '--pretty',
+            action='store_true',
+            default=False,
+            help='Pretty format (default: False)'
+        )
+        self.args = parser.parse_args()
+
+    def get_host_info(self):
+        inventory = self.get_inventory()
+        if self.args.host in inventory['_meta']['hostvars']:
+            return inventory['_meta']['hostvars'][self.args.host]
+
+    def get_inventory(self):
+        inventory = dict(_meta=dict(hostvars=dict()))
+
+        conn = libvirt.openReadOnly(self.libvirt_uri)
+        if conn is None:
+            print "Failed to open connection to %s" % libvirt_uri
+            sys.exit(1)
+
+        domains = conn.listAllDomains()
+        if domains is None:
+            print "Failed to list domains for connection %s" % libvirt_uri
+            sys.exit(1)
+
+        arp_entries = self.parse_arp_entries()
+
+        for domain in domains:
+            hostvars = dict(libvirt_name=domain.name(),
+                            libvirt_id=domain.ID(),
+                            libvirt_uuid=domain.UUIDString())
+            domain_name = domain.name()
+
+            # TODO: add support for guests that are not in a running state
+            state, _ = domain.state()
+            # 2 is the state for a running guest
+            if state != 1:
+                continue
+
+            hostvars['libvirt_status'] = 'running'
+
+            root = ET.fromstring(domain.XMLDesc())
+            ns = {'ansible': 'https://github.com/ansible/ansible'}
+            for tag_elem in root.findall('./metadata/ansible:tag', ns):
+                tag = tag_elem.text
+                self.push(inventory, "tag_%s" % tag, domain_name)
+                self.push(hostvars, 'libvirt_tags', tag)
+
+            # TODO: support more than one network interface, also support
+            # interface types other than 'network'
+            interface = root.find("./devices/interface[@type='network']")
+            if interface is not None:
+                mac_elem = interface.find('mac')
+                if mac_elem is not None:
+                    mac = mac_elem.get('address')
+                    if mac in arp_entries:
+                        ip_address = arp_entries[mac]['ip_address']
+                        hostvars['ansible_ssh_host'] = ip_address
+                        hostvars['libvirt_ip_address'] = ip_address
+
+            inventory['_meta']['hostvars'][domain_name] = hostvars
+
+        return inventory
+
+    def parse_arp_entries(self):
+        arp_entries = dict()
+        with open('/proc/net/arp', 'r') as f:
+            # throw away the header
+            f.readline()
+
+            for line in f:
+                ip_address, _, _, mac, _, device = line.strip().split()
+                arp_entries[mac] = dict(ip_address=ip_address, device=device)
+
+        return arp_entries
+
+    def push(self, my_dict, key, element):
+        if key in my_dict:
+            my_dict[key].append(element)
+        else:
+            my_dict[key] = [element]
+
+    def json_format_dict(self, data, pretty=False):
+        if pretty:
+            return json.dumps(data, sort_keys=True, indent=2)
+        else:
+            return json.dumps(data)
+
+LibvirtInventory()