Merge branch 'master' into mbruzek-openshift-openstack

8 files changed, 172 insertions, 28 deletions

diff --git a/playbooks/openstack/README.md b/playbooks/openstack/README.md
index f3b691790..2eb668dd1 100644
--- a/playbooks/openstack/README.md
+++ b/playbooks/openstack/README.md
@@ -184,7 +184,9 @@ Then run the provision + install playbook -- this will create the OpenStack
 resources:
 
 ```bash
-$ ansible-playbook --user openshift -i inventory \
+$ ansible-playbook --user openshift \
+  -i openshift-ansible/playbooks/openstack/inventory.py
+  -i inventory \
   openshift-ansible/playbooks/openstack/openshift-cluster/provision_install.yml \
   -e openshift_repos_enable_testing=true
 ```
@@ -192,6 +194,11 @@ $ ansible-playbook --user openshift -i inventory \
 Note, you may want to use the testing repo for development purposes only.
 Normally, `openshift_repos_enable_testing` should not be specified.
 
+In addition to *your* inventory with your OpenShift and OpenStack
+configuration, we are also supplying the [dynamic inventory][dynamic] from
+`openshift-ansible/inventory`. It's a script that will look at the Nova servers
+and other resources that will be created and let Ansible know about them.
+
 If you're using multiple inventories, make sure you pass the path to
 the right one to `-i`.
 
@@ -235,3 +242,4 @@ advanced configuration:
 [loadbalancer]: ./advanced-configuration.md#multi-master-configuration
 [external-dns]: ./advanced-configuration.md#dns-configuration-variables
 [cinder-registry]: ./advanced-configuration.md#creating-and-using-a-cinder-volume-for-the-openshift-registry
+[dynamic]: http://docs.ansible.com/ansible/latest/intro_dynamic_inventory.html
diff --git a/playbooks/openstack/advanced-configuration.md b/playbooks/openstack/advanced-configuration.md
index 2c9b70b5f..e8f4cfc32 100644
--- a/playbooks/openstack/advanced-configuration.md
+++ b/playbooks/openstack/advanced-configuration.md
@@ -1,9 +1,8 @@
 ## Dependencies for localhost (ansible control/admin node)
 
-* [Ansible 2.3](https://pypi.python.org/pypi/ansible)
-* [Ansible-galaxy](https://pypi.python.org/pypi/ansible-galaxy-local-deps)
-* [jinja2](http://jinja.pocoo.org/docs/2.9/)
-* [shade](https://pypi.python.org/pypi/shade)
+* [Ansible](https://pypi.python.org/pypi/ansible) version >=2.4.0
+* [jinja2](http://jinja.pocoo.org/docs/2.9/) version >= 2.10
+* [shade](https://pypi.python.org/pypi/shade) version >= 1.26
 * python-jmespath / [jmespath](https://pypi.python.org/pypi/jmespath)
 * python-dns / [dnspython](https://pypi.python.org/pypi/dnspython)
 * Become (sudo) is not required.
@@ -133,7 +132,7 @@ You can also access the OpenShift cluster with a web browser by going to:
 https://master-0.openshift.example.com:8443
 
 Note that for this to work, the OpenShift nodes must be accessible
-from your computer and it's DNS configuration must use the cruster's
+from your computer and its DNS configuration must use the cluster's
 DNS.
 
 
@@ -153,7 +152,7 @@ openstack stack delete --wait --yes openshift.example.com
 Pay special attention to the values in the first paragraph -- these
 will depend on your OpenStack environment.
 
-Note that the provsisioning playbooks update the original Neutron subnet
+Note that the provisioning playbooks update the original Neutron subnet
 created with the Heat stack to point to the configured DNS servers.
 So the provisioned cluster nodes will start using those natively as
 default nameservers. Technically, this allows to deploy OpenShift clusters
@@ -162,7 +161,7 @@ without dnsmasq proxies.
 The `openshift_openstack_clusterid` and `openshift_openstack_public_dns_domain`
 will form the cluster's public DNS domain all your servers will be under. With
 the default values, this will be `openshift.example.com`. For workloads, the
-default subdomain is 'apps'. That sudomain can be set as well by the
+default subdomain is 'apps'. That subdomain can be set as well by the
 `openshift_openstack_app_subdomain` variable in the inventory.
 
 If you want to use a two sets of hostnames for public and private/prefixed DNS
@@ -334,7 +333,7 @@ or your trusted network. The most important is the `openshift_openstack_node_ing
 that restricts public access to the deployed DNS server and cluster
 nodes' ephemeral ports range.
 
-Note, the command ``curl https://api.ipify.org`` helps fiding an external
+Note, the command ``curl https://api.ipify.org`` helps finding an external
 IP address of your box (the ansible admin node).
 
 There is also the `manage_packages` variable (defaults to True) you
@@ -372,6 +371,112 @@ In order to set a custom entrypoint, update `openshift_master_cluster_public_hos
 Note than an empty hostname does not work, so if your domain is `openshift.example.com`,
 you cannot set this value to simply `openshift.example.com`.
 
+
+## Using Cinder-backed Persistent Volumes
+
+You will need to set up OpenStack credentials. You can try putting this in your
+`inventory/group_vars/OSEv3.yml`:
+
+    openshift_cloudprovider_kind: openstack
+    openshift_cloudprovider_openstack_auth_url: "{{ lookup('env','OS_AUTH_URL') }}"
+    openshift_cloudprovider_openstack_username: "{{ lookup('env','OS_USERNAME') }}"
+    openshift_cloudprovider_openstack_password: "{{ lookup('env','OS_PASSWORD') }}"
+    openshift_cloudprovider_openstack_tenant_name: "{{ lookup('env','OS_PROJECT_NAME') }}"
+    openshift_cloudprovider_openstack_domain_name: "{{ lookup('env','OS_USER_DOMAIN_NAME') }}"
+    openshift_cloudprovider_openstack_blockstorage_version: v2
+
+**NOTE**: you must specify the Block Storage version as v2, because OpenShift
+does not support the v3 API yet and the version detection is currently not
+working properly.
+
+For more information, consult the [Configuring for OpenStack page in the OpenShift documentation][openstack-credentials].
+
+[openstack-credentials]: https://docs.openshift.org/latest/install_config/configuring_openstack.html#install-config-configuring-openstack
+
+**NOTE** the OpenStack integration currently requires DNS to be configured and
+running and the `openshift_hostname` variable must match the Nova server name
+for each node. The cluster deployment will fail without it. If you use the
+provided OpenStack dynamic inventory and configure the
+`openshift_openstack_dns_nameservers` Ansible variable, this will be handled
+for you.
+
+After a successful deployment, the cluster is configured for Cinder persistent
+volumes.
+
+### Validation
+
+1. Log in and create a new project (with `oc login` and `oc new-project`)
+2. Create a file called `cinder-claim.yaml` with the following contents:
+
+```yaml
+apiVersion: "v1"
+kind: "PersistentVolumeClaim"
+metadata:
+  name: "claim1"
+spec:
+  accessModes:
+    - "ReadWriteOnce"
+  resources:
+    requests:
+      storage: "1Gi"
+```
+3. Run `oc create -f cinder-claim.yaml` to create the Persistent Volume Claim object in OpenShift
+4. Run `oc describe pvc claim1` to verify that the claim was created and its Status is `Bound`
+5. Run `openstack volume list`
+   * A new volume called `kubernetes-dynamic-pvc-UUID` should be created
+   * Its size should be `1`
+   * It should not be attached to any server
+6. Create a file called `mysql-pod.yaml` with the following contents:
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: mysql
+  labels:
+    name: mysql
+spec:
+  containers:
+    - resources:
+        limits :
+          cpu: 0.5
+      image: openshift/mysql-55-centos7
+      name: mysql
+      env:
+        - name: MYSQL_ROOT_PASSWORD
+          value: yourpassword
+        - name: MYSQL_USER
+          value: wp_user
+        - name: MYSQL_PASSWORD
+          value: wp_pass
+        - name: MYSQL_DATABASE
+          value: wp_db
+      ports:
+        - containerPort: 3306
+          name: mysql
+      volumeMounts:
+        - name: mysql-persistent-storage
+          mountPath: /var/lib/mysql/data
+  volumes:
+    - name: mysql-persistent-storage
+      persistentVolumeClaim:
+        claimName: claim1
+```
+
+7. Run `oc create -f mysql-pod.yaml` to create the pod
+8. Run `oc describe pod mysql`
+   * Its events should show that the pod has successfully attached the volume above
+   * It should show no errors
+   * `openstack volume list` should show the volume attached to an OpenShift app node
+   * NOTE: this can take several seconds
+9. After a while, `oc get pod` should show the `mysql` pod as running
+10. Run `oc delete pod mysql` to remove the pod
+   * The Cinder volume should no longer be attached
+11. Run `oc delete pvc claim1` to remove the volume claim
+   * The Cinder volume should be deleted
+
+
+
 ## Creating and using a Cinder volume for the OpenShift registry
 
 You can optionally have the playbooks create a Cinder volume and set
@@ -415,7 +520,7 @@ OpenStack)[openstack] for more information.
 [openstack]: https://docs.openshift.org/latest/install_config/configuring_openstack.html
 
 
-Next, we need to instruct OpenShift to use the Cinder volume for it's
+Next, we need to instruct OpenShift to use the Cinder volume for its
 registry. Again in `OSEv3.yml`:
 
     #openshift_hosted_registry_storage_kind: openstack
@@ -470,12 +575,12 @@ The **Cinder volume ID**, **filesystem** and **volume size** variables
 must correspond to the values in your volume. The volume ID must be
 the **UUID** of the Cinder volume, *not its name*.
 
-We can do formate the volume for you if you ask for it in
+The volume can also be formatted if you configure it in
 `inventory/group_vars/all.yml`:
 
     openshift_openstack_prepare_and_format_registry_volume: true
 
-**NOTE:** doing so **will destroy any data that's currently on the volume**!
+**NOTE:** Formatting **will destroy any data that's currently on the volume**!
 
 You can also run the registry setup playbook directly:
 
diff --git a/playbooks/openstack/sample-inventory/inventory.py b/playbooks/openstack/inventory.py
index ad3fd936b..76e658eb7 100755
--- a/playbooks/openstack/sample-inventory/inventory.py
+++ b/playbooks/openstack/inventory.py
@@ -9,6 +9,7 @@ environment.
 
 from __future__ import print_function
 
+from collections import Mapping
 import json
 
 import shade
@@ -42,7 +43,10 @@ def build_inventory():
            if server.metadata['host-type'] == 'node' and
            server.metadata['sub-host-type'] == 'app']
 
-    nodes = list(set(masters + infra_hosts + app))
+    cns = [server.name for server in cluster_hosts
+           if server.metadata['host-type'] == 'cns']
+
+    nodes = list(set(masters + infra_hosts + app + cns))
 
     dns = [server.name for server in cluster_hosts
            if server.metadata['host-type'] == 'dns']
@@ -59,6 +63,7 @@ def build_inventory():
     inventory['nodes'] = {'hosts': nodes}
     inventory['infra_hosts'] = {'hosts': infra_hosts}
     inventory['app'] = {'hosts': app}
+    inventory['glusterfs'] = {'hosts': cns}
     inventory['dns'] = {'hosts': dns}
     inventory['lb'] = {'hosts': load_balancers}
 
@@ -84,16 +89,25 @@ def build_inventory():
         # TODO(shadower): what about multiple networks?
         if server.private_v4:
             hostvars['private_v4'] = server.private_v4
+            hostvars['openshift_ip'] = server.private_v4
+
             # NOTE(shadower): Yes, we set both hostname and IP to the private
             # IP address for each node. OpenStack doesn't resolve nodes by
             # name at all, so using a hostname here would require an internal
             # DNS which would complicate the setup and potentially introduce
             # performance issues.
-            hostvars['openshift_ip'] = server.private_v4
-            hostvars['openshift_hostname'] = server.private_v4
+            hostvars['openshift_hostname'] = server.metadata.get(
+                'openshift_hostname', server.private_v4)
         hostvars['openshift_public_hostname'] = server.name
 
+        if server.metadata['host-type'] == 'cns':
+            hostvars['glusterfs_devices'] = ['/dev/nvme0n1']
+
         node_labels = server.metadata.get('node_labels')
+        # NOTE(shadower): the node_labels value must be a dict not string
+        if not isinstance(node_labels, Mapping):
+            node_labels = json.loads(node_labels)
+
         if node_labels:
             hostvars['openshift_node_labels'] = node_labels
 
diff --git a/playbooks/openstack/openshift-cluster/install.yml b/playbooks/openstack/openshift-cluster/install.yml
index 3211f619a..2ab7d14a0 100644
--- a/playbooks/openstack/openshift-cluster/install.yml
+++ b/playbooks/openstack/openshift-cluster/install.yml
@@ -9,4 +9,7 @@
 # some logic here?
 
 - name: run the cluster deploy
+  import_playbook: ../../prerequisites.yml
+
+- name: run the cluster deploy
   import_playbook: ../../deploy_cluster.yml
diff --git a/playbooks/openstack/openshift-cluster/prerequisites.yml b/playbooks/openstack/openshift-cluster/prerequisites.yml
index 0356b37dd..8bb700501 100644
--- a/playbooks/openstack/openshift-cluster/prerequisites.yml
+++ b/playbooks/openstack/openshift-cluster/prerequisites.yml
@@ -2,11 +2,11 @@
 - hosts: localhost
   tasks:
   - name: Check dependencies and OpenStack prerequisites
-    include_role:
+    import_role:
       name: openshift_openstack
       tasks_from: check-prerequisites.yml
 
   - name: Check network configuration
-    include_role:
+    import_role:
       name: openshift_openstack
       tasks_from: net_vars_check.yaml
diff --git a/playbooks/openstack/openshift-cluster/provision.yml b/playbooks/openstack/openshift-cluster/provision.yml
index 583e72b51..a38d7bff7 100644
--- a/playbooks/openstack/openshift-cluster/provision.yml
+++ b/playbooks/openstack/openshift-cluster/provision.yml
@@ -3,7 +3,7 @@
   hosts: localhost
   tasks:
   - name: provision cluster
-    include_role:
+    import_role:
       name: openshift_openstack
       tasks_from: provision.yml
 
@@ -36,7 +36,7 @@
   hosts: localhost
   tasks:
   - name: Populate DNS entries
-    include_role:
+    import_role:
       name: openshift_openstack
       tasks_from: populate-dns.yml
     when:
@@ -47,20 +47,28 @@
   hosts: oo_all_hosts
   become: yes
   gather_facts: yes
-  roles:
-  - role: rhel_subscribe
+  tasks:
+  - name: Subscribe RHEL instances
+    import_role:
+      name: rhel_subscribe
     when:
     - ansible_distribution == "RedHat"
-    - rhsub_user | default(False)
-    - rhsub_pass | default(False)
+    - rhsub_user is defined
+    - rhsub_pass is defined
+
+  - name: Enable required YUM repositories
+    import_role:
+      name: openshift_repos
+    when:
+    - ansible_distribution == "RedHat"
+    - rh_subscribed is defined
 
-  tasks:
   - name: Install dependencies
-    include_role:
+    import_role:
       name: openshift_openstack
       tasks_from: node-packages.yml
 
   - name: Configure Node
-    include_role:
+    import_role:
       name: openshift_openstack
       tasks_from: node-configuration.yml
diff --git a/playbooks/openstack/sample-inventory/group_vars/OSEv3.yml b/playbooks/openstack/sample-inventory/group_vars/OSEv3.yml
index 933117127..a8663f946 100644
--- a/playbooks/openstack/sample-inventory/group_vars/OSEv3.yml
+++ b/playbooks/openstack/sample-inventory/group_vars/OSEv3.yml
@@ -14,12 +14,13 @@ openshift_hosted_router_wait: True
 openshift_hosted_registry_wait: True
 
 ## Openstack credentials
-#openshift_cloudprovider_kind=openstack
+#openshift_cloudprovider_kind: openstack
 #openshift_cloudprovider_openstack_auth_url: "{{ lookup('env','OS_AUTH_URL') }}"
 #openshift_cloudprovider_openstack_username: "{{ lookup('env','OS_USERNAME') }}"
 #openshift_cloudprovider_openstack_password: "{{ lookup('env','OS_PASSWORD') }}"
 #openshift_cloudprovider_openstack_tenant_name: "{{ lookup('env','OS_TENANT_NAME') }}"
-#openshift_cloudprovider_openstack_region="{{ lookup('env', 'OS_REGION_NAME') }}"
+#openshift_cloudprovider_openstack_region: "{{ lookup('env', 'OS_REGION_NAME') }}"
+#openshift_cloudprovider_openstack_blockstorage_version: v2
 
 
 ## Use Cinder volume for Openshift registry:
diff --git a/playbooks/openstack/sample-inventory/group_vars/all.yml b/playbooks/openstack/sample-inventory/group_vars/all.yml
index c7afe9a24..d63229120 100644
--- a/playbooks/openstack/sample-inventory/group_vars/all.yml
+++ b/playbooks/openstack/sample-inventory/group_vars/all.yml
@@ -7,6 +7,7 @@ openshift_openstack_dns_nameservers: []
 # # - set custom hostnames for roles by uncommenting corresponding lines
 #openshift_openstack_master_hostname: "master"
 #openshift_openstack_infra_hostname: "infra-node"
+#openshift_openstack_cns_hostname: "cns"
 #openshift_openstack_node_hostname: "app-node"
 #openshift_openstack_lb_hostname: "lb"
 #openshift_openstack_etcd_hostname: "etcd"
@@ -30,6 +31,7 @@ openshift_openstack_external_network_name: "public"
 # # - note: do not remove openshift_openstack_default_image_name definition
 #openshift_openstack_master_image_name: "centos7"
 #openshift_openstack_infra_image_name: "centos7"
+#openshift_openstack_cns_image_name: "centos7"
 #openshift_openstack_node_image_name: "centos7"
 #openshift_openstack_lb_image_name: "centos7"
 #openshift_openstack_etcd_image_name: "centos7"
@@ -37,6 +39,7 @@ openshift_openstack_default_image_name: "centos7"
 
 openshift_openstack_num_masters: 1
 openshift_openstack_num_infra: 1
+openshift_openstack_num_cns: 0
 openshift_openstack_num_nodes: 2
 
 # # Used Flavors
@@ -44,6 +47,7 @@ openshift_openstack_num_nodes: 2
 # # - note: do note remove openshift_openstack_default_flavor definition
 #openshift_openstack_master_flavor: "m1.medium"
 #openshift_openstack_infra_flavor: "m1.medium"
+#openshift_openstack_cns_flavor: "m1.medium"
 #openshift_openstack_node_flavor: "m1.medium"
 #openshift_openstack_lb_flavor: "m1.medium"
 #openshift_openstack_etcd_flavor: "m1.medium"
@@ -57,6 +61,7 @@ openshift_openstack_default_flavor: "m1.medium"
 # # - note: do not remove docker_default_volume_size definition
 #openshift_openstack_docker_master_volume_size: "15"
 #openshift_openstack_docker_infra_volume_size: "15"
+#openshift_openstack_docker_cns_volume_size: "15"
 #openshift_openstack_docker_node_volume_size: "15"
 #openshift_openstack_docker_etcd_volume_size: "2"
 #openshift_openstack_docker_lb_volume_size: "5"