5月 312016
 

After installing a new OpenStack environment, it is useful to validate the initial high availability configuration.
This example checklist assumes there are an architecture with three controllers, but it can be adapted as needed.

General Tests

Test Scenario Description Check Activity Result Notes
Firstboot Reboot all controllers at the same time. Check that all services start as expected.
  1. Create an instance.

  2. Create and assign a cinder volume to the instance.

  3. Check IP connectivity to the instance.

  4. Check instance access to metadata service.

  5. Disconnect the cinder volume from instance (and destroy the cinder volume).

  6. Connect to the dashboard and terminate the instance.

Controlled reboot Reboot each controller. Check proper service relocation. Same as firstboot.
Crash Crash each controller. Check proper service relocation. Same as firstboot.
Compute node reboot Reboot each compute node.
  • After the compute node is rebooted, check if the instances hosted on it are running. Otherwise, hard reboot them.

  • Check IP connectivity to the instances.

  • Check instance access to metadata service.

Service-Specific Tests

Keystone Tests

Test Scenario Description Check Activity Result Notes
Stop Keystone on controller 1. Stop Keystone on controller 1. Check load balancer behavior. keystone user-list works (record recovery time if possible)
Stop Keystone on controller 2. Start Keystone on controller 1, stop Keystone on controller 1. Check load balancer behavior. keystone user-list works (record recovery time if possible)
Stop Keystone on controller 3. Start Keystone on controller 2, stop Keystone on controller 3. Check load balancer behavior. keystone user-list works (record recovery time if possible)
Start Keystone on controller 3. Start Keystone on controller 3. Check load balancer behavior. keystone user-list works (record recovery time if possible)

Glance Tests

Test Scenario Description Check Activity Result Notes
Stop Glance on controller 1. Stop Glance on controller 1. Check load balancer behavior. glance image-list works (record recovery time if possible)
Stop Glance on controller 2. Start Glance on controller 1, stop Glance on controller 1. Check load balancer behavior. glance image-list works (record recovery time if possible)
Stop Glance on controller 3. Start Glance on controller 2, stop Glance on controller 3. Check load balancer behavior. glance image-list works (record recovery time if possible)
Start Glance on controller 3. Start Glance on controller 3. Check load balancer behavior. glance image-list works (record recovery time if possible)

Cinder Tests

Test Scenario Description Check Activity Result Notes
Stop Cinder on controller 1. Stop Cinder on controller 1. Check load balancer behavior. cinder list and cinder create --name test 5 works (record recovery time if possible).
Stop Cinder on controller 2. Start Cinder on controller 1, stop Cinder on controller 1. Check load balancer behavior. cinder list and cinder create --name test 5 works (record recovery time if possible).
Stop Cinder on controller 3. Start Cinder on controller 2, stop Cinder on controller 3. Check load balancer behavior. cinder list and cinder create --name test 5 works (record recovery time if possible).
Start Cinder on controller 3. Start Cinder on controller 3. Check load balancer behavior. cinder list and cinder create --name test 5 works (record recovery time if possible).

Neutron Tests

Test Scenario Description Check Activity Result Notes
Stop Neutron on controller 1. Stop Neutron on controller 1. Check load balancer behavior. neutron net-list and neutron floatingip-list works (record recovery time if possible).
Stop Neutron on controller 2. Start Neutron on controller 1, stop Neutron on controller 1. Check load balancer behavior. neutron net-list and neutron floatingip-list works (record recovery time if possible).
Stop Neutron on controller 3. Start Neutron on controller 2, stop Neutron on controller 3. Check load balancer behavior. neutron net-list and neutron floatingip-list works (record recovery time if possible).
Start Neutron on controller 3. Start Neutron on controller 3. Check load balancer behavior. neutron net-list and neutron floatingip-list works (record recovery time if possible).

Nova Tests

Test Scenario Description Check Activity Result Notes
Stop Nova on controller 1. Stop Nova on controller 1. Check load balancer behavior. nova list and nova secgroup-list works (record recovery time if possible).
Stop Nova on controller 2. Start Nova on controller 1, stop Nova on controller 1. Check load balancer behavior. nova list and nova secgroup-list works (record recovery time if possible).
Stop Nova on controller 3. Start Nova on controller 2, stop Nova on controller 3. Check load balancer behavior. nova list and nova secgroup-list works (record recovery time if possible).
Start Nova on controller 3. Start Nova on controller 3. Check load balancer behavior. nova list and nova secgroup-list works (record recovery time if possible).

Apache Tests

Test Scenario Description Check Activity Result Notes
Stop httpd on controller 1. Stop httpd on controller 1. Check load balancer behavior. Log onto Horizon and check if the GUI is working
Stop httpd on controller 2. Start httpd on controller 1, stop httpd on controller 1. Check load balancer behavior. Log onto Horizon and check if the GUI is working.
Stop httpd on controller 3. Start httpd on controller 2, stop httpd on controller 3. Check load balancer behavior. Log onto Horizon and check if the GUI is working.
Start httpd on controller 3. Start httpd on controller 3. Check load balancer behavior. Log onto Horizon and check if the GUI is working.

Final Tests

Test Scenario Description Check Activity Result Notes
Load test: 10 VMs Start 10 VMs simultaneously. Check the success rate.
  1. Create an instance.

  2. Create and assign a cinder volume to the instance.

  3. Check IP connectivity to the instance.

  4. Check instance access to metadata service.

  5. Disconnect the cinder volume from instance (and destroy the cinder volume).

  6. Connect to the dashboard and terminate the instance.

Load test: 20 VMs Start 20 VMs simultaneously. Check the success rate.
  1. Create an instance.

  2. Create and assign a cinder volume to the instance.

  3. Check IP connectivity to the instance.

  4. Check instance access to metadata service.

  5. Disconnect the cinder volume from instance (and destroy the cinder volume).

  6. Connect to the dashboard and terminate the instance.

Load test: 50 VMs Start 50 VMs simultaneously. Check the success rate.
  1. Create an instance.

  2. Create and assign a cinder volume to the instance.

  3. Check IP connectivity to the instance.

  4. Check instance access to metadata service.

  5. Disconnect the cinder volume from instance (and destroy the cinder volume).

  6. Connect to the dashboard and terminate the instance.

Final reboot Reboot the whole environment. Check proper behavior.
  1. Create an instance.

  2. Create and assign a cinder volume to the instance.

  3. Check IP connectivity to the instance.

  4. Check instance access to metadata service.

  5. Disconnect the cinder volume from instance (and destroy the cinder volume).

  6. Connect to the dashboard and terminate the instance.

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)