如何配置sheepdog作为openstack Cinder的存储后端

小编给大家分享一下如何配置sheepdog作为openstack Cinder的存储后端，相信大部分人都还不怎么了解，因此分享这篇文章给大家参考一下，希望大家阅读完这篇文章后大有收获，下面让我们一起去了解一下吧！

Cinder是Openstack的Software Defined Storage(SDS)开源方案，支持多种存储后端。其中就包括sheepdog，一种分布式的开源块设备。具有具有零配置、Thin-Provision、高可靠、智能节点管理、容量线性扩展、虚拟机感知(底层支持冷热迁移和快照、克隆等)、支持计算与存储混合架构的特点等，可扩展到上千级别的物理节点。

1. Openstack环境说明

开始研究openstack的第一步当然是安装openstack。 对于一般用户来说，最方便的获取openstack环境就是直接使用各大linux 发行版的openstack package。Ubuntu，Fedora/Centos/RHEL, SUSE都已经作了大量打包工作。

例如 Redhat的PDO： http://openstack.redhat.com/Quickstart

http://docs.openstack.org/trunk/install-guide/install/yum/content/

对于开发者开说，最好是使用devstack，它是一整套的BASH脚本，从源码一步步的部署好openstack 开发环境，因此升级到最新代码相当方便。

2. 安装sheepdog

参考官方wiki: https://github.com/sheepdog/sheepdog/wiki/Getting-Started
sheepdog和qemu项目高度集成，可以直接支持qemu的后端存储。

例如：
qemu-system-x86_64 -drive file=sheepdog:192.168.10.2:7000:Alice
意思是启动一个虚拟机，后端的文件位于192.168.10.2这台sheepdog cluster node上的Alice VDI

因为sheepdog没有自己开发集群管理，依赖的是corosync。所以务必安装wiki的说明源码安装，并保证服务启动。
否则sheepdog daemon会启动不了

[root@fedora ~]# cat /var/log/sheepdog/sheep.log Mar 25 22:56:53   INFO [main] md_add_disk(337) /var/log/sheepdog/obj, vdisk nr 68, total disk 1Mar 25 22:56:55  ERROR [main] corosync_init(733) failed to initialize cpg (6) - is corosync running?Mar 25 22:56:55  ERROR [main] main(867) failed to create sheepdog cluster

3. Congiure Cinder use sheepdog as back-end storage driver

修改Cinder的配置文件，默认的比如"/etc/cinder/cinder.conf"
volume_driver=cinder.volume.drivers.sheepdog.SheepdogDriver
修改后，重启cinder服务 openstack-service restart cinder

4. 创建一个Cinder volume

我这里测试的是把一个glance image转化成bootable volume

[ray@fedora cinder]$ cinder create --image-id 285ac2bf-e2d4-4bbd-b438-f2fa4f24bb27 --display-name CirrosBootVol 2+---------------------+--------------------------------------+|       Property      |                Value                 |+---------------------+--------------------------------------+|     attachments     |                  []                  ||  availability_zone  |                 nova                 ||       bootable      |                false                 ||      created_at     |      2014-03-26T14:55:50.631075      || display_description |                 None                 ||     display_name    |            CirrosBootVol             ||          id         | 1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 ||       image_id      | 285ac2bf-e2d4-4bbd-b438-f2fa4f24bb27 ||       metadata      |                  {}                  ||         size        |                  2                   ||     snapshot_id     |                 None                 ||     source_volid    |                 None                 ||        status       |               creating               ||     volume_type     |                 None                 |+---------------------+--------------------------------------+[ray@fedora post]$ cinder list+--------------------------------------+----------------+-----------------+------+-------------+----------+--------------------------------------+|                  ID                  |     Status     |   Display Name  | Size | Volume Type | Bootable |             Attached to              |+--------------------------------------+----------------+-----------------+------+-------------+----------+--------------------------------------+| 1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 |   available    |  CirrosBootVol  |  2   |     None    |   true   |                                      || c1dfc42c-32c1-4104-a7cc-effc4ce86385 |     in-use     |     Vdisk3G     |  3   |     None    |  false   | b358aa2e-a8e9-4cb8-9f2a-e7c6e4d4f477 |+--------------------------------------+----------------+-----------------+------+-------------+----------+--------------------------------------+# use dog or collie command to show [ray@fedora post]$ collie vdi list  Name        Id    Size    Used  Shared    Creation time   VDI id  Copies  Tag    volume-c1dfc42c-32c1-4104-a7cc-effc4ce86385     0  3.0 GB  1.2 GB  0.0 MB 2014-03-25 23:28   31433a      1    volume-1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911     0  2.0 GB   32 MB  0.0 MB 2014-03-26 22:55   cd4679      1

以上步骤结束之后，我们就可以在系统中看到创建好的cinder volume。 我在操作过程实际上碰到一个问题。

从glance image创建volume没有成功，打开cinder的log文件，会看到具体的错误信息：

2014-03-26 22:51:14.335 5821 DEBUG cinder.openstack.common.processutils [req-1e55cd8c-d4b4-45d4-9f2b-9bf535d5446b 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/log/conversion/tmpqXIy_Y execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:1422014-03-26 22:51:14.414 5821 DEBUG cinder.openstack.common.processutils [req-1e55cd8c-d4b4-45d4-9f2b-9bf535d5446b 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Running cmd (subprocess): collie vdi delete volume-c1cb8b40-82fa-4722-9424-2368ac5dc187 execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:1422014-03-26 22:51:14.597 5821 DEBUG cinder.openstack.common.processutils [req-1e55cd8c-d4b4-45d4-9f2b-9bf535d5446b 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw ', mode 'w+b' at 0x341c7a0> sheepdog:volume-c1cb8b40-82fa-4722-9424-2368ac5dc187 execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:1422014-03-26 22:51:14.682 5821 DEBUG cinder.openstack.common.processutils [req-1e55cd8c-d4b4-45d4-9f2b-9bf535d5446b 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Result was 1 execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:1672014-03-26 22:51:14.686 5821 ERROR cinder.volume.flows.create_volume [req-1e55cd8c-d4b4-45d4-9f2b-9bf535d5446b 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Failed to copy image 285ac2bf-e2d4-4bbd-b438-f2fa4f24bb27 to volume: c1cb8b40-82fa-4722-9424-2368ac5dc187, error: qemu-img: Unknown protocolqemu-img: Could not open '', mode 'w+b' at 0x341c7a0>': No such file or directoryqemu-img: Could not open '', mode 'w+b' at 0x341c7a0>'

从log显示来看，cinder创建volume的过程对应的代码位于 volume/drivers/sheepdog.py 中的 copy_image_to_volume函数。

102     def copy_image_to_volume(self, context, volume, image_service, image_id):103         # use the image_conversion_dir as a temporary place to save the image104         conversion_dir = CONF.image_conversion_dir105         self._ensure_dir_exists(conversion_dir)106         with tempfile.NamedTemporaryFile(dir=conversion_dir) as tmp:107             # (wenhao): we don't need to convert to raw for sheepdog.108             image_utils.fetch_verify_image(context, image_service,109                                            image_id, tmp.name)110 111             # remove the image created by import before this function.112             # see volume/drivers/manager.py:_create_volume113             self._delete(volume)114             # convert and store into sheepdog115             image_utils.convert_image(tmp, 'sheepdog:%s' % volume['name'],116                                       'raw')117             self._resize(volume)

解释下就是这么写步骤：

1. 从glance 下载image到本地目录。cinder有对应的下载目录配置：

 486 # 487 # Options defined in cinder.image.image_utils 488 # 489  490 # Directory used for temporary storage during image conversion 491 # (string value) 492 image_conversion_dir=/var/log/conversion

2. 下载到目录之后，用qemu-img info 命令取回image disk的信息，主要是检查cinder输入的volume szie参数，应当不能小于image disk的大小。这你懂的，不然不能够。。。

image/image_utils.py：fetch_verify_image

238         # NOTE(xqueralt): If the image virtual size doesn't fit in the239         # requested volume there is no point on resizing it because it will240         # generate an unusable image.241         if size is not None and data.virtual_size > size:242             params = {'image_size': data.virtual_size, 'volume_size': size}243             reason = _("Size is %(image_size)dGB and doesn't fit in a "244                        "volume of size %(volume_size)dGB.") % params245             raise exception.ImageUnacceptable(image_id=image_id, reason=reason)

3. 到了这一步就要用qemu-img convert 把image 转化成sheepdog VDI。 正确的命令是：

qemu-img convert -O raw /var/log/conversion/tmpf9sUFs sheepdog:volume-1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911

成功的话，就会在sheepdog系统中create一个名字是"volume-1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911"的VDI。 结果代码有个bug，实际生成的命令如上面错误log显示：

qemu-img convert -O raw ', mode 'w+b' at 0x341c7a0> sheepdog:volume-c1cb8b40-82fa-4722-9424-2368ac5dc187

这里明显是参数传错了，应该传入的是下载到本地目录的image 文件名。在tempfile.NamedTemporaryFile 这个context中，tmp就是tempfile的object instance。 改成tmp.name就ok了。

115             image_utils.convert_image(tmp.name, 'sheepdog:%s' % volume['name'],116                                       'raw'

修改后，重新操作，成功后的log：

2014-03-26 22:55:51.628 10137 DEBUG cinder.openstack.common.processutils [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/log/conversion/tmpf9sUFs execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:1422014-03-26 22:55:51.710 10137 DEBUG cinder.openstack.common.processutils [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Running cmd (subprocess): collie vdi delete volume-1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:1422014-03-26 22:55:52.036 10137 DEBUG cinder.openstack.common.processutils [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw /var/log/conversion/tmpf9sUFs sheepdog:volume-1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:1422014-03-26 22:55:53.729 10137 DEBUG cinder.openstack.common.processutils [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Running cmd (subprocess): collie vdi resize volume-1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 2147483648 execute /usr/lib/python2.7/site-packages/cinder/openstack/common/processutils.py:1422014-03-26 22:55:53.751 10137 DEBUG cinder.volume.flows.create_volume [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Downloaded image 285ac2bf-e2d4-4bbd-b438-f2fa4f24bb27 ((None, None)) to volume 1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 successfully _copy_image_to_volume /usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__init__.py:13832014-03-26 22:55:53.752 10137 DEBUG cinder.volume.flows.create_volume [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Marking volume 1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 as bootable _enable_bootable_flag /usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__init__.py:13232014-03-26 22:55:53.839 10137 DEBUG cinder.volume.flows.create_volume [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Copying metadata from image 285ac2bf-e2d4-4bbd-b438-f2fa4f24bb27 to 1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 _handle_bootable_volume_glance_meta /usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__init__.py:12862014-03-26 22:55:53.840 10137 DEBUG cinder.volume.flows.create_volume [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Creating volume glance metadata for volume 1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911 backed by image 285ac2bf-e2d4-4bbd-b438-f2fa4f24bb27 with: {'container_format': u'bare', 'min_ram': 0, 'disk_format': u'qcow2', 'image_name': u'CirrOS 0.3.1', 'image_id': u'285ac2bf-e2d4-4bbd-b438-f2fa4f24bb27', 'checksum': u'd972013792949d0d3ba628fbe8685bce', 'min_disk': 0, 'size': 13147648} _capture_volume_image_metadata /usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__init__.py:14202014-03-26 22:55:54.441 10137 DEBUG cinder.volume.flows.create_volume [req-2e51dcbc-af6a-4a5f-9a92-008b230acb45 44d5eb0d5b1d4c4189affed764b24c60 b738ad4284ed466f91e677af13e71815] Volume 1e3eb4f2-c60c-4eeb-9bc0-41f1afe7c911: creating export __call__ /usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__init__.py:1519

至于为什么tempfile.NamedTemporaryFile 抛出来的异常显示是', mode 'w+b' at 0x341c7a0>。 主要是因为openstack代码中使用了Greenlet python库来启动进程跑本地命令。

以上是"如何配置sheepdog作为openstack Cinder的存储后端"这篇文章的所有内容，感谢各位的阅读！相信大家都有了一定的了解，希望分享的内容对大家有所帮助，如果还想学习更多知识，欢迎关注行业资讯频道！