iSCSI initiator on VDS_CentOS51_r8

[obriendp]

I'm trying to have several CentOS VDS instances connect to our Intel NAS via iSCSI, but I'm running into problems. It appears the kernel isn't loading the modules. These are the steps I've taken to connect to the NAS from CentOS.

# yum install iscsi-initiator-utils
---> Package iscsi-initiator-utils.i386 0:6.2.0.868-0.18.el5_3.1 set to be updated

edit the /etc/iscsi/iscsid.conf

start the service:
# /etc/init.d/iscsi start

But I get an error:

iscsid is stopped
Turning off network shutdown. Starting iSCSI daemon: [FAILED]
Setting up iSCSI targets: iscsiadm: No records found!
[ OK ]

/var/log/messages shows:

Sep 23 13:28:04 vds_centos51 iscsid: iSCSI logger with pid=3154 started!
Sep 23 13:28:04 vds_centos51 iscsid: Missing or Invalid version from /sys/module/scsi_transport_iscsi/version. Make sure a up to date scsi_transport_iscsi module is loaded and a up todate version of iscsid is running. Exiting...


I found a guy who had an issue with Fedora getting the wrong version of the iscsi-initiator and had to load modules by hand before it would work:

http://forums.fedoraforum.org/showthread.php?t=204703

And a Citrix forum shows a Debian Virtual server that cannot use the iscsi modules because of the kernel...
http://forums.citrix.com/thread.jspa?threadID=151228&tstart=0

The kernel version that shows for my VDS is 2.6.18.8-xenU, am I having the same problems as the Citrix guy?

Can I switch to a different kernel?

I can't even build open-iscsi from source since I don't have a /lib/modules/2.6.18.8-xenU/build directory....

When it comes to kernels and modules, I'm pretty lost.

Thanks!

[agathongroup]

obriendp,

The source code for the running kernels is available here on the forum at http://forum.3tera.com/showthread.php?t=164 — if you need to rebuild anything, you can do so with the appropriate version from that thread. However, that may not be necessary in this case.

After doing the initial "service iscsi start", we have to do the following:

* iscsiadm -m discovery -t sendtargets -p $HOST (where $HOST is the IP address of your Intel NAS)
* service iscsi restart

That should only be necessary once, as the appropriate discovery happens and "service iscsi start" will find the mount at your next boot. In any case, the device should be found and information dumped into /var/log/messages.

Give that a shot and see how that works!

Peter

[obriendp]

obriendp,

The source code for the running kernels is available here on the forum at http://forum.3tera.com/showthread.php?t=164 — if you need to rebuild anything, you can do so with the appropriate version from that thread. However, that may not be necessary in this case.

After doing the initial "service iscsi start", we have to do the following:

* iscsiadm -m discovery -t sendtargets -p $HOST (where $HOST is the IP address of your Intel NAS)
* service iscsi restart

That should only be necessary once, as the appropriate discovery happens and "service iscsi start" will find the mount at your next boot. In any case, the device should be found and information dumped into /var/log/messages.

Give that a shot and see how that works!

Peter

Thanks Peter, I'll take look at the kernel info.

Unfortunately, when I start the iscsid and it fails due to the "Missing or Invalid version from /sys/module/scsi_transport_iscsi/version.", trying to discover targets fails as well, the error states that the daemon isn't running.

Thanks again! I'll post again if I find a solution.

[obriendp]

I'm not sure what I should be doing with the Kernel source.

I've never built my own kernel. And I'm not sure what I need to do with the RPM's I built.

I tried installing the latest version of open-iscsi using the source for 2.7.4:

make KSRC=/root/rpm/BUILD/xen-3.2.2-src

but it ends in error saying that there is a kernel check:

make -C kernel
make[1]: Entering directory `/root/open-iscsi-2.0-871/kernel'
make[1]: *** No rule to make target `linux_2_6_', needed by `kernel_check'. Stop.
make[1]: Leaving directory `/root/open-iscsi-2.0-871/kernel'
make: *** [kernel] Error 2


The README for open-iscsi also says:

You need to enable "Cryptographic API" under "Cryptographic options" in the
kernel config. And you must enable "CRC32c CRC algorithm" even if
you do not use header or data digests. They are the kernel options,
CONFIG_CRYPTO and CONFIG_CRYPTO_CRC32C, respectively.

How do I go about doing that?

I think I'm in over my head.
But luckily, I have a cloud full of VDS instances to break and restore!

[LeoKalev]

Here is some info on building modules for the kernel used in AppLogic virtual appliances and VDS-es. Please note that I don't know anything about ISCSI, this is general info about building modules against the kernel.

You need to enable "Cryptographic API" under "Cryptographic options" in the
kernel config. And you must enable "CRC32c CRC algorithm" even if
you do not use header or data digests.

This should be already enabled in the default configuration used by AppLogic. You can see the options used to build your installed kernel in this file: /boot/config-`uname -r`.

I'm not sure what I should be doing with the Kernel source.
I've never built my own kernel. And I'm not sure what I need to do with the RPM's I built.

In your case, you probably do not need to rebuild the kernel at all. The 'xen-devel' package provides the /lib/modules/*/build directory that you need to build modules.

Just install the xen-devel package and build the modules following the instructions that came with the module sources.

If this doesn't work and you need the full kernel sources, anyway:

The procedure provided in the thread quoted above will leave the kernel source files under $rpm_root/BUILD/xen-3.2.2-src/linux-2.6.18-xen.hg (not in BUILD/xen-3.2.2-src/). Also, the RPM build uses what's called a "separate build directory", and the path to the build directory needs to be given to the module builder instead of the path to the sources, therefore, your build command line may need to be like this:

make KSRC=/root/rpm/BUILD/xen-3.2.2-src/build-linux-2.6.18-xenU_i386

If the make file for the ISCSI modules is not designed to work this way, you may need to give it the paths both to the sources and to the build directory, e.g., like this:

make KSRC=/root/rpm/BUILD/xen-3.2.2-src/linux-2.6.18-xen.hg KOBJ=/root/rpm/BUILD/xen-3.2.2-src/build-linux-2.6.18-xenU_i386

NOTE that the 'KOBJ' option is a guess on my side, the folks that wrote the ISCSI modules may have used a different name (could be O=..., or something else - check the ISCSI readme files).

[obriendp]

Leo, Thank you so much!

You've pointed me down the correct path!

Looking at the README for open-iscsi I found what they experted for the output directory.

By default the kernel source found at
/lib/modules/`uname -a`/build
will be used to compile the open-iscsi modules. To specify a different
kernel to build against use:

make KSRC=<kernel-src>

or cross-compilation:

make KSRC=<kernel-src> KARCH="ARCH=um"

To compile on SUSE Linux you'll have to use

make KSRC=/usr/src/linux \
KBUILD_OUTPUT=/usr/src/linux-obj/<arch>/<config>

where <config> is the kernel configuration to use (eg. 'smp').

For Red Hat/Fedora and Debian distributions open-iscsi can be installed by
typing "make install". This will copy iscsid and iscsiadm to /usr/sbin, the
init script to /etc/init.d, and the kernel modules: iscsi_tcp.ko,
libiscsi_tcp.ko, libiscsi.ko and scsi_transport_iscsi to
/lib/modules/`uname -r`/kernel/drivers/scsi/ overwriting existing iscsi modules.

....

Also, please be aware that the compatibility patches that enable these
iscsi modules to run on kernels older than 2.6.25 will not update the
ib_iser module; you may get warnings related to mismatched symbols on
this driver, in which case you'll be unable to load ib_iser and
open-iscsi simultaneously.


Trying different "make" combinations in the open-iscsi folder showed the following command gave the most promise for building the module.

make KSRC=/root/rpm/BUILD/xen-3.2.2-src/linux-2.6.18-xen.hg \ KBUILD_OUTPUT=/root/rpm/BUILD/xen-3.2.2-src/build-linux-2.6.18-xenU_x86_32

But at the end I get errors:

make[2]: Entering directory `/root/rpm/BUILD/xen-3.2.2-src/linux-2.6.18-xen.hg'
CC [M] /root/open-iscsi-2.0-870.3/kernel/scsi_transport_iscsi.o
In file included from /root/open-iscsi-2.0-870.3/kernel/scsi_transport_iscsi.h:34,
from /root/open-iscsi-2.0-870.3/kernel/scsi_transport_iscsi.c:33:
/root/open-iscsi-2.0-870.3/kernel/open_iscsi_compat.h:190: error: redefinition of ‘shost_priv’
/root/rpm/BUILD/xen-3.2.2-src/linux-2.6.18-xen.hg/include/scsi/scsi_host.h:644: error: previous definition of ‘shost_priv’ was here
/root/open-iscsi-2.0-870.3/kernel/open_iscsi_compat.h:218: error: redefinition of ‘scsi_set_resid’
/root/rpm/BUILD/xen-3.2.2-src/linux-2.6.18-xen.hg/include/scsi/scsi_cmnd.h:150: error: previous definition of ‘scsi_set_resid’ was here
/root/open-iscsi-2.0-870.3/kernel/open_iscsi_compat.h:223: error: redefinition of ‘scsi_get_resid’
/root/rpm/BUILD/xen-3.2.2-src/linux-2.6.18-xen.hg/include/scsi/scsi_cmnd.h:155: error: previous definition of ‘scsi_get_resid’ was here
/root/open-iscsi-2.0-870.3/kernel/scsi_transport_iscsi.c: In function ‘__iscsi_unblock_session’:
/root/open-iscsi-2.0-870.3/kernel/scsi_transport_iscsi.c:551: warning: unused variable ‘ihost’
make[4]: *** [/root/open-iscsi-2.0-870.3/kernel/scsi_transport_iscsi.o] Error 1
make[3]: *** [_module_/root/open-iscsi-2.0-870.3/kernel] Error 2
make[2]: *** [modules] Error 2
make[2]: Leaving directory `/root/rpm/BUILD/xen-3.2.2-src/linux-2.6.18-xen.hg'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/root/open-iscsi-2.0-870.3/kernel'
make: *** [all] Error 2

Are the redefinition errors I'm getting at the end of the make process telling me that I can't build this module for this kernel? Or am I doing something wrong? Do I need more utilities installed?

[PavelGeorgiev]

If it is just iSCSI support that you are looking for, its already part of the domU kernel that will be available in AppLogic 2.7 production release which is coming in a few weeks.

Here are the domU kernel rpms from the release in case you want to use them *now*:

http://download2.3tera.net/GridU/xen-ukrnl-3.2.2-16.i386.rpm
http://download2.3tera.net/GridU/xen-ukrnl-3.2.2-16.x86_64.rpm
http://download2.3tera.net/GridU/xen-umods-3.2.2-16.i386.rpm
http://download2.3tera.net/GridU/xen-umods-3.2.2-16.x86_64.rpm

[obriendp]

Awesome!
Thanks!

Just two more quick questions:
1: Do I just force the rpm install and ignore the errors I see when I try to install?
rpm -i xen-ukrnl-3.2.2-16.i386.rpm
file /boot/System.map-2.6.18.8-xenU from install of xen-ukrnl-3.2.2-16 conflicts with file from package xen-ukrnl-3.2.2-2
file /boot/config-2.6.18.8-xenU from install of xen-ukrnl-3.2.2-16 conflicts with file from package xen-ukrnl-3.2.2-2
file /boot/vmlinuz-2.6.18.8-xenU from install of xen-ukrnl-3.2.2-16 conflicts with file from package xen-ukrnl-3.2.2-2

2: Once I've installed the rpm's (ignoring the errors and forcing the install) and installed the iscsi-initiator-utils via yum, do I need to create a place holder volume on the VDS? I'm not getting a block device in /dev for the new mount after discovering and connecting to the iSCSI target.
here's my /var/log/messages:

Sep 30 13:45:57 localhost Installed: iscsi-initiator-utils.i386 6.2.0.868-0.18.el5_3.1
Sep 30 13:47:49 localhost kernel: SCSI subsystem initialized
Sep 30 13:47:49 localhost kernel: Loading iSCSI transport class v1.1-646.<5>iscsi: registered transport (tcp)
Sep 30 13:47:49 localhost iscsid: iSCSI logger with pid=14817 started!
Sep 30 13:47:50 localhost iscsid: transport class version 1.1-646. iscsid version 2.0-868
Sep 30 13:47:50 localhost iscsid: iSCSI daemon with pid=14818 started!
Sep 30 13:48:13 localhost iscsid: iscsid shutting down.
Sep 30 13:48:13 localhost kernel: SCSI subsystem initialized
Sep 30 13:48:13 localhost kernel: Loading iSCSI transport class v1.1-646.<5>iscsi: registered transport (tcp)
Sep 30 13:48:13 localhost iscsid: iSCSI logger with pid=14913 started!
Sep 30 13:48:14 localhost kernel: scsi0 : iSCSI Initiator over TCP/IP
Sep 30 13:48:14 localhost iscsid: transport class version 1.1-646. iscsid version 2.0-868
Sep 30 13:48:14 localhost iscsid: iSCSI daemon with pid=14914 started!
Sep 30 13:48:15 localhost kernel: Vendor: LEFTHAND Model: iSCSIDisk Rev: 65
Sep 30 13:48:15 localhost kernel: Type: Direct-Access ANSI SCSI revision: 05
Sep 30 13:48:15 localhost iscsid: received iferror -38
Sep 30 13:48:15 localhost last message repeated 8 times
Sep 30 13:48:15 localhost iscsid: received iferror -22
Sep 30 13:48:15 localhost last message repeated 2 times
Sep 30 13:48:15 localhost iscsid: connection0:0 is operational now

Here are the commnads I ran:

[root@node1 ~]# /etc/init.d/iscsi start
iscsid is stopped
Turning off network shutdown. Starting iSCSI daemon: [ OK ]
[ OK ]
Setting up iSCSI targets: iscsiadm: No records found!
[ OK ]

[root@node1 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.0.2
192.168.0.2:3260,1 iqn.1986-03.com.intel:test:74:node1

[root@node1 ~]# /etc/init.d/iscsi restart
Stopping iSCSI daemon:
iscsid dead but pid file exists
Turning off network shutdown. Starting iSCSI daemon: [ OK ]
[ OK ]
Setting up iSCSI targets: Logging in to [iface: default, target: iqn.1986-03.com.intel:test:74:node1, portal: 192.168.0.2,3260]
Login to [iface: default, target: iqn.1986-03.com.intel:test:74:node1, portal: 192.168.0.2,3260]: successful
[ OK ]


here are the documents I'm working from to configure iSCSI.
http://www.cyberciti.biz/tips/rhel-centos-fedora-linux-iscsi-howto.html
http://www.cuddletech.com/articles/iscsi/ar01s04.html
(They've worked for me in the past)

[Boris1]

Here is some info on building modules for the kernel used in AppLogic virtual appliances and VDS-es. Please note that I don't know anything about ISCSI, this is general info about building modules against the kernel.