[clug] NFSv4 "Invalid argument"

Tue Oct 14 05:33:24 MDT 2014

     Hi Scott,

# mount -t nfs  -v
192.168.0.12:/mnt/vg2/vol1/san12_nfs_ds1/kvm-images/images
/var/lib/libvirt/images
mount.nfs: timeout set for Tue Oct 14 21:41:41 2014
mount.nfs: trying text-based options
'vers=4,addr=192.168.0.12,clientaddr=192.168.0.137'

# chown libvirt-qemu:libvirt-qemu /var/lib/libvirt/images/mc54.img 
chown: changing ownership of `/var/lib/libvirt/images/mc54.img':
Invalid argument

"You haven't answered the question of why the shares need to be owned
by
root. I'm still uncertain this is not an x/y problem." - well I
thought I had when I reference the statement "This ownership issue was
affecting
the creation of VM images on the NFS share" ?

I want the NFS share to be the permissions of the person/service or
group of the person/service who creates or modifies the file. Is this
possible with NFS ?  I am confident that NFS can do this, at least
with NFSv4.  The problem that exists is that I want to mount an NFS
share to "/var/lib/libvirt/images" so that I can have virt-manager
from any hypervisor-server that is on my network, access the NFS share
and run my virtual machines that are in the NFS share. However when I
attempt to do this the owner of the images is nobody:nogroup with
permissions of 600, and virt-manager cannot access the virtual machine
image files.

"Error starting domain: internal error Process exited while reading
console log output: char device redirected to /dev/pts/2
kvm: -drive
file=/var/lib/libvirt/images/MC43.img,if=none,id=drive-virtio-disk0,format=raw:
could not open disk image /var/lib/libvirt/images/MC43.img: Permission
denied"

It seems that I am unable change the ownership of the files, but since
you asked me why I wanted to do this, I have now tried to change the
permissions of the files, and this worked, and virt-manager can run
the virtual machine images. Though now security is some what degraded.
chmod 666 /var/lib/libvirt/images/MC43.img

"Please just click on the "Reply" button in your MUA instead of
creating
a new post" - it seems I am doing something wrong, just not sure
what.  If I click on my reply button, I only get the email of the
person who responded, not  linux at lists.samba.org, so I click reply
all, though I don't think this is causing the issue you are referring
to.  Would you be referring to me removing the "Re:" part of the
email?  I can stop that but then the email will not start with [clug]
which I thought it was supposed to ?  Apologies, I am still not sure
of the email forum conventions, I am more used to phpbb3 style of
forums. I can learn, so please let me know.

Thank you for your responses and comments.  Sometimes I try things
that just don't make sense in a linux world because I do not fully
understand how the system works. For example, while I wanted to map
"/var/lib/libvirt/images" to an NFS share, this is evidently not how
virt-manager works with virtual machine images, it needs more than
just the virtual machine's *.img file, and the xml file that it also
uses is not located in "/var/lib/libvirt/images", so it is not as easy
as just sharing the directory that the *.img file resides in. In fact
I don't think that virt-manager is designed to work this way (sadly),
but I will learn more about virt-manager as I get to spend more time
working with it.  At this point I can say, I am impressed how well 
Debian Wheezy running KVM and virt-manager runs in VMware Workstation
(on Windows 7) which I am currently using for this testing. I have
about 3TB of NFS share in an OpenFiler virtual machine also running in
VMware Workstation as this is where I had previously been testing ESXi
VM migration, HA, etc, so since I had the NFS share, it was quick for
me to use it in transferring VMs from a physical
Debian-KVM-Virt-Manager server to my VMware Workstation hosted
Debian-KVM-Virt-Manager server (e.g. my test environment).

If anyone is familiar with virt-manager, I am interested if and how it
is designed to work with NFS Shared storage for virtual machines.  It
seems that virt-manager supports "dir:", "disk:", "iscsi:",
"logical:", "mpath:", "netfs:", and "scsi:" ??

http://osdir.com/ml/libvir-list/2009-05/msg00080.html  - please read
this, maybe it has something to do with my issues ?
First, we've confirmed that Fedora 11 preview (fully updated) has an
issue with 
properly mounting NFS shares without the noacl option. Fedora 10 works
fine 
with regard to mounting NFS shares. That issue causes problems when
attempting 
to write files to an NFS share and more importantly, for this group,
causes 
problems when attempting to create an NFS based storage pool and then 
subsequently creating volumes on said NFS share. That bug is 499178.
This was 
all figured out with Openfiler support and with the support of the
fine folks 
working on libvirt and virt-manager. There is a somewhat ugly
workaround for 
this and I can describe it in case anyone is interested. 

Thanks...

is 
the shares do not need to be owned by "root" as far as I know, but I
answered the best I could, by referring to;
"Before I specified a domain on my servers, files created on the NFS
shares had ownership nobody.nobody, regardless of 
what user created it (even root). This ownership issue was affecting
the creation of VM images on the NFS share, which 
was fixed after I specified a domain in /etc/idmapd.conf. My VM images
now have ownership qemu.qemu, as it should be."

As you can see by the 

At Tuesday, 14-10-2014 on 14:05 Scott Ferguson wrote:

Hasty reply.

You haven't answered the question of why the shares need to be owned
by
root. I'm still uncertain this is not an x/y problem.

You haven't posted /etc/exports. Please post the output of "showmount
-e" (anonymise if you feel it's necessary)

You are using references that are:-
;seriously outdated - best ignore advice that doesn't apply to v4[*1]
;RedHat based - which is not the Debian way[*2]

I don't understand the domain name references - that's not the problem
you referred to in the OP. "rpc.nfsd -H hostname"??

Please just click on the "Reply" button in your MUA instead of
creating
a new post - I'm busy and a "bit thick" so the more work I've got to
do
to try and follow the conversation *and* make sense of it - the less
likely any reply will help you.
http://www.catb.org/esr/faqs/smart-questions.html

[*1]
I'd suggest filtering out results earlier than 2003 (RFC 3530)

[*2]
http://manpages.debian.org/cgi-bin/man.cgi?query=exports&apropos=0&sektion=0&manpath=Debian+7.0+wheezy&format=html&locale=en
The section on User ID Mapping may be informative

http://manpages.debian.org/cgi-bin/man.cgi?query=exportfs&sektion=8&apropos=0&manpath=Debian+7.0+wheezy&locale=en

On 14/10/14 13:55, George at Clug wrote:
>     _Maybe after a few more weeks of working on NFS I will
understand
> it better._

... or forming the same opinion of NFS v4 as Theo de Raadt :D

> 
> _Below I attempt to answer to some of the questions that people
asked
> me; _
> 
> _I am using OpenFiler v2.99 as the NFS server, so I am limited as to
> personally modifying the NFS server side._
>
http://www.vmwarebits.com/content/install-and-configure-openfiler-esxi-shared-storage-nfs-and-iscsi

A possible resource might be https://forums.openfiler.com/ if you
haven't tried already. It's a NAS I've avoided, and I haven't touched
VMWare for a looong time.

-------------------88-----------------------------
>
=========================================================================
> 
> http://debian-handbook.info/browse/stable/sect.nfs-file-server.html
> Example 11.23. The /etc/default/nfs-common file
> # Do you want to start the idmapd daemon? It is only needed for
NFSv4.
> NEED_IDMAPD=
> 
> 
> 

An excellent project and book.

Kind regards

-- 
linux mailing list
linux at lists.samba.org
https://lists.samba.org/mailman/listinfo/linux