NFS Server->Client: I/O device error

Discussion:

(too old to reply)

Peter Karlsson

2005-01-10 10:23:06 UTC

I am NFS exporting 'home' from a Win2003 server to a number of Linux clients.
The Linux clients mounts 'home' via automount. This setup has worked fine
until I reinstalled the Win2003 server and installed SFU3.5 instead of SFU3.

This is what I did and what happened:
1. backed up 'home' to tape
2. Reinstalled server
3. Installed and configured SFU3.5
4. Created and exported the 'home' share
5. Restored a few home directories from tape as to confirm that everything
works OK
6. Logged in to a Linux client with a user that had its home directory
restored in (5). *NFS MOUNT WORKS!!*
7. Restored the rest of the home directories from tape
8. Logged in to a Linux client with the same user as in (6). *NFS MOUNT
FAILS!!*

The log file on the Linux client says "can't read superblock..."

According to some Linux users who also got their home directories restored
in (5) NFS mounts stopped working after a few hours. By that time I had
restored about ~200 home directories from tape.

In my efforts to trouble-shoot the problem and rule out the possibility that
the Linux clients are doing something strange I installed and configured "NFS
Client" on a Windows XP SP2 computer. I then try to map 'home' (Windows
Explorer->Tools->Map Network Drive) to Y: but get the following error
message: "Y:\ is not accessible. The request could not be performed because
of an I/O device error."
In the event viewer on the Windows XP client computer I got the following
event, but only once, som maybe that's not relevant: Event ID: 3019 Source:
MRxSmb Description: "The redirector failed to determine the connection type."

I have an additional NFS share on the Win2003 server, "testhome", set up the
same way only with fewer (~10) home directories. This share I can map.

The Win2003 server is not under heavy load and I don't think there is
anything wrong with the physical disks.

Any ideas?!?

Configuration:
The Windows 2003 Server is a DC for an Active Directory domain, fully
patched, Dell PowerEdge 2650 with SCSI raid5 disks, 2GB ram.

Jijo Jose [msft]

2005-01-11 01:25:53 UTC

Permalink

Can you try ininstalling SFU3.5 altogehter and configuring it afresh?

Post by Peter Karlsson
I am NFS exporting 'home' from a Win2003 server to a number of Linux clients.
The Linux clients mounts 'home' via automount. This setup has worked fine
until I reinstalled the Win2003 server and installed SFU3.5 instead of SFU3.
1. backed up 'home' to tape
2. Reinstalled server
3. Installed and configured SFU3.5
4. Created and exported the 'home' share
5. Restored a few home directories from tape as to confirm that everything
works OK
6. Logged in to a Linux client with a user that had its home directory
restored in (5). *NFS MOUNT WORKS!!*
7. Restored the rest of the home directories from tape
8. Logged in to a Linux client with the same user as in (6). *NFS MOUNT
FAILS!!*
The log file on the Linux client says "can't read superblock..."
According to some Linux users who also got their home directories restored
in (5) NFS mounts stopped working after a few hours. By that time I had
restored about ~200 home directories from tape.
In my efforts to trouble-shoot the problem and rule out the possibility that
the Linux clients are doing something strange I installed and configured "NFS
Client" on a Windows XP SP2 computer. I then try to map 'home' (Windows
Explorer->Tools->Map Network Drive) to Y: but get the following error
message: "Y:\ is not accessible. The request could not be performed because
of an I/O device error."
In the event viewer on the Windows XP client computer I got the following
MRxSmb Description: "The redirector failed to determine the connection type."
I have an additional NFS share on the Win2003 server, "testhome", set up the
same way only with fewer (~10) home directories. This share I can map.
The Win2003 server is not under heavy load and I don't think there is
anything wrong with the physical disks.
Any ideas?!?
The Windows 2003 Server is a DC for an Active Directory domain, fully
patched, Dell PowerEdge 2650 with SCSI raid5 disks, 2GB ram.

Charlie Russel - MVP

2005-01-11 16:49:01 UTC

Permalink

And if you do, please run SFUCLEAN.EXE from the SFU installation media.
--
Please, all replies to the newsgroup.
======================
Charlie Russel - MVP
NFS Authentication issues? See:
http://www.microsoft.com/technet/interopmigration/unix/sfu/nfsauth.mspx
RSH Problems? See:
http://www.microsoft.com/technet/interopmigration/unix/sfu/sfu35rsh.mspx

Post by Jijo Jose [msft]
Can you try ininstalling SFU3.5 altogehter and configuring it afresh?

Post by Peter Karlsson
I am NFS exporting 'home' from a Win2003 server to a number of Linux clients.
The Linux clients mounts 'home' via automount. This setup has worked
fine until I reinstalled the Win2003 server and installed SFU3.5
instead of SFU3.
1. backed up 'home' to tape
2. Reinstalled server
3. Installed and configured SFU3.5
4. Created and exported the 'home' share
5. Restored a few home directories from tape as to confirm that
everything works OK
6. Logged in to a Linux client with a user that had its home
directory restored in (5). *NFS MOUNT WORKS!!*
7. Restored the rest of the home directories from tape
8. Logged in to a Linux client with the same user as in (6). *NFS
MOUNT FAILS!!*
The log file on the Linux client says "can't read superblock..."
According to some Linux users who also got their home directories
restored in (5) NFS mounts stopped working after a few hours. By
that time I had restored about ~200 home directories from tape.
In my efforts to trouble-shoot the problem and rule out the
possibility that
the Linux clients are doing something strange I installed and
configured "NFS
Client" on a Windows XP SP2 computer. I then try to map 'home'
(Windows Explorer->Tools->Map Network Drive) to Y: but get the
following error message: "Y:\ is not accessible. The request could
not be performed because
of an I/O device error."
In the event viewer on the Windows XP client computer I got the
following event, but only once, som maybe that's not relevant: Event
MRxSmb Description: "The redirector failed to determine the
connection type."
I have an additional NFS share on the Win2003 server, "testhome", set up the
same way only with fewer (~10) home directories. This share I can
map. The Win2003 server is not under heavy load and I don't think there is
anything wrong with the physical disks.
Any ideas?!?
The Windows 2003 Server is a DC for an Active Directory domain, fully
patched, Dell PowerEdge 2650 with SCSI raid5 disks, 2GB ram.

Peter Karlsson

2005-01-12 12:23:05 UTC

Permalink

I reinstalled, but I didn't run SFUCLEAN.EXE first. I will do it over again
later.
I am almost tempted to install SFU3 instead, which used to work, and see
what happens. But since the Win2003 server is a NIS slave, the NIS master
runs SFU3.5 and the DC:s have NFSServerAuth from SFU3.5 installed this might
be a bad idea..?

/Peter

Post by Charlie Russel - MVP
And if you do, please run SFUCLEAN.EXE from the SFU installation media.
--
Please, all replies to the newsgroup.
======================
Charlie Russel - MVP
http://www.microsoft.com/technet/interopmigration/unix/sfu/nfsauth.mspx
http://www.microsoft.com/technet/interopmigration/unix/sfu/sfu35rsh.mspx

Post by Jijo Jose [msft]
Can you try ininstalling SFU3.5 altogehter and configuring it afresh?

Post by Peter Karlsson
I am NFS exporting 'home' from a Win2003 server to a number of Linux clients.
The Linux clients mounts 'home' via automount. This setup has worked
fine until I reinstalled the Win2003 server and installed SFU3.5
instead of SFU3.
1. backed up 'home' to tape
2. Reinstalled server
3. Installed and configured SFU3.5
4. Created and exported the 'home' share
5. Restored a few home directories from tape as to confirm that
everything works OK
6. Logged in to a Linux client with a user that had its home
directory restored in (5). *NFS MOUNT WORKS!!*
7. Restored the rest of the home directories from tape
8. Logged in to a Linux client with the same user as in (6). *NFS
MOUNT FAILS!!*
The log file on the Linux client says "can't read superblock..."
According to some Linux users who also got their home directories
restored in (5) NFS mounts stopped working after a few hours. By
that time I had restored about ~200 home directories from tape.
In my efforts to trouble-shoot the problem and rule out the
possibility that
the Linux clients are doing something strange I installed and configured "NFS
Client" on a Windows XP SP2 computer. I then try to map 'home'
(Windows Explorer->Tools->Map Network Drive) to Y: but get the
following error message: "Y:\ is not accessible. The request could
not be performed because
of an I/O device error."
In the event viewer on the Windows XP client computer I got the
following event, but only once, som maybe that's not relevant: Event
MRxSmb Description: "The redirector failed to determine the
connection type."
I have an additional NFS share on the Win2003 server, "testhome", set up the
same way only with fewer (~10) home directories. This share I can
map. The Win2003 server is not under heavy load and I don't think there is
anything wrong with the physical disks.
Any ideas?!?
The Windows 2003 Server is a DC for an Active Directory domain, fully
patched, Dell PowerEdge 2650 with SCSI raid5 disks, 2GB ram.

Charlie Russel - MVP

2005-01-12 17:28:28 UTC

Permalink

The version of NFS Server Auth doesn't matter. But I'd still go with SFU3.5.
There are substantial, but less visible, improvements in it.
--
Please, all replies to the newsgroup.
======================
Charlie Russel - MVP
NFS Authentication issues? See:
http://www.microsoft.com/technet/interopmigration/unix/sfu/nfsauth.mspx
RSH Problems? See:
http://www.microsoft.com/technet/interopmigration/unix/sfu/sfu35rsh.mspx

Post by Peter Karlsson
I reinstalled, but I didn't run SFUCLEAN.EXE first. I will do it over
again later.
I am almost tempted to install SFU3 instead, which used to work, and
see what happens. But since the Win2003 server is a NIS slave, the
NIS master runs SFU3.5 and the DC:s have NFSServerAuth from SFU3.5
installed this might be a bad idea..?
/Peter

Post by Jijo Jose [msft]
Can you try ininstalling SFU3.5 altogehter and configuring it afresh?

Post by Peter Karlsson
I am NFS exporting 'home' from a Win2003 server to a number of Linux clients.
The Linux clients mounts 'home' via automount. This setup has worked
fine until I reinstalled the Win2003 server and installed SFU3.5
instead of SFU3.
1. backed up 'home' to tape
2. Reinstalled server
3. Installed and configured SFU3.5
4. Created and exported the 'home' share
5. Restored a few home directories from tape as to confirm that
everything works OK
6. Logged in to a Linux client with a user that had its home
directory restored in (5). *NFS MOUNT WORKS!!*
7. Restored the rest of the home directories from tape
8. Logged in to a Linux client with the same user as in (6). *NFS
MOUNT FAILS!!*
The log file on the Linux client says "can't read superblock..."
According to some Linux users who also got their home directories
restored in (5) NFS mounts stopped working after a few hours. By
that time I had restored about ~200 home directories from tape.
In my efforts to trouble-shoot the problem and rule out the
possibility that
the Linux clients are doing something strange I installed and configured "NFS
Client" on a Windows XP SP2 computer. I then try to map 'home'
(Windows Explorer->Tools->Map Network Drive) to Y: but get the
following error message: "Y:\ is not accessible. The request could
not be performed because
of an I/O device error."
In the event viewer on the Windows XP client computer I got the
following event, but only once, som maybe that's not relevant: Event
MRxSmb Description: "The redirector failed to determine the
connection type."
I have an additional NFS share on the Win2003 server, "testhome", set up the
same way only with fewer (~10) home directories. This share I can
map. The Win2003 server is not under heavy load and I don't think
there is anything wrong with the physical disks.
Any ideas?!?
The Windows 2003 Server is a DC for an Active Directory domain,
fully patched, Dell PowerEdge 2650 with SCSI raid5 disks, 2GB ram.

Peter Karlsson

2005-01-12 12:11:04 UTC

Permalink

I have tried that now, but unfortunately It didn't do any good.

/Peter

Post by Jijo Jose [msft]
Can you try ininstalling SFU3.5 altogehter and configuring it afresh?

Jijo Jose [msft]

2005-01-13 00:53:30 UTC

Permalink

Can you confirm you reconfigured mapping or anonymous access as it was
before?
Do you see an RPC error 7 from your Linux clients?

Post by Peter Karlsson
I have tried that now, but unfortunately It didn't do any good.
/Peter

Post by Jijo Jose [msft]
Can you try ininstalling SFU3.5 altogehter and configuring it afresh?

Post by Peter Karlsson
I am NFS exporting 'home' from a Win2003 server to a number of Linux clients.
The Linux clients mounts 'home' via automount. This setup has worked fine
until I reinstalled the Win2003 server and installed SFU3.5 instead of SFU3.
1. backed up 'home' to tape
2. Reinstalled server
3. Installed and configured SFU3.5
4. Created and exported the 'home' share
5. Restored a few home directories from tape as to confirm that everything
works OK
6. Logged in to a Linux client with a user that had its home directory
restored in (5). *NFS MOUNT WORKS!!*
7. Restored the rest of the home directories from tape
8. Logged in to a Linux client with the same user as in (6). *NFS MOUNT
FAILS!!*
The log file on the Linux client says "can't read superblock..."
According to some Linux users who also got their home directories restored
in (5) NFS mounts stopped working after a few hours. By that time I had
restored about ~200 home directories from tape.
In my efforts to trouble-shoot the problem and rule out the possibility that
the Linux clients are doing something strange I installed and
configured
"NFS
Client" on a Windows XP SP2 computer. I then try to map 'home' (Windows
Explorer->Tools->Map Network Drive) to Y: but get the following error
message: "Y:\ is not accessible. The request could not be performed because
of an I/O device error."
In the event viewer on the Windows XP client computer I got the following
MRxSmb Description: "The redirector failed to determine the connection type."
I have an additional NFS share on the Win2003 server, "testhome", set
up
the
same way only with fewer (~10) home directories. This share I can map.
The Win2003 server is not under heavy load and I don't think there is
anything wrong with the physical disks.
Any ideas?!?
The Windows 2003 Server is a DC for an Active Directory domain, fully
patched, Dell PowerEdge 2650 with SCSI raid5 disks, 2GB ram.

Patrick Hopp

2005-01-18 16:48:05 UTC

Permalink

I'd be interested to know if you ever figure this out because I'm having
almost the same problem. Server is running 2k3, linux and sun clients
trying to mount. Sun clients get "nfs mount: mount: <servername>: I/O
error" the linux clients get: "mount: wrong fs type, bad option, bad
superblock on <servername>:/dump, or too many mounted file systems"

Showmount from the linux box shows:
showmount -e <servername>
Export list for <servername>:
/dump (everyone)

The log on the 2k3 server says:
Microsoft Server For NFS Activity Log

-------------------------------------------------------------------------

DATE TIME TASK RESULT ADDRESS DESCRIPTION...

-------------------------------------------------------------------------

01-18-2005 11:37:23 MOUNT SUCCESS <client1 IP> Z:\
01-18-2005 11:37:28 MOUNT SUCCESS <client2 IP> Z:\
01-18-2005 11:37:28 MOUNT SUCCESS <client2 IP> Z:\

The linux clients /var/log/messages spits out these lines, for EACH time
you try to mount something(no such errors on the sun):
Jan 18 11:44:13 <linuxbox> kernel: nfs_get_root: getattr error = 5
Jan 18 11:44:13 <linuxbox> kernel: nfs_get_root: getattr error = 5
Jan 18 11:44:13 <linuxbox> kernel: nfs_read_super: get root inode failed
Jan 18 11:44:13 <linuxbox> kernel: nfs warning: mount version older than
kernel
Jan 18 11:44:13 <linuxbox> kernel: nfs_get_root: getattr error = 5
Jan 18 11:44:13 <linuxbox> kernel: nfs_read_super: get root inode failed

Gary

2005-02-18 20:29:16 UTC

Permalink

Patrick,

Were you able to resolve this issue? I am having the same error when
attempting to mount a >1TB share which is stored on a Windows 2003 storage
server box from Suse 9.1 via standard NFS mount. I also see the NFS logs
reporting a successful mount.

Gary

Post by Patrick Hopp
I'd be interested to know if you ever figure this out because I'm having
almost the same problem. Server is running 2k3, linux and sun clients
trying to mount. Sun clients get "nfs mount: mount: <servername>: I/O
error" the linux clients get: "mount: wrong fs type, bad option, bad
superblock on <servername>:/dump, or too many mounted file systems"
showmount -e <servername>
/dump (everyone)
Microsoft Server For NFS Activity Log
-------------------------------------------------------------------------
DATE TIME TASK RESULT ADDRESS DESCRIPTION...
-------------------------------------------------------------------------
01-18-2005 11:37:23 MOUNT SUCCESS <client1 IP> Z:\
01-18-2005 11:37:28 MOUNT SUCCESS <client2 IP> Z:\
01-18-2005 11:37:28 MOUNT SUCCESS <client2 IP> Z:\
The linux clients /var/log/messages spits out these lines, for EACH time
Jan 18 11:44:13 <linuxbox> kernel: nfs_get_root: getattr error = 5
Jan 18 11:44:13 <linuxbox> kernel: nfs_get_root: getattr error = 5
Jan 18 11:44:13 <linuxbox> kernel: nfs_read_super: get root inode failed
Jan 18 11:44:13 <linuxbox> kernel: nfs warning: mount version older than
kernel
Jan 18 11:44:13 <linuxbox> kernel: nfs_get_root: getattr error = 5
Jan 18 11:44:13 <linuxbox> kernel: nfs_read_super: get root inode failed

Marimuthu. P

2005-01-18 22:57:53 UTC

Permalink

Hi,

Check if the SYSTEM account (NT_AUTHOURITY\SYSTEM ) should have a full
contorl NTFS permission to root of the drive.

For example:
If the folder shared is F:\test then cacls F:\ should list that system
account has full control otherwise add it to the NTFS ACL.

How big the NFS share is ?

Is some share working with unix -unix machine ?
--
Regards,
Marimuthu.P

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This posting is provided "AS IS" with no warranties, and confers no rights.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Peter Karlsson

2005-02-23 14:37:02 UTC

Permalink

Latest update:

I had no choice but to abandon the idea of a Windows2003 based NFS solution
for now and go with a dedicated RedHat Linux file server. The file server is
now up and running without any problems.

However, it's still interesting for me to get SFU working in order to
intergrate our Windows/Linux environments. I installed Windows2003 server +
SFU3.5 in our test environment and configured it according to the original
setup. Using the original data (home directories restored from backup tape) I
was able to duplicate the exact same errors as described in my original post.

I have verified that NT_AUTHORITY\SYSTEM has full control at the root of the
drive and setting up a new share containing data other than those home
directories works.
It looks to me that the problem is either (1) size of the data, (2) number
of files or (3) contents.
I then uninstalled SFU3.5, ran 'sfuclean.exe', installed SFU3 instead and
*THE PROBLEM WENT AWAY!*

I captured network traffic as I was connecting the NFS share from SFU3.5 NFS
Client and found the following.

With SFU3.5 NFS Server (which doesn't work):
--------------------------------------------------
Source Destination Protocol Info
--------------------------------------------------
Client -> Server MOUNT V3 MNT Call
Server -> Client MOUNT V3 MNT Reply
(Status: OK, Type: NetApp file handle)
Client -> Server NFS V3 GETATTR Call
Server -> Client NFS V3 GETATTR Reply
(Status: ERR_IO (5))

With SFU3 NFS Server (which works):
--------------------------------------------------
Source Destination Protocol Info
--------------------------------------------------
Client -> Server MOUNT V3 MNT Call
Server -> Client MOUNT V3 MNT Reply
(Status: OK, Type: unknown)
Client -> Server NFS V3 GETATTR Call
Server -> Client NFS V3 GETATTR Reply
(Status: OK)

Facts about the data in 'home':
* 1988 home directories
* Total of 304441 files in 54621 folders
* 26GB on disk

James Robinson

2005-03-03 15:29:07 UTC

Permalink

I have a similar problem

We used to run SFU3.0 on Windows 2000, but had to rebuild the windows server.
The disks that held the data were contained in a SAN and were moved to the
new server, with no data change - W2K3 SFU3.5
We have not configured (and never have) any username mappings, instead all
shares are opened RW anonymous (not my decision), the only restrictions being
the client groups.

We are able mount the shares and access most of the data, however we have
certain folders that give I/O Errors from Unix. Once a folder has this error,
the only way to fix it is to delete it. If we create new folders and copy the
data file by file it eventually stops working again. The error is not caused
by a specific file, as we can reproduce by just doubling up on the existing
files in the new working folder. Also, depending on the files we copy, we can
contain different numbers of files before it breaks. Also we can show that it
is not related to total size of content or length of file names. Some limit
is reached but I cannot identify what it is. We have other folders that
contain many more files and much more data which do not exhibit any problems.

My original thought was that there was a problem with the permissions -
folders created from the Unix system show the "Null SID" group (from Windows)
as having permissions, but folders created from Windows do not. However, I
have managed to add full control to the "Null SID" group using XCACLS to all
files/folders, also "Anonymous Users", "everyone", "SYSTEM" and local
Administrators all have Full Control.

Any help/suggestions would be appreciated

Jijo Jose [msft]

2005-03-05 02:36:45 UTC

Permalink

Are you still facing this issue? (i was offline for a while...)

Post by James Robinson
I have a similar problem
We used to run SFU3.0 on Windows 2000, but had to rebuild the windows server.
The disks that held the data were contained in a SAN and were moved to the
new server, with no data change - W2K3 SFU3.5
We have not configured (and never have) any username mappings, instead all
shares are opened RW anonymous (not my decision), the only restrictions being
the client groups.
We are able mount the shares and access most of the data, however we have
certain folders that give I/O Errors from Unix. Once a folder has this error,
the only way to fix it is to delete it. If we create new folders and copy the
data file by file it eventually stops working again. The error is not caused
by a specific file, as we can reproduce by just doubling up on the existing
files in the new working folder. Also, depending on the files we copy, we can
contain different numbers of files before it breaks. Also we can show that it
is not related to total size of content or length of file names. Some limit
is reached but I cannot identify what it is. We have other folders that
contain many more files and much more data which do not exhibit any problems.
My original thought was that there was a problem with the permissions -
folders created from the Unix system show the "Null SID" group (from Windows)
as having permissions, but folders created from Windows do not. However, I
have managed to add full control to the "Null SID" group using XCACLS to all
files/folders, also "Anonymous Users", "everyone", "SYSTEM" and local
Administrators all have Full Control.
Any help/suggestions would be appreciated