All Forums |
Register |
Login |
Search |
Subscriptions |
My Profile |
Inbox |
Tool Warehouse |
FAQs |
Resources |
Help |
Member List |
Address Book |
Logout |
|
|
Slow NFS
|
Logged in as: Guest |
Users viewing this topic: none |
|
Login |
|
|
Slow NFS - May 24, '06, 11:18:20 AM
|
|
|
asfoster
Posts: 36
Joined: Oct. 18, '05,
From: Lincolnshire UK
Status: offline
|
I have software ported to run under interix that uses data from our own proprietary database. When the database is on a local disk it rips along at an acceptable rate. The same app running on linux also runs very fast - but with the database located locally or over the network. The problem I am having is when running under interix with the db located on an NFS mounted disk - the performance is seriously degraded. On larger databases it is taking our apps minutes to load up where they normally load up in just a few seconds. I have just made a further test using an SMB networked windows disk (ie a disk from another PC mapped as a network-drive locally) and this runs fine too. Is there any way of improving on this. I have looked through the forum pages but not got anything conclusive. If it is relevant, I am running user-name mapping using NIS maps from a unix NIX server.
Thanks for any help.
_____________________________
Andy Foster
|
|
|
RE: Slow NFS - May 25, '06, 12:20:44 PM
|
|
|
james_bray
Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
|
Following on from Andys post here, I have performed some additional tests to try to isolate this problem.
What I have found is that the Windows NFS client uses an extremely large cache for reading data from the remote filesystem; about 4MB. When a file is opened in readonly mode (fopen "r" flag) and the file is accessed, the cache is used and no problems occur. If the file is opened in readwrite mode (fopen "r+" flag) and read from, again the cache is used as for reading.
However, there seem to be various operations in read-write mode that stop the NFS client from using the cache. These include writing to the file, as you would expect, but also seeking in the file and closing/reopening the file.
This is causing us major problems as our database typically opens files in read-write mode and then uses seek to perform random access to rows. This results in a full read of 4MB for every tiny read of a few bytes from the database.
None of the settings in Services for Unix Administrator (including NFS performance tab) seem to relate to this cache size setting. Overriding the BUFSIZ macro in the calling application also has no effect.
Do you know of any workarounds for this issue, or any way to set the cache size to something more sensible (SMB appears to use 4KB).
TIA,
James Bray
PS: Below is the output from truss when reading byte by byte from an 11MB file and performing a seek to the current location after each read. The first truss output is when using the "r" fopen flag, the second is from "r+". Note that they both use a 4MB cache, but the first performs 3 reads whilst the second performed thousands.
OUPUT FROM TRUSS WHEN READING A BYTE AT A TIME IN "R" MODE (Cache used, takes a few seconds)
open("/dev/fs/T/bigfile.dat", 0x1) open returned 3
fstat(3, 0xA40710, 0xA20710) fstat ret: 0 dev: 0x41020001718c54 ino: 0x000d4004
isatty(3) isatty returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 1, 0) lseek returned 0
lseek(3, 0, 4194304) lseek returned 0
lseek(3, 0, 4194304) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 8388608) lseek returned 0
lseek(3, 0, 8388608) lseek returned 0
read(3, 0x830020, 4194304) read returned 3038094 0x2E5B8E
lseek(3, 0, 11426702) lseek returned 0
lseek(3, 0, 11426702) lseek returned 0
read(3, 0x830020, 4194304) read returned 0
lseek(3, 0, 11426702) lseek returned 0
lseek(3, 0, 11426702) lseek returned 0
close(3) close returned 0
lseek(0, 1, 0) lseek returned 0
lseek(0, 0, 0) lseek returned 0
exit(0) process exited with status 0
OUPUT FROM TRUSS WHEN READING A BYTE AT A TIME IN "R+" MODE (Cache not used, still going 5 minutes later)
open("/dev/fs/T/bigfile.dat", 0x3) open returned 3
fstat(3, 0xA50710, 0xA30710) fstat ret: 0 dev: 0x41020001718c54 ino: 0x000d4004
isatty(3) isatty returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 1, 0) lseek returned 0
lseek(3, 0, 1) lseek returned 0
lseek(3, 0, 1) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 2) lseek returned 0
lseek(3, 0, 2) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 3) lseek returned 0
lseek(3, 0, 3) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 4) lseek returned 0
lseek(3, 0, 4) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 5) lseek returned 0
lseek(3, 0, 5) lseek returned 0
(output trimmed here for brevity - goes on for pages)
|
|
|
RE: Slow NFS - May 25, '06, 10:37:25 PM
|
|
|
Rodney
Posts: 3714
Joined: Jul. 9, '02,
From: /Tools lab
Status: offline
|
Trying reading through this to see if anything helps:
http://www.microsoft.com/technet/interopmigration/unix/sfu/perfnfs.mspx
|
|
|
RE: Slow NFS - May 26, '06, 7:03:32 AM
|
|
|
james_bray
Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
|
Thanks for the link Rodney.
Unfortunately, the settings provided relate to the NFS Server in SFU and not the NFS client.
I have performed some further investigation into this and have narrowed down the issue somewhat.
It would appear the cache size is a bit of a misnomer. Connecting to different NFS servers results in different cache sizes - I suspect the cache size is specified by the server and not the client.
I have performed some testing using the Linux NFS client and strace (truss equivalent). Interestingly, the NFS client on Linux does *not* invalidate the cache for seek operations in r+ mode. This explains why this problem only occurs on SFU - both the Solaris and Linux NFS clients are smart enough to realise that a seek operation in r+ mode is not necessarily a precursor to a write operation.
I am now looking into alternative NFS clients for Windows to see if they exhibit the same problem. However, having to provide a separate NFS client will be a major pain for us and we would prefer to avoid this.
James Bray
|
|
|
RE: Slow NFS - Aug. 13, '06, 3:47:18 PM
|
|
|
Rodney
Posts: 3714
Joined: Jul. 9, '02,
From: /Tools lab
Status: offline
|
Way back for 4.4BSD I wrote ed and discovered the stdio buffer
was getting reloaded too many times. It really slowed things down.
So back in 1991 I wrote the comments:
quote:
/* these variables are here (instead of main and ed.h) because they
* are only used with the buffer when run under STDIO. STDIO is a
* resource pig with most of the OS's I've tested this with. The
* use of these variables improves performance up to 100% in several
* cases. The piggyness is thus: fseek causes the current STDIO buf
* to be flushed out and a new one read in...even when it is not necessary!
* Read 512 (or 1024) when you don't have to for almost every access
* and you'll slow down too. So these variable are used to control unneeded
* fseeks.
* I've been told the newer BSD STDIO has fixed this, but don't
* currently have a copy.
*/
It was Keith who said the newer STDIO would fix it. But it never did happen;
that may be where that (erroneous) comment on fopen() man page came from.
My work around was good enough that a number of other programs copied it.
Though only a couple gave credit for it.
The work-around was primarily the read side since writes to the buffer were all appends.
I think that what you are indicating is a similar issue (but for a different reason) of
trying to avoid unneeded fseek()'s because of the overhead (particularly with NFS).
|
|
|
New Messages |
No New Messages |
Hot Topic w/ New Messages |
Hot Topic w/o New Messages |
|
Locked w/ New Messages |
Locked w/o New Messages |
|
Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
|
|