Free Downloads, Community Forum,
FAQs and Developer Resources


Make /Tools Your Home | Link to us

Today's posts | Posts since last visit | Most Active Topics

All Forums Register Login Search Subscriptions My Profile Inbox
Tool Warehouse FAQs Resources Help Member List Address Book Logout

Slow NFS

 
Logged in as: Guest
Users viewing this topic: none
  Printable Version
All Forums >> [SFU / Interix / SUA Technology] >> SFU / Interix - Getting Started >> Slow NFS Page: [1]
Login
Message << Older Topic   Newer Topic >>
Slow NFS - May 24, '06, 11:18:20 AM   
asfoster

 

Posts: 36
Joined: Oct. 18, '05,
From: Lincolnshire UK
Status: offline
I have software ported to run under interix that uses data from our own proprietary database. When the database is on a local disk it rips along at an acceptable rate. The same app running on linux also runs very fast - but with the database located locally or over the network. The problem I am having is when running under interix with the db located on an NFS mounted disk - the performance is seriously degraded. On larger databases it is taking our apps minutes to load up where they normally load up in just a few seconds. I have just made a further test using an SMB networked windows disk (ie a disk from another PC mapped as a network-drive locally) and this runs fine too. Is there any way of improving on this. I have looked through the forum pages but not got anything conclusive. If it is relevant, I am running user-name mapping using NIS maps from a unix NIX server.

Thanks for any help.

_____________________________

Andy Foster
Post #: 1
RE: Slow NFS - May 25, '06, 12:20:44 PM   
james_bray

 

Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
Following on from Andys post here, I have performed some additional tests to try to isolate this problem.

What I have found is that the Windows NFS client uses an extremely large cache for reading data from the remote filesystem; about 4MB. When a file is opened in readonly mode (fopen "r" flag) and the file is accessed, the cache is used and no problems occur. If the file is opened in readwrite mode (fopen "r+" flag) and read from, again the cache is used as for reading.

However, there seem to be various operations in read-write mode that stop the NFS client from using the cache. These include writing to the file, as you would expect, but also seeking in the file and closing/reopening the file.

This is causing us major problems as our database typically opens files in read-write mode and then uses seek to perform random access to rows. This results in a full read of 4MB for every tiny read of a few bytes from the database.

None of the settings in Services for Unix Administrator (including NFS performance tab) seem to relate to this cache size setting. Overriding the BUFSIZ macro in the calling application also has no effect.

Do you know of any workarounds for this issue, or any way to set the cache size to something more sensible (SMB appears to use 4KB).

TIA,

James Bray

PS: Below is the output from truss when reading byte by byte from an 11MB file and performing a seek to the current location after each read. The first truss output is when using the "r" fopen flag, the second is from "r+". Note that they both use a 4MB cache, but the first performs 3 reads whilst the second performed thousands.

OUPUT FROM TRUSS WHEN READING A BYTE AT A TIME IN "R" MODE (Cache used, takes a few seconds)

open("/dev/fs/T/bigfile.dat", 0x1) open returned 3
fstat(3, 0xA40710, 0xA20710) fstat ret: 0 dev: 0x41020001718c54 ino: 0x000d4004
isatty(3) isatty returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 1, 0) lseek returned 0
lseek(3, 0, 4194304) lseek returned 0
lseek(3, 0, 4194304) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 8388608) lseek returned 0
lseek(3, 0, 8388608) lseek returned 0
read(3, 0x830020, 4194304) read returned 3038094 0x2E5B8E
lseek(3, 0, 11426702) lseek returned 0
lseek(3, 0, 11426702) lseek returned 0
read(3, 0x830020, 4194304) read returned 0
lseek(3, 0, 11426702) lseek returned 0
lseek(3, 0, 11426702) lseek returned 0
close(3) close returned 0
lseek(0, 1, 0) lseek returned 0
lseek(0, 0, 0) lseek returned 0
exit(0) process exited with status 0

OUPUT FROM TRUSS WHEN READING A BYTE AT A TIME IN "R+" MODE (Cache not used, still going 5 minutes later)

open("/dev/fs/T/bigfile.dat", 0x3) open returned 3
fstat(3, 0xA50710, 0xA30710) fstat ret: 0 dev: 0x41020001718c54 ino: 0x000d4004
isatty(3) isatty returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 1, 0) lseek returned 0
lseek(3, 0, 1) lseek returned 0
lseek(3, 0, 1) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 2) lseek returned 0
lseek(3, 0, 2) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 3) lseek returned 0
lseek(3, 0, 3) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 4) lseek returned 0
lseek(3, 0, 4) lseek returned 0
read(3, 0x830020, 4194304) read returned 4194304 0x400000
lseek(3, 0, 5) lseek returned 0
lseek(3, 0, 5) lseek returned 0
(output trimmed here for brevity - goes on for pages)

(in reply to asfoster)
Post #: 2
RE: Slow NFS - May 25, '06, 10:37:25 PM   
Rodney

 

Posts: 3714
Joined: Jul. 9, '02,
From: /Tools lab
Status: offline
Trying reading through this to see if anything helps:
http://www.microsoft.com/technet/interopmigration/unix/sfu/perfnfs.mspx

(in reply to james_bray)
Post #: 3
RE: Slow NFS - May 26, '06, 7:03:32 AM   
james_bray

 

Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
Thanks for the link Rodney.

Unfortunately, the settings provided relate to the NFS Server in SFU and not the NFS client.

I have performed some further investigation into this and have narrowed down the issue somewhat.

It would appear the cache size is a bit of a misnomer. Connecting to different NFS servers results in different cache sizes - I suspect the cache size is specified by the server and not the client.

I have performed some testing using the Linux NFS client and strace (truss equivalent). Interestingly, the NFS client on Linux does *not* invalidate the cache for seek operations in r+ mode. This explains why this problem only occurs on SFU - both the Solaris and Linux NFS clients are smart enough to realise that a seek operation in r+ mode is not necessarily a precursor to a write operation.

I am now looking into alternative NFS clients for Windows to see if they exhibit the same problem. However, having to provide a separate NFS client will be a major pain for us and we would prefer to avoid this.

James Bray

(in reply to Rodney)
Post #: 4
RE: Slow NFS - Aug. 13, '06, 3:47:18 PM   
Rodney

 

Posts: 3714
Joined: Jul. 9, '02,
From: /Tools lab
Status: offline
Way back for 4.4BSD I wrote ed and discovered the stdio buffer
was getting reloaded too many times. It really slowed things down.
So back in 1991 I wrote the comments:
quote:

/* these variables are here (instead of main and ed.h) because they
* are only used with the buffer when run under STDIO. STDIO is a
* resource pig with most of the OS's I've tested this with. The
* use of these variables improves performance up to 100% in several
* cases. The piggyness is thus: fseek causes the current STDIO buf
* to be flushed out and a new one read in...even when it is not necessary!
* Read 512 (or 1024) when you don't have to for almost every access
* and you'll slow down too. So these variable are used to control unneeded
* fseeks.
* I've been told the newer BSD STDIO has fixed this, but don't
* currently have a copy.
*/


It was Keith who said the newer STDIO would fix it. But it never did happen;
that may be where that (erroneous) comment on fopen() man page came from.
My work around was good enough that a number of other programs copied it.
Though only a couple gave credit for it.
The work-around was primarily the read side since writes to the buffer were all appends.
I think that what you are indicating is a similar issue (but for a different reason) of
trying to avoid unneeded fseek()'s because of the overhead (particularly with NFS).

(in reply to james_bray)
Post #: 5
Page:   [1]
All Forums >> [SFU / Interix / SUA Technology] >> SFU / Interix - Getting Started >> Slow NFS Page: [1]
Jump to:





New Messages No New Messages
Hot Topic w/ New Messages Hot Topic w/o New Messages
Locked w/ New Messages Locked w/o New Messages
 Post New Thread
 Reply to Message
 Post New Poll
 Submit Vote
 Delete My Own Post
 Delete My Own Thread
 Rate Posts


Search All Forums -

Advanced search


SPONSORS



Forum Software © ASPPlayground.NET Advanced Edition 2.5 ANSI

0.031