All Forums |
Register |
Login |
Search |
Subscriptions |
My Profile |
Inbox |
Tool Warehouse |
FAQs |
Resources |
Help |
Member List |
Address Book |
Logout |
|
|
Problems while reading and writing to a file
|
Logged in as: Guest |
Users viewing this topic: none |
|
Login |
|
|
Problems while reading and writing to a file - Aug. 11, '06, 10:59:40 AM
|
|
|
james_bray
Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
|
Hi,
We have been experiencing various file corruption issues from our Interix ported application. The application makes heavy use of random reading and writing to files.
Basically, there appears to be a flaw in the file position logic. The following steps illustrate the problem:
1. An existing file of say 8 bytes is opened in r+ mode (read/write).
2. The first four bytes of data are read using fread. The application only receives the four bytes, but the system has actually read a preset buffer amount (i.e. 4096 bytes). At this point, the file pointer should be at position 4, regardless of the amount of data read.
3. The application attempts to write a further 4 bytes out at the current file position. This file position is 4, performing an "ftell" returns 4. However, Interix has actually written the bytes out at the position it has reached after reading 4096 bytes, either pos 4096 or the end of the file.
This is not correct and results in corrupted data.
Below is a simple program that illustrates this fact:
#include <stdio.h>
int main(int argc, char *argv[])
{
FILE *file=fopen("test.dat","r+");
int dat;
if (!file)
{
printf("Failed to open file for RW\n");
return 1;
}
if (fread(&dat,4,1,file)<1)
{
printf("Failed to read 4 bytes from file\n");
fclose(file);
return 2;
}
if (fwrite(&dat,4,1,file)<1)
{
printf("Failed to write 4 bytes to file\n");
fclose(file);
return 3;
}
fclose(file);
return 0;
}
To test the bug, do the following.
1. Build the test application.
2. Create a file called "test.dat" in the current directory, containing less than 4096 bytes of any data - make a note of the file size.
3. Run the program.
4. Recheck the file size.
You will see the file size has increased by 4. This is because Interix is incorrectly positioning the file pointer at the end of the file, rather than after the initial read. Therefore, the file is enlarged by 4 bytes.
I also confirmed this using the FileMon utility from System Internals (www.sysinternals.com).
This seems to be a fairly serious and low level flaw. Are you aware of this, and/or of anything we can do about it?
Cheers,
James Bray
Badley Geoscience Limited
< Message edited by james_bray -- Aug. 11, '06, 11:31:54 AM >
|
|
|
RE: Problems while reading and writing to a file - Aug. 11, '06, 12:40:24 PM
|
|
|
Rodney
Posts: 3714
Joined: Jul. 9, '02,
From: /Tools lab
Status: offline
|
Which version of Interix are you running? (o/p from uname -X)
|
|
|
RE: Problems while reading and writing to a file - Aug. 11, '06, 2:43:18 PM
|
|
|
Rodney
Posts: 3714
Joined: Jul. 9, '02,
From: /Tools lab
Status: offline
|
I see this behavior with Interix 3.5. But at the same time I see this
behavior on an older version of OpenBSD. So this leads me to read the man
page on fopen() on OBSD which reads:
quote:
Reads and writes cannot be arbitrarily intermixed on read/write streams.
ANSI C requires that a file positioning function intervene between output
and input, unless an input operation encounters end-of-file.
Strictly speaking for portablity (by ANSI) you should be doing the fseek()
between fread() and fwrite() calls.
However, on Interix the fopen() man page reads differently (same as FBSD):
quote:
Reads and writes may be intermixed on read/write streams in any order, and
do not require an intermediate seek as in previous versions of stdio. This
is not portable to other systems, however; ANSI C requires that a file
positioning function intervene between output and input, unless an input
operation encounters end-of-file.
Interix stdio comes from 4.4BSD (not FBSD or OBSD).
So the fopen() man page is wrong.
You can do one of three things then:
1) Follow strict ANSI C (use fseek() calls).
or
2) Commission Interop Systems to extend libport to override this
strict ANSI need (similar to how sscanf() is updated for %qd/%lld).
or
3) Go through Microsoft support (PSS) and see where it goes...
< Message edited by Rodney -- Aug. 11, '06, 2:47:47 PM >
|
|
|
RE: Problems while reading and writing to a file - Aug. 11, '06, 3:17:00 PM
|
|
|
eperea
Posts: 70
Joined: Apr. 6, '05,
Status: offline
|
quote:
ORIGINAL: Rodney
I see this behavior with Interix 3.5. But at the same time I see this behavior on an older version of OpenBSD.
FWIW, OpenBSD -current (as of two hours ago) still behaves this way.
|
|
|
RE: Problems while reading and writing to a file - Aug. 13, '06, 5:05:25 AM
|
|
|
james_bray
Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
|
Hi,
Thanks for the feedback Rodney and eperea, its good to finally nail this down!
Interestingly, we *were* previously performing seek operations, but we removed these for our Interix port due to this issue.
Anyway, I will speak with my colleague on Monday to decide which option to take.
Cheers,
James Bray
Badley Geoscience Limited
|
|
|
RE: Problems while reading and writing to a file - Aug. 14, '06, 6:26:56 AM
|
|
|
james_bray
Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
|
Rodney,
Bearing in mind that the fseek removal was done purely to improve performance for NFS (and we dont currently have any NFS users), for the moment I think we'll simply back out the changes and recommend that users stick to storing project data on the local filesystem.
Thanks for your all your help in looking into this, at least we know where we stand should we need to pursue this issue further in the future.
Cheers,
James Bray
Badley Geoscience Limited
|
|
|
New Messages |
No New Messages |
Hot Topic w/ New Messages |
Hot Topic w/o New Messages |
|
Locked w/ New Messages |
Locked w/o New Messages |
|
Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
|
|