Free Downloads, Community Forum,
FAQs and Developer Resources


Make /Tools Your Home | Link to us

Today's posts | Posts since last visit | Most Active Topics

All Forums Register Login Search Subscriptions My Profile Inbox
Tool Warehouse FAQs Resources Help Member List Address Book Logout

Problems while reading and writing to a file

 
Logged in as: Guest
Users viewing this topic: none
  Printable Version
All Forums >> [SFU / Interix / SUA Technology] >> Interix Advanced Forum >> Problems while reading and writing to a file Page: [1]
Login
Message << Older Topic   Newer Topic >>
Problems while reading and writing to a file - Aug. 11, '06, 10:59:40 AM   
james_bray

 

Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
Hi,

We have been experiencing various file corruption issues from our Interix ported application. The application makes heavy use of random reading and writing to files.

Basically, there appears to be a flaw in the file position logic. The following steps illustrate the problem:

1. An existing file of say 8 bytes is opened in r+ mode (read/write).
2. The first four bytes of data are read using fread. The application only receives the four bytes, but the system has actually read a preset buffer amount (i.e. 4096 bytes). At this point, the file pointer should be at position 4, regardless of the amount of data read.
3. The application attempts to write a further 4 bytes out at the current file position. This file position is 4, performing an "ftell" returns 4. However, Interix has actually written the bytes out at the position it has reached after reading 4096 bytes, either pos 4096 or the end of the file.

This is not correct and results in corrupted data.

Below is a simple program that illustrates this fact:

#include <stdio.h>

int main(int argc, char *argv[])
{
    FILE *file=fopen("test.dat","r+");
        int dat;

    if (!file)
    {
        printf("Failed to open file for RW\n");
        return 1;
    }

    if (fread(&dat,4,1,file)<1)
        {
                printf("Failed to read 4 bytes from file\n");
                fclose(file);
                return 2;
        }

        if (fwrite(&dat,4,1,file)<1)
        {
                printf("Failed to write 4 bytes to file\n");
                fclose(file);
                return 3;
        }

        fclose(file);
        return 0;
}


To test the bug, do the following.

1. Build the test application.
2. Create a file called "test.dat" in the current directory, containing less than 4096 bytes of any data - make a note of the file size.
3. Run the program.
4. Recheck the file size.

You will see the file size has increased by 4. This is because Interix is incorrectly positioning the file pointer at the end of the file, rather than after the initial read. Therefore, the file is enlarged by 4 bytes.

I also confirmed this using the FileMon utility from System Internals (www.sysinternals.com).

This seems to be a fairly serious and low level flaw. Are you aware of this, and/or of anything we can do about it?

Cheers,

James Bray
Badley Geoscience Limited

< Message edited by james_bray -- Aug. 11, '06, 11:31:54 AM >
Post #: 1
RE: Problems while reading and writing to a file - Aug. 11, '06, 12:40:24 PM   
Rodney

 

Posts: 3714
Joined: Jul. 9, '02,
From: /Tools lab
Status: offline
Which version of Interix are you running? (o/p from uname -X)

(in reply to james_bray)
Post #: 2
RE: Problems while reading and writing to a file - Aug. 11, '06, 2:43:18 PM   
Rodney

 

Posts: 3714
Joined: Jul. 9, '02,
From: /Tools lab
Status: offline
I see this behavior with Interix 3.5. But at the same time I see this
behavior on an older version of OpenBSD. So this leads me to read the man
page on fopen() on OBSD which reads:
quote:

Reads and writes cannot be arbitrarily intermixed on read/write streams.
ANSI C requires that a file positioning function intervene between output
and input, unless an input operation encounters end-of-file.

Strictly speaking for portablity (by ANSI) you should be doing the fseek()
between fread() and fwrite() calls.
However, on Interix the fopen() man page reads differently (same as FBSD):
quote:

Reads and writes may be intermixed on read/write streams in any order, and
do not require an intermediate seek as in previous versions of stdio. This
is not portable to other systems, however; ANSI C requires that a file
positioning function intervene between output and input, unless an input
operation encounters end-of-file.

Interix stdio comes from 4.4BSD (not FBSD or OBSD).
So the fopen() man page is wrong.

You can do one of three things then:
1) Follow strict ANSI C (use fseek() calls).
or
2) Commission Interop Systems to extend libport to override this
strict ANSI need (similar to how sscanf() is updated for %qd/%lld).
or
3) Go through Microsoft support (PSS) and see where it goes...

< Message edited by Rodney -- Aug. 11, '06, 2:47:47 PM >

(in reply to Rodney)
Post #: 3
RE: Problems while reading and writing to a file - Aug. 11, '06, 3:17:00 PM   
eperea

 

Posts: 70
Joined: Apr. 6, '05,
Status: offline
quote:

ORIGINAL: Rodney

I see this behavior with Interix 3.5. But at the same time I see this behavior on an older version of OpenBSD.


FWIW, OpenBSD -current (as of two hours ago) still behaves this way.

(in reply to Rodney)
Post #: 4
RE: Problems while reading and writing to a file - Aug. 13, '06, 5:05:25 AM   
james_bray

 

Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
Hi,

Thanks for the feedback Rodney and eperea, its good to finally nail this down!

Interestingly, we *were* previously performing seek operations, but we removed these for our Interix port due to this issue.

Anyway, I will speak with my colleague on Monday to decide which option to take.

Cheers,

James Bray
Badley Geoscience Limited

(in reply to james_bray)
Post #: 5
RE: Problems while reading and writing to a file - Aug. 14, '06, 6:26:56 AM   
james_bray

 

Posts: 14
Joined: Dec. 8, '05,
From: Lincolnshire, UK
Status: offline
Rodney,

Bearing in mind that the fseek removal was done purely to improve performance for NFS (and we dont currently have any NFS users), for the moment I think we'll simply back out the changes and recommend that users stick to storing project data on the local filesystem.

Thanks for your all your help in looking into this, at least we know where we stand should we need to pursue this issue further in the future.

Cheers,

James Bray
Badley Geoscience Limited

(in reply to james_bray)
Post #: 6
Page:   [1]
All Forums >> [SFU / Interix / SUA Technology] >> Interix Advanced Forum >> Problems while reading and writing to a file Page: [1]
Jump to:





New Messages No New Messages
Hot Topic w/ New Messages Hot Topic w/o New Messages
Locked w/ New Messages Locked w/o New Messages
 Post New Thread
 Reply to Message
 Post New Poll
 Submit Vote
 Delete My Own Post
 Delete My Own Thread
 Rate Posts


Search All Forums -

Advanced search


SPONSORS



Forum Software © ASPPlayground.NET Advanced Edition 2.5 ANSI

0.031