| All Forums |
Register |
Login |
Search |
Subscriptions |
My Profile |
Inbox |
| Tool Warehouse |
FAQs |
Resources |
Help |
Member List |
Address Book |
Logout |
|
|
RE: Several "common" utils gone in 5.2; 3.5 versions unusable
|
Logged in as: Guest |
| Users viewing this topic: none |
|
Login  |
|
|
 RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 9:32:02 AM
|
|
|
breiter
Posts: 343
Joined: Jun. 14, '04,
From: Washington, DC
Status: offline
|
Another reason that symlinks, hardlinks and junction points aren't exposed to the UI--beyond the support nightmare of exposing that to Joe Average--is that Windows supports FAT32 partitions. And I believe that is even the default on Windows XP (not sure though). FAT32 doesn't have built-in support for these constructs; just NTFS.
See some of the discussion in The Old New Thing: Why are there two copies of Notepad.exe?
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 12:50:52 PM
|
|
|
jerker_back
Posts: 68
Joined: Jul. 7, '05,
From: Sweden
Status: offline
|
quote:
No they aren't. Hard links are hard links ...
Yes, they are Microsoft implementations. But we need a MS file system developer to explain how this is done really. To even more complicate things: The implementations seems to change over time. As you sad:
quote:
A reparse point can do whatever you want ...
True, just compare the different implementations of junctions in Mikael Nordell's MakeLink.exe and SysInternal's junction.exe.
I'm quite happy with Interix symlinks. It seems to be a very simple and reliable solution - and it works. I wonder: Is the ln utility in GNU CoreUtils related to our Interix ln? Anybody tried CoreUtils ln? psxdll.dll exports symlink - anybody tried that?
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 1:34:08 PM
|
|
|
woehlkmp
Posts: 102
Status: offline
|
Yup, and as far as I can tell, it works like a charm, although I did have to add an explicit call to symlink() to get it to compile/link properly. If anyone knows why that happens, I'd love to hear the reason, just out of curiosity.
Since this is using the symlink() API call, I assume that answers both your questions?
I actually just submitted a bug to savannah listing the tweaks needed to get coreutils to build on Interix. Note that you do not get the df, hostid, shred, su or sync utilities (shred and sync have to be disabled because there is no sync() API on Interix; the others don't build by default - or maybe 'configure' disables them? - but are also broken if you try build them by hand).
Oh, and the reason I'm using the GNU version is because it supports implicit targets, which is nice (i.e. 'ln [-s] /usr/bin/foo' works; the second arg is automatically '`basename <source>`').
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 1:36:14 PM
|
|
|
Rodney
Posts: 3695
Joined: Jul. 9, '02,
From: /Tools lab
Status: online
|
Hard links have been part of NTFS since day 1 to specifically support Posix.
NTFS has developed over time through several versions. Some early-on claims
didn't stand-up (e.g. you'll never need to defrag) others have been pretty
good. The hard links are actually table based references by id (aka inode).
The method is similar to the way FFS did it. I've not heard anyone mention
the implementation getting a radical change or using a differet methodology.
The ln utility is not GNU based or related.
The support for symlinks in based in the Interix subsystem.
This allows all programs to "just work" with whatever pathname.
So any program can use the API's (which are standard across systems).
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 2:15:50 PM
|
|
|
jerker_back
Posts: 68
Joined: Jul. 7, '05,
From: Sweden
Status: offline
|
quote:
Since this is using the symlink() API call, I assume that answers both your questions?
Great! Very interesting
Is there any differences of the link (the link file as you see it in Windows Explorer)?
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 2:25:22 PM
|
|
|
Rodney
Posts: 3695
Joined: Jul. 9, '02,
From: /Tools lab
Status: online
|
> Is there any differences of the link (the link file as you see it in Windows Explorer)?
Win32 doesn't know the file has a special layout (that it's a symlink).
So it (and Win Explorer) can't interpret it for parsing or display.
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 2:32:38 PM
|
|
|
jerker_back
Posts: 68
Joined: Jul. 7, '05,
From: Sweden
Status: offline
|
quote:
or maybe 'configure' disables them?
What about 'configure' - is it finally working now?
When I tried, I had serious problem with 'configure' - partly solved it by replace interix sed with GNU sed.
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 4:19:44 PM
|
|
|
woehlkmp
Posts: 102
Status: offline
|
Eek! I didn't even THINK about replacing sed... I'm going on the assumption that not having GNU sed/gawk/grep/make/etc is just plain asking for problems (ok, I'm not sure why grep matters, but I've heard it does for some packages). If you read my bug report, that contains 100% of what I had to do (or at least, remember having to do!) to get it from './configure' to 'make' to 'make install'.
I would first replace(*) sed, grep, awk, and make with their GNU versions, and then give it a whirl. If you're still having problems, I'd be happy to offer what advice I can. I'll also assume you aren't getting stuck on config.guess, as that's a fairly obvious fix (that rather explicitly tells you what the problem is, as I recall); that was another of my assumptions.
( * It's always a good idea to drop them in /usr/gnu/bin or some such and set PATH accordingly! This is where '--prefix=<somewhere>' comes in handy.)
Also, here is more of a 'how-to' that explains better what you need to do, rather than generalizing what the change needs to be:
[EDIT] Incidentally, I'm a huge fan of the GNU tools... they're a great way to increase portability by removing ideosyncracies of various operating systems (Including Interix!). We actually depend on gawk for some of our build processes; not to mention that our make process is entirely built on GNU make. [/EDIT]
< Message edited by woehlkmp -- Apr. 12, '06, 4:21:07 PM >
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 7:04:39 PM
|
|
|
DrPizza
Posts: 40
Joined: Mar. 27, '03,
Status: offline
|
quote:
Yes, they are Microsoft implementations. But we need a MS file system developer to explain how this is done really.
No, we really don't. People have done the grunt work.
quote:
To even more complicate things: The implementations seems to change over time. As you sad:
When has it changed? The only thing that's changed is there's now a Win32 API to make hard links when in the olden days there was not; this never bothered Interix, as it never used Win32 anyway.
quote:
True, just compare the different implementations of junctions in Mikael Nordell's MakeLink.exe and SysInternal's junction.exe.
Hard links and junction points have nothing to do with each other, though. Junction points are implemented using reparse points; MS have a "mount point" reparse point that can be used to graft volumes and other folders into a directory. Hard links do not use such a mechanism; they are implemented equivalently to hard links on any unix.
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 10:00:26 PM
|
|
|
jerker_back
Posts: 68
Joined: Jul. 7, '05,
From: Sweden
Status: offline
|
quote:
I'm going on the assumption that not having GNU sed/gawk/grep/make/etc is just plain asking for problems
Ha, ha - so, are you actually planning to follow the line to the end and have a (what will we call it?) GNU posix subsystem?
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 12, '06, 11:46:37 PM
|
|
|
jerker_back
Posts: 68
Joined: Jul. 7, '05,
From: Sweden
Status: offline
|
OK, DrPizza - I will not argue in this matter - I only partly know what I'm talking about - as I expect most of us are in the inner insights of file system development. In fact I'm not disagreeing with you in any what you have said - you're right! But - so am I. I once did a deep dig in these things (well you're right, deep is always relative) - the subject is huge, complicated and comes with very little available info.
Besides, my original point was really very simple: Reparse points are basically for file system filter driver development, MS have made some basic implementations for us geeks to explore but they didn't really release anything for public use. The hardlinks I referred to was the kind made by the WIN32 API function CreateHardLink in kernel32.dll. As I said there are other ways: See Andrej Ashedel's ln (NT4) (Windows DDK req for compile) and there are at least one more solution what I know of. The interesting thing with Andrej Ashedel's ln is the ability to create both hardlinks and symbolic links.
Despite this is way beyond the scope of this forum, I'll give my understanding - not that I promise that anything is either correct, true or functional:
* The NTFS filesystem can be understood as database with entries - the MFT records. All files on a volume are represented in the MFT. The contents of the files, if any, are stored as one or more streams scattered around on the volume. File attributes is also stored as streams.
* Teoretically a hard link can be made by simply duplicate a MFT record, giving it another file name, but pointing to the same data as the original entry. Any change of either file would show identically in both files.
* But how to do that in the real world?
Ashedel fill a FILE_LINK_INFORMATION structure and apply the target file with NtSetInformationFile using FileLinkInformation as FILE_INFORMATION_CLASS. A hard link is made.
You're right - no reparse point handling here.
< Message edited by jerker_back -- Apr. 12, '06, 11:48:01 PM >
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 10:48:21 AM
|
|
|
woehlkmp
Posts: 102
Status: offline
|
quote:
quote:
I'm going on the assumption that not having GNU sed/gawk/grep/make/etc is just plain asking for problems
Ha, ha - so, are you actually planning to follow the line to the end and have a (what will we call it?) GNU posix subsystem?
Why not? 
Hmm...
$ ls /usr/gnu/bin
[ bunzip2 cp false groups lookbib paste rm tee vimdiff
aclocal bzcat csplit fgrep gunzip ls pathchk rmdir test vimtutor
aclocal-1.8 bzcmp cut fmt gzexe m4 pdfroff rview tfmtodit wc
addftinfo bzdiff date fold gzip make pfbtops rvim touch whatis
afmtodit bzegrep dd gawk head man pgawk sed tr who
apropos bzfgrep dir gawk-3.1.5 hostname man2dvi pgawk-3.1.5 seq troff whoami
autoconf bzgrep dircolors gdiffmk hpftodit md5sum pic sha1sum true xxd
autoheader bzip2 dirname grap2graph id mkdir pic2graph sleep tsort yes
autom4te bzip2recover du grep ifnames mkfifo pinky soelim tty zcat
automake bzless echo grn igawk mknod post-grohtml sort uname zcmp
automake-1.8 bzmore egrep grodvi indxbib mmroff pr split unexpand zdiff
autoreconf cat env groff install mv pre-grohtml stat uniq zforce
autoscan chgrp eqn groffer join neqn printenv stty unlink zgrep
autoupdate chmod eqn2graph grog kill nice printf sum uptime zmore
awk chown ex grolbp link nl ptx tac users znew
basename chroot expand grolj4 lkbib nohup pwd tail vdir
bash cksum expr grops ln nroff readlink tar view
bashbug comm factor grotty logname od refer tbl vim
Granted, a few of those aren't strictly "GNU", but I have coreutils, bash, sed, gawk, make, groff, autoconf/automake, m4, and tar. Of the non-GNU-but-still-OSS, I have gzip, zlib, vim (vim-6.4) and man (man-1.6c), all of which I built from sources. Just for kicks I tried 'for f in `ls /bin` ; do [ -f /usr/gnu/bin/$f ] || echo $f ; done' and realized I'm missing a few things I wouldn't mind having GNU versions of (diff, for one)  .
Note: in case this wasn't clear, when I said "asking for problems" I meant "w.r.t. trying to build other GNU utilities", not "using Interix in general".
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 12:06:41 PM
|
|
|
jerker_back
Posts: 68
Joined: Jul. 7, '05,
From: Sweden
Status: offline
|
That is really impressive - and a lot of work
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 12:20:56 PM
|
|
|
woehlkmp
Posts: 102
Status: offline
|
It's a LOT less than you might think. Mostly it's a matter of building dependencies in the right order... and sitting around staring at builds running. My list of 'changes needed to build on Interix' is only 30 lines long, and half of that was for coreutils.
In general, stuff either built 'out of the box' or with only one or two minor changes (tar, for example, needs a copy of sysexits.h; the header is nothing but a bunch of exit status constants, so can be simply copied from any computer that has it). Several things need C[PP]FLAGS=-D_ALL_SOURCE to behave. Stuff like that. But that's about it.
I just built less, findutils and diffutils, and am moving on to patch, ctags and ncurses.
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 1:07:07 PM
|
|
|
jerker_back
Posts: 68
Joined: Jul. 7, '05,
From: Sweden
Status: offline
|
So, you can actually replace the whole Interix SDK and building a GNU environment on pure R2? You'll need the psxdll headers and libs of course but besides that?
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 1:29:58 PM
|
|
|
Rodney
Posts: 3695
Joined: Jul. 9, '02,
From: /Tools lab
Status: online
|
Just for reference:
diff is already a GNU based version.
You do need utilities from /bin that exist that have no GNU equivalents.
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 1:40:54 PM
|
|
|
jerker_back
Posts: 68
Joined: Jul. 7, '05,
From: Sweden
Status: offline
|
quote:
You do need utilities from /bin that exist that have no GNU equivalents
Did you ever test this in OpenNT? As an alternative to BSD I mean.
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 1:49:07 PM
|
|
|
woehlkmp
Posts: 102
Status: offline
|
I don't think you can replace "the entire SDK", although you can probably get pretty close (if you can get your own gcc built, that is). I know for a fact that you're (currently) stuck with Microsoft's 'df', and probably their 'su'. But, in terms of the shell and basic utilities, then yes, you can replace (and I guess I have replaced) most of it with GNU or other OSS versions.
Here is a list(*) of things in /bin that are not in /usr/gnu/bin:
$ for f in `ls /bin` ; do [ -f /usr/gnu/bin/$f ] || echo $f ; done | column
addr cpio getopt mail pg rsh time
at crontab hexdump mailx ping rstartd tip
atq dc host man2html posixpath2nt runwin32 truss
atrm df iconv more ps script tzselect
banner dig ipcrm mpack pstat service unifdef
batch dircmp ipcs mt psxoffset showmount unixpath2win
bc dnsquery ispell munpack ranlib size unvis
bp ed last mvwtmpx rcp spell uudecode
c89 file lex newgrp rdist ssimda uuencode
cal fileinfo line nslookup rdistd strerror vis
calendar fileinfo.exe locale nsupdate read strsignal winpath2unix
chgpath finger localedef ntpath2posix regpwd su wvisible
clear flip logger nvi renice syslogd yacc
col ftp login passwd rlogin talk yearistype
column gencat loginenv pax rpcgen telnet
command getconf lp pdomain rpcinfo tftp
* I culled from that list things from binutils, shells, shell built-ins, curses/term{info|cap} programs, things where the GNU version has a different name, etc. Specifically:
alias cat32.exe fc infotocap sh toe vi
ar cc fg jobs strings tput wait
bg cd getopts ksh strip tset
captoinfo chsh iman nm tcsh umask
cat32 csh infocmp reset tic unalias
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 2:01:38 PM
|
|
|
breiter
Posts: 343
Joined: Jun. 14, '04,
From: Washington, DC
Status: offline
|
quote:
if you can get your own gcc built, that is)
Another forum member, Andreas/jumpigh, has built gcc 4.0 with c, c++, objc, f95 and treelang which is available as a beta package from ftp://ftp.interopsystems.com/pkgs/beta/.
Personally, I haven't tried it, but you're all gung-ho and everything...
|
|
|
|
|
RE: Several "common" utils gone in 5.2; 3.5 v... - Apr. 13, '06, 2:04:12 PM
|
|
|
Rodney
Posts: 3695
Joined: Jul. 9, '02,
From: /Tools lab
Status: online
|
> Did you ever test this in OpenNT? As an alternative to BSD I mean.
The programs in particular are not BSD-based (written from scratch) or
were heavily/specifically modified for the Interix (nee OpenNT) platform.
They wouldn't make sense on any other platform. Many of the utilities that
have GNU equivalants have Interix specific changes for better functionality
and/or speed, but, again, the changes aren't applicable to other platforms.
|
|
|
|
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
|
 Locked w/ New Messages |
 Locked w/o New Messages |
|
 Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
|
|
Printable Version
