Grant wrote:

> You gonna tell the spell when you find it?

sure, but s/when/if/ ... :-(

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

In <slrne5hm51.838r.syl@alcor.concordia.ca>
Sylvain Robitaille <syl@alcor.concordia.ca> writes:


>[...]
>NOTE: "58500 (44) innfeed-dropped.A029106" above suggests that my
>relinking the file was successful, no? Yet ...

>; ls -l /news/misc/innfeed
>ls: /news/misc/innfeed/innfeed-dropped.A029106: No such file or directory

This may be a stupid question, but have you created a directory entry
to match the link count? It wouldn't need the same name (though it
would of course have to be in the same file system), but it would
certainly need the same inode number.

- Steven
__________________________________________________ ______________________
Steven Winikoff |
Concordia University |
Montreal, QC, Canada | There's an exception to every rule,
smw@alcor.concordia.ca | except this one.
http://alcor.concordia.ca/~smw |

Steven Winikoff wrote:

> This may be a stupid question, but have you created a directory entry
> to match the link count?

Within debugfs, you mean? If so, yes, but I've been unable to list it
(or otherwise "use" it).

> It wouldn't need the same name ..., but it would certainly need the
> same inode number.

Agreed, and if you know of a way to do that, that I'm not seeing, I'd
greatly appreciate the advice. I had thought that debugfs was the right
tool for the job, but if it is, it appears to not do what it does as
well on a live filesystem as on an unmounted one.

Unless ... debugfs is working on the filesystem on disk, and my reads of
the directory have come from in-memory buffers. Does that sound
possible, and if so, do you have any suggestions for forcing a reread
into the buffer from the on-disk file system?

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

In <slrne5tfgf.fakj.syl@alcor.concordia.ca>
Sylvain Robitaille <syl@alcor.concordia.ca> writes:

>> This may be a stupid question, but have you created a directory entry
>> to match the link count?

>Within debugfs, you mean?

Or any way at all, I suppose. :-)


>If so, yes,

Sorry, I didn't read your debufs trace carefully enough yesterday.


>but I've been unable to list it (or otherwise "use" it).

Within debugfs, did you try using the ncheck command to verify that the
file system was consistent after the new directory entry was created?

Another possibility (though I wouldn't think it likely) is that perhaps
the inode had already been marked free in the inode bitmap. I wouldn't
expect that to be done until the last close, but it may be worth trying
the seti command.

If that turns out to be a dead end, there may be a roundabout route to
accomplish the same goal. For example, suppose you start by copying any
data that may have been written to the existing inode (I know it was
empty yesterday, but just in case it isn't still empty now), using the
write command of debugfs:

[...]
write source_file out_file
Create a file in the filesystem named out_file, and copy
the contents of source_file into the destination file.
[...]

After doing that, you could delete the existing inode and create and
allocate a new one of the same number.


>Unless ... debugfs is working on the filesystem on disk, and my reads of
>the directory have come from in-memory buffers. Does that sound
>possible,

I won't say it's impossible, but if it is the man page is broken, and
arguably the code is also.


>and if so, do you have any suggestions for forcing a reread
>into the buffer from the on-disk file system?

I'm afraid not.

- Steven
__________________________________________________ ______________________
Steven Winikoff |
Concordia University |
Montreal, QC, Canada | There's an exception to every rule,
smw@alcor.concordia.ca | except this one.
http://alcor.concordia.ca/~smw |

Sylvain Robitaille <syl@alcor.concordia.ca> wrote:
In article <slrne5tfgf.fakj.syl@alcor.concordia.ca> you wrote:
> Unless ... debugfs is working on the filesystem on disk, and my reads of
> the directory have come from in-memory buffers.

Could well be, it is a well-known problem with fdisk and fsck,
that they change/fix the DISK, not the in-memory caches/buffers,
so those may be out-of-date.

> and if so, do you have any suggestions for forcing a reread
> into the buffer from the on-disk file system?

It is call a "reboot", grin, and especially one without rewriting
the in-memory caches (reboot -n, look it up).
For instance, it is needed after e2fsck fixed the root file system
(and the startup scripts will automatically reboot IF the root was
that much corrupt that only a reboot and REread of the on-disk
structures will help).
Pat's rc.S doesn't use the -n, though, it remounts the root
"read-only" as to NOT write back anything to it:
echo "Unmounting file systems."
/sbin/umount -a -r # all (other) file systems
/sbin/mount -n -o remount,ro / # remount root read-only
echo "Rebooting system."
sleep 2
reboot -f
--
************************************************** ******************
** Eef Hartman, Delft University of Technology, dept. EWI/TW **
** e-mail: E.J.M.Hartman@math.tudelft.nl, fax: +31-15-278 7295 **
** snail-mail: P.O. Box 5031, 2600 GA Delft, The Netherlands **
************************************************** ******************

Steven Winikoff wrote:

> Within debugfs, did you try using the ncheck command to verify that
> the file system was consistent after the new directory entry was
> created?

In this particular case, the process that was holding the file I was
interested in open has been restarted automatically by weekly
maintenance scripts, so I'm no longer trying to chase this specific file
down, but I would still like to recover from this kind of thing in the
future, if I make the same mistake again, so I'm playing around on /tmp
(separate file system) on the same system.

In this case, I've _added_ a link to a file, adjusted the link count,
and found that if I issue a "close" command within debugfs, my new file
link shows up in a directory listing. However, note the following that
is rather interesting:

: newsflash[syl] ~; ls -li /tmp
....
14 -rw-r--r-- 1 syl iits 21142 Feb 4 2004 lynx.test
14 -rw-r--r-- 1 syl iits 21142 Feb 4 2004 lynx.trace
....
^^ ^
same inode, but only 1 link???

Within debugfs, I get the following:

debugfs: ncheck 14
Inode Pathname
14 /lynx.trace

debugfs: stat <14>
Inode: 14 Type: regular Mode: 0644 Flags: 0x0 Generation: 3163345761
User: 1143 Group: 100 Size: 21142
File ACL: 0 Directory ACL: 0
Links: 2 Blockcount: 48
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x40217300 -- Wed Feb 4 17:32:32 2004
atime: 0x44605e0d -- Tue May 9 05:17:01 2006
mtime: 0x40217300 -- Wed Feb 4 17:32:32 2004
BLOCKS:
(0):9530, (1):9537, (2):9709, (3):10670, (4):10786, (5):16613
TOTAL: 6


NOTE: "Links: 2" ...

> Another possibility (though I wouldn't think it likely) is that
> perhaps the inode had already been marked free in the inode bitmap.

With a running process holding the file open? That would be a serious
bug, no?

> I wouldn't expect that to be done until the last close, but it may be
> worth trying the seti command.

(caveat: now I'm dealing with a different file, so this may be
completely inconclusive ...)

debugfs: seti <14>
seti: Warning: inode already set

> ... suppose you start by copying any data that may have been written
> to the existing inode ... After doing that, you could delete the
> existing inode and create and allocate a new one of the same number.

With a running process holding the file open at the same inode, and
potentially writing to that file while the above operation is in
progress, I would expect Very Bad Things(tm) to happen, if it's even
possible to do the above. I'm not willing to try that on this system,
but perhaps later, if I get my hands on a system whose purpose is
testing the OS under various conditions, that might be worth trying.

I'm starting to wonder if "debugfs - ext2 file system debugger" (quoted
from tha manual page) doesn't bear the implied meaning
"debugfs - NOT ext3 file system debugger" ...

It may just be that I _can't_ do what I'm trying to do on an actively
mounted ext3 filesystem, using this tool. :-(

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

Eef Hartman wrote:

>> and if so, do you have any suggestions for forcing a reread
>> into the buffer from the on-disk file system?
>
> It is call a "reboot", grin, ...

Well, a "remount", I suppose ... I obviously could have done that, but
in that case, I frankly wouldn't have needed to, as restarting the
process would have been all that was needed. In the case that I might
have lost real data, the "remount" idea, with a pass through debugfs to
relink the file first would no doubt have worked, but that's the
situation that I'm looking to avoid if at all possible.

No harm done this time, but I'm trying to be prepared, because if I made
this error once, I'm likely to do it again. :-(

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

In <slrne61k0b.1f3d.syl@alcor.concordia.ca>
Sylvain Robitaille <syl@alcor.concordia.ca> writes:

>In this case, I've _added_ a link to a file, adjusted the link count,
>and found that if I issue a "close" command within debugfs, my new file
>link shows up in a directory listing. However, note the following that
>is rather interesting:
>[...]
> same inode, but only 1 link???

>Within debugfs, I get the following:
>[...]
>NOTE: "Links: 2" ...

This certainly does suggest that debugfs is broken with respect to the
way it handles link counts.


>> Another possibility (though I wouldn't think it likely) is that
>> perhaps the inode had already been marked free in the inode bitmap.

>With a running process holding the file open? That would be a serious
>bug, no?

I agree, but it was still something that needed to be ruled out.


>debugfs: seti <14>
>seti: Warning: inode already set

I'm relieved to see that. :-)


>> ... suppose you start by copying any data that may have been written
>> to the existing inode ... After doing that, you could delete the
>> existing inode and create and allocate a new one of the same number.

>With a running process holding the file open at the same inode, and
>potentially writing to that file while the above operation is in
>progress, I would expect Very Bad Things(tm) to happen, if it's even
>possible to do the above.

Anything is possible when you have write access to the raw device. :-)

As for the worst case... well, you'd get to find out how robust the
ext3fs algorithms really are. :-)

Seriously, what *does* happen when the ext3 write() function attempts to
write to an open inode that suddenly doesn't exist? Does the function
check first before trying to write? The only way to know for sure is to
read the source.

No, I don't recommend trying this at home (or anywhere else, really,
unless you have root access on a machine that belongs to someone you
really don't like :-) unless you're desperate.

The level of desperation involved is left as an exercise for the
reader, and I'm glad that this case wasn't desperate enough.


>I'm starting to wonder if "debugfs - ext2 file system debugger" (quoted
>from tha manual page) doesn't bear the implied meaning
>"debugfs - NOT ext3 file system debugger" ...

A valid question, of course. As I understand it, ext3 is just ext2 plus
journalling. Hmm. Could the problem be that you were trying to undo a
transaction that was uncommitted in the journal at the time?

- Steven
__________________________________________________ ______________________
Steven Winikoff |"Q: What's tiny and yellow and very,
Concordia University | very dangerous?"
Montreal, QC, Canada | A: A canary with the super-user
smw@alcor.concordia.ca | password."
http://alcor.concordia.ca/~smw | - fortune(6)

Steven Winikoff wrote:

> Could the problem be that you were trying to undo a transaction that
> was uncommitted in the journal at the time?

It's certainly a possibility, I think. I don't know how I would confirm
or disprove it, though. :-(

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

Sylvain Robitaille <syl@alcor.concordia.ca> wrote:
> Eef Hartman wrote:
>
>>> and if so, do you have any suggestions for forcing a reread
>>> into the buffer from the on-disk file system?
>>
>> It is call a "reboot", grin, ...
ed (my fault)
>
> Well, a "remount", I suppose ...

IF the filesystem is one that could have been UNmounted, you could have
run the debugfs on the UNmounted fs and wouldn't have to worry about
in-memory caches etc.

A "remount" of a MOUNTED fs doesn't reread the on-disk structures, you only
can change some selected mount attributes (like read-only/read-write,
suid/nosuid, etc) that way, so it won't help against cached data of the fs.
A umount/mount, of course, will, but the umount may REwrite some of the
cache data, so could undo your debugfs modifications.
That's why the best way, but it isn't always possible, especially on the
root fs, is to unmount first, make the changes, and mount again, because
then the disk structure will be read at the mount.
--
************************************************** ******************
** Eef Hartman, Delft University of Technology, dept. EWI/TW **
** e-mail: E.J.M.Hartman@math.tudelft.nl, fax: +31-15-278 7295 **
** snail-mail: P.O. Box 5031, 2600 GA Delft, The Netherlands **
************************************************** ******************