No Space Left- No i-Nodes!

[Ruler2112]

I have a box running RedHat 7.3 that today started having serious problems. I finally figured out that the root file system was out of inodes. I've seen online that the only way to correct this is to reformat the file system and start over, setting a greater number of inodes when you format. However, I'm puzzled as to what causes this.

My root partition is 1.9 gig in size and has 1.5 gig free. It has 256512 inodes, which seems like it'd be enough IMO. Anybody have any ideas that will save me?

[nukes]

It is similar to the amount of file entries you're allowed. Say, most distros format with a density of 4096 or so, which means that the disk is organised into blocks of 4096 bytes, and if you have a small file (like a device node or symlink) then some of the space is wasted. (I think)
The best way around this is to use ReiserFS, but I don't think RH7.3 supports it.
Another way would be to split your filesystem across a number of formatted partitions, so use one for /home, one for /usr and another one for /opt, say, that way you have less files on each filesystem but they still all look the same to the OS.

[Ruler2112]

OK, I worked under the assumption that there was something creating lots and lots of very small or empty files and set out searching my HD directory by directory with ls -l |wc -l I tracked down the problem to the fact that there are 231,972 files, all with size 0, in /var/log/mailman Each of them has the format error.a.b.c.d.e.f

Right now, I'm going to zip them all up onto a separate file system (just in case I need them for some reason I cannot fathom at this point in time). Does anybody know why these are being created and how to fix it? (I'm guessing that all I'm doing is putting a new fuse on the time bomb by deleting them.)

[nukes]

They're just logs, so unless you're running a mail server and want to keep track of the logs, then it should be safe to delete them. If you're not running a mail server, check why all those error files appeared, it could happen again after a certain length of time.
One possible cause would be that you have a misconfigured daemon running loose in the background. Not a good idea when they have those pointy things
Glad to see you've worked it out though. Sure beats having to redo the system.

[Ruler2112]

OK, forget about zipping them up- there are way too many for zip to handle. (Argument list too long error.) It's too long for rm as well, so I'm doing a bunch at a time.

I have my file systems spread over multiple logical partitions already. Any idea as to how I determine what daemon is creating them? (Other than looking for the one with the pitchfork? ) Thanks for your help dude.

[Ruler2112]

And does anybody know how to delete a LOT of files at once? I've been doing em in chunks and it's literally going to take hours to get rid of them all. I've been doing rm -f error.1.1.1.1* then rm -f error.1.1.1.2* and so on until I get back to the main branch, but there are just so damn many of them. I get argument list too long if I do too big a block at once. Anyway around that?

[Ruler2112]

Nevermind- script took care of it. Man, sometimes I think inside the box so much, it's scary!

[Scott Tiger]

For what it's worth, RedHat 7.3 does support Reiserfs. All you need to do is type "linux enable reiserfs" the boot prompt when you install Linux.

In my experience (and I think Sanwiches has a thread that confirmed this) it makes RedHat quite a bit faster when you do this.

[Ruler2112]

OK, I've gotten all the log files erased, but do not know what daemon created them. They were owned by root with a group of mailman. There is not anything with mail in it other than sendmail in /etc/init.d There are no processes running with mail in their names. There was no process that was touching the file (via fuser -u) when I erased the log files. man mailman is useless, and man -k mailman returns nothing appropriate.

Anybody know what uses the mailman group or how to find out what created the log files?


*edit* - I just did a google search for mailman and came up with a ton of hits (4.59 million). I found it amusing that the very first page of results had 'Jenna Jameson- Free Porn' as the title of the document! No, really- I working! Really!

[storm2k]

actually, it's just linux reiserfs at bootime. and from my experience, redhat can be much faster with reiser, just because they tweak ext3 to be very conservative (since their primary business is on the server end, they tweak their software to be very very stable and production ready since this is what businesses need).