This behavior is fairly general to Unix! A cute thing on Linux is that there are also references to open files under /proc, including synthetic kernel-generated filesystem links to the files, even if they've been deleted.
So if you delete a file from the filesystem that's open by PID 12345, you can find a reference to that file still present in /proc/12345/fd, and you can actually make a new copy of the file with cp or something.
And also when deleting a file that's on NFS, but it's reference count is non-zero in the kernel (because at least one process has it open) you end up renaming not deleting the file.
A file is held open read only by a process, and then you delete the file. Under memory pressure and without swap, can Linux evict those pages to reclaim memory?
The file's blocks exist in the filesystem. "Those pages" just refers to the ordinary buffer cache for any other disk-based file. No cached pages are necessary to hold open a file on disk.
i actually had a situation recently where i wanted to delete a file that a process had open because i needed to free up space.
i had a process that wrote a massive log file (200GB) filling up the disk. i didn't want to kill or restart the process, but i needed to remove the log somehow. there was no log rotating option without killing the process. and i knew that just removing the file would not work, so instead i did "echo -n > logfile" causing the file to be truncated.
since the process still had the file open for writing it kept writing to the position it had written to last, creating a 200GB sparse block of virtual zeroes. not exactly what i wanted, but in hindsight, not surprising, and it did solve the problem of freeing up 200GB of space.
except this morning, a few weeks after i truncated the file the space was gone again, and the file was now 400GB large. how did those 200GB come back? and where even did they come from? it is not another sparse block, it actually has data (maybe junk data, but not all zero)
it is unlikely that the process wrote 200GB of logs in two weeks. the other 200GB were written over 2 years. i have yet to analyze what that data is. so far i only confirmed that it isn't all zero (by using cp --sparse=always, ensuring that all zero blocks would be made sparse which would then be easy to detect) (further cursory inspection suggests that the process did in fact write 200GB of logs in two weeks. the error being "Failed to bind socket". i wonder if that was caused by the disk being full the first time around.)
TL;DR: if you need to free up space from a file opened by another process then truncating works, but beware of side effects.
The content of the file remains accessible to the process that has it open and continues to take up disk space.
... and a new file with the same name (and path) and different content can be created that exists at the same time.
This can be confusing.
"I'v updated that file on the server and it has not taken effect. I'v ssh'ed in and I can see the new version. What is going on? If I reboot that box the phone will start ringing..."
I've had the opposite problem, where I assumed shell scripts were cached, but an instance already running started using my modified version, while I was still working on it
I can testify that it's bonkers when a shell script is overwritten in place, and the interpreter bombs out with an irreproducible error message, because the contents shifted around that much. Or I suppose it would be even worse if the script silently continued to execute after that same trauma.
Also, let's not limit our scenarios to one process and one file descriptor. Each of those is a reference count. So your process may open the same file 3 times. And/or 10 processes on the system may hold the file open. 30 file descriptors might as well be 30 hardlinks in the filesystem. The file has life until all of those references go away.
Well I did not read the article, but here is what I think.
In the old days on an old 16 bit UNIX, I had to execute unlink to delete a file.
So I would say the inode # will be removed from the directory entry, leaving the file in place. So the program will continue on happily. On Linux, I think it depends on the file system. But I think it will continue until the file is overwritten. If the file is closed and re-opened after rm(1), the program will fail.
EDIT: I took the quiz. But I have to wonder if on a fs like zfs/btfs something different will happen.
This behavior is fairly general to Unix! A cute thing on Linux is that there are also references to open files under /proc, including synthetic kernel-generated filesystem links to the files, even if they've been deleted.
So if you delete a file from the filesystem that's open by PID 12345, you can find a reference to that file still present in /proc/12345/fd, and you can actually make a new copy of the file with cp or something.
And also when deleting a file that's on NFS, but it's reference count is non-zero in the kernel (because at least one process has it open) you end up renaming not deleting the file.
Followup quizz:
A file is held open read only by a process, and then you delete the file. Under memory pressure and without swap, can Linux evict those pages to reclaim memory?
The file's blocks exist in the filesystem. "Those pages" just refers to the ordinary buffer cache for any other disk-based file. No cached pages are necessary to hold open a file on disk.
Correct! It's still an inode, even without a path.
Shame that you can't then link /proc/self/fd/N to a different node in the filesystem namespace. That'd be neat and symmetric.
story time:
i actually had a situation recently where i wanted to delete a file that a process had open because i needed to free up space.
i had a process that wrote a massive log file (200GB) filling up the disk. i didn't want to kill or restart the process, but i needed to remove the log somehow. there was no log rotating option without killing the process. and i knew that just removing the file would not work, so instead i did "echo -n > logfile" causing the file to be truncated.
since the process still had the file open for writing it kept writing to the position it had written to last, creating a 200GB sparse block of virtual zeroes. not exactly what i wanted, but in hindsight, not surprising, and it did solve the problem of freeing up 200GB of space.
except this morning, a few weeks after i truncated the file the space was gone again, and the file was now 400GB large. how did those 200GB come back? and where even did they come from? it is not another sparse block, it actually has data (maybe junk data, but not all zero)
it is unlikely that the process wrote 200GB of logs in two weeks. the other 200GB were written over 2 years. i have yet to analyze what that data is. so far i only confirmed that it isn't all zero (by using cp --sparse=always, ensuring that all zero blocks would be made sparse which would then be easy to detect) (further cursory inspection suggests that the process did in fact write 200GB of logs in two weeks. the error being "Failed to bind socket". i wonder if that was caused by the disk being full the first time around.)
TL;DR: if you need to free up space from a file opened by another process then truncating works, but beware of side effects.
The content of the file remains accessible to the process that has it open and continues to take up disk space.
... and a new file with the same name (and path) and different content can be created that exists at the same time.
This can be confusing.
"I'v updated that file on the server and it has not taken effect. I'v ssh'ed in and I can see the new version. What is going on? If I reboot that box the phone will start ringing..."
I've had the opposite problem, where I assumed shell scripts were cached, but an instance already running started using my modified version, while I was still working on it
I can testify that it's bonkers when a shell script is overwritten in place, and the interpreter bombs out with an irreproducible error message, because the contents shifted around that much. Or I suppose it would be even worse if the script silently continued to execute after that same trauma.
Also, let's not limit our scenarios to one process and one file descriptor. Each of those is a reference count. So your process may open the same file 3 times. And/or 10 processes on the system may hold the file open. 30 file descriptors might as well be 30 hardlinks in the filesystem. The file has life until all of those references go away.
Well I did not read the article, but here is what I think.
In the old days on an old 16 bit UNIX, I had to execute unlink to delete a file.
So I would say the inode # will be removed from the directory entry, leaving the file in place. So the program will continue on happily. On Linux, I think it depends on the file system. But I think it will continue until the file is overwritten. If the file is closed and re-opened after rm(1), the program will fail.
EDIT: I took the quiz. But I have to wonder if on a fs like zfs/btfs something different will happen.