DEV Community

Clavin June
Clavin June

Posted on • Originally published at clavinjune.dev on

Linux Retains the Size of the Deleted File

#beginners #linux #file

Photo by @ilyapavlov on Unsplash

While interning, I once had an application that logged every request and response to the Linux file system. That app easily fills up our disks due to traffic. I got confused when I tried to delete log files because disk usage didn’t decrease even after the log files were deleted. So I restarted the app and finally, the disk usage decreased.

But it is ineffective because we need to kill the application which is making a downtime at that time. Recently, I learned that it was caused by something called a file descriptor.

TL;DR 

$ df -h <big file dir> # check the current usage
$ rm -f <big file> # remove the big file
$ df -h <big file dir> # it still the same
$ lsof +L1 | grep <big file> # get the running app PID
$ cd /proc/<PID>/fd/ # go to the running app fd
$ ls -l | grep <big file> # get the symlink
$ > <symlink> # empty the symlink
$ df -h <big file dir> # it backs to normal
Enter fullscreen mode Exit fullscreen mode

Let’s reproduce that incident.

Creating a big file

We need to create a big file, to fill our disk up. I think 3GB is enough. Well, you don’t need to make it big just to reproduce, but more than 1GB could show the difference easily in the df -h.

$ yes lorem ipsum dolor sit amet \
  | head -c 3GB >> big-file.txt
Enter fullscreen mode Exit fullscreen mode

Yep, now you have a 3GB file contains lorem ipsum called big-file.txt

Create a simple app

Then we need to create an app that simply read big-file.txt. But I want to show you even if the big-file.txt is deleted, our app still writing to it. So I make the app read and append the big-file.txt using Golang, you can do it with whatever you prefer. Let’s call it write-to-file.go

package main

import (
  "fmt"
  "os"
)

func main() {
  f, err := os.OpenFile("big-file.txt",
    os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)

  if err != nil {
    panic(err)
  }

  defer f.Close()

  for i := 0; ; i++ {
    fmt.Scanln()
    fmt.Fprintf(f, "%d\n", i)
    fmt.Println(i, "written")
  }
}
Enter fullscreen mode Exit fullscreen mode

It will wait for our input and loop forever until we terminate the app.

Check current disk usage

Well, because I put it in the home dir, I check the size of the /home using df.

$ df -h /home
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 913G 34G 832G 4% /home
Enter fullscreen mode Exit fullscreen mode

Tail the appended text in the file

tail -f will follow the last 10 lines in the file and it would show this

$ tail -f big-file.txt 
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lor
Enter fullscreen mode Exit fullscreen mode

Run the app

Let’s run our write-to-file.go

$ go run write-to-file.go
Enter fullscreen mode Exit fullscreen mode

For every Enter you input, it will append the big-file.txt like this

$ tail -f big-file.txt 
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lor0
1
2
3
Enter fullscreen mode Exit fullscreen mode 
$ go run write-to-file.go 

0 written

1 written

2 written

3 written
Enter fullscreen mode Exit fullscreen mode

Remove the file

Now let’s stop the tail -f and remove that big-file.txt.

$ rm big-file.txt
Enter fullscreen mode Exit fullscreen mode

Does your disk usage decreased? Unfortunately, mine is not. It might be karma working there isn’t it?

$ df -h /home
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 913G 34G 832G 4% /home
Enter fullscreen mode Exit fullscreen mode

Now, how to really remove the big-file.txt?

For doing that we need to check the current running process that still open that big-file.txt.

$ lsof +L1 | grep big-file.txt
write-to- 102333 clavinjune 3w REG 8,17 3000000008 0 7341204 /home/clavinjune/Public/file-descriptor/big-file.txt (deleted)
Enter fullscreen mode Exit fullscreen mode

Now we got the app’s name, its PID, and we know that the Linux know the big-file.txt is already deleted.

Let’s open the file descriptor of the app at /proc/<PID>/fd.

$ cd /proc/102333/fd
$ ls -l | grep big-file.txt
l-wx------ 1 clavinjune clavinjune 64 Jan 23 01:38 3 -> /home/clavinjune/Public/file-descriptor/big-file.txt (deleted)
Enter fullscreen mode Exit fullscreen mode

It seems like, we have a symlink called 3 that linked to our big-file.txt.

Now if you tail -f to the symlink and input some Enter to the app, it will show something like this

$ tail -f 3
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lorem ipsum dolor sit amet
lor0
1
2
3
4
5
6
7
8
9
10
Enter fullscreen mode Exit fullscreen mode

Interesting isn’t it? Even if the big-file.txt is deleted, the app still writing into the symlink. That symlink held our disk space, so we need to make that symlink is empty. By simply echoing nothing to the symlink in this case echoing nothing to 3.

$ > 3
Enter fullscreen mode Exit fullscreen mode

Now check the disk usage it will return to normal like we really delete the big-file.txt and the program is still running and still appending to the file.

$ df -h /home
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 913G 32G 835G 4% /home
Enter fullscreen mode Exit fullscreen mode

Let’s input some Enter to the app again then the symlink will filled up again by the app.

$ tail -f 3 
11
12
13
14
15
16
Enter fullscreen mode Exit fullscreen mode

Top comments (0)

Read next

dhyaneshsiddhartha15 profile image

Node js AsyncHandler

Dhyanesh Siddhartha -

steeve profile image

Search Component with RiotJS (Material Design)

Steeve -

dhimaskirana profile image

Best Compress Images or Photos on Ubuntu Linux

Dhimas Kirana -

yuricodesbot profile image

Supabase Bootstrap: the fastest way to launch a new project

Yuri -