Fatrace command: how to know in real time which processes are writing to a file

It is usually easy to know which process or processes are writing to a given file in Linux, since we either know its origin and its nature beforehand (for example the Apache access_log), or we can easily find it out with the fuser or lsof commands. However, sometimes it will happen that although we know the role and purpose of a file, there are so many applications accesing it simultaneously that it is very difficult to know which of them is the one that reads/writes the most or does so in a precise moment. Knowing this would be very useful to learn for example why a log file is growing excessively or which application is making an abusive use of system resources, either by mistake or intentionally.

Error: Your Requested widget " ai_widget-6" is not in the widget list.

• [do_widget_area above-nav-left]
• [do_widget_area above-nav-right]
• [do_widget_area footer-1]
  • [do_widget id="wpp-4"]
• [do_widget_area footer-2]
  • [do_widget id="recent-posts-4"]
• [do_widget_area footer-3]
  • [do_widget id="recent-comments-3"]
• [do_widget_area footer-4]
  • [do_widget id="archives-4"]
• [do_widget_area logo-bar]
  • [do_widget id="oxywidgetwpml-3"]
• [do_widget_area menu-bar]
  • [do_widget id="search-3"]
• [do_widget_area sidebar]
  • [do_widget id="search-4"]
  • [do_widget id="ai_widget-2"]
  • [do_widget id="categories-5"]
  • [do_widget id="ai_widget-3"]
  • [do_widget id="ai_widget-4"]
  • [do_widget id="ai_widget-5"]
• [do_widget_area sub-footer-1]
  • [do_widget id="text-4"]
• [do_widget_area sub-footer-2]
• [do_widget_area sub-footer-3]
• [do_widget_area sub-footer-4]
• [do_widget_area upper-footer-1]
  • [do_widget id="search-2"]
  • [do_widget id="recent-posts-2"]
  • [do_widget id="recent-comments-2"]
  • [do_widget id="archives-2"]
  • [do_widget id="categories-2"]
  • [do_widget id="meta-2"]
• [do_widget_area upper-footer-2]
• [do_widget_area upper-footer-3]
• [do_widget_area upper-footer-4]
• [do_widget_area widgets_for_shortcodes]
  • [do_widget id="search-5"]
  • [do_widget id="ai_widget-6"]
• [do_widget_area wp_inactive_widgets]
  • [do_widget id="wpp-2"]
  • [do_widget id="text-1"]
  • [do_widget id="recent-posts-3"]
  • [do_widget id="categories-3"]
  • [do_widget id="archives-3"]
  • [do_widget id="icl_lang_sel_widget-3"]

In order to get this information in Linux we have the fatrace command, which displays all file access events that occur within the whole system, giving us information about the file involved, the name and PID of the process responsible for that event and the event type, ie whether the file was opened, read, written or closed.

Normally the fatrace command will not be present in your system, but you can install it from your Linux distribution’s repositories as usual:

– Debian / Ubuntu:

$ sudo apt-get install fatrace

– Centos / Fedora / Red Hat:

$ sudo yum install -y fatrace

Once installed, you can see which processes read or write to a file in real time running the command like this:

$ sudo fatrace -f W | grep .xsession-errors
konsole(13009): W /home/daniloaz/.xsession-errors 
konsole(13009): W /home/daniloaz/.xsession-errors 
konsole(13009): W /home/daniloaz/.xsession-errors 
konsole(13009): W /home/daniloaz/.xsession-errors 
konsole(13009): W /home/daniloaz/.xsession-errors 
konsole(13009): W /home/daniloaz/.xsession-errors 
konsole(13009): W /home/daniloaz/.xsession-errors 
konsole(13009): W /home/daniloaz/.xsession-errors

In the example above you can see how the fatrace command displays lines that are repeated hundreds of times just by moving the mouse a bit, which is clearly indicating that the konsole process with PID 13009 has some kind of fault that is causing it to write inordinately to the .xsession-errors file. And this information was obtained while other processes kept writing simultaneously to the same file, which was our initial goal.