{"id":80087,"date":"2019-12-17T09:19:46","date_gmt":"2019-12-17T17:19:46","guid":{"rendered":"https:\/\/github.blog\/?p=80087"},"modified":"2024-10-15T08:13:29","modified_gmt":"2024-10-15T15:13:29","slug":"ubuntu-apport-toctou-security-vulnerability-cve-2019-7307","status":"publish","type":"post","link":"https:\/\/github.blog\/security\/vulnerability-research\/ubuntu-apport-toctou-security-vulnerability-cve-2019-7307\/","title":{"rendered":"Ubuntu apport TOCTOU security vulnerability (CVE-2019-7307)"},"content":{"rendered":"\n

I started a four-part series about Ubuntu’s crash reporting system<\/a>. In this second post, I’ll focus on apport CVE-2019-7307<\/a>, a TOCTOU vulnerability that enables a local attacker to include the contents of any file on the system in a crash report.<\/p>\n

The bug<\/span><\/a><\/h2>\n

Apport allows you to place a file in your home directory named ~\/.apport-ignore.xml<\/code>. It enables you to specify a custom list of executables that should be ignored by the crash reporter. But what happens if you replace ~\/.apport-ignore.xml<\/code> with a symlink to a file that you don’t own, such as \/etc\/shadow<\/code>? The code that handles that is at report.py, line 962<\/a>:<\/p>\n

if not os.access(ifpath, os.R_OK) or os.path.getsize(ifpath) == 0:\n    # create a document from scratch\n    dom = xml.dom.getDOMImplementation().createDocument(None, 'apport', None)\nelse:\n    try:\n        dom = xml.dom.minidom.parse(ifpath)\n    except ExpatError as e:\n        raise ValueError('%s has invalid format: %s' % (_ignore_file, str(e)))\n<\/code><\/pre>\n
As you can see, it uses os.access<\/code> to check that the user has permission to access the file. If the permission check passes, then it calls xml.dom.minidom.parse<\/code> to parse the XML. This is a classic example of a “time of check to time of use” (TOCTOU<\/a>) vulnerability. If the file is valid at the time of the os.access<\/code> check, but I quickly replace it with a symlink to a different file before the call to xml.dom.minidom.parse<\/code>, then I can trick apport into using its elevated privileges to read a file which I do not have permission to access myself.<\/p>\n
Subtleties of privilege dropping in apport<\/span><\/a><\/h3>\n
You may wonder why the os.access<\/code> check would ever fail, because apport is a root process. The reason is that apport drops privileges during its execution in two stages. The first stage happens at apport, line 455<\/a>:<\/p>\n
# Partially drop privs to gain proper os.access() checks\ndrop_privileges(True)\n<\/code><\/pre>\n
The second stage happens at line 601<\/a>:<\/p>\n
# Totally drop privs before writing out the reportfile.\ndrop_privileges()\n<\/code><\/pre>\n
What do they mean by “partially drop privs” and “totally drop privs”? This is related to the real, effective, and saved user ids<\/a> of the process:<\/p>\n
\n\n\n\n\n\n\n\n
<\/th>\nRUID<\/th>\nEUID<\/th>\nSUID<\/th>\n<\/tr>\n<\/thead>\n
root process<\/td>\n0<\/td>\n0<\/td>\n0<\/td>\n<\/tr>\n
“partially drop privs”<\/td>\n1001<\/td>\n0<\/td>\n0<\/td>\n<\/tr>\n
read files safely<\/td>\n1001<\/td>\n1001<\/td>\n0<\/td>\n<\/tr>\n
“totally drop privs”<\/td>\n1001<\/td>\n1001<\/td>\n1001<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/content-table-wrap><\/div>\n
(0 is the user id of root<\/code> and 1001 is the user id of my own unprivileged account, kev<\/code>.) The real user id (RUID<\/code>) determines the owner of the process, but the effective user id (EUID<\/code>) determines which files the process can read and write. This means that when apport is in the “partially drop privs” state, it can still read any file on the system.<\/p>\n
The correct way for apport to make sure that it doesn’t accidentally use its root privileges to read or write a file is to first enter the state that I have named “read files safely” in the table. Because the saved user id (SUID<\/code>) is still root, the process can temporarily enter the “read files safely” state and then revert back to “partially drop privs” after it’s done reading the file. Note that the transition to “totally drop privs” is, in contrast, irreversible.<\/p>\n
The os.access<\/code> check is unusual because it uses the RUID<\/code>, rather than the EUID<\/code>, to check whether the real user has permission to access the file. This is the reason why there is a TOCTOU vulnerability. Apport is in the “partially drop privs” state when os.access<\/code> is called. This means it will reject files that I don’t own, but if I can bypass the os.access<\/code> check then the subsequent call to xml.dom.minidom.parse<\/code> will be able to read any file because the EUID<\/code> is still root. I can do this by timing the attack to replace ~\/.apport-ignore.xml<\/code> with a symlink just after<\/em> the call to os.access<\/code>.<\/p>\n
Comparison to CVE-2019-11481<\/span><\/a><\/h3>\n
I found a very similar bug at fileutils.py, line 335<\/a>:<\/p>\n
def get_config(section, setting, default=None, path=None, bool=False):\n    '''Return a setting from user configuration.\n\n    This is read from ~\/.config\/apport\/settings or path. If bool is True, the\n    value is interpreted as a boolean.\n    '''\n    if not get_config.config:\n        get_config.config = ConfigParser()\n        if path:\n            get_config.config.read(path)\n        else:\n            get_config.config.read(os.path.expanduser(_config_file))\n<\/code><\/pre>\n
This code opens the file ~\/.config\/apport\/settings<\/code> with a root EUID<\/code>. At first glance, since an os.access<\/code> check doesn’t exist here, it seems easier to exploit than the other bug. After further review, I found that it isn’t, and the reason is due to a difference in
\nerror handling behavior. For example, if I want to use the bug to read the contents of \/var\/shadow<\/code>,  it’s not a valid XML file, and it also isn’t formatted correctly to be parsed as an apport settings file. So, in either case, it will trigger a parse error in apport. In the case of ~\/.config\/apport\/settings<\/code>, this causes apport to abort immediately. But in the case of ~\/.apport-ignore.xml<\/code>, the incorrectly formatted file is ignored and apport continues running. Because of this, I found it easier to exploit ~\/.apport-ignore.xml<\/code>.<\/p>\n
I reported the ~\/.config\/apport\/settings<\/code> bug to Ubuntu: bug 1830862<\/a>.
\nIt’s since been fixed and assigned CVE-2019-11481<\/a>.<\/p>\n
Exploit plan<\/span><\/a><\/h2>\n
The bug enables me to trick apport into loading any file on the system, by replacing ~\/.apport-ignore.xml<\/code> with a symlink. But any file that I’m interested in is almost certainly not going to be a valid XML file, so it will cause a parse error and apport will ignore it. How could this help me access forbidden information?<\/p>\n
Here’s my cunning plan:<\/p>\n
\"Ubuntu<\/p>\n
The main idea is that, even though the forbidden file will trigger a parse error and get ignored, it’s still loaded into apport’s heap. This means that if I crash apport then the contents of the file will be included in the crash report. This is the sequence of events in the plan:<\/p>\n
    \n
  1. I start \/bin\/sleep<\/code> and crash it by sending it a SIGSEGV<\/code>.<\/li>\n
  2. Apport starts up to generate a crash report for \/bin\/sleep<\/code>.<\/li>\n
  3. I replace ~\/.apport-ignore.xml<\/code> with a symlink at exactly the right moment, so that apport loads a forbidden file into memory.<\/li>\n
  4. I crash apport by sending it a SIGSEGV<\/code>.<\/li>\n
  5. A second apport starts up to generate a crash report for the first apport.<\/li>\n
  6. The second apport writes out a crash report for the first, containing a copy of the forbidden file in the core dump.<\/li>\n<\/ol>\n
    Obstacles<\/span><\/a><\/h3>\n
    It wasn’t quite that easy. I ran into several problems. The obvious one is that precise timing of the symlink switcheroo is crucial, so I anticipated that being difficult to get right. But there were also some unexpected problems, which I’ll cover in the following sections.<\/p>\n
    Anti-recursion mitigations<\/span><\/a><\/h4>\n
    Apport has a couple of mitigations to prevent it from running on itself. The comment at apport, line 30<\/a> explains that this is to avoid “bringing down the system to its knees if there is a series of crashes”.<\/p>\n
    The first mitigation is a lock file named \/var\/crash\/.lock<\/code>. When apport starts, it uses lockf<\/code><\/a> to set a lock on this file to prevent another apport from running at the same time.<\/p>\n
    The interesting thing is that lockf<\/code> file locks are only advisory<\/em>! In fact, as Victor Gaydov explains in this excellent overview<\/a>, the lock is actually associated with an [i-node, pid] pair. This means that if I replace \/var\/crash\/.lock<\/code> with a new file after the first apport has set its lock, then the second apport will see a different i-node, so both apports can hold locks on \/var\/crash\/.lock<\/code> at the same time!<\/p>\n
    The trick of replacing \/var\/crash\/.lock<\/code> with a new file relies on me having permission to delete or move the file. Since the \/var\/crash<\/code> directory has the sticky bit set (see the first post for more information<\/a>), this means that I must own the file. Luckily, \/var\/crash<\/code> is world-writable, so I can create \/var\/crash\/.lock<\/code> as long as it doesn’t already exist. When I first submitted my bug report<\/a> to Ubuntu on May 29, I thought that this would often make the vulnerability unexploitable. That’s because on my work laptop, \/var\/crash\/.lock<\/code> almost always exists and is owned by root. I have since discovered that \/var\/crash\/.lock<\/code> is deleted by a daily cronjob: \/etc\/cron.daily\/apport<\/code>. The lock file often exists on my work laptop because I deliberately crash applications on a fairly regular basis. But on a typical Ubuntu system, it is unlikely to exist at any given time, due to the daily cronjob.<\/p>\n
    In my bug report<\/a>, I recommended that \/var\/crash\/.lock<\/code> should always
    \nexist and be owned by root, as a mitigation against this type of exploit. While I did not regard it as a vulnerability by itself, Sander Bos has since submitted a separate     bug report<\/a> about this issue. It’s been assigned CVE-2019-11485<\/a> and fixed by changing the directory that the lock file is stored in.<\/p>\n
    The second mitigation is a slightly obscure bit of logic in the kernel<\/a>, based on RLIMIT_CORE<\/code><\/a>. RLIMIT_CORE<\/code> is a resource limit: the maximum size of the
    \ncore file. The value RLIMIT_CORE == 1<\/code> is used as a special value to indicate that the process is a crash reporter and should not generate a core dump if it crashes (to prevent recursion). I found an explanation of this mitigation in     this comment<\/a>.<\/p>\n
    I got lucky with the RLIMIT_CORE<\/code> mitigation. It turns out that you can use prlimit<\/code><\/a> to modify the RLIMIT_CORE<\/code> of another process! You need to have appropriate permissions to so do, of course, but I found that it works as soon as apport enters the “totally drop privs” state (refer to the table). Unfortunately, It isn’t possible to increase<\/em> the value of RLIMIT_CORE<\/code> with prlimit<\/code>, but I am able to drop it to zero, which is sufficient for this exploit.<\/p>\n
    Signal handling<\/span><\/a><\/h4>\n
    Part of my cunning plan was to crash apport by sending it a SIGSEGV<\/code>. That doesn’t work because apport sets a signal handler<\/a> for SIGSEGV<\/code>:<\/p>\n
    def setup_signals():\n    '''Install a signal handler for all crash-like signals, so that apport is\n    not called on itself when apport crashed.'''\n\n    signal.signal(signal.SIGILL, _log_signal_handler)\n    signal.signal(signal.SIGABRT, _log_signal_handler)\n    signal.signal(signal.SIGFPE, _log_signal_handler)\n    signal.signal(signal.SIGSEGV, _log_signal_handler)\n    signal.signal(signal.SIGPIPE, _log_signal_handler)\n    signal.signal(signal.SIGBUS, _log_signal_handler)\n<\/code><\/pre>\n
    Again, it appears that the motivation for this is to prevent apport from running recursively on itself. Luckily for me, the list of signals that setup_signals<\/code> sets handlers for isn’t sufficiently thorough. The section 7 man page for signal<\/a> has a table titled “Standard signals”. Here’s a short excerpt:<\/p>\n
    \n\n\n\n\n\n\n\n
    Signal<\/th>\nValue<\/th>\nAction<\/th>\nComment<\/th>\n<\/tr>\n<\/thead>\n
    SIGINT<\/code><\/td>\n2<\/td>\nTerm<\/td>\nInterrupt from keyboard<\/td>\n<\/tr>\n
    SIGQUIT<\/code><\/td>\n3<\/td>\nCore<\/td>\nQuit from keyboard<\/td>\n<\/tr>\n
    SIGILL<\/code><\/td>\n4<\/td>\nCore<\/td>\nIllegal Instruction<\/td>\n<\/tr>\n
    …<\/td>\n…<\/td>\n…<\/td>\n…<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/content-table-wrap><\/div>\n
    Any signal with “Core” in the “Action” column will trigger a core dump. Apport’s list of signal handlers includes the most common core-generating signals, but it’s far from comprehensive. There are several left to choose from. My exploit uses SIGTRAP<\/code>.<\/p>\n
    Exploit implementation<\/span><\/a><\/h2>\n
    I’ve posted the source code for my proof-of-concept exploit on GitHub<\/a>. It works mostly according to the plan that I described above, but with a few tweaks to account for the obstacles discussed above. This is the sequence of events in the revised plan:<\/p>\n
      \n
    1. I start<\/a> a \/bin\/sleep<\/code>.<\/li>\n
    2. I create<\/a> \/var\/crash\/.lock<\/code>, so that I can delete it later.<\/li>\n
    3. I kill<\/a> \/bin\/sleep<\/code> with a SIGSEGV<\/code>.<\/li>\n
    4. Apport starts up to generate a crash report for \/bin\/sleep<\/code>.<\/li>\n
    5. I replace<\/a> ~\/.apport-ignore.xml<\/code> with a symlink at exactly the right moment, so that apport loads a forbidden file into memory.<\/li>\n
    6. I replace<\/a> \/var\/crash\/.lock<\/code> with a new file, to bypass the file lock and enable a second apport to run at the same time as the first.<\/li>\n
    7. I use<\/a> prlimit<\/code> to set apport’s RLIMIT_CORE<\/code> to zero.<\/li>\n
    8. I crash<\/a> apport by sending it a SIGTRAP<\/code>.<\/li>\n
    9. A second apport starts up to generate a crash report for the first apport.<\/li>\n
    10. The second apport writes out a crash report for the first, containing a copy of the forbidden file in the core dump.<\/li>\n<\/ol>\n
      All that’s left to discuss is how I time the symlink switcheroo. I initially thought it would be very difficult to get the exploit working, because there is such a short time-interval between the call to os.access<\/code> and when the file is opened. But it turns out that it is hilariously easy to win a race against Python when you are programming in C. The crucial moment in the PoC, when the switcheroo happens, is at line 155<\/a>. I use inotify<\/a> for the timing. By running sudo strace -e file -tt -p <apport PID><\/code>, I discovered that a file named expatbuilder.cpython-36.pyc<\/code> is always opened immediately before ~\/.apport-ignore.xml<\/code> is parsed. By watching<\/a> for an IN_OPEN<\/code> event on that file, I can time the switcheroo very precisely.<\/p>\n
      You have got to be kidding me!<\/span><\/a><\/h2>\n
      When I was finally able to get the exploit working, I excitedly went to look at the crash report in \/var\/crash<\/code> and saw the following:<\/p>\n
      kev@constellation:~$ ls -al \/var\/crash\/\ntotal 4492\ndrwxrwsrwt  2 root whoopsie   12288 Nov  5 12:26 .\ndrwxr-xr-x 17 root root        4096 Jul 17 19:31 ..\n-rw-r-----  1 root whoopsie 4583201 Nov  5 12:26 _usr_share_apport_apport.0.crash\n<\/code><\/pre>\n
      That was definitely a facepalm moment. The file is owned by root. What happened? I was sure that it would be owned by me, because my PoC doesn’t send the SIGTRAP<\/code> until after the first apport has entered the “totally drop privs” state (refer to the table). The apport process is completely owned by me at the moment when it crashes, so surely I should be able to read the crash report? This problem is caused by a subtle detail in how apport determines the owner of the crashed process. This happens in get_pid_info<\/code><\/a>, by running os.stat<\/code> on \/proc\/[pid]\/stat<\/code>. This is explained in a couple of comments scattered throughout the source code, such as here<\/a> and here<\/a>. It’s a mitigation against accidentally leaking sensitive information when a setuid binary crashes (which is almost exactly what I’m trying to do). In my case, apport was started as a root process, so \/proc\/[pid]\/stat<\/code> is owned by root, even after the transition to the “totally drop privs” state. I haven’t been able to find any way to defeat this protection.<\/p>\n
      The consolation prize is that the exploit works. When I looked at the contents of the file, this is what I saw:<\/p>\n
      \"Core<\/p>\n
      The other good news is that the exploit is very quick and reliable. I thought that the timing of the symlink switcheroo might make it unreliable, but I found that it works perfectly every time.<\/p>\n
      So all is not lost. Although the crash report is owned by root, it’s also readable by whoopsie, which means that if I can find a vulnerability in the whoopsie daemon, I might also be able to read the contents of the crash report.<\/p>\n
      To be continued …<\/span><\/a><\/h2>\n
      Stay tuned for the next two posts in this series:<\/p>\n