Today I was trying to update a program called snoopy logger, which is a program to log all executed commands on a system. It does this by wrapping the function execve(), and it get’s called every time a program is started because it’s listed in the ld.so.preload file.
This update was however very unfortunate, I was trying to fix a bug and when I had compiled the new version without any compiler errors I replaced the old snoopy.so with the new one. Then, when I tried executing a command it just froze. This is where the panic starts 🙂
I realized that the new snoopy file must be the culprit, but what should I do about it? I can’t execute a single command. No ls, no mv or cp. Tough one. I had really fucked up the system, no one could start any new processes and if they tried the process would consume 100% CPU and make the system painfully slow.
I logged in to the vserver host and managed to get in that way, I guess it ignored the preload file or something since it’s not a regular login, and changed back to the previous snoopy-binary. However, all programs on the system seemed to have reloaded the bad version of snoopy and wouldn’t load back the old one. Therefore I had to restart all processes on the system (ie reboot). This is what caused your screen being dead! Sorry for any inconveniance! 🙂
