Monitoring Your Wordpress Administrator User List

Overview

Many Wordpress attacks will create a backdoor Wordpress account that has administrative privileges. One way to tell if your Wordpress instance has been compromised is to check for the presence of an administrator account that you did not add.

The latest Wordpress worm, as described in Matt Mullenweg’s recent post titled How to Keep WordPress Secure, uses javascript on the user page to hide the backdoor administrator account. It’s not enough to log in and check the list of users. You need to check the database itself.

Dougal Campbell had a good post, Checking Your Wordpress Security, that talks about how to find the current list of accounts having administrative privileges in your Wordpress instance. He provided an SQL query that you could run to get the list.

I’d rather have an email sent to me if the list of administrators changes, rather than having to check manually. I created a simple setup that checks my database every 10 minutes and sends me an email if the list changes. It checks to see if new administrator accounts are added and also checks to see if one was removed and replaced.

Security

At first I thought I could use the Wordpress environment itself to make database access easier, but I realized that by doing so I could potentially run compromised code. I chose to directly connect to the database using the credentials in wp-config.php and run my queries using the mysql_* PHP functions.

Implementation

Directory Structure

First I got the directory structure in place. As root:

mkdir /root/etc
mkdir /root/bin

Files

  • admin_list_common.php – Common Code – This code is shared by the other two utilities.
  • check_admin_list.php – Admin List Checker – This checks to see if the current list of admins matches the known good list.
  • create_admin_list_file.php – Admin List Creator – This script will create the initial list of admin users.

Common Include File

IMPORTANT NOTE: Make sure to update the database variables to match your setup. At a minimum, you will need to modify $wordpress_docroot and the database credentials.

In /root/bin/admin_list_common.php I put this:

<?php
 
$wordpress_docroot = "/www/llamalabs.com/content";
$admin_list_file = "/root/etc/myblog.wp_admin_list";
 
// get these variables from wp-config.php, which is in the document root for your blog.
$db_host = "localhost";
$db_user = "user";
$db_pass = "password";
$db_name = "db_name";
 
$db = mysql_connect($db_host, $db_user, $db_pass);
if ( ! $db ) {
    echo "Unable to connect to DB: " . mysql_error() . "\n";
    exit(1);
}
 
if ( ! mysql_select_db($db_name, $db) ) {
    echo "Unable to select db: " . mysql_error() . "\n";
    exit;
}
 
// based off of get_users_of_blog in wp-includes/user.php
function get_admin_users() {
    global $db;
    $sql = "SELECT um.user_id AS ID, u.user_login "
         . "FROM wp_users u, wp_usermeta um "
         . "WHERE u.ID = um.user_id "
         . "    AND um.meta_key = 'wp_capabilities' "
         . "    AND um.meta_value LIKE '%administrator%' "
         . "ORDER BY um.user_id";
 
    $res = mysql_query( $sql, $db );
    if ( ! $res ) {
        echo "Could not execute query: " . mysql_error() . "\n";
        exit(1);
    }
 
    if ( mysql_num_rows( $res ) == 0 ) {
        return array();
    }
 
    $users = array();
    while ( $row = mysql_fetch_assoc( $res ) ) {
        $users[] = $row;
    }
    return $users;
}
 
function get_prev_admin_users() {
    global $admin_list_file;
    $fp = fopen( $admin_list_file, "r" );
    if ( ! $fp ) {
        echo  "Error opening file.\n";
        exit( 1 );
    }
 
    $data = "";
    while ( ! feof( $fp ) ) {
        $data .= fgets( $fp, 4096 );
    }
    return unserialize( $data );
}

Check Admin List

In /root/bin/check_admin_list.php I put this:

<?php
 
require_once 'admin_list_common.php';
 
$prev_admin_users = get_prev_admin_users();
$admin_users = get_admin_users();
 
if ( count( $prev_admin_users ) != count( $admin_users ) ) {
    show_warning_and_exit( $prev_admin_users, $admin_users );
}
 
$admin_count = count( $admins );
 
for ( $i = 0 ; $i < $admin_count ; $i++ ) {
    if ( $admin_users[$i]["ID"] != $prev_admin_users[$i]["ID"] ) {
        show_warning_and_exit( $prev_admin_users, $admin_users );
    }
}
 
function show_warning_and_exit( $prev_admin_users, $admin_users ) {
    echo "Warning! The list of admin users differs.\n\n";
 
    echo "Current Admin Users\n";
    print_admin_users( $admin_users );
 
    echo "Previous Admin Users\n";
    print_admin_users( $prev_admin_users );
 
    exit(1);
}
 
function print_admin_users( $admin_users ) {
    $admin_count = count( $admin_users );
    echo "------------------------------------\n";
    for ( $i = 0 ; $i < $admin_count ; $i++ ) {
        echo "ID({$admin_users[$i]["ID"]}) user_login({$admin_users[$i]["user_login"]})\n";
    }
    echo "\n";
}

Create Admin List File

In create_admin_list_file.php I put this:

<?php
 
require_once 'admin_list_common.php';
 
$fp = fopen( $admin_list_file, 'w' );
if ( ! $fp ) {
    echo "Error opening file .\n";
    exit( 1 );
}
 
$admin_users = get_admin_users();
if ( fwrite( $fp, serialize( $admin_users ) ) === FALSE ) {
    echo "Cannot write to file ($admin_list_file)";
    exit( 1 );
}
 
fclose($fp);

Setup and Testing

Run the command to populate the known list of admin accounts. Then verify that the file was created:

[root@host bin]# php create_admin_list_file.php 
[root@host bin]# cat /root/etc/myblog.wp_admin_list 
a:3:{i:0;a:2:{s:2:"ID";s:1:"1";s:10:"user_login";s:5:"admin";}i...<snipped>
[root@host bin]#

Run the check script and make sure you get an exit status of 0.

[root@host bin]# php check_admin_list.php 
[root@host bin]# echo $?
0
[root@host bin]#

Now, check to make sure that the check works. To do this you can erase the contents of the known good list, test, and then restore. Make sure you get an exit status of 1 from your check scrpt. This will cause cron to mail you the report.

[root@host bin]# php check_admin_list.php 
Warning! The list of admin users differs.
 
Current Admin Users
------------------------------------
ID(1) user_login(admin)
ID(2) user_login(bob)
ID(3) user_login(harry)
 
Previous Admin Users
------------------------------------
ID() user_login()
 
[root@host bin]# echo $?
1
[root@host bin]# php create_admin_list_file.php 
[root@host bin]# php check_admin_list.php 
[root@host bin]# echo $?
0
[root@host bin]#

Set up the Recurring Check

Set up the crontab entry using the crontab command:

crontab -u root -e

The contents of my entry are:

*/10 * * * * php /root/bin/check_admin_list.php

Final Notes

It’s important to test your setup. Make sure that when you add a new admin account you are alerted about the change.

Some people may not want to run the check as root. That’s fine, simply set it up under a different account.

Make sure that cron emails for the user running the cron job are making it to your inbox. Alternatively you could use the PHP mail functions to send yourself an email with the warning.

If you have nonstandard table prefixes you will need to update the SQL queries to reflect your custom table prefixes.

If I didn’t have full root access (ie: shared hosting, etc), I’d look into whether or not my hosting provider gave me the ability to create cron jobs. If so, I’d use whatever facilities were available, keeping the files out of the document root for the site. Yes, you could run these php scripts under your document root, but you should obfuscate them and understand that it’s not as safe to do so.

If you have PHP installed in a non-standard location, make sure to specify the path as well as the binary name when editing your crontab.

If you’re not getting cron emails, check out your system’s maillog and other log files to see what’s happening.

If you do need to add a new administrator account and you get emails about the difference, simply run the create_admin_list_file.php file again in order to update your list of known admin accounts.

Related Links

Using Keyword Substitution in Subversion

Many times I’ve been looking at a configuration file on a server, wondering who made the last change to this file? What did they change? Whatever they did, it broke something.

When there’s more than one person making changes to system configuration, I like to keep that system configuration under version control. It makes it easier to track who is making configuration changes, it provides a history of the changes, and you can set up a post-commit hook to notify team members about the changes that are being committed.

One nice feature in Subversion is Keyword Substitution. Keyword substitution allows you to embed keyword anchors into your file that get expanded to show useful information about the file such as the revision, revision date, URL.

I like to embed the Id and the Url near the top of configuration files in a comment block. The Id contains filename, revision, revision time, and user and the Url describes the full URL to the latest version of the file in the repository.

Whenever I encounter a configuration file in a system that has these substitutions in place, I can immediately tell where I need to go in order to check out and commit changes to the file.

Below is an example of how to use Subversion’s keyword substitution. I’ll set up a local subversion repository, add a file and do the keyword substitution. I’m going to do this on my macbook.

First, create an empty repository called ’system’:

macbook ~ $ sudo mkdir /usr/local/svn
macbook ~ $ sudo chown $USER /usr/local/svn
macbook ~ $ sudo chgrp $USER /usr/local/svn
macbook ~ $ svnadmin create /usr/local/svn/system
macbook ~ $

Next, check out the repository and change directory into the checked out copy. Note, I’m skipping the normal repository structure (trunk, tags, branches) to keep things simple.

macbook ~ $ svn co file:///usr/local/svn/system system
Checked out revision 0.
macbook ~ $ cd system/
macbook ~ $

Make a sample configuration file full of comments:

macbook system $ cat >sample.cfg<<EOD
> # this is a sample configuration file
> #
> # \$Id\$
> # \$URL\$
> #
> EOD
macbook system $

Add the file to the repository:

macbook system $ svn add sample.cfg 
A         sample.cfg
macbook system $ svn ci -m 'adding sample config file to demonstrate keyword substitution'
Adding         sample.cfg
Transmitting file data .
Committed revision 1.
macbook system $

Notice how the keyword anchors are in the file, not the substitutions.

macbook system $ cat sample.cfg 
# this is a sample configuration file
#
# $Id$
# $URL$
#
macbook system $

We’ll need to enable the keyword substitution for sample.cfg using ’svn propset svn:keywords’.

macbook system $ svn propset svn:keywords "Id URL" sample.cfg
property 'svn:keywords' set on 'sample.cfg'
macbook system $ svn st
 M     sample.cfg
macbook system $ svn ci -m 'enabling keyword substitution for Id and URL on sample.cfg'
Sending        sample.cfg
 
Committed revision 2.
macbook system $

Now let’s look at our configuration file:

macbook system $ cat sample.cfg 
# this is a sample configuration file
#
# $Id: sample.cfg 2 2009-07-22 06:46:35Z dustin $
# $URL: file:///usr/local/svn/system/sample.cfg $
#
macbook system $

There’s our Id and URL. Note, if you checked out an existing repository over HTTP(S) or SSH, the URL would reflect that URL. So for a team accessing the SVN repository through Apache+WebDAV the example may look like this:

macbook system $ cat sample.cfg 
# this is a sample configuration file
#
# $Id: sample.cfg 2 2009-07-22 06:46:35Z dustin $
# $URL: https://svn.example.com/svn/system/sample.cfg $
#
macbook system $

Make sure to read through the docs at Keyword Substitution for more examples and additional details.

ZenOSS: Living with EC2…

1:32 AM Subject: [zenoss] ec2-75-101- … Command timed out on device ec2-75-101… check_http
1:37 AM Subject: [zenoss] CLEAR: ec2-75-101… HTTP OK HTTP/1.1 200 OK – 1241 bytes in 7.617 seconds

This seems to be my life living with EC2 instances…

Dice Game – Hosting Failure

The Dice Game is hosted in an OpenVZ container on a server in Renton, WA. I should rephrase that. It *was* hosted in an OpenVZ container on a server in Renton, WA.

Last night the server hosting the Dice Game failed. At first there were some filesystem errors. Then the root filesystem was remounted in read-only mode. All this happened while I was pushing out some changes to the game.

I issued a reboot of the underlying server and waited for it to come back up. I heard nothing from it. Remote console access wasn’t working; nothing was showing up on the console. How long should I wait before finding a new home for the Dice Game? The server was powercycled and there was still no output on the console. I think it’s dead.

It just so happens that I have another OpenVZ container at a hosting provider in Seattle, but it’s running a rather old version of CentOS. It would have to do.

I removed the incumbent version of Ruby and downloaded, built and installed the latest stable release from scratch. I installed rubygems and all of the required gems to make the Dice Game work. I had to rebuild Apache from scratch to support mod_proxy and mod_proxy_reverse (so it could send requests to the mongrel processes). I configured Apache, Mongrel Cluster and Mysql. I restored the Dice Game database backup from my last hourly backup. I installed Git so I could push out my local repository to the new server. I created a new hostname in DNS and updated the Facebook application configuration to use the new hostname.

It took about two hours to get everything built and restored on the new system, which is less time than it would have taken to order and have a brand new dedicated server delivered. The Dice Game seemed a bit more snappy on the new system too.

I found out the next morning that the original server was not completely dead. I’m not going to move the game back to it though, unless the new system dies. If I have to do so, I know that the old system is ready to be used as a replacement.

Links:

The Filesystem Errors:


EXT3-fs warning (device sda2): ext3_rmdir: empty directory has nlink!=2 (-1)
EXT3-fs warning (device sda2): ext3_rmdir: empty directory has nlink!=2 (-2)
EXT3-fs warning (device sda2): empty_dir: bad directory (dir #17991246) - no `.' or `..'
EXT3-fs warning (device sda2): ext3_rmdir: empty directory has nlink!=2 (1)
EXT3-fs warning (device sda2): empty_dir: bad directory (dir #17991247) - no `.' or `..'
EXT3-fs warning (device sda2): ext3_rmdir: empty directory has nlink!=2 (8)
EXT3-fs unexpected failure: !buffer_revoked(bh);
inconsistent data on disk
ext3_forget: aborting transaction: IO failure in __ext3_journal_revoke
ext3_abort called.
EXT3-fs error (device sda2): ext3_forget: error -5 when attempting revoke
Remounting filesystem read-only
Aborting journal on device sda2.
EXT3-fs error (device sda2) in ext3_free_blocks_sb: Journal has aborted
EXT3-fs error (device sda2) in ext3_reserve_inode_write: Journal has aborted
EXT3-fs error (device sda2) in ext3_truncate: IO failure
EXT3-fs error (device sda2) in ext3_reserve_inode_write: Journal has aborted
EXT3-fs error (device sda2) in ext3_orphan_del: Journal has aborted
EXT3-fs error (device sda2) in ext3_reserve_inode_write: Journal has aborted
EXT3-fs error (device sda2) in ext3_delete_inode: IO failure

Wiping hard drives – DBAN – Darik’s Boot And Nuke

I recently went on a quest to find a tool to wipe four hard drives in some machines at work. I hadn’t wiped a drive in years, and was pleasantly surprised to get more than one suggestion for DBAN, or Darik’s Boot and Nuke.

From http://www.dban.org/:

Darik’s Boot and Nuke (“DBAN”) is a self-contained boot disk that securely wipes the hard disks of most computers. DBAN will automatically and completely delete the contents of any hard disk that it can detect, which makes it an appropriate utility for bulk or emergency data destruction.

I’d have to say, it has definitely lived up to its name. I downloaded the ISO image, burned it to a CD and several unattended hours later I had four clean machines. You boot one, remove the CD, and move on to the next machine.

Now I just need to find a new home for these machines.

Solitaire on EC2 – Windows Server Available

Amazon announced the availability of their Windows Server 2003 AMI’s. Read more at Amazon EC2 Running Microsoft Windows Server. The current version of Elasticfox has the new API built in, so it was a good experiment to try it out too. Read through the links below to quickly get your Windows instance up. I went with the most basic 32 bit version available for my solitaire test. Click on any of the images for a larger version.

Once you read the directions and get your instance started, you can use your favorite Remote Desktop client to access your instance:

EC2 Windows Login

The new API allows you to get the administrator password. Log in as Administrator with the password you get back and you’re greeted with the desktop.

EC2 Windows Desktop

Big deal? I want to play solitaire on it. Oh, it doesn’t come with Solitaire, you’ll have to supply that yourself (Thanks Gmail for not letting me send Windows executables). It also seems that they don’t provide an easy way to access the installation media for Windows either (Not that I could install it from the install media if I wanted to, and I’ll probably be proven wrong on this anyhow).

EC2 Windows Solitaire

Wasn’t that exciting? ec2kill time.

Quercus PHP performance compared to Apache mod_php + APC

“Quercus is Caucho Technology’s fast, open-source, 100% Java implementation of the PHP language” [1].

It has been demonstrated that Quercus outperforms a straight-out-of-the box installation of mod_php.

  1. “Performance: Quercus outperforms a straight mod_php implementation by about 4x (for Mediawiki and Drupal).” [1]
  2. “Resin backed PHP drives 4x performance improvements for Drupal” [2]

I had always wondered how Quercus compared to APC though, specifically for running a Drupal instance. I had planned on performing a bunch of tests, but they state clearly enough in the docs that it roughly matches PHP performance with accelerators like APC. [1]

That’s exactly what I needed to know. I’ll stick with APC until I need to write some new PHP functions in Java. So sorry to disappoint you with a lack of no new conclusions, but mod_php + APC keeps me pretty happy.

Links:
[1] http://www.caucho.com/resin-3.0/quercus/
[2] http://www.workhabit.com/labs/resin-backed-php-drives-4x-performance-improvements-drupal

Notes on Google Apps Premier

I’ve been collecting notes about my experiences with Google Apps Premier (The service where you pay $50 per seat). It includes Google Mail, Calendar, Documents, and Sites.

Yesterday’s outage reminded me of my list and I wanted to get around to posting it.

Hopefully these notes will be useful to someone who is evaluating Google Apps for their email/calendar/docs solution.

Without any specific order, here are the notes I have collected.

There’s no way to enforce communication over HTTPS through the admin control panel.

Before the recent announcement of the option to require https, by default you would be redirected to http://mail.google.com after signing in. With employees working remotely, not being able to force communication using HTTPS was a real bummer. There is now the option to always use HTTPS in Google Apps Premier. It’s still possible to switch back to plain old HTTP once logged in.

Here are a few pages discussing the issue. It’s no really such an issue anymore, but they are in my notes, so I’m including them here.

You are forced to go through the Google Checkout process to add new seats

There’s no easy way to add a new user account without going through the Google Checkout process. If you’re purchasing single seats each time you have a new hire, that means that you need to go through the Google Checkout process in addition to adding the account. You could avoid this hassle by purchasing seats in advance.

Ideally there would be an option to charge the credit card on file. It’s already stored in the account used for Google Checkout, so why not automate the process?

Can’t forward one user’s email to another user in the same domain

So, when someone leaves the company, and you need to forward new messages destined for her account to her supervisor, you’d usually put a forward in place.

When I try to do so in Google Apps I get the message “Forwarding to Google Apps hosted email addresses is not supported”. I’m not the only person having this problem.

I’m able to do so by logging in as that user and putting the forward in place under the user’s Gmail settings.

I’ve been going back and forth with Google Apps Premier support for a couple weeks now attempting to troubleshoot this issue. I’m assured it’s a problem specific to my account.

Disabling account access.

When you change the password for an account, it does not invalidate existing logged-in sessions. If the user is logged in, she will be able to to still use her account.

Why not suspend the account? That disables logging in, and will log a user out after several minutes, but it also blocks incoming mail to that user.

A third option is to log in as the user and click the “Details” link that is part of the account activity messaging. That opens a pop-up that shows you existing logged in sessions and gives you the opportunity to log those sessions out. It would be nice if that feature was in the Admin control panel.

Unreliable IMAP

I don’t personally use IMAP for my email, but there have been confirmed cases of messages being delayed for 20+ minutes when using IMAP. The messages will show up instantly through the Gmail interface,
but in an external mail reader the new messages won’t show up. YMMV.

Some messages silently ignored during migration

If you use their IMAP migration tool (Which is very handy), you may notice that some messages (containing zip files) are not migrated.

Support

It’s been hit and miss when dealing with Google Apps Technical Support.

Customer: We're having problems checking mail using IMAP.
Support: Which web browser are you using?

The online documents are nice.

Feature Suggestions

Make sure to take a look at the Feature Suggestions to make sure some critical feature you require isn’t in the pipeline.

Google Groups not included

Before signing up I looked into whether or not Google Groups was one of the included applications.

Pine Star said:

Hello everyone interested in this integration:

I had a chat with the Google Rep of Commercial (paid) version of
Google Apps yesterday.
He said they are going to release a “Premier” edition of Google Apps
by end of 2007 that wil have Google Groups integrated.

So, they are also thinking on these lines….

If you’re looking for Google Groups for your projects in Google Apps Premier, you will be disappointed to find out that it’s not included.

We were really hoping to have it so we could have archived discussions for projects.

Email Lists

You can create email lists that deliver to multiple recipients, both members of the domain and external email addresses.

A couple notes on the lists.

  • No subject prefixes.
  • No List-id field.
  • Not as cool as having Google Groups included as an app.

Calender Importing

Aside from using the API, there appears to be no way to import an entire calendar into a new instance.

Branding with a Custom Logo

You can define a custom logo to be displayed instead of the default Google logo.

Conclusion?

Like any service, Google Apps Premier has its pros and cons. Research thoroughly before making the leap.

Git and .profile – Aliases and config

Many thanks to Tim for his article Installing git on Mac OS X 10.5 Leopard. It’s a great starting point for git use on OS X Leopard.

In addition to his configuration and aliases, I have a couple of my own that I have added. Here’s an excerpt from my .profile:

git config --global user.name "Your Name"
git config --global user.email "your email"
git config --global color.diff auto
git config --global color.status auto
git config --global color.branch auto
git config --global color.interactive auto
git config --global alias.st status
git config --global alias.ci commit
git config --global alias.co checkout
git config --global alias.br branch
git config --global merge.tool opendiff
git config --global merge.summary true
git config --global core.excludesfile ~/.gitignore

alias g='git'
alias gst='g st'
alias gci='g ci'
alias gdiff='g diff'
alias gadd='g add'

I don’t have anything against the space bar, but it does seem to be easier with those aliases. See Tim’s article for some additional comments on some of the git configuration.

The color configuration commands will also add quite a bit of richness to your git experience.

Using ssh-agent for password-less ssh access

Instead of typing in the password to decrypt my private SSH key every time I want to ssh to a host I use a program called ssh-agent, which is included in the openssh-clients package in CentOS.

When you run ssh-agent it creates a long-running process that holds decrypted keys and spits out some environment variables that the ssh client can use:

SSH_AUTH_SOCK=/tmp/ssh-Ksfjq28070/agent.28070; export SSH_AUTH_SOCK;
SSH_AGENT_PID=28071; export SSH_AGENT_PID;
echo Agent pid 28071;

The problem is that you want to have those environment variables sourced in when you open new terminal windows. I added the following to my .profile to write those lines out to a file so they could be sourced in easily.

test -e ~/.ssh_agent && . ~/.ssh_agent
need_to_start_ssh_agent="0"
if test "" != "$SSH_AGENT_PID" ; then
  /bin/ps -p $SSH_AGENT_PID | /usr/bin/grep ssh-agent > /dev/null
  res=$?
  if test "1" == "$res" ; then
    echo "ssh-agent is not running. We need to start the agent"
    need_to_start_ssh_agent="1"
  else
    echo "ssh-agent running - pid: $SSH_AGENT_PID"
  fi
else
  echo "ssh-agent is not running. We need to start the agent"
  need_to_start_ssh_agent="1"
fi
if test "1" == "$need_to_start_ssh_agent" ; then
  `which ssh-agent` | grep -v echo > ~/.ssh_agent
  . ~/.ssh_agent
  `which ssh-add`
fi

After using this for a while I found Daniel Robbin’s article about his keychain utility http://www.ibm.com/developerworks/library/l-keyc2/. It’s basically a more-refined version of my addition to .profile. I’m still going to stick with my solution for now because it’s so lightweight.

  • September 2010
    M T W T F S S
    « Sep    
     12345
    6789101112
    13141516171819
    20212223242526
    27282930  
  • Author

    A little something about you, the author. Nothing lengthy, just an overview.