When you run ssh-agent it creates a long-running process that holds decrypted keys and spits out some environment variables that the ssh client can use:

SSH_AUTH_SOCK=/tmp/ssh-Ksfjq28070/agent.28070; export SSH_AUTH_SOCK;
SSH_AGENT_PID=28071; export SSH_AGENT_PID;
echo Agent pid 28071;

The problem is that you want to have those environment variables sourced in when you open new terminal windows. I added the following to my .profile to write those lines out to a file so they could be sourced in easily.

test -e ~/.ssh_agent && . ~/.ssh_agent
need_to_start_ssh_agent="0"
if test "" != "$SSH_AGENT_PID" ; then
  /bin/ps -p $SSH_AGENT_PID | /usr/bin/grep ssh-agent > /dev/null
  res=$?
  if test "1" == "$res" ; then
    echo "ssh-agent is not running. We need to start the agent"
    need_to_start_ssh_agent="1"
  else
    echo "ssh-agent running - pid: $SSH_AGENT_PID"
  fi
else
  echo "ssh-agent is not running. We need to start the agent"
  need_to_start_ssh_agent="1"
fi

if test "1" == "$need_to_start_ssh_agent" ; then
  `which ssh-agent` | grep -v echo > ~/.ssh_agent
  . ~/.ssh_agent
  `which ssh-add`
fi

After using this for a while I found Daniel Robbin’s article about his keychain utility http://www.ibm.com/developerworks/library/l-keyc2/. It’s basically a more-refined version of my addition to .profile. I’m still going to stick with my solution for now because it’s so lightweight.