After you've setup ssh to access your remote server and things are working fine you might still be challenged for a password for each connection. To remove this add the following to your $HOME/.ssh/config.
Host *
AddKeysToAgent yes
UseKeychain yes
You'll be prompted one more time but subsequent connection requests will be passwordless.