For some reason I couldn't log into a remote server after recreating my ssh keys and cleaning up my keychain on my system.
The error that kept coming up was
Permission denied (publickey,gssapi-keyex,gssapi-with-mic)
After searching around, I found this unanswered post
on Stack Exchange that lead me to the solution that worked for me:
First generate the key-pair on your Ubuntu machine.
After, copy the contents of the generated .pub file in your ssh folder (~/.ssh/id_rsa) and paste it to the username/.ssh/id_rsaauthorized_keys file, on a new line, on your CentOS for the specific user you are logging in with.