Git credential helper - specified cache but still using git-credential-osxkeychain

Git credential helper - specified cache but still using
git-credential-osxkeychain

I set the credential.helper to cache with
git config credential.helper 'cache --timeout=300'
but when I type in my credentials, git-credential-osxkeychain prompts for
my password to cache the credentials into the Keychain.
How can I prevent git-credential-osxkeychain from taking over?
Some background info, I don't think I had this problem before I installed
Github for Mac (but I may just not have noticed). I've uninstalled it now,
but this behavior is still happening.