You have frightened him/her.  Maybe he/she won't invite you next time.