Would you have drawn?

I was talking to a friend of mine outside tonight when a homeless man I've dealt with regularly approached us and mumbled something about me talking to his wife (my friend.) I asked him to repeat himself and he mumbled it again and then stepped towards me encroaching on my personal space. My friend told him to not get in our personal spaces like that and leave but as we kept walking away from him he kept following us. He then started yelling at me about having a ccw in his backpack saying he felt threatened by my hand being in my pocket and he was going to shoot me. He kept holding up his backpack but never tried to open it or reach inside. He kept repeating himself and was obviously intoxicated in addition to his usual craziness and kept approaching me and then walking away all while repeatedly telling me I was threatening him. I kept putting space between us but couldn't go inside because that would require turning my back to him to open the door. He was also between me and my car so I couldn't drive away. He finally left after saying his sister would come shoot me and his "friends" would come find my home and shoot me.The police were notified and are looking for him and I never drew because he was just using verbal threats and didn't make a motion to actually draw his alleged gun that I highly doubt he possessed. Would you have drawn? How would you have handled it?Edit: I should add the hand that was in my pocket was my weak hand, my dominant hand was at my side in case he made a motion to draw a weapon. via /r/CCW http://ift.tt/2bT7AHD