I feel like I'm being victimized by one of my friends...?

For starters, I've known this friend for about 2 or 3 years now. He didn't used to be like this, but recently he's started to treat me worse than usual. I may be overreacting, but I don't feel like I am. It began when he decided he liked me and I liked him back, but I knew he was kind of a player, however, he felt bad about this as well and said he wanted to change. We never really went out-- it never got to that stage because neither of us had the courage to ask each other out, so we remained as good friends. Unfortunately, we do fight a lot. So much so that it has even lead me to break up our friendship, but we always end up back together somehow. Right now... I just feel like he doesn't care. He's all "buddy buddy" with all of his other friends(girls and guys) and treats them perfectly fine, but when it comes to me... He always has some rude comment to say or accuses me of something. The funny thing that I've noticed is that during the day(when his other friends are mainly around), he treats me horribly, but when it's later at night, he's actually nice to me and we can carry on a conversation without him making up some rude comment about me. Of course.. he usually only comes to me about his problems and "the girl who just ditched him or deleted him from facebook". He only comes to me to complain about his issues basically.. and then later says to me that "I complain to much and that the world doesn't only revolve around me." He came up with that response after I simply asked him a question. and I didn't get an answer either. I'm afraid if I try speaking to him-- he'll come up with a lame response and say I'm being absurd. Lately I've been thinking of breaking up the friendship, but I'm kind of afraid too. What do you think? See any reasons as to why he would be treating me this way? It has also come out again that he likes me, however, he has said before "Been there, Done that, Won't work." in regards to a relationship with me. I might add that he's still a player...