Should I break up with my boyfriend?!

I was truly in love with my boyfriend for a long time before we got together, but then maybe 6 months after we got together things went downhill from there. I found out he wasn't the same guy I fell in love with, and there were a lot of things that bothered me... Mainly I found out he has a temper. When he gets mad he usually becomes very condescending and harsh. This past weekend was the breaking point though. I basically went and told him when I was angry that I'm breaking up with him because he doesn't care about me etc etc. Then he grabbed my arm really hard squeezing it and wouldn't let go while he was talking to me. So I slapped him to make him stop... Which I shouldn't have done.. After that, he go so mad he was shaking and started yelling and cussing at me and I just walked away. He followed me and stared me down and kept telling me to hit him again(which I didn't). Later on he told me he can't get mad like that because he could hurt me, and things were going through his mind that he wasn't happy with at all.

The fact that it has reached this point now, I don't feel like we should be together. But I also can't bring myself to leave him... I know I would miss him. When he doesn't have any anger or anything it's really good between us at times. I feel close with his family and friends and I've never had that before in past relationships. I wish I could be completely sure of either staying or breaking up. But it's just in between.