should i or should i not?

I have a little dilemma, I am dealing with mixed emotions and am not sure what i should do.  I am debating on letting the little sperm donor of my daughter come around when he feels like it or just keep telling him no.  He hasn't been around too much since she was born and he only wants to come around everyonce in awhile so it doesnt seem right to me that i should treat him like her father and give him rights to see her.  He has already technically signed over his rights in february and still wants to come see her, what a joke. No they aren't from any special lawyer but they are notarized making them legal so for all i care he really shouldnt be able to come see my daughter.  I have been doing good without him for eight months and she is living a happy life with having her adopted daddy around, my boyfriend is pretty much like her dad. She won't know any difference any time soon.  Well this subject is making me so confused that sometimes thinking about it just makes me sick to my stomach, that and i'm not sure if I am going over my head about my boyfriend. Things are going good in my head but not hundred percent if it's just in my head or if it really is going good.  I am afraid that i'm going to get too comfortable with him and than all of a sudden things will get ruined, why do i feel this way if things are going great between us? Am i just being paranoid, am i just being insecure? I am not sure but i hope am not being annoying to him lol.  Well i'm off to do whatever just had to get that off my chest =)