It is something to worry about if your leaving and the amount of competition is worth enough money.
I agree with the earlier post, it'll be much betterr to consult a lawyer first, just to be on the safe side.
I had a similar case like yours and what i did was talked to our HR and asked her to kindly define what exactly is their definition of "competition". I told her that I respect the non-compete clause but I think I do have the right to make a decent living based on what I can do.
Wishing enough peace of mind and good luck!