Some cats can be very sensitive to the energy of the Fae, so while it's possible that the ones turning up are not well intentioned, it could also just be that the cat is responding to the higher level of energy carried by the Fae. There's no one cure-all for this since each cat and each situation is different, but it couldn't hurt to make sure the cat has a special place where s/he feels safe and on his/her own territory. This could be anything from a kitty condo, to a box with some special toys and blankets in it, to your bed. The important thing about it is that it has to have scents that are familiar and comforting to your cat – either his/her own scent, or there is a special product you can buy at pet stores called 'Feliway' which you can spray on areas where you want your cat to feel comfortable and at home. This may help your cat feel calmer when the Fae come calling, as well as when any other stressful event happens.
Another thing you can do is reassure your cat in a calm voice that everything is ok (if you're pretty sure that it really is). Cats pick up on our moods and thoughts very easily, and so if you are calm yourself, it may help him/her to also calm down.
It's possible your cat is feeling uneasy and territorial about the 'invading' fae. Then again, maybe s/he is feeling playful? You didn't really post any details, but some cats like to pounce and play with Fae, especially the smaller ones who fly, so if your cat is going berzerk maybe that's what's going on?