The classic answer would be some sort of exceptional (childhood) event that brings about some sort of sexual reaction or connotation, even when people are too young to know what sexual even means.
That's more or less the classic textbook answer I would guess.
But I would think it incomplete because people can develop sexual fetishes after childhood and often enough without any specific event being the prime cause.
I know for myself that I picked up a few sexual peculiarities/fetishes simply by reading about them and then imagining myself experiencing them. One would be spanking, medfet or various aspects of transgendering. I cannot for the life of me recall any childhood instance where such things happened to me in any way. And yet through reading or viewing imagery, these things struck a chord in me somehow and ended up developing into a sexual like.
I know that is just a partial answer because how does a person by just reading about a certain type of unusual sexual activity, develop a fetish without necessarily having experienced anything linked to it, except in the mind?
I guess the classic answer to that is sublimation of memories and forgetting. But that often sounds like a cop-out or far too easy an explanation.
Anyway, this is something I've pondered deeply on, wondering why in the world I have been burdened/blessed with such less than usual sexual attractions. Why me and not my friends?
I hope to hear other members take.