Every human has the genes to create both a man and a woman, sex hormones are what tells your cells what sex they are, by telling them what genes to actually transcribe and express. So on a very real and material and biologically important level, if you replace your sex hormones, you also alter your sex (to a degree). A lot of trannies get fucked by morphology though, as sex hormones differentiate primary sexual structures before birth, and secondary sex characteristics during puberty. By mid 20s, they don't have the growth hormones necessary for sex hormones to really make major morphological changes.
In short, from an actual biological standpoint, calling trannies males is wrong, they're more like varying degrees of intersex depending on how early they started.