I’m in the other camp. Used to have a really bad fear of heights and have always been pretty awful at flying. Nowadays I’m more or less okay, but on average it’s still one of the more difficult things for me to do in dreams. The only things I can think of that are harder are any kind of “violent” expressions of dream control. Energy blasts or forcing my will on dream characters, things like that.
My problem was kind of like that. It was getting off of the ground that gave me trouble. If I was in a position where I could start off in the air (jumping off of something, for example) I could at least glide, but if I started on the ground the best I could do was a kind of semi-weightless bounding jump. When I could get off of the ground, I’d sometimes reach some kind of invisible altitude limit, where I would lose speed and start getting lower after reaching a certain height - though that wasn’t as common.
I slowly got over my fear as I got older, and got better at flying as time went on, but I can’t be sure if those two things were connected. I’d been getting creative with takeoff in dreams for a couple years - using gusts of air to push me upward (thankfully I grew up in the middle of the desert. Being shoved around by wind was a very familiar feeling), shapeshifting into a bird (changing shapes in dreams has always been effortless for me) or walking/sliding up imaginary stairs or ramps. It might have just been a matter of practice, or maybe my practice in dreams helped my fear of heights. Wasn’t really paying attention to it as it happened, so I can’t say with confidence.