I hear you, brother - I've never found it easy to gain while losing fat, and even maintaining without SOME muscle or strength loss is really tough for me.
The few people I know who have had it "easier" when it comes to losing fat while keeping all their hard-earned gains all had one thing in common - they kept their protein intake REALLY high, especially when cutting calories beyond more than just a slight amount. For example, one guy I know experimented with a vegan cutting diet (he's normally omni, but wanted to see how he'd fare going vegan for 3 months to cut down), he managed to drop nearly 30 lbs. total weight with almost all of it being fat, but he had REALLY high protein intake, roughly 280-320g on just 2200-2500 calories per day he was taking in during that time (he also did some cardio and plenty of weight training). Same goes for the others I've known, they all cut carbs to some degree and kept protein high, and the results spoke for themselves.
Me, I now never expect to make any real progress for gains when trying to cut a bit, now I just focus on trying not to lose anything in the way of muscle or strength and that's as good as it gets. Wish I had some better feedback, but I'm still trying to figure things out after all this time. Every so often I learn a bit more, so about the time I'm 70, I guess I'll finally know how to eat to get that ripped 6-pack midsection!
"A 'hardgainer' is merely someone who hasn't bothered to try enough different training methods to learn what is actually right for their own damned body." - anonymous