Unlike most cancers, adrenocortical carcinomas (ACCs) are more frequent in women than in men, but the underlying mechanisms of this sexual dimorphism remain elusive. Here, we show that inactivation of Znrf3 in the mouse adrenal cortex, recapitulating the most frequent alteration in ACC patients, is associated with sexually dimorphic tumor progression. Although female knockouts develop metastatic carcinomas at 18 months, adrenal hyperplasia regresses in male knockouts. This male-specific phenotype is associated with androgen-dependent induction of senescence, recruitment, and differentiation of highly phagocytic macrophages that clear out senescent cells. In contrast, in females, macrophage recruitment is delayed and dampened, which allows for aggressive tumor progression. Consistently, analysis of TCGA-ACC data shows that phagocytic macrophages are more prominent in men and are associated with better prognosis. Together, these data show that phagocytic macrophages are key players in the sexual dimorphism of ACC that could be previously unidentified allies in the fight against this devastating cancer.