I agree It’ll certainly contribute towards that yes, in my experience a decent diet can certainly help things like SHBG and Estrogen especially if said diet keeps you lean, which in turn can support testosterone levels.
To be fair anything which raises dopamine can inhibit prolactin to some degree, that will effect sex drive, also lower prolactin means a tiny bump in LH (luteinizing hormone), which one could hypothesize will then also increase testosterone to some degree by giving a stronger call to the gonads, the Endocrine system is a delicate balancing acting, where one thing will effect another thing etc etc…
Thats why low Testosterone and low dopamine go hand in hand, and can lead to lovely untoward illnesses no one wants.
source, me. (male hormone activist…) plus a quick Google search would confirm the above, but I don’t want to quote several sites without permission, so apologies for not backing up what I’m saying with evidence (which there is plenty of).