I'm not sure what circles you're mixing in there but this is absolutely the opposite of my experience. Try getting along to an APS event would be my best advice for a start. https://www.psychedelicsociety.org.au/
And "Drug Bros"? There are quite a few sisters about too, so again, maybe you've just been at the wrong events.
EGA - another great org and more specifically plant orientated: https://www.entheogenesis.org/
I'm not saying the type of person you describe does not exist, but i'd say "very rarely" instead of "very often".