It could be almost any of the suggestions already suggested, but my hunch from the information provided (in particular, that the complaints "are mostly about the organization of the teaching or the course being demanding"), is that it may tend to be one of two things:
Communication style. You may organize and explain ideas in a style that more typical men in your culture use more and follow more easily than typical women in your culture do.
The minor/major point mentioned in the comment by @darijgrinberg . For example, if your male students are mostly majoring in related subjects, but your female students are mainly taking your course becauseto meet some requirement but they're not particularly interested in the subject, or if they have less experience with courses in your field, your course may seem less intelligible to them because you may not be explaining things for people less familiar with the assumptions and language of your subject, making the teaching seem more disorganized and harder from their perspective.