I think your sizing is wrong. It took me three purchases of the shorts to get it right. I have a 36 inch waist but it took me an extra large to get a size that fitted, never before have I been an extra large.
All is not lost, I should have known better when I ordered the medium but a local teenager was pleased with them.
I figured a large would fit me fine, wrong again, I passed them on to "she who must be obeyed" and they fit her comfortably.
So it took three attempts to get it right so I am satisfied.