I Don't Need a Pair, I'd Just Like to Buy One Jean Please

When we look at the terms we use for closed crotch garments worn below the waist, we almost always use the plural -- pants, trousers, jeans, dungarees, Bermuda shorts, cut-offs, capris, leggings, tights, a pair of underwear, panties, bloomers, jockey shorts, boxer shorts, briefs,...  Yet, when we examine the terms for clothing worn on the torso, the terms are always in the singular -- shirt, blouse, sweater, vest, jacket, coat, tank top, camisole, even brassiere (the one where it would make the most sense to use the plural).  Open bottomed clothes that cover lower half -- skirt, kilt, dress, robe -- are singular.  We generally have two arms, just like two legs, so why the syntactical difference? The only counterexample I can think of is the two piece bathing suit where the lower half is the bikini bottom (singular) or is it generally bottoms?  For men, we use the singular bathing suit, but also the plural swim trunks.  Explanations or further examples and counterexamples?