I'm shrinking!

Back in high school, I use to wear X-large t-shirts. Okay... so they were a bit baggy. Then I got to college and started wearing Large, which actually fit. Then about 3-5 years ago, I started having to get size Medium. Yesterday, I had to buy Small! I've been the same height and +/- 5 lbs for the last 14 years or so.... I hate how they adjust clothing sizes.