Why are dirtbikes so expensive?

Specifically used dirtbikes. I've ridden dirtbikes plenty of times, and currently own 3 road bikes(a Honda Shadow, a Yamaha FZ1, and a Honda 300f), but have been considering buying a dirtbike recently and can't see why they're so pricey. I was wanting to just get a smaller trail bike to have some fun in the woods with, but for the prices I've seen on used dirtbikes I could easily pick up a KLR650 or similar dualsport and it'd be street legal too. Obviously I know the dual sport won't be as good on the trails, especially with it being a larger bike than I was planning to get, but if I'm gonna be spending the same amount I'd rather go for the street legal one and get the extra on-road use out of it too. Would it be that much of a negative having a larger dual sport over a smaller off road only bike?
Added (1). Ian - I'm not looking for motocross/track bikes or trying to get 20' of air, just something to zip through some trails in the woods, so softer suspension set up anyways. If that was the case then by no means would I consider the KLR. So far the biggest downside I can see is in maneuverability through the trails with a larger bike, but still can't see why I'd pay the same or more for the smaller bike just for that.

Read more: Why are dirtbikes so expensive?