Yes, we were incredibly adaptable little critters once upon a time. I'm such a wuss in my old age!
That's how time works. At a certain point, getting older means your body gets tired and you're not as strong as before. Even if we could change that though, the world is going to shit anyway, so it doesn't matter how old we are, the weather is all crazy.
I wish winters were getting colder - here in Galveston, we hardly have any winter at all, but dang, the summers are blazing humid HOT. And, yeah, summers have gotten hotter since being older.