I agree with your latter points - as little as 30 years ago you could go to a different country and feel like you are in another world. Even in parts of your own town. But with the internet, google maps, street view, and all the world-wide communication, the unknown and sense of exploration is dead.
I'm old enough to have experienced all the utopian expectations that surrounded the internet, but like most technological advancements it's been a mixed bag. The television seemed a revolution in the 40s, but on the other hand you got the phenomenon of the couch potato.
That said, don't look at being a caveman with rose-colored glasses. Try going out into the wilderness for a week with no supplies, and you will see that it really, really, really, fucking sucks. Going caveman is too far, I would say anything before the 19th century would be a bad idea. I mean if you could have modern medicine maybe it would be ok to go back further.
Maybe you should look at becoming Amish as a last resort?
Otherwise, try to find meaning in our modern world. To paraphrase somebody, even today there are many aisles and facets of life that remain undiscovered.