To get a message from one place to another incurs a cost per mile.
More miles of empire to send messages to means more money spent on just communicating with the empire.
Since Augusts Rome was in debt till Trajan conquered Dacia then it was in debt again after he died.
Also people see Eastern Europe with a modern bias, after we have developed good fertilizer, hardy crops, the potato and the iron plow. Before the 8th century even with the warmer climate of the 1st century AD you would still need to transport food from Egypt, Carthage and Moesia to feed all of your new provinces since they won't feed themselves. And you would hardly even have any resources to get from there. Since there aren't many iron or gold deposits that were known to the locals.
It would make the Imperial over extension worse.
The only place the Empire should have conquered is the whole of the British Isles so they would no longer waste resources manning a wall in the middle of nowhere to stop like 3 tribes from pillaging the Nowhere colony.