When does abstraction become over abstraction?

If there is little evidence that it is the correct abstraction.

Sometimes two similar things are more similar now than they logically will be in the future. Creating an abstraction from the pair of similar pieces of code can get you headed down the wrong path.

The moment you use a framework and create a "Base" anything. For instance, with Spring why would you need a BaseController...it comes out of the box! If it's so you can implement cross cutting concerns, like logging and analytics...try AOP instead.

Almost immediately.
You should only introduce abstractions if you are absolutely sure they are the right ones. Otherwise you are better off writing the code twice.
Its a good idea to discuss abstractions with your teammates before Introducing them to the codebase.

Uncle Bob has put this into a nice perspective, even if in a different context: when two things evolve at a different pace.

If you have two things that look quite similar, but might change at different speed into different directions, it will be harder to take the step back in future and see, that these two need to be separated again. It is more likely that you will add complexity to handle this instead.

This is rather about estimating and evaluating what might happen in the future and to be prepared for that, as in most architectural decisions, where there is less right and wrong and more compromise between goals that can't be achieved at the same time (easy understanding, simplicity, speed, ability to change, reuse etc.)