The number obviously varies across individuals, there's no correct answer. That said, I'd put it at around 4-5 before there are some fairly noticeable complications.

If you consider the pregnancy process, eventually what happens is the fertilized zygote attaches to a newly formed and reinforced endometrium, which is different from what normal uteral lining is like in the first place. Then as the embryo develops, placentation occurs in the form of embryonic cells invading into this new tissue (called the decidua), eventually causing drastic morphological and physiological changes to this tissue. This is what forms the placenta. After birth, the remaining tissue in the uterus "heals" itself (namely, vasculature has to go back from venous-like structures to arterial structures). After a few times this process becomes less efficent and the efficacy fails too. That's one of the complications.

The other, more obvious one is the fact that a full term baby is huge, pushing many many things out of the way and practically causing them to take on another shape for a few months. This movement tends to stretch and pull at things which hold other things together. After a few times of this, plus age-related degeneration, prolapse happens.