We have a site with (as of right now) 77,613 elements. 9,954 of which are entries.
We found that we had to be careful about using Craft's {% cache %}
functionality because clearing the element cache criteria took a long time. Instead we use a Varnish cache and it all works fine.
There are times when it's not as fast as it should be, particular templates that pull in a lot of data from different places struggle if there's a bit of traffic to it at any one time - but Varnish resolves that. There's probably lots in those templates that we could be doing to make them more efficient too, it's just not been a high priority because it works okay.]
Just make sure your hosting is reasonably beefy. Depending on your traffic mysqld
might get called upon a lot and that can be a resource hog. I recommend using something like DigitalOcean where you can simply increase the VPS specs if things are struggling and tweaking Craft settings or templates isn't helping.
And with Craft 3's performance boosts coming this year, I think you'll be fine.
Hope that helps!