0

I'm looking to use a Craft3 installation as a bridge app to synch a client's contacts from their CRM server to their Email Service Provider, via respective APIs.

I'll be storing their contacts as users in Craft. They have four contact lists with numbers (there may be some overlap) totaling:

  • 12,000 contacts
  • 55,000 contacts
  • 23,000 contacts
  • 22,000 contacts

Is storing these in Craft as users a realistic option? We'll be editing/adding new users in small batches on a daily basis, slicing them off the full list on the source side by last updated date, and then filtering records to push to the destination using the Craft API.

Improve this question
1

1 Answer 1

Reset to default
3

It depends more on how you are going to create and display them. I had to import 20k elements this weekend and I had to be careful to store them step by step.

Depending on your server/hardware - you can't just insert 10k elements in one request. There is a function App::maxPowerCaptain() to disable the timeout but I still faced overflow errors.

And of course you can't display so many users in one single page without using a pagination or Ajax

Besides that - Craft is able to handle such an amount of elements if you can do it in your code as well

Improve this answer
4
  • I'm accustomed to batching them on the way out, as the Email Service Provider has a 1000 contact limit on imports. We'd be working with them entirely via API code—it's just a bridge app between two systems—or using the Control Panel for troubleshooting. Everything else would be aggregate numbers, as it's intended to be an automated system.
    – Mark J. Reeves
    Commented May 2, 2018 at 11:33
  • There's be a big one-time load, but I could do that manually, offline. Subsequent loads would be much, much smaller date range-filtered slices.
    – Mark J. Reeves
    Commented May 2, 2018 at 11:41
  • Populating 1000 entries won't be a problem, so you won't run into critical issues (unless you want to store - let's say - 100 redactor fields in each entry). If you are able to batch your inserts to a smaller amount than 1000 it will be fine as well. Otherwise I would highly suggest you to store your new entries as simple rows/records in a custom table and include them in Crafts ecosystem via cronjob. Creating a new Entry/Element means to create a record in at least 4 different tables - if you have a matrix field even more. So you'll reach 6000 queries while storing 1000 entries easily
    – Robin Schambach
    Commented May 2, 2018 at 11:42
  • Alright... So you'll be fine
    – Robin Schambach
    Commented May 2, 2018 at 11:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.