I'm aware of this thread on Feed Me slow performance, but I'm raising it as a separate thread in case there is anything particular about my circumstance that is pertinent.
I'm running Craft 3.4.18 and trying to import a large number of users (about 1,500) into a specific user group using Feed Me:
- I'm using
.csvsources on the local file system above web root
- I'm importing:
- Username (as email address string)
- First name
- Last name
- Shared password
- User group string
- I'm batching imports in 500s but because things are so slow will now try smaller batches
- I have read and attended to the performance aspects of the Feed Me troubleshooting docs
- There is plenty of memory and execution time allocation to Craft and the instance is on a dedicated Linux server with 8Gb of RAM
Imports are working without error, but paaaaaainfully slowly -- we're talking hours -- to the point of the import process being effectively useless*.
I have looked into Andrew Welch's #1 solution to running the queue as a background process, but have bumped into installation issues with the Async Queue plugin.
Any suggestions much appreciated.
I'd be happy in principle to do SQL inserts directly into the
craft_usergroups_users tables but I infer that's not an option since each hash of the shared password is unique and there is a unique UID to generate for each row too.
Update on #3 above:
I'm currently running a batch of 100 instead of 500 and that makes no difference to speed whatsoever.
Next step is perhaps to try a JSON feed instead of CSV?
*'Effectively useless' in the sense that although I have time to run extended imports right now, if it's going to generally take this long it's a non-starter for time-pressured imports.
Second update on #3 above:
An import batch of 100 I started over 2 hours ago has crept along incrementally, with me periodically reviewing its status during that time and seeing barely any change, and then…
I have just revisited it and it suddenly went from 64% complete and unchanging to 100%. Now onto the next batch to see what happens.