I have 80,000+ files on S3 that need to be added to Craft. The Update Asset Indexes works, but takes over 24 hours and cannot be interrupted. Does anybody have a way to programmatically index assets like this in batches?
1
-
1Seems to be this sort of thing is best left to background Tasks. Would love to find the time to write a plugin that did this, as we have a similar situation.– Chad WFeb 10, 2015 at 19:29
Add a comment
|
2 Answers
Your best bet for a quick hacky solution would be to interrupt the indexing process, take a look at craft_assetindexdata table and then write a small plugin that calls craft()->assetindexing->processIndexForSource().
You would get the sessionId from the craft_assetindexdata table and you'd be able to deduce how far the indexing has gotten by looking at the same table. All the entries that have recordId field set to a value have been processed already.
This way you can batch it up however you want.
After seeing this and hitting the same problem ourselves, we decided to create a plugin to index the assets from S3 in batches. We have not yet moved to using tasks in the plugin but it is on our roadmap. We know that Craft 3.x will have improvements to asset indexing but until then, please feel free to give our plugin a go if you are still experiencing issues with this. Our plugin can be downloaded from: https://github.com/a-digital/assetindexer
