2

I have a folder with 10GB of files that I need to sync with Craft. It has 12 subfolders, one for each year:

/1999
  /01
  /02
  /03
  /...

It's possible to run Update Asset Indexes on the top-level folder but this process takes several hours and can't be interrupted. If things get interrupted, for whatever reason, some of the sub-folders are successfully imported but the other subfolders aren't, and there is no simple way to pick things up where they left off without restarting the full process. Ouch.

Is it possible to sync a specific asset subfolder, or list of subfolders? How would I approach this programmatically?

It seems I may want to call the AssetIndexTool::performAction method but only pass in the subfolder IDs or something like that. I haven't interfaced with the Tool classes at all. Is this a reasonable direction to consider or am I not considering another potential direction?

3

There is no good solution in Craft 2.x, but you probably can put together a plugin or a console app that does what you want it to do. Here's all the background info you, hopefully, need.

The whole indexing process is controlled by javascript as well but let's just pretend you've instantiated the AssetIndexTool class and are operating on that via performAction()

Craft Asset indexer operates strictly on the data in craft_assetindexdata table. When you start an indexing task, the very first thing it does is query the Asset Sources for all the actual files that are there, stores a list of them in the said table and from that point on uses that table to reference what files have been indexed and which not. As it does it, Craft also creates all the folders already (because, apparently, a long time ago it seemed like a good idea to me).

After that, the tool returns an array for each of the steps it needs to take, where one step is generated per file and also an additional "overview" step at the end. Each array entry basically contains parameters that you should throw back at the Index Tool.

What you're interested in each of those "step" arrays (if you want to understand how the whole process works) is sessionId variable there to track the indexing session in the indexing table, the sourceId variable so you know which source you're operating on and the offset variable that tells the Index Tool which file in the indexing table to look up and actually index.

You should definitely also follow through with the last overview step. Never mind that it presents the user with a HTML dialog for all the files that are in the DB but are not present in the source, but, more importantly, it cleans up the asset indexing table and removes the obsolete data in there.

The great news is that it will be significantly less convoluted and way more accessible programmatically in Craft 3.

|improve this answer|||||
  • Thanks Andris. Glad to hear this will be improving in the future. – Ben Parizek Oct 2 '16 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.