Redesign of existing commerce site with content/structure changes - whats the best way to manage this?

Question

So I am currently working on an existing clients craft and craft commerce website and looking to completely redesign and revamp this. Earlier in the yer, this underwent a major upgrade in terms of the craft versions, moving this from Commerce 1 and Craft 2 both onto Craft 3 for commerce and craft core.

Now, has come the time, where the website is required to be updated. This will involve a redesign of the overall site, changes to certain structures and content modelling (including products) and adjustments to fields, structures and channels (creating new, removing old etc).

Given, this is a live commerce website, it is constantly moving. So, what would be the best approach to manage this. I want to ensure that I can design, develop and work locally, but when I am ready, move everything live with no commerce data loss.

I work with git and work on a local, staging, production strategy.

Is there a way to just move the commerce data over to the new build or is it too ingrained and would this have problems with the changes to the product structures/content modelling.

One option, I was considering is to roll out structure changes, new plugin installs and structural changes on the live site, so that the data can be populated, then syncing down the live database, into a separate repo/branch where the new design takes place. However, this could be tricky to manage and has a higher risk with managing on the live site.

The ideal situation is to setup a new repo, design, build and structure the new website and CMS and then just to pull in the latest/lie commerce data to marry the two up for launch.

Really looking for some guidance and best practices on how to handle this.

---

EDITED - 15 December 2021 Some ideas and thoughts on the approach I am considering.

Having given this some thought over the past two days, constantly swirling around my head. I was considering the following approach, but I am not settled or tied to this so welcome any further input.

The Approach

Create a new repo (v5) that is an empty craft install. Add in all the project and configure files, and create a new local database that is initially synced with the live database initially. This v5 repo will be where all my initial front-end template development would happen. This keeps the new template/code/CSS/assets all separate from the live project. This is where I would do initial coding/testing and building in a local development setup.

This gives me a couple of options when coming to the end and launching.

• I could either launch and deploy from this entirely new v5 repo and this then becomes the master after launch.
• I could create a new branch on the current repo (v4) and simply move over the files and templates and restructure this as required. Give then change to the build tools, assets and packages, I suspect option 1 would be preferable.
• If at any stage, I decide it would be easier working from one repo, again I can create a new branch and migrate the files and folders over as required.

The way I see the benefit of the separate repo vs a new branch, is that I can have both installed and open locally and easier see/compare and move files around. Having separate branches proves much more cumbersome for doing this.

So what about the CMS?
Content Modelling, Fields, Structures Plug-ins etc.

So this is where things get tricky. My considerations regarding this are:

• Minimise duplication of work/effort and/or minimise discarded work/effort.
• Ensure nothing breaks the live site.
• Keeping a handle on the changes.

What is going to change:

• New fields
• New structures/channels
• Remove structures/channels
• New Globals
• Updates to product content modelling
• Updates to other static pages/singles content modelling
• New Plugins

So to ensure that the commerce data and all the current information is correct and up to date, the approach I am considering at the moment is that any 'project config' changes would be tested locally and then merged into the live website'. How will I do this?

So, my idea is that I would do any changes that affect the project config on the new V5 repo, once tested and ensured no breakages, I would then copy the project config folder over to the development branch on the v4 repo and then deploy these and test the site. If all is ok then I will merge this branch into the live site and deploy here.

Why this approach?
Because I need to constantly sync with the live database (due the constant movement of data from the commerce aspect), I need to ensure any new content I am adding goes into the live website (even if it is not used in any capacity initially). This ensure I can add new content as I build out the new layouts/structures etc. Without doing this locally and overwriting it all when I do a db sybc. So reduce loss of data, effort and time. This would allow me to then pull this live database back into my new repo (v5) add content, test, build and then once happy - add the final content into the live site, re-sync and move onto the next change.

Naturally, I need to be quite pragmatic in this approach and I am not 100% site how viable it will be. It means moving between two repos frequently and almost doing parts of the site in sprints to avoid confusion or oversights.

Whilst I know it is not typically sensible to touch and develop on the live website, I do feel this is fairly safe in that none of the new stuff will be used in any capacity in the live website. I would need to keep extensive notes and then pre-launch clean up anything that needs to be removed.

At this point I can't see a viable alternative, unless someone points to that fact that only certain database tables can be merged back in at the end to ensure no data loss.

Answer 1

So this is without doubt a thorny problem. I've been thinking about this a lot recently too, and here's just a bit of a brain dump....

I think there are two main ways to go about this:

1. Use Craft's migrations to handle the transform of the old data into the new forms.

2. Create essentially a new site, and then use FeedMe to import the old data in to the new structures.

When updating to Craft 3/Commerce 3, I used approach 1. And this worked pretty well, but the amount of re-structuring of fields/data modelling was relatively minor. What I did here was create a new branch/DB, and began the transition work. Periodically (at least once a week) - I would re-import/re-migrate the C2/C2 Dbs into the new C3/C3 site (which was constantly kept up to date with Craft/plugin versions so as to be bang up to date at launch). It turned out to be really important to do this frequently, as there ended up (even with relatively minor modelling changes) - being quite a lot of migration work. And especially so if you're aiming for a just released Craft/Commerce combo - as the ground often shifts under your feet in surprising ways, I found.

(By far the most complex part was writing migrations for lazy plugin authors who did not, and should have, supplied these - I am talking here of paid plugins, of course...still irritated about this!).

As I begin to contemplate the C4/C4 transition - and for us, this will be pretty much a complete front end re-build - I am thinking I will likely use option 2. I suspect this will end up the easier approach as I anticipate more modelling changes (particularly to our page builder stuff) - than the last one.

The approach will be much the same though - I will create a new branch, DB, and then periodically pull the live content in. Given this may involve a bunch of FeedMe imports, it will be considerably slower I suspect, than simply importing the DB followed by a bunch of migrations. So only time will tell if this really works. If it is too complicated, I might do the site re-build first and save the data modelling changes - which would improve things, but are less essential than addressing our out-of-date front end approach - to the next round of work. We'll see! There may well be enough work in just re-writing the checkout side of things (given the massive changes to Customers/Users and Addresses that are coming) - that this will make sense, rather than trying to do everything all at once.

(Fortunately, this new transition won't, it seems, involve a monumental re-write of all our plugin/module code. Honestly if that had to be done I'd probably contemplate jumping ship altogether - it was a massive, unexpected burden last time, involving months of work. I understand such things are needed but I really would think twice about doing all that work again within a few years...)

Unfortunately, I suspect I may in fact have to use a combination of the two approaches - import the live db, run some migrations AND import some of the old content into new places. Because I am not sure FeedMe will support bringing in all the things I want to bring in...I've yet to really look into that.

No matter what, I tried and will try to not disturb the live site during all this. I personally think it is better to have one clean, possibly painful upgrade session (say, over a weekend) - when finally switching, rather than constant disruption and potential for breakage along the way. (As the actual shop owner...I desire for my income to be safe whilst all this is going on, basically!).