Over the last two years, I’ve worked with one of the world’s largest automotive manufacturers. Early in that relationship, they presented me with a challenge: to automate a process that moved thousands of web pages to the Sitefinity Content Management System (CMS). Understandably, they didn’t want to build and populate each page manually. Instead, I built a custom program to reach their goal.

It’s a common challenge for corporate enterprise.

Many Marketing and IT teams want to use modern web practices, but before a website is changed, they have to navigate the corporate structure. They push for support through early wins that feel manageable and show a tangible benefit. An easy first step is a “lift and shift” to migrate website content into a new CMS. Then, you can build out features in the new system piece-by-piece. It’s a long-term approach, but you can point to ROI throughout the process and keep momentum.

While many companies take this approach, the amount of work varies greatly. Businesses with a dozen web pages may choose to migrate the content themselves, but when it comes to enterprise content management, with hundreds or thousands of pages, that can be a big lift. It’s not really surprising that our client asked me to automate their website content migration.

Why move to a CMS at all?

In order to make a complicated website useful, structure matters. And it certainly helps if the people who create the content can control it.

When our client changed one of their products. The Marketing team would update a spec sheet and send it to an outside technical firm to change the website. Our client couldn’t make changes to the website themselves without knowing how to code. So the technical firm would perform a search and replace, to find where the product was listed on each page. That assumes every page was named correctly and consistently. I think most people can agree, that’s not always the case. You may have a website created years ago with different goals, a different team, and different terminologies. So pages were missed, which meant that the next time a change happened, the problem just compounded. The client ended up with a dated website that didn’t function well.

The client’s Marketing Manager saw a well-structured CMS as the solution. One that non-technical people could edit. One where content was repurposed not duplicated. With that in mind, our client chose the Sitefinity CMS.

How did we automate the “lift and shift” into Sitefinity?

This project made sense in theory, but as I looked for resources to help complete the task, I found little help. There are resources on the process of migrating website content, which helped conceptualize the process, but I needed to build a system to create a page and fill it with custom widgets and content. I saw little information out there (it’s the main reason I thought this article would be a good idea).

So I started at the beginning. I broke everything down into small steps to reach to my goal.

1. I created a system to crawl each page of the old website. I needed to know exactly how much we were migrating. My system found thousands of pages, but most pages followed one of three templates. Of course there were some exceptions, but it was much easier to record the one-off pages, create them manually, and then write rules on how to read them.

2. I built a web crawler that created a page in Sitefinity based on the content of the original page. It identified the page template, built the template, and imported the content accordingly. We first built the 3 main templates we found in Sitefinity. Then I used content blocks within the templates to insert html from the old page into its new location within the template.

3. We needed to keep the current structure of the site so that old URLs would still go to the correct page. If there were previous pages linked in social media, we wanted to avoid 404 [Page Not Found] issues. Other than creating some vanity redirect pages in the system, I created pages as they were scrapped with the program.

4. Of course, there were images and documents to move from the old site into Sitefinity. To manage that shift, I used the crawler to detect images and documents in the HTML. If it found any, a process would begin where the file was downloaded from the original site and uploaded to Sitefinity. The crawler then modified the new HTML using links from Sitefinity’s file storage system and save that modified data to the new page.

5. Anticipating future iterations of the site, I laid the groundwork for our client to take advantage of the product catalog features within Sitefinity. I leveraged the old structure to build out products within the Ecommerce module of the site. Each product would fall under a product category, so the URL and product properties should reflect that order. It would use something like companyname.com/product-family/product-type/produce-name.

6. Then I was ready to test it all. I had my system built to take a page from the old website, analyze the page template it used, build that page in Sitefinity, and then put the content in the new Sitefinity page. Once the system was working, it took about 24 hours.

Life after the Lift and Shift

Today, the Marketing team manages the content on their website. They make changes by going to just one page, and then any page with related content is automatically updated. The customers have a better experience too. The Sitefinity navigation allows you to easily move between products.

The most important thing

Piece by piece, I continue to help our client continue to improve their website. They could choose to use any firm with Sitefinity knowledge, but they choose to keep working with us. Now we have a relationship. Although this content migration was just a piece of their new website, it was a vital first step. A step that set them up for success, more budget, and more time to start making the other changes they want.

Technologies I used along the way:

• JavaScript to create the web crawler leveraging jQuery selectors to grab specific sections of a webpage (looking back python would’ve been a better choice…)
• C# - Create an API to send our front end data through
• Sitefinity API to integrate our data within the Sitefinity CMS