#Screaming frog log analyzer update#

Some platforms that can dynamically update sitemaps don’t give you much control over what’s included in it, they just dump everything in there. The next key motivator for using the Screaming Frog SEO Spider to auto-generate your XML sitemaps is to customise them. But what if the site uses a proprietary CMS without a built-in system or a huge publicly-created plugin library? The solution I’m about to explain is CMS-agnostic, meaning you can set it up for any site, regardless of what it’s built on.

Platforms like WordPress and Shopify offer great solutions to keeping your XML sitemap fresh. The first reason is because you use an uncommon CMS.

Establishing a central location for storing SF output files.

What are the steps for automating your XML sitemaps?

A dedicated machine (this isn’t necessary but it’ll make your life a lot easier).

An IT/tech team that can implement reverse proxies.

The Paid version of the Screaming Frog SEO Spider.

You have limited access to servers/devs.

You want more control over the contents of your XML sitemaps.

Use the Screaming Frog SEO Spider to automate your XML sitemaps if: I’m sure there are more I haven’t come across yet, and I’d love to hear from you on more use-cases. My motivators were three-fold – sites on a CMS without dynamically generated XML sitemaps, no access to the server, and efficiency. One of the quickest wins was automating XML sitemaps doing so would give us more control and efficiency. The more processes we could take into our own hands, the better. This article is a guest contribution from Logan Ray of Beacon.Īround the same time, I was exploring ways for our team to gain more autonomy. Then it was on to find some more advanced uses. I familiarized myself with the feature by setting up a standard weekly crawl for each of my clients. I knew there were some great use-cases and couldn’t wait to find them. When Screaming Frog first rolled out the scheduled crawl feature, I was thrilled.

Screaming frog log analyzer