The Genesis Framework comes with built-in SEO, which is the ability to manage settings important to search engines for your WordPress powered site. Let’s be clear about something: Genesis SEO is rock-solid at managing those settings and putting them in the HTML markup of your pages, where search engines grab and make use of them. But it has a big shortcoming, and after hearing about a little test we conducted recently, you’ll probably agree.
We wanted to see how quickly a site that’s newly visible to search engines would attract visits from them. Those visits would initially be to index our site’s content, which is the first step to getting search engine traffic.
With some quick copy and paste, we had quickly published a few posts on our new site. As the first step in our test, we changed our search engine visibility to let the spiders in. Nearby is a screenshot of the search engine visibility setting from Settings->Reading on your WordPress dashboard. We sat back and waited for search engines to index our content and visitors who found our fascinating content on search engines to show up in droves.
We waited. And then we waited some more. The search engines and the traffic they bring still hadn’t shown up, and this wait was looking like it would be measured in days rather than hours.
Therein lies the critical shortcoming in Genesis SEO. It doesn’t generate an XML sitemap integrated with its SEO settings or provide an ability to ping (tech-speak for notify) search engines to let them know our new site exists or that we’ve published new content. Without those things, we’re at the mercy of search engines deciding to stop by. Eventually, search engines will visit and discover our content, but that might take some time. For a new site that wants traffic right away, those days can seem like an eternity.
The lag to index a new site also applies to an already-index site when it publishes new content. Because XML sitemaps generators are normally configured to ping search engines when content is published or updated, search engines arrive quickly to index new content. Without the ping request, content discovery by search engines takes longer, and the time difference is generally longer for newer sites or sites that publish less frequently. For example, Google has sophisticated algorithms to determine how frequently or deeply to crawl a site, but those settings are beyond the control of a site owner. For sites without an XML sitemap and an established track record of regular publishing, Google’s visit frequency will be lower, and that means the chances of discovering new content are lower.
For businesses that generate revenue from search engine visitors, days when your site or its content is not in a search index equal days when you don’t get revenue from search visitors. Those lost days can never be recouped.
After several days of waiting, the search engines still hadn’t dropped by. So we activated Yoast’s WordPress SEO plugin which includes an integrated XML sitemap, configured it, and in under 10 minutes Google’s crawler had paid a visit to our site. That visit would be repeated many times in the days that followed. What changed? Our XML sitemap pointing to fresh-to-the-search-engine content (even if that content was silly) plus the invitation put forward by WordPress SEO. Without that combination, our small amount of content and no incoming links would likely have left us waiting for a while.
Some will say that not including an XML sitemap as part of SEO isn’t a shortcoming and certainly not a critical one. After all, there are plenty of good XML sitemap plugins in the WordPress plugin repository, such as Google XML Sitemaps. That viewpoint ignores the problem of separating the XML sitemap from other SEO settings. It also assumes that new site owners are aware of their need for an XML sitemap. If they assume that the Genesis SEO settings completely satisfy recognized SEO needs, they’d be starting off based on an incorrect assumption.
When you activate a plugin such as Google XML Sitemaps in a Genesis child theme, your sitemap plugin doesn’t talk to Genesis SEO, the source of your SEO settings. That means that posts and pages that you opt to noindex appear in your sitemap. In effect, with your sitemap you’re inviting search engines to a noindex’ed post or page, only to have them arrive and discover the content has a noindex tag.
While the world won’t end if search engine crawlers waste a few trips to your site, that’s not an ideal approach. If you have a lot of content that’s tagged noindex and your server is having problems keeping up with normal traffic, this unnecessary traffic makes your site slower. (The slower speed from extra spider traffic might only be an issue if your site is hosted elsewhere; for sites hosted on our network, there wouldn’t be an impact.) That traffic is a recurring event because the search engines will keep repeating this unproductive task over and over. Since search engines use site speed in determining where your site will rank, the slowness from unnecessary visits has a long-lasting impact on your site’s organic search traffic. Moreover, for newer sites where search engines such as Google might drop by less frequently and attempt to index fewer pages, you want to maximize the impact of those visits by having them visit content you want indexed, not wasting time on content that you don’t.
Still think not having an XML sitemap that’s integrated with your SEO settings isn’t a problem? Respected SEO authority Joost de Valk aka Yoast shares the view that XML sitemaps are part of SEO and shouldn’t be separated. Genesis SEO doesn’t generate an XML sitemap. According to Yoast:
XML Sitemaps are an essential part of current day SEO, and can thus not be excluded from a complete WordPress SEO plugin. While there are other WordPress XML Sitemap Generators out there, they don’t talk to your SEO plugin. Meaning that if you noindex a page, preventing it from showing up in Google, it might still be in your sitemap.
He says they’re not just a part, but an “essential part….” When you don’t include something that’s essential or make it clear that what you’re offering is an incomplete solution, you end up with a critical shortcoming. If you fix that flaw by activating a sitemap plugin, you end up with a less-than-ideal solution. That’s why Yoast included integrated XML sitemap functionality in his WordPress SEO plugin. Yoast is a fan of Genesis and runs his own site with the framework. But he says that Genesis SEO doesn’t beat his own WordPress SEO plugin. Even factoring in a dollop of possible bias in Yoast’s assessment, he’s right. There’s room for Genesis to improve here.
While we don’t like the fact that Genesis doesn’t have an integrated XML sitemap capability, we give its developers a lot of credit for making it easy to switch to and from different SEO plugins using the SEO Data Trasporter plugin. That makes it easy for users to switch to Yoast’s WordPress SEO or to switch back to Genesis SEO if they’re unhappy. And there are good reasons to keep XML sitemap functionality separate from the main framework. After all, many framework users may have already activated an XML sitemap plugin and wouldn’t benefit from duplicate functionality.
But the story doesn’t have to end here, because there are steps that the developers of Genesis can take to address this shortcoming. The first step is easy and involves adding educational information to the Genesis SEO settings page that describes what an XML sitemap does, briefly explains why it’s important, and encourages use of a third-party plugin to create one. Such information puts SEO newcomers on notice that while Genesis SEO is a good solution, it’s not a complete one.
The second step involves more work: creating a helper plugin similar to SEO Data Transporter to adjust how popular sitemap plugins create the sitemap based on Genesis SEO settings. Google XML Sitemaps might be a good starting point; with nearly 12 million downloads as of late 2013, it has a large installed base. Some may argue that a helper plugin won’t satisfy every user because it can’t possibly cover every XML sitemap plugin. But it will add considerable value for many, just as the SEO Data Transporter does. Compared to building a Genesis-specific XML sitemap plugin from scratch, we think the helper plugin approach is the easier route to go here.
A healthy dose of encouragement to new users to use an XML sitemap plus a helper plugin to integrate Genesis SEO settings with a few popular XML sitemap plugins would address the lack of an integrated XML sitemap, a big shortcoming in Genesis SEO.