Quick Tips for Getting your Pages Found & Indexed by Search Engines

Those of you who know me will be aware of how much of a fan I am of content – now, more than ever, content really is king and, let’s face it: what’s not to love? High converting sales content for web pages; useful, valuable and informational content for blog marketing; chatty content for social media marketing; formal content for article marketing; content for link building; structured and internal linked content for Content Clusters. I love it all! Content, content, content!

And those who know me are also aware of how much we invest in each and every piece of content here at LeadGeneratorsDigital.  All our content must be comprehensively researched (no fluff from us!), be accompanied by high quality, interesting images and must be long form. Very long form. The good old days of banging out 400-600 words per page are long gone. In today’s world of Content Marketing and SEO success, pages usually have to be many thousands of (high quality) words each – at least!

So, with all that in mind, you’ll understand how frustrating it can be to invest so much time, energy and passion into a piece of content, only to have it ignored by Google. It makes me want to … well perhaps I shouldn’t say how it really makes me feel or we might find the LeadGeneratorsDigital website getting ranked for something a little off-brand! 

Anyway, all that valuable and painstakingly researched content that we create each and every month for our clients has to achieve two important marketing objectives.  

• The first is to get well ranked so that it can generate more traffic.  
• And the second is to convert that traffic into the desired actions (enquiries, phone calls, sales etc).

Put in very simple terms, it’s critical that Google is able to see those content pages (we call that indexing) in order to judge them and to then rank them. 

So, how do you know if that’s happening? Glad you asked.

How to Check if your Pages are Indexed

There are many ways of checking if one of your edited or newly created pages has been seen or indexed by Google.  

One of the easiest, non technical ways, which I tend to use as a first check, is to type the full URL of that page into Google and, if it appears, then look at the title and description in the snippet (aka the meta title and meta description, if you’d like to get ever-so-slightly technical). 

If a page we’ve created or edited has been indexed, the meta title and meta description that we created for that page usually appears in the SERPS (search engine results page). 

However, be aware that this is not entirely foolproof – I mean, what is when it comes to Google? I’ve seen instances when Google has only used the title or just the description or, in some cases, neither. But if I see at least one of them (title or description) that is ours, then I am happy and consider it ‘job done’, and I can move on.

If not though, it’s time to use another way to skin the cat and I’ll check the page in the site’s Google Search Console account using Google’s URL Inspection Tool.

Our Go-To Options for Getting Client Pages Indexed In Google

So far, so good, right? It’s pretty straightforward stuff. But if, however, your pages are not being indexed, it doesn’t matter how much lovely juicy content you’ve created, it will prevent them climbing the rankings.  

The bad news is it’s basically a deal-breaker, the good news is it can be fixed fairly easily. Phew, I hear you say! (Or perhaps you were just saying, OK, get on to the good stuff Frank!)  There are multiple methods for getting those pages indexed, but I’m not going to bombard you with an exhaustive list in this article – I’m going to tell you about the handful that I use on a regular basis. Because they work.

Technical Blocks

The first thing I tend to look at, especially if we didn’t actually develop the site ourselves, are the robot.txt files. Robot texts give the search engine various instructions, and one of them is what is called “noindex”. This, in essence, tells a search engine not to index the page, so that it (and its contents) will not show up in SERPS.

Strange? Well, not really. There are often very good reasons for wanting to keep the contents of a page from being indexed, but I won’t go into that now because it’s a whole different issue. For now, let’s just keep things easy and focus on finding out if a page is indeed blocked or not.  

A simple “View Source” of the page and a “Ctrl F” for the word “noindex” will often reveal if the page is being blocked to search engines or not. It really is that easy.

There are other technical issues that can prevent or slow down the indexing of pages by search engines. Some examples of these include server issues or crawl budget issues, both of which can slow the indexing process. (I did promise not to get too technical, but it helps to know these issues exist.)

Request Indexing

If you can’t find a specific issue that’s preventing your page from being indexed (such as your robot texts), then Google Search Console is the next step. There’s a wealth of under-accessed information available if you know where to look, and Google will often tell you (either in its reports or in its message centre) if it has a problem with something on your site.

If you’ve done a URL Inspection as above and can’t see any reason, I suggest simply requesting that Google index the page again. Yes, again, it’s that easy. Just use the REQUEST INDEXING button, as shown in the graphic below. Of course, this doesn’t guarantee that Google will go ahead and index it, and it also doesn’t indicate how long it will take if it does (Google loves to keep us guessing, as always), but it’s definitely worth adding this as one of the strings to your bow.

Submit a Sitemap to Google and Bing

The Request Indexing option above is best used if you only need to index a few pages of your site. Maybe you’ve just created a new page or perhaps you’ve edited an existing page. 

However, if you have multiple pages (or the entire site) to index, I find the best thing to do is to use an XML Site Map, then submit it to Google Search Console and Bing Webmaster Tools.  

Signposting

While Google Search Console and Bing Webmaster Tools are the technical starting points for getting your whole site indexed, there are some basic and simple SEO tactics that need to be implemented as well. Stay with me, though, because it’s not as scary as it might sound. 

Firstly, it helps to understand that search bots are in the business of following links (both internal and external), which is how they discover, find and index pages and content across the internet. Never thought you’d want to be friends with a bot, did you? But you do, and with this in mind, it’s really important to give the bots the opportunity to follow a clear link path toward that page that needs indexing.  

Simply put, I use signposting to achieve this by linking to that page from a number of social media posts – and also from other pages that are already properly indexed and regarded by Google. Expanding the friendship network, if you like. 

At LeadGenerators Digital, whenever we create new content (so, basically every day), or edit existing content, we always follow up by providing strong inbound, external link support to help the search bots find that page and index it for us. We’ve made it a standard part of our practice for the sole reason that we know it works. 

Optimise Your Interlinking Scheme

OK, let’s take this one step further. I’ve explained that getting links from external web pages is important (as long as the pages providing the links are relevant to your page), but don’t forget how vital internal linking is, as well.  

When we create content for our clients, we tend to use the Content Silo or Content Cluster information architecture approach, which helps us organise the content around certain topics or phrases. We’re very careful to ensure that all pages in these silos or clusters interlink with each other, because this helps our bot friends understand the relevance of pages to each other. In my experience, Google really loves it when we organise and interlink content in this way. 

Train the Spider by Updating Content Regularly

Stop the presses, search bots can be trained? Yes, I thought I’d keep this one up my sleeve to impress you! Here’s how it works. Every time a bot visits your site, they look to see if there are any changes from the last time they called in. If they find there’s been no or few changes, they’ll lose interest and downgrade your site and visit you less (very short attention spans, our bot friends). However, if every time they visit they see more changes, they’ll then upgrade you and come by more often.  

So, the idea is to train your bot to keep on popping in by regularly publishing new content on your site. There are plenty of quick and easy ways of doing this: change your pricing, change your products, increase the texts on pages of your site and, importantly, create a few new blog posts every month. The more new content you have, the more often the bots will visit and index that new content. If you think about it in those terms, it’s not hard to understand the concept of consistently creating fresh new content to keep the bots interested in visiting your site. You wouldn’t serve guests to your house stale biscuits and cake and then expect them to come back in a hurry, would you?

And Lastly, Don’t Forget Content Quality 

Gone are the days (thank goodness) of churned out cookie-cutter content that just filled a page for the sake of it. The quality of your content, along with a clear understanding of what you need that content to achieve, is actually a lot more important than you think.  

You might well tick off all the boxes I’ve mentioned above and you might get indexed. Great, well done. However, I urge you to try to remember why you want to be indexed in the first place.  

Not to show your technical prowess (although, good for you). Not to try to outsmart Google (it’ll never happen). The real marketing reason is that you want your compelling salesmanship or your expert blog content to be found and seen by real human beings; the human beings who are your potential customers. 

Always remember that and I can promise you, the rest becomes so much easier. 

Need Help with Getting your Pages Indexed?

If you would like more information or more help with indexing your pages, feel free to get in touch with me on frankorman@leadgenerators.co.uk.  I look forward to helping you get the most out of your website.