This post was collaboratively researched and written by Ben Swofford and Ben Pearl.
Properly submitting a website or webpage to Google (and other search engines) is a critical step to ensure your content can successfully drive traffic and other business goals for your website. You can’t simply hit “publish” and expect your URLs to immediately appear in search results. Submitting your content to the search engine directly will encourage the page to be indexed in a timely manner.
This process is most critical when you launch a new website or when you’re transitioning following a redesign or domain change. However, submitting your site to search engines is also a recommended step after you’ve added a new high-priority page to an existing site or edited important older pages.
Skipping this step may prevent or delay indexation of this new content, and that means there is real money on the line: One study found that organic search is responsible for more than 50% of all website traffic. Without that significant source of traffic, the revenue driven by your website may decrease.
In this post, I’ll share some tips on how to accomplish this process. You can skip ahead here:
Why You Should Submit Your Site to Google
If your website never gets indexed by Google, then it won’t show up in any search results. Submitting directly to Google ensures that important changes to your site’s content are recognized and updated in the Google index. This also helps you see and respond to results of recent optimization efforts more quickly.
Additional benefits of manually submitting your website include:
- Directly control the process – Gain peace of mind knowing your content efforts are not being wasted because of slow indexation or other unanticipated issues with automatic crawling.
- See quick results – If you need a quick SEO win, submitting to a search engine is potentially an easy improvement opportunity.
- It’s free – Besides time, there is no cost to submitting a sitemap or URL to Google. (Warning: Do NOT pay for a submission service to help you with this.)
Why You Might Be Tempted to Skip This Step
It’s important to note that even if you don’t manually submit your site to Google, your webpages will most likely (eventually) be indexed and show up in search results. This is because Google’s web crawlers will find your site and index it by other means (see below). This is especially true if you are maintaining an up-to-date sitemap.xml file that has already been previously submitted to Google.
However, even though these more automatic processes exist, manually submitting your URLs is a great option when you need to see results quickly. It also ensures that Google finds updates that it might not recognize on its own right away. We have seen some pages take weeks to be updated in search results when waiting for the crawlers to do the work on their own.
How Google Finds Content
“Crawling” is the method used by Google Search and other search engines to automatically find, “read,” and index websites on the internet. A crawler (also sometimes called a “bot,” “robot,” or “spider;” Google’s is also known as the “Googlebot”) is a program that is on a never-ending quest to discover and index information on the web. These bots do follow directives from code known as robots meta tags, so it is possible to tell Google to ignore content that you don’t want indexed, too.
These are the basic steps of how a bot will interact with your website:
- The bot will visit your site based on the URLs it already indexed from previous crawls.
- On those pages, it will look for changes that need to be updated in the index.
- The bot will follow both internal and external links on that page, finding new pages to review and index.
- The indexed page will also include the page metadata such as Page Titles and structured Schema markup used to produce rich search results.
LEARN MORE ABOUT CRAWLERS AND SEARCH ENGINES
One of the best resources to better understand this complex process is Google’s handy How Search Works website. Check it to learn more about the process of how Google builds and maintains search results.
This 5-minute video also provides a good starting point:
If you want to dive deeper, read the How Search Works for Site Owners articles on Google’s developer website.
How to Check If Your Site is Already Listed on Google
Think your site might already be indexed? The easiest way to check is by using the site search function:
- Go to Google and type in “site:yoursitename.com” and hit enter.
Every page that Google has indexed from that domain will be shown.
You can also do this for subdirectories or specific pages by simply entering the full URL, such as site:fourfront.us/blog.
Another quick method for checking your current indexation status is by using the Coverage report in Google Search Console. This report will tell you how many pages have been indexed, as well as those with errors or other issues. This is a great place to troubleshoot indexing problems, since Google will tell you why pages have been skipped over.
Getting Started with Submitting Your Site
For best results, here are a few things you will need before you get started:
- Admin access to your website (or to the site’s developers/managers who can help you out) in order to add things like sitemap XML files and robots.txt files, or editing plugins that manage an automatically generated sitemap file.
- Access to active webmaster tools for your chosen search engines, most notably including Google Search Console. These accounts will need to be verified, meaning they have been properly connected to your website and your account has verified website ownership.
Submitting Your Site to Search Engines
Broadly, there are 2 basic, proactive options for getting your site indexed across all search engines:
- Submit an updated sitemap.xml file
- Manually request indexing for individual URLs, such as with Google’s URL Inspection Tool
Between these two options, the manual request is best for one page at a time, while the sitemap approach is best for indexing multiple URLs in one go. (Maintaining an updated sitemap file is also best practice for sustaining your site’s “basic SEO.” I personally like using the Yoast WordPress plugin to automate this.)
Below are more specific instructions for accomplishing these tasks on the major search engines, with the emphasis on the all-important Google.
Submitting Your Site to Google
Submitting a sitemap to Google can be done by completing a few simple steps:
- Visit your website’s Search Console dashboard.
- Navigate to the Sitemaps page under “Index” in the sidebar menu.
- Enter your sitemap’s URL and click submit. That’s it!
Below the “Add a new sitemap” area, you can see your previously submitted sitemap(s) including the date of submission, the last time Google “read” (crawled) the sitemap, and how many URLs it found there.
After submitting, head over to the Coverage report to see the current status of your sitemap’s pages. Be sure to use the dropdown in the top left corner to “filter to sitemaps.”
If you’re not sure how to read this report, check out Google’s Index Coverage report Help page.
Requesting indexation for individual URLs is currently done using the URL Inspection Tool in Search Console. This can be done by simply entering the URL in the search bar at the top of the page.
- Copy and paste your URL into the “Inspect any URL” search bar.
- Once the Inspection report loads, click “Request Indexing.”
As mentioned earlier, this method is preferred if you just want to index a few URLs at once. Be aware that there is a quota for submitting individual URLs. Although the exact limit is not stated, one Help forum post mentioned getting cut off after around 5-10 submissions. (They apparently wanted to submit 50 per day—this would be a big waste of your time, so please don’t try that approach.)
After submitting (with either method) you have a few ways you can check the indexation status:
- Use a “site:” search to see your pages directly in search results.
- Check the Coverage report in Search Console.
- Use the URL Inspection tool in Search Console.
- For previously indexed pages that you’ve recently updated, you can check the cached version of your page to see if updates have been indexed yet.
Submitting Without a Google Account?
My keyword research for this topic has informed me that some people would like a way to submit a URL to Google without signing in. However, I have not found a reliable way to do this process without a Google account. If you’re really against giving Google your information, I suggest creating a Gmail account solely for the purpose of managing SEO efforts. This will surely make your life easier than looking for outdated or unreliable URL submission techniques.
Submitting Your Site to Bing
Bing Search is the search engine owned and operated by Microsoft. It doesn’t have the enormous market share that Google has, but it is the default search engine within the Edge browser, Cortana, and other prominent products. As mentioned in other sections below, Bing also feeds into other search engines. So, it is best practice to ensure your website is properly indexed in Bing results.
To submit a sitemap to Bing, visit Bing Webmaster Tools. Then:
- Go to the Sitemaps page in the sidebar menu.
- Your sitemap may have been detected automatically. If not, click the “Submit sitemap” button.
- Enter your sitemap URL and click “Submit.”
Note: If you’ve never used this tool before, you will first need to verify your site. One great feature if you are new to Bing is the ability to import sites directly from Google Search Console to Bing Webmaster Tools. You can even do this for up to 100 websites at once if needed.
Requesting indexation for individual URLs is currently done using Bing’s URL Inspection menu:
- Click “URL Inspection” in the left sidebar menu.
- Click “Submit URLs” to open a sidebar submission area.
- Enter each URL you’d like to index on a new line in the form. You can submit up to 100 URLs to Bing at one time.
- Click “Submit.”
Note: Bing, perhaps in an effort to encourage more engagement, has a 100 URL submission quota that resets every 3 hours. Very generous! However, you can only submit root domain URLs (pages on a subdomain like subdomain.site.com/page will not be accepted).
It’s not entirely clear how long Bing will take to index your pages, but some sources mention it should take 1-2 days. If you’re having issues, check out the Why is my site not in the Bing Index? Help page.
Submitting Your Site to Other Search Engines
Yahoo!
For those readers who like their search engines shouting(!) at them, you are in luck: Once you have submitted your site to Bing, Yahoo(!) search is already taken care of thanks to Yahoo’s integration with Bing Webmaster Tools. Therefore, there is no special action needed to submit a site or URL to Yahoo.
DuckDuckGo
DuckDuckGo is the search network known for its commitment to user privacy.
The DuckDuckGo index is automatically updated and maintained using information from “over four hundred sources” including Bing, Wikipedia, and its own crawler known as DuckDuckBot.
There is no manual submission option for DuckDuckGo, so similar to Yahoo, no special action is needed to get your website on this search platform. There is also currently no way to add or change information (such as correcting issues you may see) beyond correcting the “misinformation at the source.”
Baidu
Baidu is the primary search engine in China. Submitting sitemaps and URLs to this platform is crucial for any website operating in China. The submission process is very complex and is only recommended if you are targeting users in China.
To learn more about this process, check out this guide.
Yandex
Yandex is the primary search engine in Russia, accounting for over 60% of market share in that country. However, it is not used anywhere else, so submitting your website to Yandex is only necessary if you’re targeting Russian users.
Submitting your site can be accomplished using the Yandex Webmaster Tools website. Yoast has a helpful walkthrough for how to use this tool.
How Long Does Indexing Take?
When you’re in a rush, you’re likely to be asking how long it will take for Google to index your site. Unfortunately, you don’t have a ton of control here. According to Google, “crawling can take anywhere from a few days to a few weeks” (once your site is actually crawled, the indexing happens right away).
If you’re feeling impatient, keep in mind that requests are cued and requesting multiple times will not speed up the process.
Another important thing to keep in mind is that just by taking a proactive approach to submitting your website, you are already likely speeding up indexation significantly. One Moz study (and a subsequent HubSpot analysis) found that, on average, it took 23 hours for a URL to be crawled when it wasn’t included in a sitemap versus 14 minutes for a URL that was present in a sitemap.
Source: Moz
Common Indexing Issues
If you’re running into issues, there’s no secret bullet for getting indexing done faster. But here are a few things you can do if you’re having trouble:
- Check your page: Use the URL Inspection Tool and be sure there are no issues such as misplaced noindex tags, canonical tags, or other issues.
- Optimize internal links: Add high-quality internal links to your page, especially from your existing high-ranking pages.
- Double-check your sitemap file: Be sure there are no errors. If you fix any, be sure to resubmit to Google.
- Double-check your robots.txt file: Ensure crawlers can properly access your site by testing the robots.txt file using Google’s tester tool. If you have access to ScreamingFrog, you can also use their custom robots.txt feature to test your file.
- Double-check that your URL isn’t redirecting: For example, you might be submitting a version without “www” when the final URL actually includes it. You should also be aware of the presence of trailing slashes.
- Generate backlinks: High-quality backlinks from other websites could help your page get crawled more quickly while also boosting its authority.
- Share on social: In the spirit of getting more links to your page out there, consider also sharing the link on social media as well as publishing a Google Business Profile post. (Note: This is not guaranteed to speed up indexing, but when I’ve been in a rush, it felt like something I could do that might help a bit—and it certainly couldn’t hurt).
Why Submitting Your Site is Not Enough
Now that you’ve submitted your website to all the search engines, you are good to go! All your SEO is perfect, right?
Of course, this is not usually the case. Reliable search engine optimization is built on foundational things like indexing, but that is only the beginning. In order to drive the most business from your website, additional strategy and tactics are required.
For example, an unoptimized page might be indexed, but if it sits then you are essentially giving up business to the competitors who outrank you. With Strong SEO, the traffic you get from organic search can increase exponentially with each ranking improvement. One study found that moving up one spot increases click-through rate (CTR) by 31%!
To help you get to the next level, explore FourFront’s data-powered SEO services, or contact us to learn more.
