If you’re a self-publishing author, one of the important chores you can do to avoid having to actually write anything is to see how the two most popular search engines report back on you and your work. The dominant US search engine is of course Google, with about 66% of the search market, while Microsoft’s Bing makes up most of the remainder. (Bing powers not only the Bing.com site, which is 16% of US search, but also Yahoo.com’s search engine, which accounts for roughly 12 percent.)
If you’re not already obsessed with your personal and professional rankings on search engines and social networks, the good news is that you don’t have to become your own favorite celebrity in order to make sure people can find you. All you need is a basic understanding of how search engines work, and how people may try to find you using various words and phrases — like, say, your name or the title of something you wrote.
While it may seem as if all search engines see the internet the same way, that’s not actually the case. In order to return hits for any search you conduct, the search engine you’re using must have already visited the page you’re looking for in order to point you to it. This process of scouring the web for content is done automatically by what are called web crawlers, which follow links from one page to the next. In general web crawlers do a good job of indexing most of what’s available on the web, but depending on how often a search engine crawls a particular site there can be some lag between when a page is published (or updated) and when that page is indexed.
To get around this lag it’s possible to go to most search engines and submit pages and sites directly so search engines know where to find new content. Since this is a bit of a chore you can also use various aggregating services to submit new pages or sites to most of the popular search engines at once, albeit often for a fee. In my own experience it’s almost never necessary to submit URL’s to search engines yourself, and in no case would I pay to have this done. In a relatively short amount of time almost any new content will show up after the web crawlers make their next sweep.
From the point of view of authorship the internet has changed everything. Not only is it now possible to distribute content to almost anyone, it’s possible to make yourself visible to people who are looking for you. As we’ve just seen, however, the internet is not the omniscient portal it purports to be, and search engines are not infallible seers.
While the internet allows self-publishing authors to get around the gatekeeping inherent in the traditional publishing industry, every service provider on the internet has their own agenda and their own means of driving, throttling or precluding traffic if that traffic is respectively friendly, potentially risky or openly hostile to their corporate goals. As previously discussed relative to ISBN’s and my decision to opt out of that monopolistic racket, it is possible for self-published authors to use the internet itself as a means of making themselves available. If you have your own author site and your own point of sale (POS), would-be customers should be able to find your work either by searching for your name, which will lead to your website, which should have links to the POS for each of your works, or by searching for titles directly, which should lead to the POS for that work, which should also include a link back to your author site.
In theory it’s as easy as it sounds. In practice, however, bad things can still happen to good people, which is why it’s important to do regular searches to make sure all the services you depend on (including internet search) are actually doing what they’re supposed to be doing. Which brings me to efforts I made over the past week to make my grandmother’s book available via CreateSpace, and how one particular search engine is impeding that goal.
After making the book available for sale via CreateSpace, I waited a couple of days to see if Google and Bing would pick it up on their own. When that didn’t happen I first submitted the page URL to Google, then a few hours later I submitted it to Bing. After about a day or so the page became available through highly targeted searches using Google, but still failed to show up on Bing. After another day I submitted the page to Bing again, but as of this post Bing is still not returning the book page as a hit, even if I specifically limit the search to the CreateSpace site.
After trying every trick I knew to get Bing to see the page for my grandmother’s book I decided to use Bing to try to find other pages on CreateSpace, including, particularly, a collection of my own short stories that has been available on that site for over two years. Unfortunately, when I searched for the title of my book (The Year of the Elm) and Createspace, Bing refused to return any hits for that relatively longstanding page. (Google had long ago reliably indexed the page.)
If you want to perform a targeted search on either Google or Bing, you can do so by prepending site: to the name of the specific site you want to search. Yet even when I did that, limiting the search for the title of my book to the CreateSpace site, I still got no hits to the Year of the Elm page even though I had that page loaded in another browser tab. According to Bing, my book, which is clearly visible and available on CreateSpace, is not visible or available on CreateSpace:
Obviously this kind of technological gatekeeping — regardless of the cause — is as much a threat to building an audience and reaching customers as were any of the gatekeeping practices of the traditional publishing industry. Yet even knowing as much as I do about search engines I don’t have any idea why Bing cannot see my book or my grandmother’s book on CreateSpace. The only thing I can say with certainty is that it seems as if the CreateSpace site has not been fully indexed by Bing for some reason. (If you’re the nerdy type, you can read CreateSpace’s robots.txt file here.)
If you’re a self-publishing writer and have a book available on CreateSpace you may want to see if Google and Bing can find your work using both basic keyword searches and the site-specific method discussed above. If you’re having the same problem, feel free to drop a note in the comments because I’m curious if this problem is widespread.
— Mark Barrett