To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots. When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.[46]

Your website should be the cornerstone of your content building strategy. Your website is the one piece of real-estate on the internet that you truly own. Facebook, Twitter, and Google are nice, but you don’t own anything on those platforms other than your intellectual property rights. The services they provide now may change in the future. Invest in the content on your website.


Solid analysis on this tough topic Rand. It will definitely be interested to see what in-serp features Google continues to add to keep you on their site as opposed to clicking through to a website. I think SEOs need to take more consideration into branding and content marketing tactics in order to supplement potential lost organic traffic as time goes on.
If you take full advantage of social media to promote your top quality content, you want to keep in mind that engagement matters for SEO. Engagement helps to not only improve your online reputation but also to make connections and generate leads for your business. Content that gets tons of engagement on social media platforms will rank for the topics they cover. According to Moz, “to determine success, an algorithm looks at whether users engaged. If more people engage that’s a clear sign that their algorithm is showing this right content; if not, their systems will audition other content instead to find something that generates that interest.”

(function(){"use strict";function s(e){return"function"==typeof e||"object"==typeof e&&null!==e}function a(e){return"function"==typeof e}function u(e){X=e}function l(e){G=e}function c(){return function(){r.nextTick(p)}}function f(){var e=0,n=new ne(p),t=document.createTextNode("");return n.observe(t,{characterData:!0}),function(){t.data=e=++e%2}}function d(){var e=new MessageChannel;return e.port1.onmessage=p,function(){e.port2.postMessage(0)}}function h(){return function(){setTimeout(p,1)}}function p(){for(var e=0;et.length)&&(n=t.length),n-=e.length;var r=t.indexOf(e,n);return-1!==r&&r===n}),String.prototype.startsWith||(String.prototype.startsWith=function(e,n){return n=n||0,this.substr(n,e.length)===e}),String.prototype.trim||(String.prototype.trim=function(){return this.replace(/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g,"")}),String.prototype.includes||(String.prototype.includes=function(e,n){"use strict";return"number"!=typeof n&&(n=0),!(n+e.length>this.length)&&-1!==this.indexOf(e,n)})},"./shared/require-global.js":function(e,n,t){e.exports=t("./shared/require-shim.js")},"./shared/require-shim.js":function(e,n,t){var r=t("./shared/errors.js"),i=(this.window,!1),o=null,s=null,a=new Promise(function(e,n){o=e,s=n}),u=function(e){if(!u.hasModule(e)){var n=new Error('Cannot find module "'+e+'"');throw n.code="MODULE_NOT_FOUND",n}return t("./"+e+".js")};u.loadChunk=function(e){return a.then(function(){return"main"==e?t.e("main").then(function(e){t("./main.js")}.bind(null,t))["catch"](t.oe):"dev"==e?Promise.all([t.e("main"),t.e("dev")]).then(function(e){t("./shared/dev.js")}.bind(null,t))["catch"](t.oe):"internal"==e?Promise.all([t.e("main"),t.e("internal"),t.e("qtext2"),t.e("dev")]).then(function(e){t("./internal.js")}.bind(null,t))["catch"](t.oe):"ads_manager"==e?Promise.all([t.e("main"),t.e("ads_manager")]).then(function(e){undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined}.bind(null,t))["catch"](t.oe):"publisher_dashboard"==e?t.e("publisher_dashboard").then(function(e){undefined,undefined}.bind(null,t))["catch"](t.oe):"content_widgets"==e?Promise.all([t.e("main"),t.e("content_widgets")]).then(function(e){t("./content_widgets.iframe.js")}.bind(null,t))["catch"](t.oe):void 0})},u.whenReady=function(e,n){Promise.all(window.webpackChunks.map(function(e){return u.loadChunk(e)})).then(function(){n()})},u.installPageProperties=function(e,n){window.Q.settings=e,window.Q.gating=n,i=!0,o()},u.assertPagePropertiesInstalled=function(){i||(s(),r.logJsError("installPageProperties","The install page properties promise was rejected in require-shim."))},u.prefetchAll=function(){t("./settings.js");Promise.all([t.e("main"),t.e("qtext2")]).then(function(){}.bind(null,t))["catch"](t.oe)},u.hasModule=function(e){return!!window.NODE_JS||t.m.hasOwnProperty("./"+e+".js")},u.execAll=function(){var e=Object.keys(t.m);try{for(var n=0;n=c?n():document.fonts.load(l(o,'"'+o.family+'"'),a).then(function(n){1<=n.length?e():setTimeout(t,25)},function(){n()})}t()});var w=new Promise(function(e,n){u=setTimeout(n,c)});Promise.race([w,m]).then(function(){clearTimeout(u),e(o)},function(){n(o)})}else t(function(){function t(){var n;(n=-1!=y&&-1!=v||-1!=y&&-1!=g||-1!=v&&-1!=g)&&((n=y!=v&&y!=g&&v!=g)||(null===f&&(n=/AppleWebKit\/([0-9]+)(?:\.([0-9]+))/.exec(window.navigator.userAgent),f=!!n&&(536>parseInt(n[1],10)||536===parseInt(n[1],10)&&11>=parseInt(n[2],10))),n=f&&(y==b&&v==b&&g==b||y==x&&v==x&&g==x||y==j&&v==j&&g==j)),n=!n),n&&(null!==_.parentNode&&_.parentNode.removeChild(_),clearTimeout(u),e(o))}function d(){if((new Date).getTime()-h>=c)null!==_.parentNode&&_.parentNode.removeChild(_),n(o);else{var e=document.hidden;!0!==e&&void 0!==e||(y=p.a.offsetWidth,v=m.a.offsetWidth,g=w.a.offsetWidth,t()),u=setTimeout(d,50)}}var p=new r(a),m=new r(a),w=new r(a),y=-1,v=-1,g=-1,b=-1,x=-1,j=-1,_=document.createElement("div");_.dir="ltr",i(p,l(o,"sans-serif")),i(m,l(o,"serif")),i(w,l(o,"monospace")),_.appendChild(p.a),_.appendChild(m.a),_.appendChild(w.a),document.body.appendChild(_),b=p.a.offsetWidth,x=m.a.offsetWidth,j=w.a.offsetWidth,d(),s(p,function(e){y=e,t()}),i(p,l(o,'"'+o.family+'",sans-serif')),s(m,function(e){v=e,t()}),i(m,l(o,'"'+o.family+'",serif')),s(w,function(e){g=e,t()}),i(w,l(o,'"'+o.family+'",monospace'))})})},void 0!==e?e.exports=a:(window.FontFaceObserver=a,window.FontFaceObserver.prototype.load=a.prototype.load)}()},"./third_party/tracekit.js":function(e,n){/**

As of 2009, there are only a few large markets where Google is not the leading search engine. In most cases, when Google is not leading in a given market, it is lagging behind a local player. The most notable example markets are China, Japan, South Korea, Russia and the Czech Republic where respectively Baidu, Yahoo! Japan, Naver, Yandex and Seznam are market leaders.
Click through rates: Searches using terms that denote high purchase intent such as product or brand-specific keywords will get more clicks than organic results. The advantage of paid search can clearly be seen in the Internet retailers MarketLive Performance Index data. For the year 2013 as a whole, PPC accounted for 36.5% of search traffic but an outsized 47.9% of revenue from search.
James, you give a great template for how a business needs to move forward in their chosen niche online.  Quite informative and the meeting of minds has been something a number of us have done online and in person to gain better insight into our small similar businesses.  Thank you for sharing your detailed approach to increasing organic traffic...content still is king.
Website saturation and popularity, or how much presence a website has on search engines, can be analyzed through the number of pages of the site that are indexed by search engines (saturation) and how many backlinks the site has (popularity). It requires pages to contain keywords people are looking for and ensure that they rank high enough in search engine rankings. Most search engines include some form of link popularity in their ranking algorithms. The following are major tools measuring various aspects of saturation and link popularity: Link Popularity, Top 10 Google Analysis, and Marketleap's Link Popularity and Search Engine Saturation.
The first is that you look at who are your current customers, who are your prospects, what groups are they in, and join those same groups if they’re relevant and not too title specific to a marketer or a different field that you’re not in. But really starting with the folks that you already are working with and using them to figure out which groups are most relevant. This is also especially important because I think it was about a year ago, LinkedIn now made all the groups private. So you can’t go in and look at what’s going on in the group. And it’s really hard to know which ones are the best ones to join just by doing simple searches on say the title of the group. So, start with the folks that you know.
Hey, Matt! Thank you for your sharing, and I learned much from it, but I still have a question. We began to do SEO work for our site 2 years ago, and our organic traffic grew 5 times ( from 8K to 40K every day). But two years later, it is very difficult to get it grow more, even it drop to 3.2K every day. So can you give me any advice to make our site's traffic grow again? Thank you in advance!

I am too much late to commenting on this article. I want to read "How much get Organic traffic by SEO", found your your article on top & really very interesting. James Norquay, you did good research.I think Now days google block mostly SEO activities. Is this worthy for current marketing scnerio?If any other post to related strategy for increasing Organic traffic, you can reffer me.

Like having a good book title, it’s always important to have a strong title for your content. According to Moz, having a strong SEO title is one of the most important on-page SEO element. A proven method to ensure that you’re writing the best possible title is to position the most important keyword first, followed by the second most important, and then the brand name. It’s also important to note that title tags should only be around 50-60 characters long and be very relevant to the content on the page. Apply this seo optimization can help you jump the ranks on search engines.


The most common form of organic SEM is search engine optimization (SEO). SEO refers to a variety of techniques designed to help your website rank higher in search engine results. Optimizing your website involves doing a little bit of research on what keywords or phrases your customers or potential customers are searching for when they are looking for your products or services online. It then involves writing web content using those keywords in a way that is both easy for search engines to pick up but still readable and pleasant for your website visitors.
Google is currently been inundated with reconsideration requests from webmasters all over the world. On public holidays the Search Quality teams do not look at reconsideration requests. See below analysis. From my experience it can take anywhere from 15-30+ days for Google to respond to reconsideration requests; during peak periods it can even take longer.
Facebook Ads and other social media ad platforms, for example, are pay-per-click platforms that do not fall under the SEM category. Instead of showing your ads to people who are searching for similar content like search ads do, social media sites introduce your product to people who happen to be just browsing through their feeds. These are two very, very different types of online advertising.

So for the last 19 years or 20 years that Google has been around, every month Google has had, at least seasonally adjusted, not just more searches, but they've sent more organic traffic than they did that month last year. So this has been on a steady incline. There's always been more opportunity in Google search until recently, and that is because of a bunch of moves, not that Google is losing market share, not that they're receiving fewer searches, but that they are doing things that makes SEO a lot harder.
While there are several HTML tagging techniques that improve a page’s Google SEO results, creating relevant page content is still the best way to rank high. A big part of content creation is your use of targeted keywords. You should include important keywords in your first 50 words, since early placement can be a signal of relevance. And while you should never repeat keywords too often at the expense of good writing, you should repeat keywords in your content two or three times for short pages and four to six times for longer pages. Also, you may wish to use some keyword variation in your content – such as splitting keywords up – as this could potentially improve your ranking.
Small business owners sometimes think that search engine marketing (SEM), also known as pay-per-click advertising (PPC), is not lucrative option for them. They may think they can’t afford it, or that their online presence is not important if they are a local or service-based business. The truth is, as search engines have undeniably become a part of our lifestyles as consumers, there are many ways to leverage them for businesses of any size. This post will introduce you to the basics and benefits of search engine marketing (SEM).

The first way that social media can (indirectly) help your search engine ranking is through content promotion. We might write, film, or record tons of great quality, keyword-optimized content but still not get many eyes and ears consuming it. Social media allows you to take the quality content you have worked hard to produce and promote it on several channels.
There are many reasons explaining why advertisers choose the SEM strategy. First, creating a SEM account is easy and can build traffic quickly based on the degree of competition. The shopper who uses the search engine to find information tends to trust and focus on the links showed in the results pages. However, a large number of online sellers do not buy search engine optimization to obtain higher ranking lists of search results, but prefer paid links. A growing number of online publishers are allowing search engines such as Google to crawl content on their pages and place relevant ads on it.[16] From an online seller's point of view, this is an extension of the payment settlement and an additional incentive to invest in paid advertising projects. Therefore, it is virtually impossible for advertisers with limited budgets to maintain the highest rankings in the increasingly competitive search market.
Though it can take a lot of time and effort to develop one, having an organic marketing strategy is important for any business to find success. Essentially, organic marketing is about growing your audience naturally versus using paid advertising and marketing tactics. To build a successful strategy, you’ll need to accumulate a variety of string organic marketing ideas that includes social media, search, and email marketing. But why should I look into organic marketing when I can just pay for everything? That’s a great question. The thing is organic marketing can bring a lot of benefits for your business including more authentic customer engagement and brand loyalty. Now, we’re not saying to cut paid advertising from your plan but a mix of organic and paid marketing strategies will be very effective. Here are some tips that you can use to improve your organic marketing strategy.
You could get even more specific by narrowing it down to customer base. Is there a specific group of clients you tend to serve? Try including that in your long-tail key phrase. For example: “SEO agency for non-profits in Albuquerque NM.” That’s a key phrase you’re a lot more likely to rank for. Not to mention it will also attract way more targeted, organic traffic than a broad key phrase like “SEO agency.”
hey james - congrats on your success here. just a question about removing crummy links. for my own website, there are hundreds of thousands of backlinks in webmaster tools pointing to my site. The site has no penalties or anything  - the traffic seems to be growing every week. would you recommend hiring someone to go through the link profile anyway to remove crummy links that just occur naturally?
In today’s complex organizations, IT departments are already overburdened and experiencing expertise gaps, shrinking budgets and only so many hours to get IT done. Migrating to O365 is a top priority for many organizations, but it can also be costlier, more complicated and more time-consuming than expected — especially when internal IT resources are already stretched … Continue Reading...
Many people know which search results are paid versus natural and often give precedence to natural search options. On the other hand, if your website shows up on later pages of the search results, you might have better luck using PPC to get seen. Because PPC costs money, you might consider using it for items that you sell, as opposed to promoting a free offer or your blog. With that said, many people have successfully used PPC marketing to promote a free lead magnet.

Search engine marketing’s greatest strength is that it offers advertisers the opportunity to put their ads in front of motivated customers who are ready to buy at the precise moment they’re ready to make a purchase. No other advertising medium can do this, which is why search engine marketing is so effective and such an amazingly powerful way to grow your business.
Link building is a catchall term for the practice of creating new external links to your site. Beyond creating great content people want to share, guest blogging and asking webmasters from authoritative sites relevant to your business to link back to your pages are great ways to build links. When possible, use keywords are the anchor text for your links, as this will help send signals to Google that your pages are relevant for those terms.

The first one is pretty powerful and pretty awesome, which is investing in demand generation, rather than just demand serving, but demand generation for brand and branded product names. Why does this work? Well, because let's say, for example, I'm searching for SEO tools. What do I get? I get back a list of results from Google with a bunch of mostly articles saying these are the top SEO tools. In fact, Google has now made a little one box, card-style list result up at the top, the carousel that shows different brands of SEO tools. I don't think Moz is actually listed in there because I think they're pulling from the second or the third lists instead of the first one. Whatever the case, frustrating, hard to optimize for. Google could take away demand from it or click-through rate opportunity from it.

Pay Per Click (PPC) advertising is the most common form of paid SEM. PPC ads are the ones you see at the top of your Google search with the word “ad” written discreetly next to the link. Search engines such as Google sell keywords to the highest bidder. One of the nice things about this form of advertising is that – as the name suggests – you only pay for the ad when someone actually clicks on it.
Organic-search traffic is the only search traffic you should be interested in. This is the traffic you get without any direct action on your part. It’s not something you can achieve with ease, though. The statistics say that 83 percent of organic clicks go to the first four results under most categories on search engines. So, to see your site attain that status, lay the foundations well in advance for the traffic you want.
Organic traffic is the primary channel that inbound marketing strives to increase. This traffic is defined as visitors coming from a search engine, such as Google or Bing. This does not include paid search ads, but that doesn’t mean that organic traffic isn’t impacted by paid search or display advertising, either positively or negatively. In general, people trust search engines, and sayings such as “just Google it” reinforce that humans are tied to the search engine. Thus, paid search, display, or even offline campaigns can drive searches, which may increase organic traffic while those campaigns are running.
×